[linux] How to generate a core dump in Linux on a segmentation fault?

I have a process in Linux that's getting a segmentation fault. How can I tell it to generate a core dump when it fails?

This question is related to linux bash unix coredump tcsh

The answer is


This depends on what shell you are using. If you are using bash, then the ulimit command controls several settings relating to program execution, such as whether you should dump core. If you type

ulimit -c unlimited

then that will tell bash that its programs can dump cores of any size. You can specify a size such as 52M instead of unlimited if you want, but in practice this shouldn't be necessary since the size of core files will probably never be an issue for you.

In tcsh, you'd type

limit coredumpsize unlimited

As explained above the real question being asked here is how to enable core dumps on a system where they are not enabled. That question is answered here.

If you've come here hoping to learn how to generate a core dump for a hung process, the answer is

gcore <pid>

if gcore is not available on your system then

kill -ABRT <pid>

Don't use kill -SEGV as that will often invoke a signal handler making it harder to diagnose the stuck process


What I did at the end was attach gdb to the process before it crashed, and then when it got the segfault I executed the generate-core-file command. That forced generation of a core dump.


There are more things that may influence the generation of a core dump. I encountered these:

  • the directory for the dump must be writable. By default this is the current directory of the process, but that may be changed by setting /proc/sys/kernel/core_pattern.
  • in some conditions, the kernel value in /proc/sys/fs/suid_dumpable may prevent the core to be generated.

There are more situations which may prevent the generation that are described in the man page - try man core.


By default you will get a core file. Check to see that the current directory of the process is writable, or no core file will be created.


Ubuntu 19.04

All other answers themselves didn't help me. But the following sum up did the job

Create ~/.config/apport/settings with the following content:

[main]
unpackaged=true

(This tells apport to also write core dumps for custom apps)

check: ulimit -c. If it outputs 0, fix it with

ulimit -c unlimited

Just for in case restart apport:

sudo systemctl restart apport

Crash files are now written in /var/crash/. But you cannot use them with gdb. To use them with gdb, use

apport-unpack <location_of_report> <target_directory>

Further information:

  • Some answers suggest changing core_pattern. Be aware, that that file might get overwritten by the apport service on restarting.
  • Simply stopping apport did not do the job
  • The ulimit -c value might get changed automatically while you're trying other answers of the web. Be sure to check it regularly during setting up your core dump creation.

References:


To check where the core dumps are generated, run:

sysctl kernel.core_pattern

or:

cat /proc/sys/kernel/core_pattern

where %e is the process name and %t the system time. You can change it in /etc/sysctl.conf and reloading by sysctl -p.

If the core files are not generated (test it by: sleep 10 & and killall -SIGSEGV sleep), check the limits by: ulimit -a.

If your core file size is limited, run:

ulimit -c unlimited

to make it unlimited.

Then test again, if the core dumping is successful, you will see “(core dumped)” after the segmentation fault indication as below:

Segmentation fault: 11 (core dumped)

See also: core dumped - but core file is not in current directory?


Ubuntu

In Ubuntu the core dumps are handled by Apport and can be located in /var/crash/. However, it is disabled by default in stable releases.

For more details, please check: Where do I find the core dump in Ubuntu?.

macOS

For macOS, see: How to generate core dumps in Mac OS X?


In order to activate the core dump do the following:

  1. In /etc/profile comment the line:

    # ulimit -S -c 0 > /dev/null 2>&1
    
  2. In /etc/security/limits.conf comment out the line:

    *               soft    core            0
    
  3. execute the cmd limit coredumpsize unlimited and check it with cmd limit:

    # limit coredumpsize unlimited
    # limit
    cputime      unlimited
    filesize     unlimited
    datasize     unlimited
    stacksize    10240 kbytes
    coredumpsize unlimited
    memoryuse    unlimited
    vmemoryuse   unlimited
    descriptors  1024
    memorylocked 32 kbytes
    maxproc      528383
    #
    
  4. to check if the corefile gets written you can kill the relating process with cmd kill -s SEGV <PID> (should not be needed, just in case no core file gets written this can be used as a check):

    # kill -s SEGV <PID>
    

Once the corefile has been written make sure to deactivate the coredump settings again in the relating files (1./2./3.) !


It's worth mentioning that if you have a systemd set up, then things are a little bit different. The set up typically would have the core files be piped, by means of core_pattern sysctl value, through systemd-coredump(8). The core file size rlimit would typically be configured as "unlimited" already.

It is then possible to retrieve the core dumps using coredumpctl(1).

The storage of core dumps, etc. is configured by coredump.conf(5). There are examples of how to get the core files in the coredumpctl man page, but in short, it would look like this:

Find the core file:

[vps@phoenix]~$ coredumpctl list test_me | tail -1
Sun 2019-01-20 11:17:33 CET   16163  1224  1224  11 present /home/vps/test_me

Get the core file:

[vps@phoenix]~$ coredumpctl -o test_me.core dump 16163

Better to turn on core dump programmatically using system call setrlimit.

example:

#include <sys/resource.h>

bool enable_core_dump(){    
    struct rlimit corelim;

    corelim.rlim_cur = RLIM_INFINITY;
    corelim.rlim_max = RLIM_INFINITY;

    return (0 == setrlimit(RLIMIT_CORE, &corelim));
}

For Ubuntu 14.04

  1. Check core dump enabled:

    ulimit -a
    
  2. One of the lines should be :

    core file size          (blocks, -c) unlimited
    
  3. If not :

    gedit ~/.bashrc and add ulimit -c unlimited to end of file and save, re-run terminal.

  4. Build your application with debug information :

    In Makefile -O0 -g

  5. Run application that create core dump (core dump file with name ‘core’ should be created near application_name file):

    ./application_name
    
  6. Run under gdb:

    gdb application_name core
    

Maybe you could do it this way, this program is a demonstration of how to trap a segmentation fault and shells out to a debugger (this is the original code used under AIX) and prints the stack trace up to the point of a segmentation fault. You will need to change the sprintf variable to use gdb in the case of Linux.

#include <stdio.h>
#include <signal.h>
#include <stdlib.h>
#include <stdarg.h>

static void signal_handler(int);
static void dumpstack(void);
static void cleanup(void);
void init_signals(void);
void panic(const char *, ...);

struct sigaction sigact;
char *progname;

int main(int argc, char **argv) {
    char *s;
    progname = *(argv);
    atexit(cleanup);
    init_signals();
    printf("About to seg fault by assigning zero to *s\n");
    *s = 0;
    sigemptyset(&sigact.sa_mask);
    return 0;
}

void init_signals(void) {
    sigact.sa_handler = signal_handler;
    sigemptyset(&sigact.sa_mask);
    sigact.sa_flags = 0;
    sigaction(SIGINT, &sigact, (struct sigaction *)NULL);

    sigaddset(&sigact.sa_mask, SIGSEGV);
    sigaction(SIGSEGV, &sigact, (struct sigaction *)NULL);

    sigaddset(&sigact.sa_mask, SIGBUS);
    sigaction(SIGBUS, &sigact, (struct sigaction *)NULL);

    sigaddset(&sigact.sa_mask, SIGQUIT);
    sigaction(SIGQUIT, &sigact, (struct sigaction *)NULL);

    sigaddset(&sigact.sa_mask, SIGHUP);
    sigaction(SIGHUP, &sigact, (struct sigaction *)NULL);

    sigaddset(&sigact.sa_mask, SIGKILL);
    sigaction(SIGKILL, &sigact, (struct sigaction *)NULL);
}

static void signal_handler(int sig) {
    if (sig == SIGHUP) panic("FATAL: Program hanged up\n");
    if (sig == SIGSEGV || sig == SIGBUS){
        dumpstack();
        panic("FATAL: %s Fault. Logged StackTrace\n", (sig == SIGSEGV) ? "Segmentation" : ((sig == SIGBUS) ? "Bus" : "Unknown"));
    }
    if (sig == SIGQUIT) panic("QUIT signal ended program\n");
    if (sig == SIGKILL) panic("KILL signal ended program\n");
    if (sig == SIGINT) ;
}

void panic(const char *fmt, ...) {
    char buf[50];
    va_list argptr;
    va_start(argptr, fmt);
    vsprintf(buf, fmt, argptr);
    va_end(argptr);
    fprintf(stderr, buf);
    exit(-1);
}

static void dumpstack(void) {
    /* Got this routine from http://www.whitefang.com/unix/faq_toc.html
    ** Section 6.5. Modified to redirect to file to prevent clutter
    */
    /* This needs to be changed... */
    char dbx[160];

    sprintf(dbx, "echo 'where\ndetach' | dbx -a %d > %s.dump", getpid(), progname);
    /* Change the dbx to gdb */

    system(dbx);
    return;
}

void cleanup(void) {
    sigemptyset(&sigact.sa_mask);
    /* Do any cleaning up chores here */
}

You may have to additionally add a parameter to get gdb to dump the core as shown here in this blog here.


Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to unix

Docker CE on RHEL - Requires: container-selinux >= 2.9 What does `set -x` do? How to find files modified in last x minutes (find -mmin does not work as expected) sudo: npm: command not found How to sort a file in-place How to read a .properties file which contains keys that have a period character using Shell script gpg decryption fails with no secret key error Loop through a comma-separated shell variable Best way to find os name and version in Unix/Linux platform Resource u'tokenizers/punkt/english.pickle' not found

Examples related to coredump

Floating point exception( core dump undefined reference to `std::ios_base::Init::Init()' How do I analyze a program's core dump file with GDB when it has command-line parameters? Core dump file is not generated Core dump file analysis Core dumped, but core file is not in the current directory? How to analyze information from a Java core dump? How to generate a core dump in Linux on a segmentation fault?

Examples related to tcsh

Changing default shell in Linux Check if a file is executable How to determine the current shell I'm working on Can a shell script set environment variables of the calling shell? How to generate a core dump in Linux on a segmentation fault?