[opensuse] coredump size
Hi, After running an update on my server, a private application crashed when told to exist. This was expected, but it dumped a huge core, then compressed it, which took several minutes, during which time I had to wait before rebooting, or it would probably stall:
Isengard:~ # ls -lh /var/lib/systemd/coredump total 2,1G -rw-r----- 1 root root 32M Aug 17 20:28 .#core.Kodi_MovistarTV.1000.50e69b420ce4463dab76ed6532adb941.21327.1502994490000000.xz17cedf1c2919f899 -rw-r----- 1 root root 153M Aug 17 20:28 .#core.Kodi_MovistarTV.1000.50e69b420ce4463dab76ed6532adb941.21327.1502994490000000c1fb45b2282ac57c -rw-r-----+ 1 root root 1,8G Aug 17 20:26 core.Kodi_MovistarTV.1000.50e69b420ce4463dab76ed6532adb941.22507.1502992827000000.xz -rw-r-----+ 1 root root 13M Aug 17 20:01 core.kodi\x2ebin.1000.50e69b420ce4463dab76ed6532adb941.1999.1502992785000000.xz -rw-r-----+ 1 root root 11M Aug 17 05:53 core.kodi\x2ebin.1000.50e69b420ce4463dab76ed6532adb941.21167.1502941816000000.xz Isengard:~ #
Notice the huge size, 1.8 GB. The questions are: can I choose compress options, so that it compresses faster and less? Can I configure not to try dump core of certain named processes? I see the configuration is done in "coredump.conf" (/etc/systemd/coredump.conf, but I don't see there I can configure any of it. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
* Carlos E. R. <robin.listas@telefonica.net> [08-17-17 17:46]: [...]
Notice the huge size, 1.8 GB.
The questions are:
can I choose compress options, so that it compresses faster and less?
Can I configure not to try dump core of certain named processes?
I see the configuration is done in "coredump.conf" (/etc/systemd/coredump.conf, but I don't see there I can configure any of it.
your /etc/systemd/coredump.conf doesn't contain [Coredump] #Storage=external #Compress=yes #ProcessSizeMax=2G #ExternalSizeMax=2G #JournalSizeMax=767M #MaxUse= #KeepFree= remove "#" and provide the value you feel necessary. -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Registered Linux User #207535 @ http://linuxcounter.net Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-18 05:09, Patrick Shanahan wrote:
* Carlos E. R. <> [08-17-17 17:46]:
your /etc/systemd/coredump.conf doesn't contain [Coredump] #Storage=external #Compress=yes #ProcessSizeMax=2G #ExternalSizeMax=2G #JournalSizeMax=767M #MaxUse= #KeepFree=
remove "#" and provide the value you feel necessary.
That's one possibility, but then some other big app might crash and I might want its core dump. It is this one particular app I do not want the core. I have the vague recollection that there is a way to run an app and impede it collecting core :-? (I don't want it because the dev is not interested in them) -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
On 2017-08-18 05:09, Patrick Shanahan wrote:
* Carlos E. R. <> [08-17-17 17:46]:
your /etc/systemd/coredump.conf doesn't contain [Coredump] #Storage=external #Compress=yes #ProcessSizeMax=2G #ExternalSizeMax=2G #JournalSizeMax=767M #MaxUse= #KeepFree=
remove "#" and provide the value you feel necessary.
That's one possibility, but then some other big app might crash and I might want its core dump. It is this one particular app I do not want the core.
I have the vague recollection that there is a way to run an app and impede it collecting core :-?
use 'ulimit -c 0' before you start the app. -- Per Jessen, Zürich (24.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-18 11:32, Per Jessen wrote:
Carlos E. R. wrote:
That's one possibility, but then some other big app might crash and I might want its core dump. It is this one particular app I do not want the core.
I have the vague recollection that there is a way to run an app and impede it collecting core :-?
use 'ulimit -c 0' before you start the app.
Thanks, that was it. That will apply only to the apps started on that shell, or to any other? I start that app via script, so it is easily done. Which is its man page? "man 1p ulimit" has only one option, "-f". 3 and 3p ara about ulimit.h 2 ? No. ?? -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
On 2017-08-18 11:32, Per Jessen wrote:
Carlos E. R. wrote:
That's one possibility, but then some other big app might crash and I might want its core dump. It is this one particular app I do not want the core.
I have the vague recollection that there is a way to run an app and impede it collecting core :-?
use 'ulimit -c 0' before you start the app.
Thanks, that was it.
That will apply only to the apps started on that shell, or to any other?
only the current and forked processes.
I start that app via script, so it is easily done.
Which is its man page?
section 1 - the bash page. -- Per Jessen, Zürich (24.9°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-18 12:03, Per Jessen wrote:
Carlos E. R. wrote:
use 'ulimit -c 0' before you start the app.
Thanks, that was it.
That will apply only to the apps started on that shell, or to any other?
only the current and forked processes.
Perfect!
I start that app via script, so it is easily done.
Which is its man page?
section 1 - the bash page.
Ah! -c The maximum size of core files created However: cer@Isengard:~> ulimit -c 0 It is already zero. Well, the limit is set now in the script. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Hello, On Fri, 18 Aug 2017, Carlos E. R. wrote:
cer@Isengard:~> ulimit -c 0
It is already zero. Well, the limit is set now in the script.
Try 'ulimit -H -c 0' or even: echo 0 > /proc/self/coredump_filter ./some_program Read up on the latter in /usr/src/linux/Documentation/filesystems/proc.txt HTH, -dnh -- Only wimps use tape backup: _real_ men just upload their important stuff on ftp, and let the rest of the world mirror it ;) -- Linus Torvalds -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-18 20:17, David Haller wrote:
Hello,
On Fri, 18 Aug 2017, Carlos E. R. wrote:
cer@Isengard:~> ulimit -c 0
It is already zero. Well, the limit is set now in the script.
Try 'ulimit -H -c 0' or even:
I don't find -H in man bash :-? Ah, I see it now, it is embedded in the text section. Hard limit. As I set the limit as user, it will be soft anyway.
echo 0 > /proc/self/coredump_filter ./some_program
Read up on the latter in /usr/src/linux/Documentation/filesystems/proc.txt
+++--------------------- 3.4 /proc/<pid>/coredump_filter - Core dump filtering settings --------------------------------------------------------------- When a process is dumped, all anonymous memory is written to a core file as long as the size of the core file isn't limited. But sometimes we don't want to dump some memory segments, for example, huge shared memory or DAX. Conversely, sometimes we want to save file-backed memory segments into a core file, not only the individual files. /proc/<pid>/coredump_filter allows you to customize which memory segments will be dumped when the <pid> process is dumped. coredump_filter is a bitmask of memory types. If a bit of the bitmask is set, memory segments of the corresponding memory type are dumped, otherwise they are not dumped. ---------------------++- I see. But it applies to all programs. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
On 2017-08-18 20:17, David Haller wrote:
Hello,
On Fri, 18 Aug 2017, Carlos E. R. wrote:
cer@Isengard:~> ulimit -c 0
It is already zero. Well, the limit is set now in the script.
Try 'ulimit -H -c 0' or even:
I don't find -H in man bash :-? Ah, I see it now, it is embedded in the text section. Hard limit.
As I set the limit as user, it will be soft anyway.
echo 0 > /proc/self/coredump_filter ./some_program
Read up on the latter in /usr/src/linux/Documentation/filesystems/proc.txt
+++--------------------- 3.4 /proc/<pid>/coredump_filter - Core dump filtering settings --------------------------------------------------------------- When a process is dumped, all anonymous memory is written to a core file as long as the size of the core file isn't limited. But sometimes we don't want to dump some memory segments, for example, huge shared memory or DAX. Conversely, sometimes we want to save file-backed memory segments into a core file, not only the individual files.
/proc/<pid>/coredump_filter allows you to customize which memory segments will be dumped when the <pid> process is dumped. coredump_filter is a bitmask of memory types. If a bit of the bitmask is set, memory segments of the corresponding memory type are dumped, otherwise they are not dumped. ---------------------++-
I see. But it applies to all programs.
When you apply it to /proc/self/coredump_filter, it only affects the current process (and children). -- Per Jessen, Zürich (20.2°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-19 17:12, Per Jessen wrote:
Carlos E. R. wrote:
I see. But it applies to all programs.
When you apply it to /proc/self/coredump_filter, it only affects the current process (and children).
Ah, self. I see. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2017-08-18 at 13:42 +0200, Carlos E. R. wrote:
On 2017-08-18 12:03, Per Jessen wrote:
Carlos E. R. wrote:
use 'ulimit -c 0' before you start the app.
Thanks, that was it.
That will apply only to the apps started on that shell, or to any other?
only the current and forked processes.
Perfect!
I start that app via script, so it is easily done.
Which is its man page?
section 1 - the bash page.
Ah!
-c The maximum size of core files created
However:
cer@Isengard:~> ulimit -c 0
It is already zero. Well, the limit is set now in the script.
Well, it is dumping core right now, it did not work. The app was using about 3 GB, so it is running now compressing those 3 GB of nothing, for 20 minutes already. Oh, finished now. 1.8 GB compressed. - -- Cheers, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlmZsGcACgkQtTMYHG2NR9V1DQCbBL4AlYYPQQYcGcl2HUN0TwL+ WLwAn2Kecl9MkrlPwwXpCxu5IMETfICJ =ht8H -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
On Friday, 2017-08-18 at 13:42 +0200, Carlos E. R. wrote:
cer@Isengard:~> ulimit -c 0
It is already zero. Well, the limit is set now in the script.
Well, it is dumping core right now, it did not work. The app was using about 3 GB, so it is running now compressing those 3 GB of nothing, for 20 minutes already. Oh, finished now. 1.8 GB compressed.
On my leap423 test system, at first it seems I have the opposite behaviour - (dump is just some code that tries to write to address 0x0). per@office36:~> ulimit -c 0 per@office36:~> ./dump Segmentation fault (core dumped) per@office36:~> coredumpctl No coredumps found. Okay. per@office36:~> ulimit -c unlimited per@office36:~> ./dump Segmentation fault (core dumped) per@office36:~> coredumpctl No coredumps found. Weird. switching to root - office36:~ # coredumpctl TIME PID UID GID SIG PRESENT EXE Sun 2017-08-20 18:41:36 CEST 9394 1000 100 11 * /home/per/dump Sun 2017-08-20 18:44:41 CEST 9420 1000 100 11 * /home/per/dump For one thing, 'ulimit' indeed seems to be ignored, for another why can't I see my own coredumps? -- Per Jessen, Zürich (21.4°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-20 18:50, Per Jessen wrote:
office36:~ # coredumpctl TIME PID UID GID SIG PRESENT EXE Sun 2017-08-20 18:41:36 CEST 9394 1000 100 11 * /home/per/dump Sun 2017-08-20 18:44:41 CEST 9420 1000 100 11 * /home/per/dump
For one thing, 'ulimit' indeed seems to be ignored, for another why can't I see my own coredumps?
The second one I know: they are stored under a root owned directory, /var/lib/systemd/coredump. That code of yours, could you post it? So as not to wait days for another fault and see if my modification works ;-) -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
On 2017-08-20 18:50, Per Jessen wrote:
office36:~ # coredumpctl TIME PID UID GID SIG PRESENT EXE Sun 2017-08-20 18:41:36 CEST 9394 1000 100 11 * /home/per/dump Sun 2017-08-20 18:44:41 CEST 9420 1000 100 11 * /home/per/dump
For one thing, 'ulimit' indeed seems to be ignored, for another why can't I see my own coredumps?
The second one I know: they are stored under a root owned directory, /var/lib/systemd/coredump.
Hmm, that rings a bell, but it sounds silly - a developer cannot access his own core dumps??
That code of yours, could you post it? So as not to wait days for another fault and see if my modification works ;-)
Sure - int main( int argc, char **argv ) { int *p; p=0; *p=99; } -- Per Jessen, Zürich (17.8°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-20 20:35, Per Jessen wrote:
Carlos E. R. wrote:
On 2017-08-20 18:50, Per Jessen wrote:
office36:~ # coredumpctl TIME PID UID GID SIG PRESENT EXE Sun 2017-08-20 18:41:36 CEST 9394 1000 100 11 * /home/per/dump Sun 2017-08-20 18:44:41 CEST 9420 1000 100 11 * /home/per/dump
For one thing, 'ulimit' indeed seems to be ignored, for another why can't I see my own coredumps?
The second one I know: they are stored under a root owned directory, /var/lib/systemd/coredump.
Hmm, that rings a bell, but it sounds silly - a developer cannot access his own core dumps??
That's right. I remember this because there was another thread where we commented this, some time ago.
That code of yours, could you post it? So as not to wait days for another fault and see if my modification works ;-)
Sure -
int main( int argc, char **argv ) { int *p;
p=0;
*p=99; }
Thanks - my C is rusty :-) -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
On 2017-08-20 23:50, Carlos E. R. wrote:
On 2017-08-20 20:35, Per Jessen wrote:
That code of yours, could you post it? So as not to wait days for another fault and see if my modification works ;-)
Sure -
Ok, testing. Compare when the code is run as user or as root TIME PID UID GID SIG PRESENT EXE Mon 2017-08-21 12:57:03 CEST 14936 1000 100 11 * /home/cer/bin/C/a.out Mon 2017-08-21 13:00:18 CEST 15165 0 0 11 * /home/cer/bin/C/a.out Mon 2017-08-21 13:01:10 CEST 15187 1000 100 11 * /home/cer/bin/C/a.out 1: no ulimit. 2 ulimit -c 0, run as root. 3 ulimit -c 0, run as user. I had to look at the header, the '0' is not the size. -rw-r-----+ 1 root root 14K Aug 21 12:57 core.a\x2eout.1000.3d0b1afec7d64446b590db4abf002a53.14936.1503313023000000.xz -rw-r-----+ 1 root root 14K Aug 21 13:01 core.a\x2eout.1000.3d0b1afec7d64446b590db4abf002a53.15187.1503313270000000.xz -rw-r-----+ 1 root root 22M Aug 21 03:36 core.kodi\x2ebin.1000.3d0b1afec7d64446b590db4abf002a53.9568.1503279207000000.xz Using -H -c 0 makes no difference. Now trying David Haller idea of "echo 0 > /proc/self/coredump_filter" cer@Isengard:~/bin/C> ./script ./script: line 3: 15338 Segmentation fault (core dumped) /home/cer/bin/C/a.out cer@Isengard:~/bin/C> cat script #!/bin/bash echo 0 > /proc/self/coredump_filter /home/cer/bin/C/a.out cer@Isengard:~/bin/C> So, doesn't work, either. Why? -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2017-08-20 at 18:50 +0200, Per Jessen wrote:
Carlos E. R. wrote:
On Friday, 2017-08-18 at 13:42 +0200, Carlos E. R. wrote:
cer@Isengard:~> ulimit -c 0
It is already zero. Well, the limit is set now in the script.
Well, it is dumping core right now, it did not work. The app was using about 3 GB, so it is running now compressing those 3 GB of nothing, for 20 minutes already. Oh, finished now. 1.8 GB compressed.
...
For one thing, 'ulimit' indeed seems to be ignored, for another why can't I see my own coredumps?
Just found a post from Andrei Borzenkov on 2 Dec 2016, subj "leap422 - where is my core dump?" +++..................
If core exceeds ulimit -c, it is not created at all and systemd-coredump does not see it. So how exactly it would be useful here?
I stay corrected. Linux kernel ignores ulimit -c when core_pattern is set to pipe; so yes, it is useful and is actually honored by systemd as of version 229 upstream. If this is not included in Leap 42.2, it is good candidate for bug report. ..................++- - -- Cheers, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlmmuWAACgkQtTMYHG2NR9UU0QCfV8QSo9u11VVK30IcoJbiY91X 6T8AoIeuUJqpAo1uxDkXVWq4Cli6UnYc =0pGp -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-08-20 17:53, Carlos E. R. wrote:
On Friday, 2017-08-18 at 13:42 +0200, Carlos E. R. wrote:
On 2017-08-18 12:03, Per Jessen wrote:
Carlos E. R. wrote:
use 'ulimit -c 0' before you start the app.
Thanks, that was it.
That will apply only to the apps started on that shell, or to any other?
only the current and forked processes.
Perfect!
I start that app via script, so it is easily done.
Which is its man page?
section 1 - the bash page.
Ah!
-c The maximum size of core files created
However:
cer@Isengard:~> ulimit -c 0
It is already zero. Well, the limit is set now in the script.
Well, it is dumping core right now, it did not work. The app was using about 3 GB, so it is running now compressing those 3 GB of nothing, for 20 minutes already. Oh, finished now. 1.8 GB compressed.
ulimit -H -c 0 also does not work. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
participants (4)
-
Carlos E. R. -
David Haller -
Patrick Shanahan -
Per Jessen