On 07/04/2019 21.35, Carlos E. R. wrote:
On Sunday, 2019-02-03 at 14:33 +0100, Carlos E. R. wrote:
Hi,
Last December I reported this problem on my laptop, and now my desktop also exhibits it.
After several days running, this morning it locked as soon as I changed workspaces. I noticed kswapd0 was busy long time, and that firefox had about 10 gigs of virtual memory. The disk activity led was solid blue. I couldn't find out more, the machine froze and had to be hard rebooted.
A week before it also froze, but that time I could not observe anything.
I'm not yet sure, but it seems that my upgrade to the kernel used by Leap 15.1 has solved the problem (and also the problem with hibernation restore crashing).
I'm now running a bunch of memory hungry applications, and using swap heavily, and no issues (besides things going slow, of course).
Firefox, Thunderbird, LibreOffice, Evince, Shotwell, Clementine, Konversation, Alpine...
top - 21:30:32 up 6 days, 1:51, 4 users, load average: 0,36, 0,46, 0,45 Tasks: 483 total, 1 running, 481 sleeping, 0 stopped, 1 zombie %Cpu(s): 1,9 us, 0,9 sy, 0,0 ni, 96,7 id, 0,5 wa, 0,0 hi, 0,0 si, 0,0 st KiB Mem : 8161136 total, 745112 free, 6326860 used, 1089164 buff/cache KiB Swap: 25165820 total, 19614704 free, 5551116 used. 1353532 avail Mem
PID USER PR NI VIRT RES SHR SWAP S %CPU %MEM TIME+ COMMAND 5066 cer 20 0 4389304 1,270g 74080 325520 S 0,000 16,32 417:05.80 thunderbird-bin 27150 cer 20 0 2583260 847756 12444 0 S 0,299 10,39 8:52.66 shotwell 7717 cer 20 0 3092444 741564 68480 233292 S 0,000 9,087 13:43.23 Web Content
Maybe related to this, I just saw something in the log. The swap in use is now above 6 GB. Apparently the kernel is having some memory problem but coping, not crashing. <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654102] iotop: page allocation stalls for 11744ms, order:0, mode:0x14000d0(GFP_TEMPORARY), nodemask=(null) <0.6> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654108] iotop cpuset=/ mems_allowed=0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654112] CPU: 3 PID: 8611 Comm: iotop Tainted: P O 4.12.14-lp151.23-default #1 openSUSE Leap 15.1 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654113] Hardware name: MICRO-STAR INTERNATIONAL CO.,LTD MS-7516/MS-7516, BIOS V1.5 10/10/2008 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654114] Call Trace: <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654123] dump_stack+0x5c/0x86 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654126] warn_alloc+0xe0/0x170 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654129] __alloc_pages_slowpath+0x732/0xc50 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654133] ? terminate_walk+0xe4/0x100 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654134] __alloc_pages_nodemask+0x20d/0x230 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654137] alloc_pages_current+0x72/0x140 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654139] __get_free_pages+0xa/0x40 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654142] proc_pid_cmdline_read+0x90/0x470 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654144] ? do_filp_open+0xa0/0xf0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654146] ? _copy_to_user+0x22/0x30 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654147] ? cp_new_stat+0x13d/0x160 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654150] ? __vfs_read+0x26/0x140 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654151] __vfs_read+0x26/0x140 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654153] vfs_read+0x89/0x130 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654155] SyS_read+0x42/0x90 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654156] ? SyS_lseek+0x80/0xa0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654159] do_syscall_64+0x7b/0x150 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654163] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654165] RIP: 0033:0x7ffaf1f4ae61 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654166] RSP: 002b:00007ffd8a292af8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654168] RAX: ffffffffffffffda RBX: 00007ffaf232a4c0 RCX: 00007ffaf1f4ae61 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654169] RDX: 0000000000002000 RSI: 000055879cdb0860 RDI: 0000000000000007 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654170] RBP: 0000000000002000 R08: 0000000000000000 R09: 0000000000000000 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654171] R10: 0000000000000100 R11: 0000000000000246 R12: 00007ffaeee71090 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654172] R13: 0000000000000007 R14: 000055879cdb0860 R15: 000055879cbb6000 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654173] Mem-Info: <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654177] active_anon:1159547 inactive_anon:197953 isolated_anon:0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654177] active_file:34229 inactive_file:19251 isolated_file:0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654177] unevictable:58 dirty:67 writeback:0 unstable:0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654177] slab_reclaimable:466310 slab_unreclaimable:37407 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654177] mapped:77276 shmem:45651 pagetables:24225 bounce:0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654177] free:30080 free_pcp:222 free_cma:0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654181] Node 0 active_anon:4638188kB inactive_anon:791812kB active_file:136916kB inactive_file:77004kB unevictable:232kB isolated(anon):0kB isolated(file):0kB mapped:309104kB dirty:268kB writeback:0kB shmem:182604kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 436224kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654181] Node 0 DMA free:15876kB min:132kB low:164kB high:196kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654186] lowmem_reserve[]: 0 2963 7954 7954 7954 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654189] Node 0 DMA32 free:54948kB min:35276kB low:41532kB high:47788kB active_anon:1304484kB inactive_anon:428380kB active_file:45396kB inactive_file:28076kB unevictable:32kB writepending:48kB present:3128896kB managed:3034748kB mlocked:32kB slab_reclaimable:1010672kB slab_unreclaimable:43544kB kernel_stack:3136kB pagetables:18108kB bounce:0kB free_pcp:820kB local_pcp:128kB free_cma:0kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654193] lowmem_reserve[]: 0 0 4990 4990 4990 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654196] Node 0 Normal free:49496kB min:50604kB low:61204kB high:71804kB active_anon:3333360kB inactive_anon:363632kB active_file:91520kB inactive_file:48928kB unevictable:200kB writepending:220kB present:5242880kB managed:5110480kB mlocked:200kB slab_reclaimable:854568kB slab_unreclaimable:106084kB kernel_stack:14720kB pagetables:78792kB bounce:0kB free_pcp:68kB local_pcp:48kB free_cma:0kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654200] lowmem_reserve[]: 0 0 0 0 0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654203] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15876kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654213] Node 0 DMA32: 3219*4kB (ME) 2677*8kB (ME) 806*16kB (ME) 127*32kB (UME) 29*64kB (UME) 7*128kB (ME) 4*256kB (ME) 1*512kB (U) 0*1024kB 0*2048kB 0*4096kB = 55540kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654224] Node 0 Normal: 7689*4kB (UMEH) 1376*8kB (UMEH) 328*16kB (UMEH) 76*32kB (UMEH) 3*64kB (U) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 49636kB <0.6> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654234] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654234] 206138 total pagecache pages <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654241] 106991 pages in swap cache <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654243] Swap cache stats: add 10069285, delete 9964855, find 19224545/21513208 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654243] Free swap = 19530224kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654244] Total swap = 25165820kB <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654244] 2096942 pages RAM <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654245] 0 pages HighMem/MovableOnly <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654245] 56658 pages reserved <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.654246] 0 pages hwpoisoned <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659843] thunderbird-bin: page allocation stalls for 11564ms, order:0, mode:0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null) <0.6> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659850] thunderbird-bin cpuset=/ mems_allowed=0 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659857] CPU: 1 PID: 5066 Comm: thunderbird-bin Tainted: P O 4.12.14-lp151.23-default #1 openSUSE Leap 15.1 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659858] Hardware name: MICRO-STAR INTERNATIONAL CO.,LTD MS-7516/MS-7516, BIOS V1.5 10/10/2008 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659858] Call Trace: <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659869] dump_stack+0x5c/0x86 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659872] warn_alloc+0xe0/0x170 <0.4> 2019-04-08 01:06:41 Telcontar kernel - - - [335302.659875] __alloc_pages_slowpath+0x732/0xc50 ... Top list of processes, sorted by memory use: top - 04:06:46 up 6 days, 8:27, 4 users, load average: 0,49, 0,37, 0,37 Tasks: 481 total, 1 running, 479 sleeping, 0 stopped, 1 zombie %Cpu(s): 2,6 us, 1,0 sy, 0,0 ni, 96,3 id, 0,1 wa, 0,0 hi, 0,0 si, 0,0 st KiB Mem : 8161136 total, 942204 free, 4910296 used, 2308636 buff/cache KiB Swap: 25165820 total, 18809328 free, 6356492 used. 2484860 avail Mem PID USER PR NI VIRT RES SHR SWAP S %CPU %MEM TIME+ COMMAND 5066 cer 20 0 4458508 1,214g 91568 388448 S 0,298 15,59 472:07.69 thunderbird-bin 7634 cer 20 0 3306468 723440 44876 370612 S 1,190 8,864 95:42.66 Web Content 7735 cer 20 0 3358296 662356 82140 361204 S 1,190 8,116 88:09.76 Web Content 7717 cer 20 0 3061056 602636 65180 303236 S 0,000 7,384 15:32.35 Web Content 7523 cer 20 0 10,262g 553668 117976 332564 S 1,190 6,784 174:10.53 firefox 7685 cer 20 0 3228388 412204 109696 283076 S 0,595 5,051 72:26.80 Web Content 4805 root 20 0 916432 145100 110972 209608 S 1,190 1,778 50:16.42 X 27918 cer 20 0 1689996 128036 21148 169300 S 0,298 1,569 4:19.68 soffice.bin 2320 named 20 0 562588 50708 0 178060 S 0,000 0,621 0:52.94 named 5036 cer 20 0 776180 47548 16168 17192 S 0,000 0,583 0:40.93 xfdesktop 5013 cer 20 0 1125572 37684 14356 78656 S 0,000 0,462 0:53.17 Thunar 3902 vscan 20 0 979648 36216 2672 621104 S 0,000 0,444 7:49.28 clamd 5852 cer 20 0 527072 35664 24744 13864 S 0,000 0,437 0:11.83 lxterminal 12761 cer 20 0 1161920 34448 17492 23984 S 0,000 0,422 1:30.48 konversation 5077 cer 20 0 687980 30520 10192 48832 S 0,298 0,374 6:11.76 xfce4-terminal 5806 cer 20 0 675152 30076 2944 33344 S 0,000 0,369 0:41.00 tracker-store 5054 cer 20 0 2536800 29036 232 220796 S 0,000 0,356 0:35.31 clementine 8153 cer 20 0 4399444 26472 3320 176564 S 0,000 0,324 2:19.17 java 26883 vscan 20 0 194348 25320 1400 37168 S 0,000 0,310 0:00.51 /usr/sbin/amavi 16342 cer 20 0 333924 20096 2964 190656 S 0,000 0,246 1:09.53 alpine 5785 cer 39 19 1133116 20016 1968 36808 S 0,000 0,245 1:15.04 tracker-miner-f 10346 news 20 0 24972 19500 1936 0 S 0,000 0,239 0:00.14 leafnode 5117 cer 20 0 597132 15520 8584 13848 S 0,000 0,190 11:20.41 panel-14-weathe 5015 cer 20 0 458336 15384 7056 19204 S 0,000 0,189 0:51.07 xfce4-panel 8611 root 20 0 57356 15136 2292 1700 S 2,679 0,185 142:04.74 iotop 5055 cer 20 0 527648 14176 7628 14972 S 0,000 0,174 0:05.90 panel-1-whisker 5098 cer 20 0 710596 13984 10000 17408 S 0,000 0,171 2:59.29 panel-19-pulsea Is somebody interested in this, to send the logs? Of course, normally I would restart Thunderbird and Firefox, but I'm keeping the system stressed for observation. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)