https://bugzilla.novell.com/show_bug.cgi?id=685777 https://bugzilla.novell.com/show_bug.cgi?id=685777#c1 --- Comment #1 from Michael Meeks <mmeeks@novell.com> 2011-04-07 09:16:13 UTC --- Created an attachment (id=423646) --> (http://bugzilla.novell.com/attachment.cgi?id=423646) hwinfo -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=685777 https://bugzilla.novell.com/show_bug.cgi?id=685777#c3 Michael Meeks <mmeeks@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Critical |Major --- Comment #3 from Michael Meeks <mmeeks@novell.com> 2011-04-07 10:07:26 UTC --- Hah - so, there is a PEBKAC element here; I had not enabled icecream - which throttles compiler processes to a manageable number - down from 60+ to one (on this machine). Having said that, the OOM killer / load balancing seems to have done a pretty terrible job in this instance; easy to reproduce just checkout libreoffice, zypper si -d libreoffice-bootstrap and autogen with: '--with-num-cpus=16' '--with-max-jobs=4' :-) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=685777 https://bugzilla.novell.com/show_bug.cgi?id=685777#c4 --- Comment #4 from Mel Gorman <mgorman@novell.com> 2011-04-15 18:56:11 UTC --- Can we see the full OOM message, particularly the memory-related information that is usually displayed after the stack trace? What would also be helpful is the contents of /proc/slabinfo as close to the time of failure as possible. Ideally include the kernels config but if you don't have it handy, I'll assume it's the default openSUSE 11.4 kernel. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=685777 https://bugzilla.novell.com/show_bug.cgi?id=685777#c6 --- Comment #6 from Mel Gorman <mgorman@novell.com> 2011-07-26 16:45:18 UTC --- In partial response to this bug, I made a number of reclaim-related fixes in mainline that are now merged. Most of them dealt with kswapd consuming too much CPU or dumping too much memory but thought they might related to this bug so prepared a backport. I configured a laptop with 3G memory and set it up for building libreoffice including the disabling of swap. The machine did not lock up but the OOM trigger did fire and I see Jul 26 17:27:08 micromek kernel: [ 1838.738311] Node 0 DMA32 free:6704kB min:6916kB low:8644kB high:10372kB active_anon:2851928kB inactive_anon:6040kB active_file:2848kB inactive_file:2900kB unevictable:2584kB isolated(anon):0kB isolated(file):1588kB present:3009804kB mlocked:0kB dirty:0kB writeback:3856kB mapped:1960kB shmem:4520kB slab_reclaimable:10596kB slab_unreclaimable:22220kB kernel_stack:2392kB pagetables:16920kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:480 all_unreclaimable? no page cache from files is reduced to negligible levels while anonymous memory is through the roof. This system is genuinely out of memory due to a lack of swap space as the available memory is close to the minimum watermarks. When I examined the OOM message, 92% of memory was consumed by cc1plus. I don't think this is a kernel issue as such. Can you check the g++ versions you are using in openSUSE 11.3 and 11.4? A difference in version could explain an increase in memory usage for newer optimisations that were just enough to push your system over the edge. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=685777 https://bugzilla.novell.com/show_bug.cgi?id=685777#c8 --- Comment #8 from Mel Gorman <mgorman@novell.com> 2011-07-28 10:05:44 UTC --- (In reply to comment #7)

Mel ! thanks so much for looking into that. Meanwhile I had had a similar problem with just overloading the machine by other means I think.

I wonder - so, it is entirely possible that my problems are related to the Intel SSD I have, is it possible that that exacerbates the paging issues ?

What is the reproduction scenario? The speed of the SSD could be masking the fact that the machine is almost out-of-memory but not enough to trigger the OOM killer. If you have no swap configured and anonymous memory is occupying a high percentage of memory (e.g. 85%) then a significant percentage of time will be spent paging to and from the SSD. On a slower disk, the machine would become extremely unresponsive. This thrashing would be visible as a high page in/out rate in "vmstat -n 1". The percentage of memory that is anonymous can be determined from the nr_active_anon and nr_inactive_anon fields in /proc/vmstat . Can you tell me if this is the case? If so, it's not a bug in the kernel unless it is a memory leak that is causing the lack of memory. Just attaching the contents of /proc/vmstat when the machine is running very slow would be helpful in determining what is going on here.

Is it possible you have a new kernel I can try to see if the issues is fixed?

I didn't build a new kernel with the backport yet. However, unless there is sufficient free memory and kswapd is still using a lot of CPU, the patches will not help. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=685777 https://bugzilla.novell.com/show_bug.cgi?id=685777#c10 --- Comment #10 from Michael Meeks <mmeeks@suse.com> 2011-11-01 16:26:33 UTC --- I was using: Linux lenovo-w500 2.6.37.1-1.2-desktop #1 SMP PREEMPT 2011-02-21 10:34:10 +0100 i686 i686 i386 GNU/Linux and had a couple of similar hangs while loading my machine over the last week: annoying. Then I wrote this test which kills it dead. Interestingly, having upgraded to 3.1.0 ( to try un-successfully to fix the churning connect debounce syslog thrash ;-) - this now crashes successfully on start: which is ideal; whereas in 2.6.37 it totalled the machine. So - progress somewhere I suppose :-) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=685777 https://bugzilla.novell.com/show_bug.cgi?id=685777#c12 --- Comment #12 from Mel Gorman <mgorman@suse.com> 2011-11-01 17:37:39 UTC --- Has swap been configured? Your test case is filling the system with 4G of anonymous pages on a machine with 3G of RAM. If there is no swap configured, I would expect it to go OOM due to overcommit being enabled by default. With swap configured, it will grind heavily while it trashes swap for a while but should recover eventually. However, if you have wireless on this laptop, it is possible that the trashing is due to trying to allocate order-1 pages for frames depending on the model of card. Similarly, fork-heavy workloads require order-1 pages. On systems with a lot of anonymous pages (such as your testcase) and without swap, 2.6.37 behaves badly and can take an extremely long time to fork new processes because basically it has little choice other than to wait for processes to exit. Enabling swap mitigates this somewhat but there were also a number of changes made between 2.6.37 and 3.1 related to allocating high-order pages quickly that would make a noticeable difference. They are not likely to get backported though any time soon. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.