In data venerdì 31 gennaio 2020 12:32:59 CET, Carlos E. R. ha scritto:
On 31/01/2020 08.04, Andrei Borzenkov wrote: | On Thu, Jan 30, 2020 at 6:53 PM Per Jessen <per@computer.org> | | wrote: |> Andrei Borzenkov wrote: |>> On Thu, Jan 30, 2020 at 4:29 PM Per Jessen <per@computer.org> |>> |>> wrote: |>>> In the process lists from dmesg, I see 266 processes using up |>>> about 1Gb (RSS) which doesn't seem like a lot? |>> |>> It's in pages, so multiply by 4K. Active anonymous memory alone |>> is around 6GB. |> |> Ah, thanks - I was thinking it was kilobytes. |> |>> This is known problem. There is a lot of active memory so every |>> time kernel has to search a lot to find free page (or page that |>> can be reclaimed). |> |> that could lead to an oom condition? | | Rather this is indication of low memory. | | In this case free RAM is below absolute minimum that kernel | attempts to keep: | | Node 0 Normal free:102036kB min:106080kB
Ah! That's the start condition? I also have that in the log of my incident.
<0.4> 2020-01-30 15:20:37 Telcontar kernel - - - [283242.921018] Node 0 Normal free:105704kB min:106040kB low:116644kB high:127248kB
That's about 100 Mb, right?
active_anon:1885752kB inactive_anon:626456kB active_file:371848kB inactive_file:1478016kB unevictable:10880kB writepending:239740kB present:5242880kB managed:5110480kB mlocked:10880kB slab_reclaimable:170100kB slab_unreclaimable:162136kB kernel_stack:21080kB pagetables:91024kB bounce:0kB free_pcp:2776kB local_pcp:1280kB free_cma:0kB
Any document you know that explains these?
And no knowing how the machine got there.
| Which means on every allocation request kernel will synchronously | attempt to free enough memory to go above watermark. It looks like | it tries it actually: | | writeback:1251356kB writeback:1671120kB writeback:1876996kB | writeback:1982080kB writeback:1961032kB
I have one instance of that word.
<0.4> 2020-01-30 15:20:37 Telcontar kernel - - - [283242.921004] Node 0 active_anon:2560096kB inactive_anon:1335884kB active_file:461648kB inactive_file:2786820kB unevictable:10880kB isolated(anon):0kB isolated(file):256kB mapped:495892kB dirty:23756kB writeback:391168kB shmem:212684kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 382976kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
| This started with almost 2GB of data that is being written to | disk.
How do you know that? :-? (I want to find it in my case)
My machine was at the time probably copying a large directory from one disk to another, using 'mc', and nobody seated.
| At some point kernel managed to complete it just to get more and | more new data to be written. If kernel needs to wait while dirty | pages are written to disk it pretty much explains those "allocation | stalled" warnings. | | This goes on until no more free swap is available (which probably | means - kernel cannot free memory to go above watermark at all) at | which point OOM kicks in.
I don't think that happened in my case, but I do not know. I have 25 GB of swap.
| So it appears like some application(s) really use that much | memory. IOW system is overloaded. | | As for "If you sudo swapoff -a the swap turns into memory, the | system turns responsive" it is far too vague to make any useful | comment. It is impossible to do "swapoff" under conditions shown in | dmesg output. If this was supposed to mean "swapoff before this | situation is reached" - sure, it will eliminate stalls due to | writing data to disk, it will also cause OOM earlier. If RAM+swap | is not enough for current workload, then RAM alone is obviously not | enough as well.
This may be because read/write to swap with rotating disk is way too slow and involves many seeks. I observed that in my machine when I switched to Leap, and stopped when I bought an SSD disk and put the swap there.
-- Cheers / Saludos,
Carlos E. R. (from 15.1 x86_64 at Telcontar) I am using a Samsung 850 Pro SSD (just for completeness.
_________________________________________________________________ ________________________________________________________ Ihre E-Mail-Postfächer sicher & zentral an einem Ort. Jetzt wechseln und alte E-Mail-Adresse mitnehmen! https://www.eclipso.de -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org