[opensuse] Emergency shell during times of intense swapping
I have 1.5 G of RAM, and from time to time I inadvertently get stuck with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out. Wouldn't it be a good idea if there were a way to get a shell with hi priority urgently? Perhaps there is. It would probably need to be on an interrupt like ctrl-alt-D. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Dne neděle 15. března 2020 21:05:34 CET, Richmond napsal(a):
I have 1.5 G of RAM, and from time to time I inadvertently get stuck
Really only 1.5 G? What are You running on it?
with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out. Wouldn't it be a good idea if there were a way to get a shell with hi priority urgently? Perhaps there is. It would probably need to be on an interrupt like ctrl-alt-D.
Did You check Magic SysRq key? https://en.wikipedia.org/wiki/Magic_SysRq_key It must be allowed first in YaST. I'm not sure where. I guess security settings or kernel could be good starting points. I haven't used them myself, I just know people use to use them for similar purposes as You. :-) -- Vojtěch Zeisek https://trapa.cz/ Komunita openSUSE GNU/Linuxu Community of the openSUSE GNU/Linux https://www.opensuse.org/
Vojtěch Zeisek wrote:
Dne neděle 15. března 2020 21:05:34 CET, Richmond napsal(a):
I have 1.5 G of RAM, and from time to time I inadvertently get stuck Really only 1.5 G? What are You running on it? Opensuse 15.1, with the IceWM. Most of the time it works fine. It's just I somehow got seamonkey and chromium going at the same time and that was too much.
with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out. Wouldn't it be a good idea if there were a way to get a shell with hi priority urgently? Perhaps there is. It would probably need to be on an interrupt like ctrl-alt-D. Did You check Magic SysRq key? https://en.wikipedia.org/wiki/Magic_SysRq_key It must be allowed first in YaST. I'm not sure where. I guess security settings or kernel could be good starting points. I haven't used them myself, I just know people use to use them for similar purposes as You. :-)
I have it enabled in Yast2 but it doesn't seem to work. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Richmond wrote:
Vojtěch Zeisek wrote:
Dne neděle 15. března 2020 21:05:34 CET, Richmond napsal(a):
I have 1.5 G of RAM, and from time to time I inadvertently get stuck Really only 1.5 G? What are You running on it? Opensuse 15.1, with the IceWM. Most of the time it works fine. It's just I somehow got seamonkey and chromium going at the same time and that was too much.
with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out. Wouldn't it be a good idea if there were a way to get a shell with hi priority urgently? Perhaps there is. It would probably need to be on an interrupt like ctrl-alt-D. Did You check Magic SysRq key? https://en.wikipedia.org/wiki/Magic_SysRq_key It must be allowed first in YaST. I'm not sure where. I guess security settings or kernel could be good starting points. I haven't used them myself, I just know people use to use them for similar purposes as You. :-)
I have it enabled in Yast2 but it doesn't seem to work.
I see cat /proc/sys/kernel/sysrq 184 So I think I need to set it to 1 or add 1 to it.... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 15/03/2020 18:31, Richmond wrote:
Richmond wrote:
Vojtěch Zeisek wrote:
Dne neděle 15. března 2020 21:05:34 CET, Richmond napsal(a):
I have 1.5 G of RAM, and from time to time I inadvertently get stuck Really only 1.5 G? What are You running on it? Opensuse 15.1, with the IceWM. Most of the time it works fine. It's just I somehow got seamonkey and chromium going at the same time and that was too much.
with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out. Wouldn't it be a good idea if there were a way to get a shell with hi priority urgently? Perhaps there is. It would probably need to be on an interrupt like ctrl-alt-D. Did You check Magic SysRq key? https://en.wikipedia.org/wiki/Magic_SysRq_key It must be allowed first in YaST. I'm not sure where. I guess security settings or kernel could be good starting points. I haven't used them myself, I just know people use to use them for similar purposes as You. :-)
I have it enabled in Yast2 but it doesn't seem to work.
I see
cat /proc/sys/kernel/sysrq 184
So I think I need to set it to 1 or add 1 to it....
These are the available SysRq functions: 0 - disable every SysRq function. 1 - enable every SysRq function. 2 - enable control of console logging level 4 - enable control of keyboard (SAK, unraw) *8 - enable debugging dumps of processes etc. *16 - enable sync command *32 - enable remount read-only 64 - enable signalling of processes (term, kill, oom-kill) *128 - allow reboot/poweroff 256 - allow nicing of all RT tasks 184 => 10111000 -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 15/03/2020 18:22, Richmond wrote:
Did You check Magic SysRq key? https://en.wikipedia.org/wiki/Magic_SysRq_key It must be allowed first in YaST. I'm not sure where. I guess security settings or kernel could be good starting points. I haven't used them myself, I just know people use to use them for similar purposes as You. :-)
I have it enabled in Yast2 but it doesn't seem to work.
+1 -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 15/03/2020 21.05, Richmond wrote:
I have 1.5 G of RAM, and from time to time I inadvertently get stuck with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out.
Mine does not time out. Is your disk rotating rust? The situation improves a lot with swap on SDD. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On 15/03/2020 17:27, Carlos E. R. wrote:
On 15/03/2020 21.05, Richmond wrote:
I have 1.5 G of RAM, and from time to time I inadvertently get stuck with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out.
Mine does not time out.
Is your disk rotating rust? The situation improves a lot with swap on SDD.
It might appear to but the reality is that it hasn't. All you've done is reduced the disk access latency time. There is still the lack of available memory and the VM subsystem is still dealing with continual page faulting and servicing in order to maintain a working set. Personal I blame Chromium. Firefox runs as one process and one set of page buffers and the ability, therefore, to easily make extra-large pages and so reduce the demands on the page mapping tables. Chromium runs as one process per accessed site/page and hence places a much greater demand on the system, both from the POV of scheduling and of the demands on the VM systems in terms of indirection/page-mapping tables. While I like the idea of process-per-webpage from a UNIX-philosophical POV it does place demands on the VM system that are going to become unreasonable unless you have a lot of memory. Once upon a time it wasn't like this, but now web pages are heavy on CSS (often machine generated and massively redundant), JS files (again often redundant) and graphics (often animated or enhanced with JS). -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 16 Mar 2020 08:23:13 -0400
Anton Aylward
On 15/03/2020 17:27, Carlos E. R. wrote:
On 15/03/2020 21.05, Richmond wrote:
I have 1.5 G of RAM, and from time to time I inadvertently get stuck with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out.
Mine does not time out.
Is your disk rotating rust? The situation improves a lot with swap on SDD.
It might appear to but the reality is that it hasn't. All you've done is reduced the disk access latency time. There is still the lack of available memory and the VM subsystem is still dealing with continual page faulting and servicing in order to maintain a working set.
Personal I blame Chromium. Firefox runs as one process and one set of page buffers and the ability, therefore, to easily make extra-large pages and so reduce the demands on the page mapping tables.
Well it used to, but now it spawns all these Web Content processes that consume resources as well.
Chromium runs as one process per accessed site/page and hence places a much greater demand on the system, both from the POV of scheduling and of the demands on the VM systems in terms of indirection/page-mapping tables.
While I like the idea of process-per-webpage from a UNIX-philosophical POV it does place demands on the VM system that are going to become unreasonable unless you have a lot of memory. Once upon a time it wasn't like this, but now web pages are heavy on CSS (often machine generated and massively redundant), JS files (again often redundant) and graphics (often animated or enhanced with JS).
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 16/03/2020 08:48, Dave Howorth wrote:
Personal I blame Chromium. Firefox runs as one process and one set of page buffers and the ability, therefore, to easily make extra-large pages and so reduce the demands on the page mapping tables.
Well it used to, but now it spawns all these Web Content processes that consume resources as well.
Sorry, I don't see those in my process table. I'm running three FF windows each with about 40-60 active pages. A lot, A LOT of youtube that have started and been stopped, but buffered. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 16 Mar 2020 08:58:25 -0400
Anton Aylward
On 16/03/2020 08:48, Dave Howorth wrote:
Personal I blame Chromium. Firefox runs as one process and one set of page buffers and the ability, therefore, to easily make extra-large pages and so reduce the demands on the page mapping tables.
Well it used to, but now it spawns all these Web Content processes that consume resources as well.
Sorry, I don't see those in my process table.
Then I suppose you're not running electrolysis for whatever reason. https://wiki.mozilla.org/Electrolysis But not doing so is a function of your setup, not a feature of Firefox.
I'm running three FF windows each with about 40-60 active pages. A lot, A LOT of youtube that have started and been stopped, but buffered.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 16/03/2020 13.23, Anton Aylward wrote:
On 15/03/2020 17:27, Carlos E. R. wrote:
On 15/03/2020 21.05, Richmond wrote:
I have 1.5 G of RAM, and from time to time I inadvertently get stuck with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out.
Mine does not time out.
Is your disk rotating rust? The situation improves a lot with swap on SDD.
It might appear to but the reality is that it hasn't. All you've done is reduced the disk access latency time.
Which is what I want.
There is still the lack of available memory and the VM subsystem is still dealing with continual page faulting and servicing in order to maintain a working set.
So what? For someone that has limited memory and can not add more, the situation improves with swap on sdd, that's a proven fact. By my personal and others experience. That maybe there are other things to improve the situation. Ok, so what? Still an sdd improves the situation. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
Anton Aylward wrote:
Personal I blame Chromium.
Chromium is causing some problems. It seems to be crashing, or maybe not exiting properly, and leaving many processes hanging around. As there is no evidence chromium is running (from the UI) I get caught out when I run something else. There is some kind of crash report showing registers etc but I need to find out how to recreate it. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered. I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that. In order to get some idea of what's happening I ran vmstat (on my reniced -19 console) and found that it wasn't actually swapping, the activity was BI and BO, which I think is to do with buffers? The page I went to in firefox was maps.google.co.uk. Eventually firefox came up with a tiny little window with nothing in it. Then some time after that a message appeared saying "something is slowing down your browser, what would you like to do?" and there were two very flat buttons (i.e. small vertical size, long horizontal size) and nothing written in them. The system did eventually stabilize without me doing anything. I think something is going wrong, but I am not sure what. I have a local installation of Mozilla Firefox 74.0, i.e. I downloaded and unpacked into /usr/local. Your thoughts? cat /proc/sys/vm/swappiness 40 uname -a 4.12.14-lp151.28.36-default #1 SMP Fri Dec 6 13:50:27 UTC 2019 (8f4a495) x86_64 x86_64 x86_64 GNU/Linux Leap 15.1 free total used free shared buff/cache available Mem: 1517608 654472 576844 64640 286292 648952 Swap: 3148796 143360 3005436 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered.
I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that.
Because "swap fragmentation" causes excesive disk head movements. You need to put swap on an SSD device instead of rotating rust. - -- Cheers Carlos E. R. (from openSUSE 15.1 (Legolas))
-----BEGIN PGP SIGNATURE----- iHIEARECADIWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCXodFuhQccm9iaW4ubGlz dGFzQGdteC5lcwAKCRC1MxgcbY1H1SeOAKCAGPGmzvx8cf5f34kKTm5zLa9O6ACg jlQHkYTvWmMB/QUQzKLI50RZ9FI= =K7Tu -----END PGP SIGNATURE-----
On 03/04/2020 10:18, Carlos E. R. wrote:
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered.
I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that.
Because "swap fragmentation" causes excesive disk head movements.
That is why this is about 'paging' and not 'swapping'. The old UNIX V7 model or roll-in/roll-out was 'swapping' one program for another and it worked by a 'DMA' (often on the PDP-11 hardware-assisted on a separate bus) of contiguous memory to contiguous disk sectors, hence one seek at the start of the transfer then just smooth head movement. Please stop referring to what is 'paging', often unoptimized one-page-at-a-time, demanding, as Carlos says, excessive head movement, as 'swapping'. Like the old adage says Virtual Memory means Virtual Performance. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Anton Aylward wrote:
On 03/04/2020 10:18, Carlos E. R. wrote:
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered. I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that. Because "swap fragmentation" causes excesive disk head movements. That is why this is about 'paging' and not 'swapping'.
The old UNIX V7 model or roll-in/roll-out was 'swapping' one program for another and it worked by a 'DMA' (often on the PDP-11 hardware-assisted on a separate bus) of contiguous memory to contiguous disk sectors, hence one seek at the start of the transfer then just smooth head movement.
Please stop referring to what is 'paging', often unoptimized one-page-at-a-time, demanding, as Carlos says, excessive head movement, as 'swapping'.
Like the old adage says
Virtual Memory means Virtual Performance.
I can see that you are making an important distinction, but I should point out that "swap" is an English word, and that "swapping" does not imply any particular object which is being swapped for any other particular object, so it could mean swapping pages. Additional confusion arises (for me at least) because the parameter is called /proc/sys/vm/swappiness. Does it mean paginess? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/04/2020 18.40, Richmond wrote:
Anton Aylward wrote:
On 03/04/2020 10:18, Carlos E. R. wrote:
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered. I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that. Because "swap fragmentation" causes excesive disk head movements. That is why this is about 'paging' and not 'swapping'.
The old UNIX V7 model or roll-in/roll-out was 'swapping' one program for another and it worked by a 'DMA' (often on the PDP-11 hardware-assisted on a separate bus) of contiguous memory to contiguous disk sectors, hence one seek at the start of the transfer then just smooth head movement.
Please stop referring to what is 'paging', often unoptimized one-page-at-a-time, demanding, as Carlos says, excessive head movement, as 'swapping'.
Like the old adage says
Virtual Memory means Virtual Performance.
I can see that you are making an important distinction, but I should point out that "swap" is an English word, and that "swapping" does not imply any particular object which is being swapped for any other particular object, so it could mean swapping pages. Additional confusion arises (for me at least) because the parameter is called /proc/sys/vm/swappiness. Does it mean paginess?
It doesn't matter what we call it. Let's call it 'X'. When the system is doing 'X', it writes and reads from the "swap" partition on the disk, with a lot of head movement, so that writing a gigabyte at random positions on the partition takes forever. If you replace the rotating disk with an SSD and put the "swap" partition on it, the speed gain is tremendous. I have seen it, I know from personal experience. On 4 machines. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
Carlos E. R. wrote:
It doesn't matter what we call it. Let's call it 'X'. When the system is doing 'X', it writes and reads from the "swap" partition on the disk, with a lot of head movement, so that writing a gigabyte at random positions on the partition takes forever.
If you replace the rotating disk with an SSD and put the "swap" partition on it, the speed gain is tremendous. I have seen it, I know from personal experience. On 4 machines.
Yes. But I don't really want to spend money on this PC. It's from around 2006. I am waiting to see if I can fix my newer pc but deliveries have been delayed. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
On 03/04/2020 18.40, Richmond wrote:
Anton Aylward wrote:
On 03/04/2020 10:18, Carlos E. R. wrote:
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered. I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that. Because "swap fragmentation" causes excesive disk head movements. That is why this is about 'paging' and not 'swapping'.
The old UNIX V7 model or roll-in/roll-out was 'swapping' one program for another and it worked by a 'DMA' (often on the PDP-11 hardware-assisted on a separate bus) of contiguous memory to contiguous disk sectors, hence one seek at the start of the transfer then just smooth head movement.
Please stop referring to what is 'paging', often unoptimized one-page-at-a-time, demanding, as Carlos says, excessive head movement, as 'swapping'.
Like the old adage says
Virtual Memory means Virtual Performance.
I can see that you are making an important distinction, but I should point out that "swap" is an English word, and that "swapping" does not imply any particular object which is being swapped for any other particular object, so it could mean swapping pages. Additional confusion arises (for me at least) because the parameter is called /proc/sys/vm/swappiness. Does it mean paginess?
It doesn't matter what we call it. Let's call it 'X'. When the system is doing 'X', it writes and reads from the "swap" partition on the disk, with a lot of head movement, so that writing a gigabyte at random positions on the partition takes forever.
If you replace the rotating disk with an SSD and put the "swap" partition on it, the speed gain is tremendous. I have seen it, I know from personal experience. On 4 machines.
I happened to have a spare 160GB Hitatchi, so I sw... er... switched it for the 80GB Samsung which was in the laptop, and it has improved things considerably. Although it is still sw.... er.. paging I am still able to do things like alt-tab between tasks. Now I wonder perhaps if that 80GB drive was faulty, i.e. getting hardware errors on read. I cannot check now. I have chromium, firefox (at google maps) and seamonkey running all at the same time and I am still able to type this. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/04/2020 17.20, Richmond wrote:
Carlos E. R. wrote:
On 03/04/2020 18.40, Richmond wrote:
Anton Aylward wrote:
On 03/04/2020 10:18, Carlos E. R. wrote:
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered. I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that. Because "swap fragmentation" causes excesive disk head movements. That is why this is about 'paging' and not 'swapping'.
The old UNIX V7 model or roll-in/roll-out was 'swapping' one program for another and it worked by a 'DMA' (often on the PDP-11 hardware-assisted on a separate bus) of contiguous memory to contiguous disk sectors, hence one seek at the start of the transfer then just smooth head movement.
Please stop referring to what is 'paging', often unoptimized one-page-at-a-time, demanding, as Carlos says, excessive head movement, as 'swapping'.
Like the old adage says
Virtual Memory means Virtual Performance.
I can see that you are making an important distinction, but I should point out that "swap" is an English word, and that "swapping" does not imply any particular object which is being swapped for any other particular object, so it could mean swapping pages. Additional confusion arises (for me at least) because the parameter is called /proc/sys/vm/swappiness. Does it mean paginess?
It doesn't matter what we call it. Let's call it 'X'. When the system is doing 'X', it writes and reads from the "swap" partition on the disk, with a lot of head movement, so that writing a gigabyte at random positions on the partition takes forever.
If you replace the rotating disk with an SSD and put the "swap" partition on it, the speed gain is tremendous. I have seen it, I know from personal experience. On 4 machines.
I happened to have a spare 160GB Hitatchi,
SSD or rotating rust?
so I sw... er... switched it for the 80GB Samsung which was in the laptop, and it has improved things considerably. Although it is still sw.... er.. paging I am still able to do things like alt-tab between tasks.
Now I wonder perhaps if that 80GB drive was faulty, i.e. getting hardware errors on read. I cannot check now.
Huh, that would be terrible, tremendous impact.
I have chromium, firefox (at google maps) and seamonkey running all at the same time and I am still able to type this.
How much ram? 1.5G? In that case, it is surprising. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
Carlos E. R. wrote:
On 04/04/2020 17.20, Richmond wrote:
Carlos E. R. wrote:
On 03/04/2020 18.40, Richmond wrote:
Anton Aylward wrote:
On 03/04/2020 10:18, Carlos E. R. wrote:
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
> Excuse me for going on about swapping. I am using the same > thread so anyone who has killed it won't be bothered. I was > running firefox and had plenty of tabs open, but reasoned that > if these are swapped out (that is to say the memory > used by them is swapped out) then it won't matter. But the > system ground to a halt. What puzzles me is how long this > goes on for. I am not sure how long it would take to swap > the whole 1.5 G of RAM out but it seems to be stuck for > much longer than that. Because "swap fragmentation" causes excesive disk head movements. That is why this is about 'paging' and not 'swapping'.
The old UNIX V7 model or roll-in/roll-out was 'swapping' one program for another and it worked by a 'DMA' (often on the PDP-11 hardware-assisted on a separate bus) of contiguous memory to contiguous disk sectors, hence one seek at the start of the transfer then just smooth head movement.
Please stop referring to what is 'paging', often unoptimized one-page-at-a-time, demanding, as Carlos says, excessive head movement, as 'swapping'.
Like the old adage says
Virtual Memory means Virtual Performance.
I can see that you are making an important distinction, but I should point out that "swap" is an English word, and that "swapping" does not imply any particular object which is being swapped for any other particular object, so it could mean swapping pages. Additional confusion arises (for me at least) because the parameter is called /proc/sys/vm/swappiness. Does it mean paginess?
It doesn't matter what we call it. Let's call it 'X'. When the system is doing 'X', it writes and reads from the "swap" partition on the disk, with a lot of head movement, so that writing a gigabyte at random positions on the partition takes forever.
If you replace the rotating disk with an SSD and put the "swap" partition on it, the speed gain is tremendous. I have seen it, I know from personal experience. On 4 machines.
I happened to have a spare 160GB Hitatchi,
SSD or rotating rust? A rotating disk. I don't think it is rusty.
so I sw... er... switched it for the 80GB Samsung which was in the laptop, and it has improved things considerably. Although it is still sw.... er.. paging I am still able to do things like alt-tab between tasks.
Now I wonder perhaps if that 80GB drive was faulty, i.e. getting hardware errors on read. I cannot check now.
Huh, that would be terrible, tremendous impact. It was terrible. Not so much poor performance as a major denial of service attack.
I have chromium, firefox (at google maps) and seamonkey running all at the same time and I am still able to type this.
How much ram? 1.5G? In that case, it is surprising.
1.5G yes. I now have quite a usable system. It takes 30 seconds to write 1.5G to the disk, so I could not see any reason why paging would go on for several minutes at a time. But I didn't think to check for SMART errors. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sat, 4 Apr 2020 20:06:20 +0100
Richmond
Carlos E. R. wrote:
On 04/04/2020 17.20, Richmond wrote:
Carlos E. R. wrote:
On 03/04/2020 18.40, Richmond wrote:
Anton Aylward wrote:
On 03/04/2020 10:18, Carlos E. R. wrote: > El 2020-04-03 a las 14:24 +0100, Richmond escribió: > >> Excuse me for going on about swapping. I am using the same >> thread so anyone who has killed it won't be bothered. I was >> running firefox and had plenty of tabs open, but reasoned that >> if these are swapped out (that is to say the memory >> used by them is swapped out) then it won't matter. But the >> system ground to a halt. What puzzles me is how long this >> goes on for. I am not sure how long it would take to swap >> the whole 1.5 G of RAM out but it seems to be stuck for >> much longer than that. > Because "swap fragmentation" causes excesive disk head > movements. That is why this is about 'paging' and not 'swapping'.
The old UNIX V7 model or roll-in/roll-out was 'swapping' one program for another and it worked by a 'DMA' (often on the PDP-11 hardware-assisted on a separate bus) of contiguous memory to contiguous disk sectors, hence one seek at the start of the transfer then just smooth head movement.
Please stop referring to what is 'paging', often unoptimized one-page-at-a-time, demanding, as Carlos says, excessive head movement, as 'swapping'.
Like the old adage says
Virtual Memory means Virtual Performance.
I can see that you are making an important distinction, but I should point out that "swap" is an English word, and that "swapping" does not imply any particular object which is being swapped for any other particular object, so it could mean swapping pages. Additional confusion arises (for me at least) because the parameter is called /proc/sys/vm/swappiness. Does it mean paginess?
It doesn't matter what we call it. Let's call it 'X'. When the system is doing 'X', it writes and reads from the "swap" partition on the disk, with a lot of head movement, so that writing a gigabyte at random positions on the partition takes forever.
If you replace the rotating disk with an SSD and put the "swap" partition on it, the speed gain is tremendous. I have seen it, I know from personal experience. On 4 machines.
I happened to have a spare 160GB Hitatchi,
SSD or rotating rust? A rotating disk. I don't think it is rusty.
According to wikipedia: Older hard disk drives used iron(III) oxide (Fe2O3) as the magnetic material, but current disks use a cobalt-based alloy.[2]
so I sw... er... switched it for the 80GB Samsung which was in the laptop, and it has improved things considerably. Although it is still sw.... er.. paging I am still able to do things like alt-tab between tasks.
Now I wonder perhaps if that 80GB drive was faulty, i.e. getting hardware errors on read. I cannot check now.
Huh, that would be terrible, tremendous impact. It was terrible. Not so much poor performance as a major denial of service attack.
I have chromium, firefox (at google maps) and seamonkey running all at the same time and I am still able to type this.
How much ram? 1.5G? In that case, it is surprising.
1.5G yes. I now have quite a usable system.
It takes 30 seconds to write 1.5G to the disk, so I could not see any reason why paging would go on for several minutes at a time. But I didn't think to check for SMART errors.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/04/2020 21.06, Richmond wrote:
Carlos E. R. wrote:
On 04/04/2020 17.20, Richmond wrote:
Carlos E. R. wrote:
On 03/04/2020 18.40, Richmond wrote:
Anton Aylward wrote:
On 03/04/2020 10:18, Carlos E. R. wrote: > El 2020-04-03 a las 14:24 +0100, Richmond escribió:
...
If you replace the rotating disk with an SSD and put the "swap" partition on it, the speed gain is tremendous. I have seen it, I know from personal experience. On 4 machines.
I happened to have a spare 160GB Hitatchi,
SSD or rotating rust? A rotating disk. I don't think it is rusty.
"Rotating rust" is slang for rotating magnetic disk :-D
so I sw... er... switched it for the 80GB Samsung which was in the laptop, and it has improved things considerably. Although it is still sw.... er.. paging I am still able to do things like alt-tab between tasks.
Now I wonder perhaps if that 80GB drive was faulty, i.e. getting hardware errors on read. I cannot check now.
Huh, that would be terrible, tremendous impact. It was terrible. Not so much poor performance as a major denial of service attack.
I have chromium, firefox (at google maps) and seamonkey running all at the same time and I am still able to type this.
How much ram? 1.5G? In that case, it is surprising.
1.5G yes. I now have quite a usable system.
It takes 30 seconds to write 1.5G to the disk, so I could not see any reason why paging would go on for several minutes at a time. But I didn't think to check for SMART errors.
If you cloned the installation, it would be on the log. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On 2020-04-03 16:18, Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered.
I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that.
Because "swap fragmentation" causes excesive disk head movements. You need to put swap on an SSD device instead of rotating rust.
- --
This is exactly how I destroyed my very first flash-disk... Swap is like the safety lan on the highway, it should be there, but never (or hardly, only in case of emergencies) been used. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 05/04/2020 13.30, suse@a-domani.nl wrote:
On 2020-04-03 16:18, Carlos E. R. wrote:
El 2020-04-03 a las 14:24 +0100, Richmond escribió:
Excuse me for going on about swapping. I am using the same thread so anyone who has killed it won't be bothered.
I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter. But the system ground to a halt. What puzzles me is how long this goes on for. I am not sure how long it would take to swap the whole 1.5 G of RAM out but it seems to be stuck for much longer than that.
Because "swap fragmentation" causes excesive disk head movements. You need to put swap on an SSD device instead of rotating rust.
This is exactly how I destroyed my very first flash-disk...
Hardware has improved much.
Swap is like the safety lan on the highway, it should be there, but never (or hardly, only in case of emergencies) been used.
Define emergency :-) If you can not add more RAM for whatever reason (machine too old so modules hard to find, no money, motherboard is already maxed, etc) then the only alternative is swap. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On 03/04/2020 09:24, Richmond wrote:
In order to get some idea of what's happening I ran vmstat (on my reniced -19 console) and found that it wasn't actually swapping, the activity was BI and BO, which I think is to do with buffers?
Sadly that UI/display doesn't tell you if the buffers were used for file system or for network. There is a 'netstat' program that you can, see the manual page, focus on a specific user, program, host or host address. Sadly you can't tell netstat what to display on that, you have to take it all in and grep what you need from the firehose. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Anton Aylward wrote:
On 03/04/2020 09:24, Richmond wrote:
In order to get some idea of what's happening I ran vmstat (on my reniced -19 console) and found that it wasn't actually swapping, the activity was BI and BO, which I think is to do with buffers? Sadly that UI/display doesn't tell you if the buffers were used for file system or for network.
There is a 'netstat' program that you can, see the manual page, focus on a specific user, program, host or host address. Sadly you can't tell netstat what to display on that, you have to take it all in and grep what you need from the firehose.
During this time of whatever-its-called the disk light was lit up, so I made the assumption that it was not waiting on the network. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/04/2020 09:24, Richmond wrote:
I was running firefox and had plenty of tabs open, but reasoned that if these are swapped out (that is to say the memory used by them is swapped out) then it won't matter.
Stop there. We've got to stop talking about 'swap'. It's an anachronism from the days when UNIX was a roll-in/roll-out system, version 5,6,,7; before Virtual Memory and long before the VM mapping technique for file access. Linux runs with TWO virtual memory queues, one for immutable data, one for mutable data. The immutable data is, essentially code. Since it can always be retrieved from the file system it is never swapped out. Pages on that queue simply 'age out' and are made available in the general pool. All memory except the kernel is on a queue, and, yes, that also goes for the mapping tables, the things that do the virtual-to-physical mapping. If you think that is an OUCH, the "Yah! You betcha!". If the system has to page-out (never say 'swap out'!) those then it really is memory hungry and the relevant process is crippled. Yes, a bad link editor that puts commonly used code on the same page as rarely used code, many times over, is idiotic. The same goes for mutable data. Perhaps the command line parameters get set but are they used thoughout the program? Link editor issues again. The problem with pages in firefox invovle HTML and that means 'rendering' and interpreting and that drags in CSS files. I've seen machine generated CSS on commercial sites that is, while humungous, could be condensed, made heirarchical, and contains items for every imaginable situation but don't actually occur. All of whcih makes rendering the page slower and takes up memory. And ultimately, these days, web pages are graphics-intense. Rendering graphics is more code, more CSS, more memory. You might ask why your phone or tablet performs so much better? A lot has t do with hardware acceleration but a lot more has to do with code libraries that are turned for the hardware. I can understand that a map HAS to be rendered with graphics, but you might compare the performance with graphic turned off or make use of Lynx to read pages where you have no interest in the banner images or other graphical eye-candy. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2020-03-16 08:23 AM, Anton Aylward wrote:
Personal I blame Chromium. Firefox runs as one process and one set of page buffers and the ability, therefore, to easily make extra-large pages and so reduce the demands on the page mapping tables.
I have also noticed Chromium uses a lot of swap, but Firefox seems to as well. I leave my computer up 24/7 and over time the swap usage increases. Then, when I kill Chromium and Firefox, the memory and swap usage both drop. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2020-03-15 21:05, Richmond wrote:
I have 1.5 G of RAM, and from time to time I inadvertently get stuck with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out.
Phew, that's really few RAM. I'm afraid it will be hard for the system to start a new shell in such a situation. The best bet will probably be to have a shell already started there beforehand, so you just have to switch to it once the system got into that pressure situation. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 15 Mar 2020 22:57:46 +0100
Bernhard Voelker
On 2020-03-15 21:05, Richmond wrote:
I have 1.5 G of RAM, and from time to time I inadvertently get stuck with swapping overload. At this time it becomes difficult even to swap to the virtual console with ctrl-alt-f1, and having got there, even though I changed the login time-out to 10 minutes, sometimes it times out.
Phew, that's really few RAM. I'm afraid it will be hard for the system to start a new shell in such a situation. The best bet will probably be to have a shell already started there beforehand, so you just have to switch to it once the system got into that pressure situation.
I think you've missed Richmond's point. Switching to an existing shell still takes many minutes in such a situation. He's suggesting a new mechanism that allows such a thing, or creation of a new shell as well, even when memory is overloaded. I don't see any fundamental problems in creating such a facility, so it's just willpower. FWIW BTW, I have 'Enable SysRq Keys' checked in YaST2 - Kernel Settings but pressing Alt-[Fn]-[PrtSc]-h on my keyboard does not bring up SysRq help as I believe it should, and I don't understand what is wrong.
Have a nice day, Berny
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 15/03/2020 23.17, Dave Howorth wrote:
FWIW BTW, I have 'Enable SysRq Keys' checked in YaST2 - Kernel Settings but pressing Alt-[Fn]-[PrtSc]-h on my keyboard does not bring up SysRq help as I believe it should, and I don't understand what is wrong.
It prints on tty10 and syslog. <0.6> 2020-03-15 23:37:10 Telcontar kernel - - - [42130.563836] sysrq: SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) show-blocked-tasks(w) dump-ftrace-buffer(z) -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On Sun, 15 Mar 2020 23:38:46 +0100
"Carlos E. R."
On 15/03/2020 23.17, Dave Howorth wrote:
FWIW BTW, I have 'Enable SysRq Keys' checked in YaST2 - Kernel Settings but pressing Alt-[Fn]-[PrtSc]-h on my keyboard does not bring up SysRq help as I believe it should, and I don't understand what is wrong.
It prints on tty10 and syslog.
Ah, thanks. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2020-03-15 23:17, Dave Howorth wrote:
I think you've missed Richmond's point. Switching to an existing shell still takes many minutes in such a situation. He's suggesting a new mechanism that allows such a thing, or creation of a new shell as well, even when memory is overloaded.
If switching alone is already taking too long, how could creating of a new shell plus switching to it be lighter for an overloaded system? It seems I really missed something. Re. MagicSysRq: you can kill processes, sync, umount and reboot, but in the long run, I think there's no way around getting more RAM to have more fun. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 16/03/2020 11.56, Bernhard Voelker wrote:
On 2020-03-15 23:17, Dave Howorth wrote:
I think you've missed Richmond's point. Switching to an existing shell still takes many minutes in such a situation. He's suggesting a new mechanism that allows such a thing, or creation of a new shell as well, even when memory is overloaded.
If switching alone is already taking too long, how could creating of a new shell plus switching to it be lighter for an overloaded system? It seems I really missed something.
No, the virtual terminal and shell would become high priority, and all graphical tasks would become tertiary and be swapped out and no cpu time allowed to them. If no way to know graphical tasks, then everything else but the emergency shell. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
"Carlos E. R."
On 16/03/2020 11.56, Bernhard Voelker wrote:
On 2020-03-15 23:17, Dave Howorth wrote:
I think you've missed Richmond's point. Switching to an existing shell still takes many minutes in such a situation. He's suggesting a new mechanism that allows such a thing, or creation of a new shell as well, even when memory is overloaded.
If switching alone is already taking too long, how could creating of a new shell plus switching to it be lighter for an overloaded system? It seems I really missed something.
No, the virtual terminal and shell would become high priority, and all graphical tasks would become tertiary and be swapped out and no cpu time allowed to them. If no way to know graphical tasks, then everything else but the emergency shell.
It ought to be possible. There is something very odd to me that I am not able to do anything with my computer because it has decided it has more important things to do than listen to me. I should have absolute and overriding control if I really want it. Certainly logging in on the console as soon as I boot up is helpful, but I keep forgetting to do it. The computer has a max of 2G of RAM. It is rather old. I am only using it because a newer one died. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 16/03/2020 07:51, Richmond wrote:
Certainly logging in on the console as soon as I boot up is helpful, but I keep forgetting to do it.
If you are running a late-model opensuse then you have systemd and you can make a systemd unit to start up a root shell on a console as part of the multi-user or graphical startup. Only a file of few tens of bytes .. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 16/03/2020 07:51, Richmond wrote:
The computer has a max of 2G of RAM. It is rather old. I am only using it because a newer one died.
Understood. I have a basement of them ... But in reality, my phone and my tablet do better access. Yes, I have my primary mail accounts and a few critical lists like this ask-9 accounts on my tablet. OUCH! My tablet only has 1.5G of RAM but performs remarkably well! -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 16 Mar 2020 11:56:29 +0100
Bernhard Voelker
On 2020-03-15 23:17, Dave Howorth wrote:
I think you've missed Richmond's point. Switching to an existing shell still takes many minutes in such a situation. He's suggesting a new mechanism that allows such a thing, or creation of a new shell as well, even when memory is overloaded.
If switching alone is already taking too long, how could creating of a new shell plus switching to it be lighter for an overloaded system? It seems I really missed something.
Yes, you missed that the kernel would simply stop/pause processing pretty much everything else in order to give priority to the emergency shell, as Carlos has pointed out. The problem is that something is grabbing too many resources. Unlike the real world, we can simply stop the world inside the computer so we can examine its state and fix whatever is emitting too much CO2 or whatever the problem is. PS please don't send me a private copy of emails. I am subscribed to the list.
Re. MagicSysRq: you can kill processes, sync, umount and reboot, but in the long run, I think there's no way around getting more RAM to have more fun.
It seems that setting up a service unit to start a root shell on a VT as the system starts and giving it high priority with nice might come close to meeting Richmond's requirements.
Have a nice day, Berny
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2020-03-16 13:46, Dave Howorth wrote:
The problem is that something is grabbing too many resources.
little swapping --> more swapping --> oom killer. The question is what processes are growing at that time. And how much the kernel is paging out to swap before starting to OOM-kill; e.g. if the system has 2G of RAM but 16G of swap, then it would probably take a loooong time (for slow swap reading/writing compared to RAM) until the kernel starts OOM-killing. I don't say that OOM is good in general, but - unless it kills the wrong process - it will make the system responsive again. Well, I think this all has been discussed here 100 times already. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (9)
-
Anton Aylward -
Bernhard Voelker -
Carlos E. R. -
Carlos E. R. -
Dave Howorth -
James Knott -
Richmond -
suse＠a-domani.nl -
Vojtěch Zeisek