Carlos E. R. wrote:

...

I wonder why "swapoff -a" takes so long (7:44 minutes). There is enough free RAM space (after resuming from hibernate). I currently have two swap partitions, one on a WD red hard disk and one on a SSD. But even before (only SSD swap space) the swapoff time was nearly the same.

mybox:~ # iotop mybox:~ # free total used free shared buff/cache available. Mem: 8104720 3435540 3510580 15192 1158600 4492152 Swap: 38281208 3277124 35004084 mybox:~ # time swapoff -a

real 7m44.407s user 0m0.000s sys 4m37.312s mybox:~ # You forgot to post the output of "free" after swapoff. My guess is that

On 2016-04-26 19:51, Bjoern Voigt wrote: there will very little free ram, and possibly less ram assigned to buff/cache.

Notice that the amount of used swap is similar to the amount of free, so that disabling swap takes long. Yes, I though this too and it sound logical. But even with 16 GB RAM I had the problem, that "swapoff -a" tooks much time. I will get my 16 GB RAM back soon, so I can post additional test cases.

Here is a new test case with 8 GB RAM: mybox:~ # free -m total used free shared buff/cache available Mem: 7914 4391 253 27 3269 3335 Swap: 37383 1377 36006 mybox:~ # time swapoff -a real 3m40.079s user 0m0.008s sys 3m0.792s mybox:~ # free -m total used free shared buff/cache available Mem: 7914 5548 57 53 2308 2152 Swap: 0 0 0 All values are in Megabytes. Swapoff takes 3:40 minutes to free 1377 MB used SWAP space. RAM is enough because 3269 MB buffer/cache space is available. 1377 MB in 3:40 minutes mean 6,25 MB per second. My 3 TB WD Red hard disk should have up to 147 MB per second. Of course this a theoretical value, but a SWAP partition can't be fragmented and there was not much other I/O activit

...

I doubt, that the long swapoff time is not only a theoretical problem. I have several phenomenons which can be related:

* System feels sluggish after resuming from hibernate. This problem became stronger after removing 8 GB RAM from 16 GB before. This is normal. After a while the system will become faster.

...

With "iotop" I found, that there is much I/O SWAPIN traffic. * Shutdown can take some minutes This one I think is unrelated. Rather, it has to be some application or service that is taking a long time

...

* Even with much less used SWAP memory swapoff takes too long

Any ideas? What is a reasonable time for swapoff? Actually, disabling swap will make matters worse. I don't plan this. But I sometimes have discussions with Linux beginners which say: "I had a lot of memory (4 GB). I do not need swap. Swap destroys my SSD or my hard disk." It's difficult to explain them, that SWAP is good anyway. But this is another topic.

Greetings, Björ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On 2016-04-27 10:31, Bjoern Voigt wrote:

...

Carlos E. R. wrote: ...

...

Yes, I though this too and it sound logical. But even with 16 GB RAM I had the problem, that "swapoff -a" tooks much time. I will get my 16 GB RAM back soon, so I can post additional test cases.

Here is a new test case with 8 GB RAM:

mybox:~ # free -m total used free shared buff/cache available Mem: 7914 4391 253 27 3269 3335 Swap: 37383 1377 36006 mybox:~ # time swapoff -a

real 3m40.079s user 0m0.008s sys 3m0.792s mybox:~ # free -m total used free shared buff/cache available Mem: 7914 5548 57 53 2308 2152 Swap: 0 0 0 Well, you see, before swapoff you have 253MB free ram, and 1377MB in swap. It just doesn't fit! It had to eat ram from buffers and cache (which makes the computer slower). This operation takes time to do, as (I understand) affected programs have to be notified. Normally cache can be freed easily. Just destroy the cache, so that the data has to be loaded again, if needed.

Unfortunately this Swapoff operation seems to be expensive in Linux. See may mail starting with "I probably found the answer. It's a Kernel problem:"

...

All values are in Megabytes. I prefer "free -h", it is easier to read. Figures have the units next to them.

...

Swapoff takes 3:40 minutes to free 1377 MB used SWAP space. RAM is enough because 3269 MB buffer/cache space is available. But not directly free.

...

1377 MB in 3:40 minutes mean 6,25 MB per second. My 3 TB WD Red hard disk should have up to 147 MB per second. Of course this a theoretical value, but a SWAP partition can't be fragmented and there was not much other I/O activit I don't think disk speed is the factor here. It is possible that the system has to juggle sections in and out of swap repeatedly. Like those sliding puzzles: https://en.wikipedia.org/wiki/Sliding_puzzle#/media/File:15-puzzle.svg

It is also possible that it involves some big/slow application that has to be told. :-? Yes, see my mail starting with "I probably found the answer. It's a Kernel problem:"

...

...

...

Any ideas? What is a reasonable time for swapoff? I don't dare to try in my machine (I have swap 1.4G in use), but I do remember that it is quite slow. I don't remember how much.

...

...

Actually, disabling swap will make matters worse. I don't plan this. But I sometimes have discussions with Linux beginners which say: "I had a lot of memory (4 GB). I do not need swap. Swap destroys my SSD or my hard disk." It's difficult to explain them, that SWAP is good anyway. But this is another topic. Then why do you want to disable it? :-? I never said, that I want to disable swap. But I want to make swap operations faster, which doesn't seem to be possible without modifying the Kernel code itself. I know the parameters swappiness, swap priority etc. They are interesting, but do not help so much here.

I only do that operation for maintenance operations with disks. Possibly kernel devs have not optimized an operation that is seldom done. Yes, probably. Also, nearly nobody seems to use swapoff. For instance, it will probably not called during shutdown/power-off. Otherwise Linux would take several minutes for shutdown/power-off.

I wonder if other swap operation are also affected by this not optimized code. Greetings, Björn -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

I don't see the need to do a swapoff during power off.

Would the disk otherwise be left as "in-use"?

Applications are just told to quit, freeing memory: thus chunks are freed, no matter if actually in ram or in swap.

After which swapoff is probably very fast. -- Per Jessen, Zürich (9.1°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-04-27 14:11, Per Jessen wrote:

Per Jessen wrote:

...

Carlos E. R. wrote:

...

I don't see the need to do a swapoff during power off.

Would the disk otherwise be left as "in-use"?

systemd-shutdown does turn off swap.

If swap is a regular file, then it needs to be swapped off before the filesystem can be unmounted. If swap is on a partition, it can just be discarded, once processes have been terminated. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-04-27 12:55, Carlos E. R. wrote:

On 2016-04-27 13:43, Bjoern Voigt wrote:

...

Carlos E. R. wrote:

...

Unfortunately this Swapoff operation seems to be expensive in Linux. See may mail starting with "I probably found the answer. It's a Kernel problem:"

Yes, I just read it.

Fascinating. I will have to investigate.

I don't see the need to do a swapoff during power off. Applications are just told to quit, freeing memory: thus chunks are freed, no matter if actually in ram or in swap.

One big problem is that programs which make use of garbage-collected memory, especially object-based systems, typically have callbacks that are called at object destruction time to clean up and this in turn can invoke the garbage collector. Unfortunately a lot of these systems don't make a distinction between routine housekeeping during normal program operation, where it's important to keep all the records straight and to free memory so it can be reallocated, and the final cleanup that occurs when the program is shutting down. In the latter case, most of the memory structures could just be forgotten entirely but instead are carefully gathered and made into neat piles before burning the lot! Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. schreef op 27-04-2016 17:35:

Yes, I see...

As Xen says, those chunks would have to be collected from swap, then discarded. If those objects assigned memory from a pool maintained by the application, then the kernel would only know about much bigger chunks assigned from the system by the application memory manager, if it exists. In that case, it would suffice to just clear out the system chunk. And maybe the system does not need to transfers the block to ram first.

This is theory: I don't know the details of how Linux applications usually work regarding memory pools.

But cleanup affecting files that have to be written on disk need operations to be done. Once that is done, remaining assigned memory could just be dumped.

That can't actually work. I don't know if you are a programmer but I think you are. But. The system cannot know whether some application has cleanup stuff to do. In the case of Java, which I know very well or at least reasonably well, and I think C++ would be likened to it. Sometimes as a user I think: screw this, I am turning it off. I employ kill -9 for these circumstances or the hard power button on my pc case. And I worry and I am sick from programs that take longer to shut down than to start up. Like what on hell are they doing? It seriously bugs me. In Windows, no. In Linux, yes, often. On a sidenote, at present I have no installed system and I am just working from a Kubuntu live DVD :P. Having a Kubuntu live dvd (or any) makes you aware of the cute little things going on that require accessing the disc. Pasting some text from the copy-paste buffer in LibreOffce. Requires DVD access. Often. Sometimes. It happens. I think tmpfs might reside in buffer space but I am not sure, it ... I think ramfs resided in buffers pace. Tmpfs can be swapped out. I have 4GB of RAM and usually some 1.6GB worth of buffers. The Entire Kubuntu DVD is about 1.4GB and the loopback mounted /rofs is 1.4GB. That's the entire filesystem. Linux, this live dvd likes to fill itself up with temporary files. The /cow (copy on write, the temporary writable root fs) is currently worth 1.4GB. That stuff resides in RAM. /var/cache is only good for 102M my home directory is good for 574M, 298M in cache (Firefox) and 170M in .local That's ~800MB located. 43M in logs. Okay I installed some programs, lets assume it makes up for the rest. tmpfs is supposed to be swappable and residing in ram, ramfs is not used. It has 2GB buffers/cache and currently it has 1GB available because 900M is in the swap (a partition on the harddisk). (I turned swappiness to 90 :P). Currently my system functions reasonably normally. But without swap and high swappiness it is really a nightmare. Why does LibreOffice read the dvd when pressing ctrl-v? Why is the dvd getting read because I run "ls" ? Isn't that supposed to be in the buffers? How are you going to have 2GB worth of buffers and not cover the entire 1.4GB filesystem? Why does Firefox read the DVD when pressing ctrl-t? Or reloading some page? Why is the DVD getting read because I click on the clipper icon? Why does *every* icon cause the DVD to be read? Pluggable devices, volume control, network window. The second time it is buffered. I do not grasp how 2GB worth of IO buffers can fail to take care of some things that should *not* require disk access. You become fully aware of just what a ludicrous amount of files are constantly being sourced from disk for the simplest of IO operations. I mean user interface operations. -------------- So on the topic of shutdown. There's just no way you can discard memory when it is not in RAM. Some programs may want to shut down a network connection in a nice way. Some want to write changed config to disk. Some want to close file handles. The operating system cannot know if some application has cleanup code to do on the ram. Not possible. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. schreef op 27-04-2016 21:13:

But the applications do. The system just tells the applications to bugger of fast, that it is going to shutdwon. It first tells so nicely, and after sometime issues kill -9 to the rest.

Right, but like Howard said, or what's his name, Howorth, the application might have a memory manager and it is going to process objects one by one. It does not know where in memory its objects reside. The application just wants to close objects. But they have to be loaded first to do so. For example if you have some tree of objects, you start by closing the leaf nodes and then work your way to the root. You don't send a signal to the root to send a signal to the leaf nodes to kill their children (well maybe you do but it is still the same). In general you don't say: "here is the list of all of my nodes (if I have that) let's now all dealloc them. In Java you can't even dealloc something. You put all your pointers to null and work your way up until you have no references to any object anymore. Then you shut down. Or maybe if you don't care you just call System.exit() or something. But I guess people like to clean stuff up nicely because it prevents other kinds of errors and complications. What if you have no cleanup code but now a usecase arises where some remote object does need to? Now you have a problem because the objects are not getting closed, so the cleanup code can't be run until you fix that.

...

Currently my system functions reasonably normally. But without swap and high swappiness it is really a nightmare. Why does LibreOffice read the dvd when pressing ctrl-v?

Maybe because the code to do it is not yet loaded in ram.

You suppose program files and libraries are loaded partially? I could never imagine that code itself would be getting lazily loaded in the sense of "parts of files" being left on disk. But maybe it's just some shared library that was not accessed yet.

...

Why is the dvd getting read because I run "ls" ?

Isn't that supposed to be in the buffers? How are you going to have 2GB worth of buffers and not cover the entire 1.4GB filesystem?

Maybe not everything is loaded into tmpfs. I wouldn't. I would put in the tmpfs only the modifications.

I know. It's just that these buffers seem to be cleaned very rapidly. Currently I am not experiencing any DVD loads while alt-tabbing, doing "ls", or even "ls /". Even "which" does not load anything so I guess parts of the filesystem are buffered now.

Yep. That's why ram dedicated to buffers is important. It would also be important to differentiate code and data buffers. Probably it differentiates r/o from r/w instead.

You know in the days of old in Microsoft Windows you could work for a long time and the harddisk was set to sleep, and you could be working and doing stuff and the harddrive would go to sleep. It was very common in e.g. Windows 98 for the harddrive to just fall asleep while you were doing stuff. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-27 23:44, Xen wrote:

Carlos E. R. schreef op 27-04-2016 21:13:

...

But the applications do. The system just tells the applications to bugger of fast, that it is going to shutdwon. It first tells so nicely, and after sometime issues kill -9 to the rest.

Right, but like Howard said, or what's his name, Howorth, the application might have a memory manager and it is going to process objects one by one. It does not know where in memory its objects reside.

The application just wants to close objects. But they have to be loaded first to do so.

Yes, I know. But if there is nothing to write to disk, of after done with writing to disk, instead of closing objects "properly" (destruct call?), simply tell the memory manager to dump all memory. To return all memory it got from the system, to the system. Probably the application memory manager gets big chunks from the system and gives small sections to parts of the program. But I don't think this is done. Objects are closed properly one by one.

...

...

Currently my system functions reasonably normally. But without swap and high swappiness it is really a nightmare. Why does LibreOffice read the dvd when pressing ctrl-v?

Maybe because the code to do it is not yet loaded in ram.

You suppose program files and libraries are loaded partially?

I know they are. I don't know how exactly this is handled in Linux, but I read somewhere that it was done that way. And Windows does the same. Or rather similarly. Even old MsDos programs could do it, but in that case it was handled by the applications, not by the system. It was called "overlays".

I could never imagine that code itself would be getting lazily loaded in the sense of "parts of files" being left on disk.

But maybe it's just some shared library that was not accessed yet.

That too.

...

...

Why is the dvd getting read because I run "ls" ?

Isn't that supposed to be in the buffers? How are you going to have 2GB worth of buffers and not cover the entire 1.4GB filesystem?

Maybe not everything is loaded into tmpfs. I wouldn't. I would put in the tmpfs only the modifications.

I know. It's just that these buffers seem to be cleaned very rapidly.

Currently I am not experiencing any DVD loads while alt-tabbing, doing "ls", or even "ls /".

Even "which" does not load anything so I guess parts of the filesystem are buffered now.

Yep.

...

Yep. That's why ram dedicated to buffers is important. It would also be important to differentiate code and data buffers. Probably it differentiates r/o from r/w instead.

You know in the days of old in Microsoft Windows you could work for a long time and the harddisk was set to sleep, and you could be working and doing stuff and the harddrive would go to sleep.

Yes, I remember. To do that nowdays in Linux you have to trick the system, to not dump write buffers into disk immediately, but delayed for hours if need be. There is a package, laptop-mode-tools or similar name that does that change if asked.

It was very common in e.g. Windows 98 for the harddrive to just fall asleep while you were doing stuff.

I remember. But not often that late. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlchRmYACgkQja8UbcUWM1zgdwD/acOVbzgBZSQiT8X3kvdpNM+9 rTHBPEWBS7O+XsZ6hEgA/0ejqWYKDoG0VgcS2XhkOBFzuMV+i+DhcQ7OtfLTk/R7 =X80p -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 4/26/16 2:42 PM, Xen wrote:

Bjoern Voigt schreef op 26-04-2016 17:51:

...

I doubt, that the long swapoff time is not only a theoretical problem. I have several phenomenons which can be related:

* System feels sluggish after resuming from hibernate. This problem became stronger after removing 8 GB RAM from 16 GB before. With "iotop" I found, that there is much I/O SWAPIN traffic. * Shutdown can take some minutes * Even with much less used SWAP memory swapoff takes too long

Any ideas? What is a reasonable time for swapoff?

I think Carlos is right that your amount of free RAM was minimal in the figures you cite and that if you want to really troubleshoot it in a way that makes more sense regarding "the swap mechanic" in general you would need to do so while you have more RAM available.

That said you could try to reduce the swappiness value to about 10. And try again.

I think you can just put a new value in: /proc/sys/vm/swappiness

To persist it you need to set it in sysctl.conf. The Swappiness value determines how eager the kernel is in putting stuff into the swap. Or how aggressively it is going to keep buffer memory available (free memory used for buffering IO).

The default for my system is 60 but they say it is only really good for servers and that a desktop would be better suited with a value of like 10.

The theory that I just concocted will be that maybe the kernel tries to put stuff back into the swap while you are trying to clear it. And this is why it takes so long. If this is true, then reducing swappiness a lot should also reduce that time a lot.

Cause I know nothing about the kernel and I don't know if it is getting inhibited (the swap) while it is getting cleared.

If you have zero memory left-over after swapoff you also have zero IO buffers.

Regards.

Oh but you say indeed sorry, that even with less swap it also takes so long. Apologies, didn't see that before.

Personally at this point I do not know how to easily fill my swap but for me swapoff has never taken long. I don't know, maybe just play with swappiness to see if you learn anything?

Regards. What you might want to look at prior to swapoff is free -m

This will tell you how much swap is actually in use. free -m total used free shared buffers cached Mem: 11956 1916 10040 11 121 713 -/+ buffers/cache: 1080 10875 Swap: 16383 0 16383 My laptop here, has 16GB ram. It's using no swap running a KDE desktop swapoff -a free -m total used free shared buffers cached Mem: 11956 1917 10039 11 121 714 -/+ buffers/cache: 1081 10875 Swap: 0 0 0 And it was almost instantaneous If the swap values are high, you need to determine what is using that memory. I suspect that long swapoff time is due to the lag for the out of memory killer to determine what you don't need. But that's just what we used to call a SWAG... sophisticated wild a** guess. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 04/26/2016 02:42 PM, Xen wrote:

The default for my system is 60 but they say it is only really good for servers and that a desktop would be better suited with a value of like 10.

Point of data: Mine is also 60, and this is on a new install (not upgrade) of 13.2. This install never made any pretense of being a server, it was a full kde install from the getgo. With very little data in swap, a test run: poulsbo:~ # time swapoff -U 86b250ea-1e75-433f-a045-73f2a117c1a6 real 0m21.188s user 0m0.000s sys 0m14.064s -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-04-27 00:19, Xen wrote:

They just said (I think it was an Ubuntu page) that the default of 60 causes unresponsiveness in programs while this is less important for a server. So if you want a responsive system, reduce the value. That page

This needs expanding :-) I mean, a server has to respond reasonably fast to a request, or everybody lags in their work. I take it that they mean fast response to the mouse or keyboard. In that case, I think that programs should classify their own code as "need fast response" and "can be delayed", so that the kernel can adjust what to swap fast or slow. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

John Andersen schreef op 26-04-2016 23:15:

To that I would add, that my system seldom uses swap. My swap is there mostly for suspend to disk.

I don't have any noticeable non-responsive incidents that are disk related.

If you have no swap and you have plenty of buffers, that makes sense. Swap is designed for systems that are low on memory.

Wiki says https://en.wikipedia.org/wiki/Swappiness 60 is the default and that value is specifically designed for desktop systems.

It actually doesn't say that that value is specifically designed for desktop systems (even if it is). It says that putting it to 0 (or in the direction of 0) improves responsiveness while putting it to 100 (or in the direction of 100) improves overall (average) performance. That's all it says there.

Setting it lower make it more resistant to swapping, which means releasing buffers when space is needed. Of course this only make sense if you have excess memory such that you essentially never need to swap at all.

No it makes sense if immediate responsiveness is more important to you than overall disk performance. If you are okay with slightly longer load times, but a more responsive system when you need it, then you can lower the value. But I will just respond to Carlos instead ;-).

I might play with other values and see if there are any improvements.

Won't help you if you don't have swap or your system doesn't use it :p. I mean... :-/. It's gotta be me right. Anyway. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward wrote:

On another thread there is the question about browser start-up. many of us have a LOT of tabs "open" in Firefox or similar, and do not start the browser 'empty' and only visit one page at a time. FF is multi-threading; the CPU is multi core. Opening one page of, say, Wikipedia, will have links to explore in parallel. One might then also have an ebay, a Facebook, a LinkedIn, a couple of mail thread, and more all open. On some matters I may check references in email I receive while having 4 or 5 email threads open.

I must admit to working like this even before X11 became dominant; job control in the shell and SCREEN(1) were a wonder for us "multi-taskers".

Linux is pretty good at sharing resources but even so having six or eight "desktops", a couple of Firefox windows each with 30-80 tabs, four or five active email threads open, a couple of documents being edited, half a dozen xterms doing various things, what is ample for a 'single threader' soon gets consumed.

I've been looking at new motherboards and I see some 'server' models with TWO CPUs and 128G of memory slot available .... Would make a nice desktop machine, eh?

I don't quite have that many Windows and/or apps open, usually 20-30 office documents, 3-4 firefox with 20 tabs each, gimp, thunderbird, knode, a bunch of konsoles with many tabs, konquerors and dolphins, okular, maybe a game somewhere. With KDE3 on 10.3, 4Gb is plenty. Probably it would require more with KDE5. Loads of memory would be good for running mprime though. -- Per Jessen, Zürich (8.3°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward wrote:

On 04/27/2016 05:34 AM, Xen wrote:

...

...

Wiki says https://en.wikipedia.org/wiki/Swappiness 60 is the default and that value is specifically designed for desktop systems.

"Default"? Where is that set?

in mm/swap.c. Can be set during start-up in: /etc/sysctl.d/ or whenever you want with "sysctl". -- Per Jessen, Zürich (8.1°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. schreef op 26-04-2016 22:40:

On 2016-04-27 00:19, Xen wrote:

...

They just said (I think it was an Ubuntu page) that the default of 60 causes unresponsiveness in programs while this is less important for a server. So if you want a responsive system, reduce the value. That page

This needs expanding :-)

I mean, a server has to respond reasonably fast to a request, or everybody lags in their work. I take it that they mean fast response to the mouse or keyboard. In that case, I think that programs should classify their own code as "need fast response" and "can be delayed", so that the kernel can adjust what to swap fast or slow.

I don't know. You would consider that overall disk performance is more important to a server. If any service requires memory to be brought out of swap that can incur a delay but the delay will a small 'spike' in a landscape of higher performance. For a server average performance is more important I think. Maybe you are right. But this tends to indicate overall a problem with prioritizing and I don't know if Linux handles it well. I think I have experienced unimportant jobs (such as high IO jobs, archiving) crippling a system where system response became impossible. One example was playing a YouTube video while a (Wine) game was running in the background. The YouTube video lagged so much that it was unplayable. The background game was not shown on screen. Certain classes of applications require almost realtime performance, but Linux is by default not a realtime system. Anything that requires fast response, or audio-video playback, could or should really have a certain amount of "realtimeness" guaranteed to it. Most playback does not require 100% CPU and even with 10% guaranteed it would be okay. Yet I am always afraid in Linux to do some background task -- last time I was just burning a DVD and the buffers (software buffer and device buffer) went down to zero as soon as I started up a network copy task (to my harddisk, or from my harddisk). That is scary and I don't know if the write would have failed. Maybe devices are protected against that these days; they weren't in the past. So I manually tried to ionice the thing (the copy) but in practical terms any cp, tar, rsync, whatever task is always going to be background, and always going to be something that could be classified as "lower priority" for IO. And CPU as well. User interface elements are just higher priority than background processes or tasks. So I think thinking about this thing is more important than swap, and if you do want it to apply to swap, you might just as well use the same measure to begin with ?. Regards. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. schreef op 27-04-2016 11:42:

No general OS is.

Funny thing is there is a Linux version for audio enthusiasts called Audiophile Linux that says it has a realtime kernel: http://www.ap-linux.com/ "Audiophile Linux is based on custom Real Time Kernel. Audio processing is the first priority of this specially crafted Linux distribution. Only the digital music playback is considered and preferred as the main process. Interested enough? Click on learn more." ;-).

...

User interface elements are just higher priority than background processes or tasks.

Not all of them. Like responding to a network packet.

Yeah but that doesn't require disk IO :). But you're right of course. Generally CPU scheduling doesn't seem to me as big a problem as IO scheduling. Thank you for your answers Carlos. Much appreciated. Xen. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

...

...

...

User interface elements are just higher priority than background processes or tasks.

Not all of them. Like responding to a network packet.

Yeah but that doesn't require disk IO :).

Except if the handler was swapped off :-p

Interrupt handlers are non-pageable. -- Per Jessen, Zürich (2.9°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

27.04.2016 13:15, Bjoern Voigt пишет:

Unfortunately nobody seems to work on this. There was a discussion 2007 on Kernel mailing list, but without results: http://copilotco.com/mail-archives/linux-kernel.2007/msg39423.html

Quoting this thread. --><-- before you go there... is this a "real life" problem? Or just a mostly-artificial corner case? (the answer to that obviously is relevant for the 'should we really care' question) --><-- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-04-28 09:22, Bjoern Voigt wrote:

Swapoff for swap partitions seems to be never used in system shutdown scripts. I use Swapoff mostly for testing.

But swapoff is called at shutdown for swap files (as opposed to swap partitions). -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Thu, Apr 28, 2016 at 11:59 AM, Per Jessen wrote:

Dave Howorth wrote:

...

On 2016-04-28 09:22, Bjoern Voigt wrote:

...

Swapoff for swap partitions seems to be never used in system shutdown scripts. I use Swapoff mostly for testing.

But swapoff is called at shutdown for swap files (as opposed to swap partitions).

I looked it up - in systemd-210 in ./src/core/umount.c, swapoff() is called, as far as I can tell regardless of whether it is a file or a device. It's possible there is a check on file or partition somewhere else, I could have missed it.

No, there is no such check. But this happens very late, when almost everything has already been stopped or killed, so swap space is almost unused. Time needed for swapoff is directly proportional to amount of swap space in use. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-28 13:28, Bjoern Voigt wrote:

Andrei Borzenkov wrote:

...

On Thu, Apr 28, 2016 at 11:59 AM, Per Jessen wrote:

...

Dave Howorth wrote:

...

...

I looked it up - in systemd-210 in ./src/core/umount.c, swapoff() is ... Then we have a good "real life" use case. Swapoff seems to delay the shutdown (and maybe the restart) process. This would explain my observations. Sometimes shutdown is fast, sometimes slow and sometimes I think, it will never finish and I do a hard power-off. I should write down the memory situation (output of "free") and the shutdown times.

I doubt it. The binary code that makes that call is, I think, happening at the end, after stopping all services, which is probably what takes longer. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlch9hAACgkQja8UbcUWM1xCJAD/Sxsba4Ahd35j0Ew7SBN+b76G GQzTg5qcF9zM5FssLlUA/09YxDbqswJ6OeDpkQXLXlFz1tOfqKyA55jFI3ZufQVD =1wEf -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-04-29 00:02, Xen wrote:

Ideally you would be able to produce a form of profiling map with a simple switch that would display results for the various stages of (say) shutdown and how much time was spent in what. I do not know if this exists for Linux.

For boot, yes. It is part of systemd: systemd-analyze does it. systemd-analyze - Analyze system boot-up performance But it does not analyze halt. Otherwise, yes, there is a debugging tool for profiling applications. The problem is that during halt things are being stopped, and logs lost. Log entries can be sent to another machine, though. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

Bjoern Voigt wrote:

Swapoff for swap partitions seems to be never used in system shutdown scripts. I use Swapoff mostly for testing.

I think systemd-shutdown will do a swapoff. According to the man page:

Before shutting down, this binary will try to unmount all remaining file systems, disable all remaining swap devices, detach all remaining storage devices and kill all remaining processes.

Presumably this is quick because no swap-space is in use at this point. -- Per Jessen, Zürich (6.4°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-28 10:46, Xen wrote:

I mean it is clear you are enthusiast and I applaud that but having like 2-4GB in swap is rare in these days and times and probably if you did want to turn it off, a more common use case is probably going to be moving swap to another partition.

Not that rare. My desktop machine has 8 GiB of ram, and swap usage yesterday was around 2.5 GB. But I wouldn't ever do a swapoff, except when moving to another disk or something maintenance wise.

Because if you have a loaded swap (except perhaps for shutdown) then it's going to be there for a reason and you might be better off activating a second swap before disabling the first one. Well maybe it doesn't make a difference seeing how it works.

Yes, it does make a difference. I spread swap over all internal hard disk, and swapof is faster, IIRC, if there is another swap partition active.

I created a second swap and activated it, then swapoff-ed the first one. All of it was loaded into RAM and none of it was moved to the second swap, 600M and it took about 20 seconds.

I thought it had different behaviour, sorry. Now I lost my IO buffers :p.

Well, that was because you had little amount in use, I think. Also the priority of each swap may make a difference. Maybe what happens is not that a block is moved from one swap to another, but that some memory blocks are swapped out again to the available spaces, thus freeing memory for the process that is doing the swap off.

20 seconds for 600M is quite long. The disk has a sequential read of 90MB/s.

But it is not a continuous read. Much processing has to be done for each block, apparently.

It's not pleasant and it's not beautiful. And if it hinders shutdown I can expect it to be an issue.

But it doesn't seem to be so. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlch664ACgkQja8UbcUWM1zrxwD/V+EJkGzh4apIsUFsS3EwYi04 vuUTIBJAv5x7qDZviz8A/iJEZwANb0QQspGa/ioSlFuwMqYnQ2ejiP7XrMn5y/CK =DcGI -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-28 12:59, Xen wrote:

On Thu, 28 Apr 2016, Carlos E. R. wrote:

...

Not that rare. My desktop machine has 8 GiB of ram, and swap usage yesterday was around 2.5 GB. But I wouldn't ever do a swapoff, except when moving to another disk or something maintenance wise.

I guess if you use VMs or other high memory applications (video photo editing?)

A long video conversion was running. But the big spenders are firefox, thunderbird, and libreoffice. But daemons such as amavis come close. Then there are a myriad of things that use silly amounts of memory for what they do.

...

...

It's not pleasant and it's not beautiful. And if it hinders shutdown I can expect it to be an issue.

But it doesn't seem to be so.

And so we all wonder what the need is. Although it would be a nice project perhaps.

Yep :-) - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlch7ugACgkQja8UbcUWM1z2aAD/Vnru0WcHyi9xcU4Ie5FHPfHW nPIQw2GYW+9xXL0h9XEA/jDYhQ6h46eSv/sR1ErR2VXmQnT6F/nwK0EDLIWBlUqV =XqAL -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org