Re: [SLE] swap space question in 9.3 install
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 El 2005-07-29 a las 00:41 +0100, Jonathan Brooks escribió: You forgot to email to the list.
Huh, what has swap got to do with suspending to disk? I thought it just mirrored what physical RAM you have installed?????
Well, ask the kernel developpers :-) They decided to use swap space as space for swapping or suspending to disk. It makes sense to me, tasks are just swapped out completely before swapping out memory pages; it is a raw partition, no drivers involved: just ideal. In fact, when restoring, some memory pages are not swapped back in, they stay off for days. Look: top - 02:42:07 up 16 days, 4 min, 25 users, load average: 0.08, 0.22, 0.27 Tasks: 166 total, 1 running, 165 sleeping, 0 stopped, 0 zombie Cpu(s): 8.3% us, 2.7% sy, 0.0% ni, 88.4% id, 0.3% wa, 0.3% hi, 0.0% si Mem: 775852k total, 682772k used, 93080k free, 13196k buffers Swap: 1052216k total, 224240k used, 827976k free, 404580k cached Notice the used swap area, and ram used, free and cached. - -- Saludos Carlos Robinson -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFC6Xw5tTMYHG2NR9URAuJFAJ9CAB5Fju519deoTTdYnrC4ytvuJQCdHa8+ F4YxwOW784L6jm3ezoLSD7k= =S9Vq -----END PGP SIGNATURE-----
On Thursday, July 28, 2005 @ 4:46 PM, Carlos Robinson wrote:
El 2005-07-29 a las 00:41 +0100, Jonathan Brooks escribió:
You forgot to email to the list.
Huh, what has swap got to do with suspending to disk? I thought it just mirrored what physical RAM you have installed?????
Well, ask the kernel developpers :-)
They decided to use swap space as space for swapping or suspending to disk. It makes sense to me, tasks are just swapped out completely before swapping out memory pages; it is a raw partition, no drivers involved: just ideal. In fact, when restoring, some memory pages are not swapped back in, they stay off for days. Look:
top - 02:42:07 up 16 days, 4 min, 25 users, load average: 0.08, 0.22, 0.27 Tasks: 166 total, 1 running, 165 sleeping, 0 stopped, 0 zombie Cpu(s): 8.3% us, 2.7% sy, 0.0% ni, 88.4% id, 0.3% wa, 0.3% hi, 0.0% si Mem: 775852k total, 682772k used, 93080k free, 13196k buffers Swap: 1052216k total, 224240k used, 827976k free, 404580k cached
Notice the used swap area, and ram used, free and cached.
Interesting. On a Windoze machine, when you start a program, it loads two copies of the program into memory, one into cache and one into active memory. If you close the program, the active memory area is freed, but the cached copy stays there until you do a restart or until the memory runs low and Windoze needs to reuse it. If you turn right around and re-start the program, the program is copied from the cache area of RAM over to the active memory area (a memory to memory copy), so that it starts faster the second time. Is this the same thing with swap on Linux? Is everything in "used" also in "cached", along with some copies of programs that are no longer executing (I e., the memory is available for re-use, if necessary, but also available for a fast load of the program)? Greg Wallace
nteresting. On a Windoze machine, when you start a program, it loads two copies of the program into memory, one into cache and one into active memory. If you close the program, the active memory area is freed, but the cached copy stays there until you do a restart or until the memory runs low and Windoze needs to reuse it. If you turn right around and re-start the program, the program is copied from the cache area of RAM over to the active memory area (a memory to memory copy), so that it starts faster the second time. Is this the same thing with swap on Linux? Is everything in "used" also in "cached", along with some copies of programs that are no longer executing (I e., the memory is available for re-use, if necessary, but also available for a fast load of the program)? In some cases yes. Remember that a program is not necessarily copied to backing store (eg. swap space), and sometimes only part of the program may be resident. I have not looked at the 2.6 kernel VM spec, but AFAIK, the system does not release all the pages. In some Unix systems in the past,
On Friday 29 July 2005 12:44 am, Greg Wallace wrote: the VM would keep programs in RAM for a while until it had to reuse those pages. -- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
Hi, On Thursday 28 July 2005 21:44, Greg Wallace wrote:
...
Interesting. On a Windoze machine, when you start a program, it loads two copies of the program into memory, one into cache and one into active memory. If you close the program, the active memory area is freed, but the cached copy stays there until you do a restart or until the memory runs low and Windoze needs to reuse it. If you turn right around and re-start the program, the program is copied from the cache area of RAM over to the active memory area (a memory to memory copy), so that it starts faster the second time. Is this the same thing with swap on Linux? Is everything in "used" also in "cached", along with some copies of programs that are no longer executing (I e., the memory is available for re-use, if necessary, but also available for a fast load of the program)?
For the way most programs are linked, the text (and some of the pre-initialized data) is considered "pure" (read-only) and thus shareable. Furthermore the linker can align the pure code and data within the object file so it's properly aligned to the hardware's paging boundaries. Then the OS need never copy that information to swap space. Demand paging retrieves it directly from the executable program or shared-object file. When a page holding pure code or data is to be replace, it's simply abandoned, never needing to be paged out.
Greg Wallace
Randall Schulz
On Friday, July 29, 2005 @ 5:32 AM, Randall Schulz wrote:
Hi,
On Thursday 28 July 2005 21:44, Greg Wallace wrote:
...
Interesting. On a Windoze machine, when you start a program, it loads two copies of the program into memory, one into cache and one into active memory. If you close the program, the active memory area is freed, but the cached copy stays there until you do a restart or until the memory runs low and Windoze needs to reuse it. If you turn right around and re-start the program, the program is copied from the cache area of RAM over to the active memory area (a memory to memory copy), so that it starts faster the second time. Is this the same thing with swap on Linux? Is everything in "used" also in "cached", along with some copies of programs that are no longer executing (I e., the memory is available for re-use, if necessary, but also available for a fast load of the program)?
For the way most programs are linked, the text (and some of the pre-initialized data) is considered "pure" (read-only) and thus shareable. Furthermore the linker can align the pure code and data within the object file so it's properly aligned to the hardware's paging boundaries. Then the OS need never copy that information to swap space. Demand paging retrieves it directly from the executable program or shared-object file. When a page holding pure code or data is to be replace, it's simply abandoned, never needing to be paged out.
Greg Wallace
Randall Schulz
So are you saying that this "pure" code is still re-read from disk, or can it be sometimes available for use directly from a previously loaded "pure" copy in RAM? I. e., is a RAM copy ever available for re-use after a program has finished, or is it always re-loaded from disk (whether from swap or from the file copy itself)? Greg Wallace
So are you saying that this "pure" code is still re-read from disk, or can it be sometimes available for use directly from a previously loaded "pure" copy in RAM? I. e., is a RAM copy ever available for re-use after a program has finished, or is it always re-loaded from disk (whether from swap or from the file copy itself)? As I had mentioned before, the text section of a program contains the instructions (and some initialized read-only data, such as strings). When a
On Friday 29 July 2005 10:05 am, Greg Wallace wrote: program is loaded, the text section is mapped into virtual memory from the place where it resides (eg. /usr/bin). For instance, if you run multiple instances of a program at the same time, only 1 physical copy of the text section will reside in physical RAM, and will not ever be swapped out to backing store (eg. swap space). Additionally, data is mapped in read-only using a technique called copy-on-write which means that when written to, the page is then marked as dirty, and will be paged out to the swap area if necessary. The third section of a program is uninitialized data. Pages are not allocated for uninitialized data until the program uses them. At that time, generally the using page is zeroed and allocated to the program, and marked dirty. Additionally, libraries used in Linux are called shared objects (.so). So, in addition to multiple instances of the same program sharing the text section, all code in a Linux and Unix system using shared libraries will share the same physical libraries in memory. Also, note that Randall mentioned that the linker must properly align the sections. The initial alignment of the sections is performed first by the compiler. There is usually a "calling standard" that defines some of these things. It is the calling standards that allow C, C++, Fortan, and other languages to call eachother and to call functions in the libraries. -- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
On Friday, July 29, 2005 @ 6:29 AM, Jerry Feldman wrote:
On Friday 29 July 2005 10:05 am, Greg Wallace wrote:
So are you saying that this "pure" code is still re-read from disk, or can it be sometimes available for use directly from a previously loaded "pure" copy in RAM? I. e., is a RAM copy ever available for re-use after a program has finished, or is it always re-loaded from disk (whether from swap or from the file copy itself)? As I had mentioned before, the text section of a program contains the instructions (and some initialized read-only data, such as strings). When a
program is loaded, the text section is mapped into virtual memory from the place where it resides (eg. /usr/bin). For instance, if you run multiple instances of a program at the same time, only 1 physical copy of the text section will reside in physical RAM, and will not ever be swapped out to backing store (eg. swap space). Additionally, data is mapped in read-only using a technique called copy-on-write which means that when written to, the page is then marked as dirty, and will be paged out to the swap area if necessary. The third section of a program is uninitialized data. Pages are not allocated for uninitialized data until the program uses them. At that time, generally the
using page is zeroed and allocated to the program, and marked dirty. Additionally, libraries used in Linux are called shared objects (.so). So, in addition to multiple instances of the same program sharing the text section, all code in a Linux and Unix system using shared libraries will share the same physical libraries in memory.
Also, note that Randall mentioned that the linker must properly align the sections. The initial alignment of the sections is performed first by the compiler. There is usually a "calling standard" that defines some of these things. It is the calling standards that allow C, C++, Fortan, and other languages to call eachother and to call functions in the libraries. -- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
Ok, I follow you now. If the last process accessing the "text section of a program" ... "instructions (and some initialized read-only data, such as strings)" closes and then you re-start the program, is that part of the program still in RAM, or would it have to be loaded from swap or from the file? Thanks, Greg Wallace
Ok, I follow you now. If the last process accessing the "text section of a program" ... "instructions (and some initialized read-only data, such as strings)" closes and then you re-start the program, is that part of the program still in RAM, or would it have to be loaded from swap or from the file? The text section is never copied over to backing store (swap). The load
On Friday 29 July 2005 8:51 pm, Greg Wallace wrote: process simply "maps" that section in from where it resides. I don't know the specific algorithm the kernel uses these days for persistence, but I would assume that if the pages are still resident in RAM that they would be reused. This has been a common technique used in Unix systems for years since Unix/Linux commands are generally small executables. The same thing applies to libraries. The core libraries, such as libc.so are used by daemons, so it should be almost continuously resident. -- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
On Monday, August 01, 2005 @ 4:51 Am, Jerry Feldman wrote:
Ok, I follow you now. If the last process accessing the "text section of a program" ... "instructions (and some initialized read-only data, such as strings)" closes and then you re-start the program, is that part of the program still in RAM, or would it have to be loaded from swap or from the file? The text section is never copied over to backing store (swap). The load
On Friday 29 July 2005 8:51 pm, Greg Wallace wrote: process simply "maps" that section in from where it resides.
I don't know the specific algorithm the kernel uses these days for persistence, but I would assume that if the pages are still resident in RAM
that they would be reused. This has been a common technique used in Unix systems for years since Unix/Linux commands are generally small executables. The same thing applies to libraries. The core libraries, such as libc.so are used by daemons, so it should be almost continuously resident. -- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
So, taking the numbers from Carlos Robinson's Friday post -- SWAP 1,052,216K TOTAL 224,240K USED 827,976K FREE 404,580K CACHED Maybe CACHED is the 224,240K USED plus 404,580K - 224,240K = 180,340K previously used; i. e., modules that were earlier called but are no longer running. If you invoke one of those modules again, it gets copied from CACHED to USED instead of from the medium to USED. Greg Wallace
So, taking the numbers from Carlos Robinson's Friday post --
SWAP 1,052,216K TOTAL 224,240K USED 827,976K FREE 404,580K CACHED
Maybe CACHED is the 224,240K USED plus 404,580K - 224,240K = 180,340K previously used; i. e., modules that were earlier called but are no longer running. If you invoke one of those modules again, it gets copied from CACHED to USED instead of from the medium to USED. The page caches are used to indicate which pages have been copied to backing store so that they do not need to be saved if they are evicted. But, the
On Monday 01 August 2005 5:40 pm, Greg Wallace wrote: page cache is actually in RAM memory. -- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
Greg, On Monday 01 August 2005 14:40, Greg Wallace wrote:
...
SWAP 1,052,216K TOTAL 224,240K USED 827,976K FREE 404,580K CACHED
Maybe CACHED is the 224,240K USED plus 404,580K - 224,240K = 180,340K previously used; i. e., modules that were earlier called but are no longer running. If you invoke one of those modules again, it gets copied from CACHED to USED instead of from the medium to USED.
Really, guessing is quite inappropriate in this case: <http://gentoo-wiki.com/FAQ_Linux_Memory_Management> gives a pretty good explanation of what these various parameters mean.
Greg Wallace
Randall Schulz
You can't really say this thread has beenhijacked. Hijackers usually know where they want to go. Perhaps "joyriding" is a more apt description. While the respondents seem to be having a good time and I wouldn't want to chill their reminiscences perhaps they could wander across to Suse-OT? michaelj -- Michael James michael.james@csiro.au System Administrator voice: 02 6246 5040 CSIRO Bioinformatics Facility fax: 02 6246 5166 No matter how much you pay for software, you always get less than you hoped. Unless you pay nothing, then you get more.
participants (6)
-
Carlos E. R. -
Greg Wallace -
Greg Wallace -
Jerry Feldman -
Michael James -
Randall R Schulz