largest mem available to a 64 bit binary, and how I can reduce buffer cache usage?
Hi folks: I have an application I am running which wants to use as much memory as possible. I have a quad 848 with 32 GB ram onn it. It has been running for a while with an RSS of 15GB, and a total memory size of 19GB. This leaves well over 11 GB for the cache and the os. Sure enouch, using the atop tool, and the vmstat tool, I see cache at 15068336k, free mem at 10432k, and it is swaped out to the maximum extent possible. From what I see, it needs about another 4 GB to do the operation entirely in RAM. I m having severe difficulty believing that keeping buffer cache so high, and swapping out pages is in any way helpful to this process. I did set user limits to unlimited where possible, adjusted the kernel.sh{max,all} to enable 20 GB of shared memory. What else can I do? I would like to be able to use 3/4 of the ram of the machine for applications, and not have 4 GB of swapped pages when I have 15 GB of disk cache. I understand the need for disk cache, but paging to disk, and filling up swap space in this particular scenario makes no sense whatsoever. I see this on SuSE 9.0 2.4.21-215-smp, as well as the RHEL kernels on the machine. This is a compute node for a cluster. Thanks. Joe -- Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com
On Sun, Oct 03, 2004 at 09:28:04AM -0400, Joe Landman wrote:
I have an application I am running which wants to use as much memory as possible. I have a quad 848 with 32 GB ram onn it. It has been running for a while with an RSS of 15GB, and a total memory size of 19GB. This leaves well over 11 GB for the cache and the os. Sure enouch, using the atop tool, and the vmstat tool, I see cache at 15068336k, free mem at 10432k, and it is swaped out to the maximum
The "cache" is your program. free mem indeed sounds a bit high (you sure it isn't a typo?)
What else can I do? I would like to be able to use 3/4 of the ram of the machine for applications, and not have 4 GB of swapped pages when I have 15 GB of disk cache. I understand the need for disk cache, but paging to disk, and filling up swap space in this particular scenario makes no sense whatsoever.
There are various VM tunables in /proc/sys/vm. Play with them. However I have tested similar configurations in the past and it definitely didn't show such behaviour as you're describing. A single process was usually able to fill up near all of memory in the default configuration, although the VM would do constant aging of pages would cost a bit of CPU time. This can be tuned however. -Andi
No, it was a cut and paste. I will go back and look. The "cache" line is not buffer cache then, but program page and data cache? I have been playing a bit with the /proc/sys/vm/ stuff. I am wondering if it is time to move this to 2.6. Joe Andi Kleen wrote:
On Sun, Oct 03, 2004 at 09:28:04AM -0400, Joe Landman wrote:
I have an application I am running which wants to use as much memory as possible. I have a quad 848 with 32 GB ram onn it. It has been running for a while with an RSS of 15GB, and a total memory size of 19GB. This leaves well over 11 GB for the cache and the os. Sure enouch, using the atop tool, and the vmstat tool, I see cache at 15068336k, free mem at 10432k, and it is swaped out to the maximum
The "cache" is your program. free mem indeed sounds a bit high (you sure it isn't a typo?)
What else can I do? I would like to be able to use 3/4 of the ram of the machine for applications, and not have 4 GB of swapped pages when I have 15 GB of disk cache. I understand the need for disk cache, but paging to disk, and filling up swap space in this particular scenario makes no sense whatsoever.
There are various VM tunables in /proc/sys/vm. Play with them.
However I have tested similar configurations in the past and it definitely didn't show such behaviour as you're describing. A single process was usually able to fill up near all of memory in the default configuration, although the VM would do constant aging of pages would cost a bit of CPU time. This can be tuned however.
-Andi
--
participants (2)
-
Andi Kleen -
Joe Landman