Have you looked into HUGEPAGES? These are allocated as 2 MEG pages instead of 4 K pages. You can set aside part of memory for HUGE pages at Boot time. Maybe SUSE doesn't recommend using HUGEPAGES I don't know. /boot/grub/menu.lst and add the following string to the kernel line: hugepages=xx where xx is the number of HUGE pages you want allocated. You can also put a value in /etc/sysctl.conf file: vm/nr_hugepages = xx then on the SHMGET adds the parameter SHM_HUGETLB And the pages will come out of these HUGE pages. Now I have never quite figure out how to calculate a good value for 'xx'. Also I can't find any documentation that states if you can give pages back if you over allocate HUGE pages. or what happens if 'xx' is too small for your requested allocation. Will part of it come out of huge pages and the rest come out of Small pages or will it just fail. --- -----Original Message----- From: Brad Dameron [mailto:brad@seatab.com] Sent: Thursday, July 07, 2005 9:12 AM To: SuSE-Linux-E@suse.com; suse-amd64@suse.com Subject: Re: [suse-amd64] mmap & /dev/shm Sorry for cross-list messaging. But I use SuSe on both i686 and 64bit. On Thu, 2005-07-07 at 11:48 +0200, Andi Kleen wrote:
On Wed, Jul 06, 2005 at 03:59:32PM -0700, Brad Dameron wrote:
We have a in-house application that uses mmap to pull a file into memory. I can't seem to find any info on where this file is stored in memory. I know that it is shared memory. But does the /dev/shm have anything to do with it. Isn't shm just a tmpfs? Also can shm be
Not directly no. Yes.
disabled? I have a machine with 16GB and 8GB then goes to the shm. Does
Yes, but you probably don't need to.
It just means that shm could grow upto 8GB if someone put that much data into it by creating files in it. Normal mmap doesn't do that.
this only allow mmap to use the other 8GB or is all 16GB accessible to it? These are the questions I have been trying to figure out for over a
All 16GB accessible for it. Actually a few percent less because there are some other data structures like page tables that need to fit into memory too.
week now. Not many answers on google and no one has seemed to ask this before.
Also is mmap the fastest way to get a file from disk to memory?
It depends. Normal read() does more aggressive readahead which can be faster. However mmap() won't copy and you cache the data only once.
-Andi
Andi, First thanks for your response. If I mmap say a 1GB file, then I mmap another 1GB file. It will then have both files in memory correct? And how long will it keep them in memory? Also I noticed it uses PAGE_SIZE as the block size to grab the files in. Is there a way to increase this just for mmap or is the default of 4096kB a good size to use. We mainly deal with 100MB to 5GB file sizes. Nothing really small. And usually everything sequential. Any further answer's appreciated. Thank you, Brad Dameron SeaTab Software www.seatab.com -- Check the List-Unsubscribe header to unsubscribe For additional commands, email: suse-amd64-help@suse.com