On Tue, 19 Mar 2019 12:59:24 -0700 L A Walsh <suse@tlinx.org> wrote:
Huge pages are the ones that are larger than the "standard" 8K pages Linux uses.
On larger memory systems, say 256GB servers or 128G desktops, it might be useful to use 'always'. At a 2MB page size, you can fit 512 "items" into 1G, so for an 8GB system, that translates to 4096 items that can fit into memory at a time (maybe a bit more if they only use hugepages for data and not code).
If you run a program that wants to store half of a million words in memory and it wants to allocate 1 memory segment for each word, it thinks that would be 8k (standard page size) * 512K = 4096 MB or 4GB.
But if each page is 2M, for 512K entries, it would take 1024*1024 MB, or about one TB of space. Most systems would have to have a large paging file to support that which is guaranteed to cause the latency associated with swapping.
If you had a system with 1-2TB of memory in it, it would still be rather slow as there is no way that memory would fit in the cpu cache.
The transparent part has to do with whether or not an application can use 'huge pages' without being aware that they exist (i.e. needing NO changes to the program, thus "transparent". madvise refers to the name of the posix call that can tell the kernel how your application wants to use memory.
One of those calls can tell the kernel what areas of memory should be best handled in 'huge pages'. That's the idea situation - programs that can run best with large contiguous memory segments ask for them and generally run faster due to a need to use 4096x less kernel pages to map their data into active memory. This is very beneficial as system memory sizes grow.
For a VM that fits into a 64m memory space, it usually would be more efficient to map that in via 32 'map' calls vs. 8096 'map' calls, meaning the host would likely be more efficient running that program with transparent huge pages (which the client would likely use normal size pages assuming it simply used memory as a linear address space), If using huge pages on the host requires that the client also use huge pages, the client would only have 32-things it could map into memory and would likely suffer performance problems.
I don't know which vm techs allow for treating memory differently from the host vs. the client viewpoint.
Hopefully this gives a more clear idea about what these options mean and when they might be useful...(?)
Yes, it makes things fairly clear. They're useful on big machines and they're useful with VMs. Since I have a modest machine and never use VMs I question why my system is set to 'always' by default? Especially given that it does apparently have negative effects on programs that I *do* run. And since a VM technology can easily request huge pages, as can other programs that use large amounts of memory in suitable ways, why do they have to be 'transparent'?
if not, feel free to ask more questions and I can try to answer them, though if you want me to respond more quickly, sending a copy directly to me would likely have me seeing it more quickly...
Feel free to modify the reply-to settings if you want personal replies. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org