-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11/10/2011 11:06 AM, milan zimmermann wrote:
On Wed, Nov 9, 2011 at 3:41 PM, Jeff Mahoney <jeffm@suse.de> wrote:
virtual memory (kbytes, -v) 29536000
... but here is your problem.
I disagree. With the switches I used, and with
Then test it. On my system, I see: jeffm@jetfire:~> ulimit -v unlimited jeffm@jetfire:~> ./a.out Hello world Allocating 29Gb... Success: 0x7fc7732d6000 jeffm@jetfire:~> ulimit -v 28000000 jeffm@jetfire:~> ./a.out Hello world Allocating 29Gb... FAILED
sysctl -w vm.overcommit_memory=0 # or 1
according to the link I posted
http://www.mjmwired.net/kernel/Documentation/vm/overcommit-accounting
the cost of allocation with these parameters should be 0, and the call
mmap((void *) NULL, someSize * (((size_t) 1) << 30), PROT_NONE, MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE, -1, 0); # someSize is integer with number of Gigs
should succeed no matter what virtual memory, and no matter what size I am allocating.
As far as the kernel generally is concerned, yes. You're hitting user resource limits, not a kernel out-of-memory condition. If you had overcommit disabled, then you might run into that issue.
So far I have no other explanation then this is a bug (anyone agrees?). On the chrome list, people indicated same issue was fixed in Ubuntu lately (mmap 84Gb succeds with way less virtual memory). I only have 4 boxes with Opensuse ... will also try Opensuse 12.1 and report
I blame Windows for perpetuating this myth that "virtual memory" = "real memory + swap" since that's not at all what it means. Virtual memory is the address space. You're allocating part of the address space but you're not using it. - -Jeff
let me know if i should run anything else, thanks
On my 11.4 64-bit system, I can't reproduce your failure. I see:
jeffm@sled2:~> ./a.out Hello world Allocating 29Gb... Success: 0x7f7f79e57000 jeffm@sled2:~> uname -a Linux sled2 2.6.37.6-0.9-desktop #1 SMP PREEMPT 2011-10-19 22:33:27 +0200 x86_64 x86_64 x86_64 GNU/Linux
-Jeff
> I am trying to resolve a failure of Google Native > Client in Opensuse 64 11.4, discussed in Google Native > Client forum: > > [1] > https://groups.google.com/forum/#!topic/native-client-discuss/7DUFfi_BxqM > > >
> To repeat the issue in a nutshell:: > --------------------------------------------- > > Google native client (Chrome) works on recent versions > of at least Ubuntu [1], but fails on Opensuse 11.4 > (with all latest updates up to Nov 4). This failure can > be reproduced in chrome 14, 15, 16 (from > http://dl.google.com/linux/chrome/rpm/stable/x86_64) > and verified by loading > > [2] > http://www.gonacl.com/dev/demos/sdk_examples/load_progress/load_progress.htm... > > >
> The problem / question for opensuse (kernel?) :: > ------------------------------------------------------------------ > > >
> There is a long discussion in the above thread, to get to the > point quickly: The Google guys identified an issue > with mmap() with MAP_NORESERVE (see below). They > believe it may be a bug or a kernel configuration > issue(?) > > A Chrome Nacl person suggest the following code should > print "Success" but it fails in my testing: > > #include <stdio.h> #include <sys/mman.h> > > int main(void) { void *addr; > > printf("Hello world\nAllocating 29Gb...\n"); addr = > mmap((void *) NULL, 29 * (((size_t) 1) << 30), > PROT_NONE, MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE, > -1, 0); /* test 29 or other values */ if (MAP_FAILED == > addr) { printf("FAILED\n"); } else { printf("Success: > %p\n", addr); } return 0; } > > This prints FAILED on Opensuse 11.4 64 bit. > > I did some experiments. On my system: > > # cat /proc/meminfo MemTotal: 15948428 kB > MemFree: 11270612 kB .... CommitLimit: 28945728 kB > Committed_AS: 4918284 kB ... > > From running the test program above, it looks like > *CommitLimit* is clearly used as upper limit of > mmap(MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE), *no > matter what vm.overcommit_memory* flag is used. > > In concrete terms: > > mmap((void *) NULL, 29 * (((size_t) 1) << 30), > PROT_NONE, MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE, > -1, 0); // always FAILS with value of 29 or higher > > mmap((void *) NULL, 28 * (((size_t) 1) << 30), > PROT_NONE, MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE, > -1, 0); // always SUCCEEDS with value of 28 or lower > > No matter what setting of sysctl -w > vm.overcommit_memory=0 # or 1 or 2 > > (This seems a Opensuse 11.4 bug in modes 0 and 1, as > according to [2] anonymous private readonly should have > 0 cost) > > Any comments or solutions or how to fix this? > > Thanks, > > Milan > > [1] > http://www.mjmwired.net/kernel/Documentation/filesystems/proc.txt > >
> [2] > http://www.mjmwired.net/kernel/Documentation/vm/overcommit-accounting > > > > >
> ======================= PS: I am attaching a few pieces of info > about my system that may be relevant: > > The hardware is AMD 4 core AMD Athlon II X4 610e and > has 16Gb (sixteen) of memory, running very little (just > KDE desktop at this point) > > # swapon -s Filename > Type Size Used Priority /dev/sda1 partition > 20971516 0 -1 > > # cat /proc/sys/vm/overcommit_memory 0 > > # ulimit -a core file size (blocks, -c) 0 data > seg size (kbytes, -d) unlimited scheduling priority > (-e) 0 file size (blocks, -f) unlimited > pending signals (-i) 123980 max locked memory > (kbytes, -l) 64 max memory size (kbytes, -m) 13556224 > open files (-n) 1024 pipe size (512 bytes, > -p) 8 POSIX message queues (bytes, -q) 819200 real-time > priority (-r) 0 stack size (kbytes, -s) > 8192 cpu time (seconds, -t) unlimited max user > processes (-u) 123980 virtual memory > (kbytes, -v) 29536000 file locks (-x) unlimited > > # df Filesystem 1K-blocks Used Available > Use% Mounted on rootfs 82567856 > 15558248 62815408 20% / devtmpfs 7934744 > 244 7934500 1% /dev tmpfs 7974212 > 1592 7972620 1% /dev/shm /dev/sda2 > 82567856 15558248 62815408 20% / /dev/sda3 377510440 > 90623252 267710692 26% /home
- -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJOu/iYAAoJEB57S2MheeWywEAP/ROIa5GQo+wasRACfKNH6xJd /4E0MBxWbKBduorirabz786vIuZZ7xKzX0ZzakBh3xO0896SFTbzukJRPGWO+3tG hf1xrS+BiRQywIgS+d4xDLspLAgiXh2RlH5ANPaeE+jEy3+lCvDM/aH0UiOcGkGd nt9fQemP3QmoUSAK1xl+tXsHvK4yP5g/If/JDFiVhmNpSTRA3w8qqwL6PCDLnzH9 dL0XqN8e72a+xXCvkgPDLHFevVs2xkjSwJmtDpDnO9z1Bxj8lyWhpjl03mUu9jh9 0gxbT/soEr8YqBT9OtmX/o3OHZ3Ex7jHweEv10DmNg2H8z7jf8LMm/mTor4OXAQH PSoEwVfauBLhNMwWc+WYmBK++Pq79DIHR0jCe7BgKshr9QyL9fGzRqxPG8lKiXyZ oraBV7Ts9GLiWrTc4r3m3/I/tSAJJjYsqeJTwdiyTPNXsrDWowl5w3gjJRYmig0q ZIdFvBw3BoxvqkB3P4EwaC5HY41kqMquPmc1onvFW05yRxXAMahKsFu0Xn4/RG7R 85rf60IvD+vnXxw+y0Vu7qF4WdYV3sA/oqFrHQA4Q3bVP+6BdLwoVRVIjikjRwyL BURYGjFPcdPHoZs2rvaTIM9ssZXjHZyuvj/qTXQ5ML4li0CmWBxV3R0BC72vO/w2 foqHlyXZ0jrmJCcQR8dv =iqrE -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org