Comment # 20 on bug 1045340 from Michal Hocko

(In reply to Vlastimil Babka from comment #19)
> (In reply to Michal Hocko from comment #18)
> > (In reply to Vlastimil Babka from comment #10)
> > > (In reply to Vlastimil Babka from comment #9)
> > > > Catchpoint 1 (call to syscall mmap2), 0xf7fd9f89 in __kernel_vsyscall ()
> > > 
> > > (gdb) bt
> > > #0  0xf7fd9f89 in __kernel_vsyscall ()
> > > #1  0xf7454508 in mmap () from /lib/libc.so.6
> > > #2  0xf7a51cdb in os::commit_memory(char*, unsigned int, bool) () from
> > > /opt/novell/groupwise/client/java/lib/i386/client/libjvm.so
> > 
> > OK, this is what I found in jdk source
> > // NOTE: Linux kernel does not really reserve the pages for us.
> > //       All it does is to check if there are enough free pages
> > //       left at the time of mmap(). This could be a potential
> > //       problem.
> 
> Um, did they hear about MAP_POPULATE?

I am pretty sure they do not want this if they plan to use really large stacks.

> 
> > bool os::commit_memory(char* addr, size_t size, bool exec) {
> >   int prot = exec ? PROT_READ|PROT_WRITE|PROT_EXEC : PROT_READ|PROT_WRITE;
> >   uintptr_t res = (uintptr_t) ::mmap(addr, size, prot,
> >                                    MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1,
> > 0);
> >   return res != (uintptr_t) MAP_FAILED;
> > 
> > }
> > 
> > which explains it I am afraid. We simply smash a new mapping in the middle
> > of the stack. And so the stack expansion cannot possibly work. :/
> 
> Yeah, I guess they think they are placing the guards below the stack, that's
> why there's mmap first and then mprotect (which could be done by single mmap
> with the right protection flags, but nevermind).

well, they could simply mprotect directly and mmap only if it failed.