Memory problems at build time
Hello all, I have now noticed with a wide variety of packages that they build in OBS itself, but not locally using osc. It comes then memory error. e.g.: RangeError: WebAssembly.Instance(): Out of memory: wasm memory As said on the OBS server online the packages are built. But not locally. Does anyone know this? Do others have this problem? Is there a solution? What do I have to do then? I would be very grateful for any help. Regards Eric
On 23.12.22 09:54, Eric Schirra wrote:
Is there a solution? What do I have to do then?
Add more memory to your build machine? -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman
Am Freitag, 23. Dezember 2022, 15:19:45 CET schrieb Stefan Seyfried:
On 23.12.22 09:54, Eric Schirra wrote:
Is there a solution? What do I have to do then?
Add more memory to your build machine?
Think this is not the reason. I have 16GB RAM. And ca. 8GB are free. Also, I can build other, much larger packages. Another possibility/tip? Regards Eric
On 23.12.22 15:56, Eric Schirra wrote:
Am Freitag, 23. Dezember 2022, 15:19:45 CET schrieb Stefan Seyfried:
On 23.12.22 09:54, Eric Schirra wrote:
Is there a solution? What do I have to do then?
Add more memory to your build machine?
Think this is not the reason. I have 16GB RAM. And ca. 8GB are free. Also, I can build other, much larger packages. Another possibility/tip?
Maybe different settings in local vs obsworker build? Are you building locally in chroot or a VM? You could try to debug with a "ulimit -a" statement (and others, "free", whatever) in your spec file before the actual make call and check what the differences are? -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman
Am Samstag, 24. Dezember 2022, 09:33:01 CET schrieb Stefan Seyfried:
On 23.12.22 15:56, Eric Schirra wrote:
Am Freitag, 23. Dezember 2022, 15:19:45 CET schrieb Stefan Seyfried:
On 23.12.22 09:54, Eric Schirra wrote:
Is there a solution? What do I have to do then?
Add more memory to your build machine?
Think this is not the reason. I have 16GB RAM. And ca. 8GB are free. Also, I can build other, much larger packages. Another possibility/tip?
Maybe different settings in local vs obsworker build? Are you building locally in chroot or a VM?
You could try to debug with a "ulimit -a" statement (and others, "free", whatever) in your spec file before the actual make call and check what the differences are?
Hm. I don't understand this. The only different point are virtual memory. Can this the reason? Directly in console on my local client is "virtual memory" unlimited. Also on OBS with ulimit in spec "virtual memory" is unlimited. But in spec on local client with occ "virtual memory" is 24839028 Is this the reason? How can i change it? Regards Eric
On 12/23/22 09:54, Eric Schirra wrote:
Hello all,
I have now noticed with a wide variety of packages that they build in OBS itself, but not locally using osc. It comes then memory error. e.g.: RangeError: WebAssembly.Instance(): Out of memory: wasm memory
The problem is kind of a hack in the build script. /usr/lib/build/build see function setmemorylimit() and remove the ulimit -d $limit echo "Memory limit set to ${limit}KB" This is kind of this legacy mechanism that is used to prevent the local build from blowing up and stalling your machine (which doesn't always work anyway). But, if you run NodeJS with webassembly, it will allocate a whole 4GB address space for every instance, even if it uses just a few bytes. So when you run a few of these concurrently, like in the nodejs unit tests, it will go past the `ulimit` set with the above error. - Adam
On Mittwoch, 25. Januar 2023, 14:35:05 CET Adam Majer wrote:
On 12/23/22 09:54, Eric Schirra wrote:
Hello all,
I have now noticed with a wide variety of packages that they build in OBS itself, but not locally using osc. It comes then memory error. e.g.: RangeError: WebAssembly.Instance(): Out of memory: wasm memory
The problem is kind of a hack in the build script.
/usr/lib/build/build
see function setmemorylimit()
and remove the
ulimit -d $limit echo "Memory limit set to ${limit}KB"
This is kind of this legacy mechanism that is used to prevent the local build from blowing up and stalling your machine (which doesn't always work anyway). But, if you run NodeJS with webassembly, it will allocate a whole 4GB address space for every instance, even if it uses just a few bytes. So when you run a few of these concurrently, like in the nodejs unit tests, it will go past the `ulimit` set with the above error.
yes ... please note that this limit is only active in chroot builds, where this is the only protection against OOM situations. You won't run into this when using a KVM build env. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Wednesday 2023-01-25 14:35, Adam Majer wrote:
On 12/23/22 09:54, Eric Schirra wrote:
Hello all,
I have now noticed with a wide variety of packages that they build in OBS itself, but not locally using osc. It comes then memory error. e.g.: RangeError: WebAssembly.Instance(): Out of memory: wasm memory
ulimit -d $limit echo "Memory limit set to ${limit}KB"
This is kind of this legacy mechanism that is used to prevent the local build from blowing up and stalling your machine (which doesn't always work anyway). But, if you run NodeJS with webassembly, it will allocate a whole 4GB address space for every instance, even if it uses just a few bytes.
But if it only uses a few bytes, then the -d limit should not normally be reached assuming standard overcommit (or should it?); rather, the -v limit would be relevant.
On 1/25/23 15:08, Jan Engelhardt wrote:
But if it only uses a few bytes, then the -d limit should not normally be reached assuming standard overcommit (or should it?); rather, the -v limit would be relevant.
The -v is for the entire address space, code, stack and data segments. The -d only for the data segment bits. But in both cases, this is about address space and not actually committed data. You can try it yourself with, #include <stdio.h> #include <stdlib.h> int main() { printf("%p\n", malloc(1e9)); return 0; } and set `ulimit -d 100000` and it will print a nice null for you :-) - Adam
On Jan 25 2023, Adam Majer wrote:
This is kind of this legacy mechanism that is used to prevent the local build from blowing up and stalling your machine (which doesn't always work anyway). But, if you run NodeJS with webassembly, it will allocate a whole 4GB address space for every instance, even if it uses just a few bytes. So when you run a few of these concurrently, like in the nodejs unit tests, it will go past the `ulimit` set with the above error.
That limit is per-process, though, so the number of instances is not relevant for this particular limit. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
Am Mittwoch, 25. Januar 2023, 14:35:05 CET schrieb Adam Majer:
On 12/23/22 09:54, Eric Schirra wrote:
Hello all,
I have now noticed with a wide variety of packages that they build in OBS itself, but not locally using osc. It comes then memory error. e.g.: RangeError: WebAssembly.Instance(): Out of memory: wasm memory
The problem is kind of a hack in the build script.
/usr/lib/build/build
see function setmemorylimit()
and remove the
ulimit -d $limit echo "Memory limit set to ${limit}KB"
This is kind of this legacy mechanism that is used to prevent the local build from blowing up and stalling your machine (which doesn't always work anyway). But, if you run NodeJS with webassembly, it will allocate a whole 4GB address space for every instance, even if it uses just a few bytes. So when you run a few of these concurrently, like in the nodejs unit tests, it will go past the `ulimit` set with the above error.
Hello Adam, this was the solution. The error message is gone and the package is also built locally. But how do I do this permanently? After an update everything is gone again. Regards Eric
participants (6)
-
Adam Majer -
Adrian Schröter -
Andreas Schwab -
Eric Schirra -
Jan Engelhardt -
Stefan Seyfried