[Bug 1202952] New: OBS ppc64 and ppc64le builders continually run out of disk space
https://bugzilla.suse.com/show_bug.cgi?id=1202952 Bug ID: 1202952 Summary: OBS ppc64 and ppc64le builders continually run out of disk space Classification: Internal Novell Products Product: openSUSE Build Service Version: master Hardware: Other OS: Other Status: NEW Severity: Major Priority: P5 - None Component: General Assignee: adrian.schroeter@suse.com Reporter: william.brown@suse.com QA Contact: adrian.schroeter@suse.com Found By: --- Blocker: --- Multiple times a week I receive build failures due to ppc64 and ppc64le builders running out of disk space. This happens so frequently, that I have come to never really trust failures on ppc64 which can mask real errors. This means we are wasting time and cpu resources on spurious failures. This is a systemic and chronic issue with our ppc64 systems that needs resolving. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1202952 https://bugzilla.suse.com/show_bug.cgi?id=1202952#c1 --- Comment #1 from Adrian Schr�ter <adrian.schroeter@suse.com> --- I suppose your sources have no constraints for required disk space? How much disk space do they need according to the statistics file? -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1202952 https://bugzilla.suse.com/show_bug.cgi?id=1202952#c2 --- Comment #2 from William Brown <william.brown@suse.com> --- (In reply to Adrian Schr�ter from comment #1)
I suppose your sources have no constraints for required disk space?
I have never needed one on any other architecture.
How much disk space do they need according to the statistics file?
There is no such thing as a "statistics file". home:firstyear:branches:network:ldap/ # find . | grep -i stat home:firstyear:branches:network:ldap/ # This affects many packages I maintain, but lets use 389-ds as an example. Doing a du -h on a build root shows that it only needs 2.4G for a completed build, including all installed packages and compiled artefacts. However the only way to make the build reliable is to request a constraints of 25G. This appears to be because OBS selects the constraints based on maximum capacity of the disk, not the amount of disk reserved for a build. Additionally, because OBS shares builders, that disk amount of 25G has to also account for potentially other builds going on at the same time AND the actually installed host itself. I think that if I build only needs 2.4G, and it somehow is able to trigger "disk out of space errors" there is a significant issue in the builders configuration. Does it need more disk? Does a _constraint for size need to change to represent the amount of space the build actually will need? Regardless, the fact that these out of space errors are so common, indicates to me that other people are likely having this issue, but aren't reporting it. It also indicates that monitoring of the builders in our fleet for disk usage errors is probably lacking because this should have been noticed before I am reporting it. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply@suse.com