[opensuse-buildservice] build loops and/or really excessive rebuilds
I continue to have build loops (not actual loops) I can't explain. They do eventually run themselves out, but sometimes I see packages rebuild 30, 40, 100 or more times. In some cases, there are explanations for why they might rebuild a few times, but not 10 times much less over a hundred. For example, one package in x86_64 has rebuilt (around) 47 times in the last 3 days. The jobhistory shows only 'rebuild counter'. The package should have only rebuilt for i586, and in that case the only explanation given by jobhistory is "meta change". Several of the underlying dependencies did change over the last 3 days (like glibc, gcc, etc...) however they all changed /at the same time/. What I mean by that is that the source repository received a dozen or more updated rpms (binary rpms) for i586 *only* - I expected that this would cause each package to rebuild, but not once for each changed dependency - that seems just silly since after the first rebuild all subsequent rebuilds are using the exact same sources. I am using obs rpms with version 1.6.0-7.2. -- Jon -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Montag, 18. Januar 2010 14:51:03 schrieb Jon Nelson:
I continue to have build loops (not actual loops) I can't explain. They do eventually run themselves out, but sometimes I see packages rebuild 30, 40, 100 or more times. In some cases, there are explanations for why they might rebuild a few times, but not 10 times much less over a hundred.
For example, one package in x86_64 has rebuilt (around) 47 times in the last 3 days. The jobhistory shows only 'rebuild counter'.
We deliver the obs in default config that i586 and x86_64 packages should always have the same release number. This means, if it builds for one architecture, the other one gets triggered as well. If you done like this, define this in your BSConfig.pm to have seperate pools: our $relsync_pool = { ... "i586" => "i586", "x86_64" => "x86_64", ... }
The package should have only rebuilt for i586, and in that case the only explanation given by jobhistory is "meta change".
have a look in the package reason (osc reason or just by looking in the :reason file on the server). It lists all changed packages which have triggered this meta change oh, wait ....
Several of the underlying dependencies did change over the last 3 days (like glibc, gcc, etc...) however they all changed /at the same time/. What I mean by that is that the source repository received a dozen or more updated rpms (binary rpms) for i586 *only* - I expected that this would cause each package to rebuild, but not once for each changed dependency - that seems just silly since after the first rebuild all subsequent rebuilds are using the exact same sources.
I am using obs rpms with version 1.6.0-7.2.
... you need to have an obs 1.7 to have the logging of the "reason". bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Monday 2010-01-18 15:10, Adrian Schröter wrote:
Am Montag, 18. Januar 2010 14:51:03 schrieb Jon Nelson:
I continue to have build loops (not actual loops) I can't explain. They do eventually run themselves out, but sometimes I see packages rebuild 30, 40, 100 or more times. In some cases, there are explanations for why they might rebuild a few times, but not 10 times much less over a hundred.
For example, one package in x86_64 has rebuilt (around) 47 times in the last 3 days. The jobhistory shows only 'rebuild counter'.
We deliver the obs in default config that i586 and x86_64 packages should always have the same release number. This means, if it builds for one architecture, the other one gets triggered as well.
If you done like this, define this in your BSConfig.pm to have seperate pools:
our $relsync_pool = { ... "i586" => "i586", "x86_64" => "x86_64", ... }
Speaking of build loops, I still seem to have some, and it is rather delicate. - gcc44.sparc64 depends(BuildRequires) on glibc-devel-32bit. - gcc44.sparcv9 depends(BuildRequires) on glibc-devel-64bit. ExportFiltering glibc-devel however will cause a meta change cycle. Building glibc-devel.sparcv9 leads to: - baselibs generation glibc-devel-32bit.sparc64 - leads to meta change in gcc44.sparc64 (rebuild) - this in turn leads to meta change in glibc.sparc64 - glibc-devel-64bit.sparcv9 baselibs generation - leads to meta change in gcc44.sparcv9, rebuild - in turn leads to meta change in glibc.sparcv9, completing the circle At least this is what it looks like. There are entries in package/reason that are not obvious at all: # /sparc64/elfutils/reason: <packagechange change="added" key="gcc44/libdb-4_5-32bit" /> Why does gcc44 depend(?) on libdb-4_5-32bit? I had made sure that there is no libdb-4_5-32bit in /srv/obs/repos/$project/sparc64 at all. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Mon, Jan 18, 2010 at 8:10 AM, Adrian Schröter <adrian@suse.de> wrote:
Am Montag, 18. Januar 2010 14:51:03 schrieb Jon Nelson:
I continue to have build loops (not actual loops) I can't explain. They do eventually run themselves out, but sometimes I see packages rebuild 30, 40, 100 or more times. In some cases, there are explanations for why they might rebuild a few times, but not 10 times much less over a hundred.
For example, one package in x86_64 has rebuilt (around) 47 times in the last 3 days. The jobhistory shows only 'rebuild counter'.
We deliver the obs in default config that i586 and x86_64 packages should always have the same release number. This means, if it builds for one architecture, the other one gets triggered as well.
If you done like this, define this in your BSConfig.pm to have seperate pools:
our $relsync_pool = { ... "i586" => "i586", "x86_64" => "x86_64", ... }
I made that change but today I was getting complaints of packages rebuilding over and over and over. The build history showed the same srcmd5, same revision, and builds every 2-5 minutes, until I shut down and restarted the scheduler. I'd love to be able to correlate the timestamps but why don't the logs have timestamps? How to I tell the ruby/perl/whatever logging system to log timestamps (without resorting to logging to syslog)? Anyway, it looks like glibc, glibc-common, and glibc-headers: !glibc !glibc-common !glibc-headers meta change, start build were the packages triggering the rebuild. However, after rebuilding, it would rebuild again, and for the same reasons! Sigh. What might be going on here? -- Jon -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Mon, Jan 18, 2010 at 9:10 AM, Adrian Schröter <adrian@suse.de> wrote:
Am Montag, 18. Januar 2010 14:51:03 schrieb Jon Nelson:
I continue to have build loops (not actual loops) I can't explain. They do eventually run themselves out, but sometimes I see packages rebuild 30, 40, 100 or more times. In some cases, there are explanations for why they might rebuild a few times, but not 10 times much less over a hundred.
For example, one package in x86_64 has rebuilt (around) 47 times in the last 3 days. The jobhistory shows only 'rebuild counter'.
...
The package should have only rebuilt for i586, and in that case the only explanation given by jobhistory is "meta change".
have a look in the package reason (osc reason or just by looking in the :reason file on the server). It lists all changed packages which have triggered this meta change
oh, wait ....
Several of the underlying dependencies did change over the last 3 days (like glibc, gcc, etc...) however they all changed /at the same time/. What I mean by that is that the source repository received a dozen or more updated rpms (binary rpms) for i586 *only* - I expected that this would cause each package to rebuild, but not once for each changed dependency - that seems just silly since after the first rebuild all subsequent rebuilds are using the exact same sources.
I am using obs rpms with version 1.6.0-7.2.
... you need to have an obs 1.7 to have the logging of the "reason".
Today it was brought to my attention that the OBS server I manage was building lots of packages for an unknown reason. I can't find a reason in the logs (which aren't time-stamped per an earlier email, making this sort of things much more difficult). I am now using 1.7.2, and I still seem to only have "meta change" listed in jobhistory. I am using osc 0.125.5 Where should I be looking? -- Jon -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Montag, 15. März 2010 16:28:52 schrieb Jon Nelson:
On Mon, Jan 18, 2010 at 9:10 AM, Adrian Schröter <adrian@suse.de> wrote:
Am Montag, 18. Januar 2010 14:51:03 schrieb Jon Nelson:
I continue to have build loops (not actual loops) I can't explain. They do eventually run themselves out, but sometimes I see packages rebuild 30, 40, 100 or more times. In some cases, there are explanations for why they might rebuild a few times, but not 10 times much less over a hundred.
For example, one package in x86_64 has rebuilt (around) 47 times in the last 3 days. The jobhistory shows only 'rebuild counter'.
...
The package should have only rebuilt for i586, and in that case the only explanation given by jobhistory is "meta change".
have a look in the package reason (osc reason or just by looking in the :reason file on the server). It lists all changed packages which have triggered this meta change
oh, wait ....
Several of the underlying dependencies did change over the last 3 days (like glibc, gcc, etc...) however they all changed /at the same time/. What I mean by that is that the source repository received a dozen or more updated rpms (binary rpms) for i586 *only* - I expected that this would cause each package to rebuild, but not once for each changed dependency - that seems just silly since after the first rebuild all subsequent rebuilds are using the exact same sources.
I am using obs rpms with version 1.6.0-7.2.
... you need to have an obs 1.7 to have the logging of the "reason".
Today it was brought to my attention that the OBS server I manage was building lots of packages for an unknown reason. I can't find a reason in the logs (which aren't time-stamped per an earlier email, making this sort of things much more difficult).
I am now using 1.7.2, and I still seem to only have "meta change" listed in jobhistory.
I am using osc 0.125.5
Where should I be looking?
Details are logged in the package. Run for example # osc triggerreason openSUSE:Factory bc standard x86_64 meta change changed keys: added bison/rpm/lua-devel/readline-devel removed rpm/lua-devel/readline-devel tells you the binary package dependency path which have changed compared to the former build. -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Mon, Mar 15, 2010 at 10:38 AM, Adrian Schröter <adrian@suse.de> wrote:
Am Montag, 15. März 2010 16:28:52 schrieb Jon Nelson: ...
I am now using 1.7.2, and I still seem to only have "meta change" listed in jobhistory.
I am using osc 0.125.5
Where should I be looking?
Details are logged in the package. Run for example
# osc triggerreason openSUSE:Factory bc standard x86_64 meta change changed keys: added bison/rpm/lua-devel/readline-devel removed rpm/lua-devel/readline-devel
tells you the binary package dependency path which have changed compared to the former build.
Aha. Found it! -- Jon -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (3)
-
Adrian Schröter -
Jan Engelhardt -
Jon Nelson