On Freitag 28 November 2008 14:51:50 Lars Marowsky-Bree wrote:
On 2008-11-28T13:43:20, Adrian Schröter
wrote: The i586 scheduler has processed it meanwhile, I saw that it had more than 2000 medium events some hours ago (means that it has to recalculate 2000 project repositories due to changed packages). If you can provide a patch which makes this faster, I am happy to apply. (small sidenote, a compiler which compiles the entire distribution instantly would be also appricated :).
It is below 100 meanwhile, that means that should currently faster catch up.
There's also a weird issue when jobs have just finished (possibly failed).
The logs shown then - for a couple of minutes - will refer to the N-1 build. (They essentially revert from the "live log" to the stored log, I guess, and the copy is not immediate, while the status got updated already.)
This can be very confusing while the sync is catching up, and noone can tell me that copying a logfile takes a long time ;-)
This is a bug, please create a report. (The file does not need even to be copied on the server, it just get renamed).
Maybe the scheduler could be partitioned, so that it doesn't have to process everything in one go? Maybe the sync could be partitioned too, so that smaller increments are synced but the status is always correct?
The data export is partitioned in a seperate part (called publisher). The status can not be always complete, except we block accepting the build result until it has been recalculated. Just to clarify, I am happy about any constructive suggesstion how to speed up the process. But we have by intention a batch job system and not a life interactive system in this regard to be handle the load at all. Why does it take so long ? When ever a build job finishes, the build result can provide and require anything. So it needs recalculate all related packages. This are all packages inside of the project repo and dependending on the build repo (while there is a prio on the packages inside of the repo). It can not start any other build in this project before this has been done. Recalculating of one openSUSE:Factory repo does take about 90 seconds. (small project repos are much faster and usually in the 0-2 seconds area). In short, the scheduler has always a complete view of all packages in all projects including all dependencies at any time. It updates the view on every change (source or binarie). These dependencies take currently about 750MB in memory. -- Adrian Schroeter SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org