Mailinglist Archive: opensuse-buildservice (158 mails)

< Previous Next >
Re: [opensuse-buildservice] project hasn't starting building for 20 hours and counting...
  • From: "Hans-Peter Jansen" <hpj@xxxxxxxxx>
  • Date: Fri, 3 Dec 2010 12:19:30 +0100
  • Message-id: <201012031219.31141.hpj@xxxxxxxxx>
Dear Stephan,

first of all thank you very much for taking your precious time to
respond to this issue.

On Friday 03 December 2010, 11:06:27 Stephan Kulow wrote:
Am Freitag 03 Dezember 2010 schrieb Hans-Peter Jansen:
OTOH, monster projects like windows:mingw:win{32,64} seem to be
preferred for some reason. At least, they built several hundred
packages during the last two days, while _none_ of mine.

Douglas, see, it still can get worser.

We'll have to be patient and should be happy, if it finally builds
something for us, too..

This is a problem we discussed just yesterday - we see it too and
want to solve it, but we're unsure how.

One thing to do surely is to clean up stuff that noone touched for
years. Adrian sent around a list of projects to get rid of
repositories, but so far it wasn't yet done - but this will only help
a bit.

There are two some key facts about the dispatching:
- repositories that see a lot of downloads are preferred
- projects that create a lot of load get punished
- packages that were touched in the last 24 hours are preferred

The number of downloads that are the base of the priorities, you can
find here: http://www.suse.de/~coolo/repo.list

Hmm, that heuristic sounds reasonable at first.

I think, it could be improved by adding a fairness/charity ratio:
a fixed amount of build resources, let's say 5%, that that weights by
project finished ratio (packages missing / packages done) preferring
high values, combined with one that prefers the longest waiting one.
The fixed amount of resources is only taken, if there are any aspirants
in this group. By turning the knob, the overall fairness can be
controlled on the fly.

You have to scroll a lot to find home:sipfoundry - which leads to the
fact that it doesn't get _any_ priority. It's just one of many home
projects and as such it gets build power only when everything else is
finished (not exactly that black & white, but towards that).

That windows:mingw:win32 gets _so_ much build power shouldn't happen
either (its openSUSE_Factory repo #1279 in the repo list, so it's ok
if it's build more often than random home projects not downloaded,
but it should get a "fair" share, which I guess is less than what it
gets right now).

One additional thing we discussed was prefering projects that have
recent changes assuming that it's more likely that people look at it.

This is a bit dangerous, since it could lead to forced changes that
might trigger _more_ rebuilds and to an forcibly increased overall
load. It might probably be combineable with the other ideas towards
fairness.

home:sipfoundry:4.4 is a good example:
The packaged changed have all built, but one scheduled package blocks
the others and the final pushing - frustrating the one wanting to
test the change. In this case, the scheduled one should get a little
higher priority because it's in a project together with recently
changed packages. At least higher than all these kernels and libqt4
packages that people linked into their home projects a year ago ;(

BTW: the dispatching didn't change and the priorities have been this
way for a long time, but we see a lot more packages building now as
we're more relaxed in what packages we checkin into openSUSE:Factory,
so a lot of repositories building against openSUSE:Factory create a
lot of build jobs - and if these repositories are even downloaded,
they get preferrence. This might be something we have to decrease.

At least, the fairness to the poor projects should be raised a bit.

Pete
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-buildservice+help@xxxxxxxxxxxx

< Previous Next >
Follow Ups