On Mai 27 2018, Giovanni Santini wrote:

I have noticed that builds for my repository are stuck to 2 days ago. Using `osc rbuild PACKAGE` doesn't work too.

My repository is: home:ItachiSan:archlinux

The builds finished only a few hours ago. $ osc jobhist -l3 home:ItachiSan:archlinux standard x86_64 time package reason code build time worker 2018-05-27 14:31:06 webkit2gtk-mse new build failed 30m 24s lamb66:3 2018-05-27 14:36:08 ffmpeg meta change succeeded 35m 25s lamb27:7 2018-05-27 15:23:37 gecode meta change succeeded 50m 25s cloud124:3 Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

On Sonntag, 27. Mai 2018, 22:02:02 CEST wrote Giovanni Santini:

Il 27/05/2018 20:03, Andreas Schwab ha scritto:

...

On Mai 27 2018, Giovanni Santini wrote:

...

I have noticed that builds for my repository are stuck to 2 days ago. Using `osc rbuild PACKAGE` doesn't work too.

My repository is: home:ItachiSan:archlinux

The builds finished only a few hours ago.

$ osc jobhist -l3 home:ItachiSan:archlinux standard x86_64 time package reason code build time worker 2018-05-27 14:31:06 webkit2gtk-mse new build failed 30m 24s lamb66:3 2018-05-27 14:36:08 ffmpeg meta change succeeded 35m 25s lamb27:7 2018-05-27 15:23:37 gecode meta change succeeded 50m 25s cloud124:3

Andreas.

You are correct, although the build for *telegram-desktop-dev* continues to be blocked...

I notice an extreme slow-down on our backend responsible for home projects atm. Will need to debug where this is comming from suddenly, but most likely not today :/ -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

On Montag, 28. Mai 2018, 12:30:51 CEST wrote Hans-Peter Jansen:

Hi Giovanni,

On Montag, 28. Mai 2018 08:47:57 Adrian Schröter wrote:

...

On Sonntag, 27. Mai 2018, 22:02:02 CEST wrote Giovanni Santini:

...

Il 27/05/2018 20:03, Andreas Schwab ha scritto:

...

On Mai 27 2018, Giovanni Santini wrote:

...

I have noticed that builds for my repository are stuck to 2 days ago. Using `osc rbuild PACKAGE` doesn't work too.

My repository is: home:ItachiSan:archlinux

The builds finished only a few hours ago.

$ osc jobhist -l3 home:ItachiSan:archlinux standard x86_64 time package reason code build time worker 2018-05-27 14:31:06 webkit2gtk-mse new build failed 30m 24s lamb66:3 2018-05-27 14:36:08 ffmpeg meta change succeeded 35m 25s lamb27:7 2018-05-27 15:23:37 gecode meta change succeeded 50m 25s cloud124:3

Andreas.

You are correct, although the build for *telegram-desktop-dev* continues to be blocked...

I notice an extreme slow-down on our backend responsible for home projects atm.

Will need to debug where this is comming from suddenly, but most likely not today :/

Have a look here: https://build.opensuse.org/monitor (bottom of page)

As long as repo recalculation count keep above 500, OBS "feels" really slow. Symptom: package builds keep in finished state for a long time. Since RRC is way above 2000 since at least two days, OBS "feels" almost stalled.

This is a rare event: during the last year RRC was bigger only once (in January), where it exceeded 10000.

Let's hope, that things settle again in the next few days.

yes. JFYI, one issue got solved and the scheduler is a bit faster again. But has still a quite large backlog. btw, this affects "only" x86_64 in home:* projects. Please use local builds (osc build ..) if you feel slowed down, this is the fastest way to test or debug something as usual and just submit your working version. That one will build on the server eventually :) -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

On Dienstag, 29. Mai 2018, 08:08:24 CEST wrote Johannes Weberhofer:

Am 28.05.2018 um 12:40 schrieb Adrian Schröter:

...

yes. JFYI, one issue got solved and the scheduler is a bit faster again. But has still a quite large backlog.

btw, this affects "only" x86_64 in home:* projects.

Please use local builds (osc build ..) if you feel slowed down, this is the fastest way to test or debug something as usual and just submit your working version. That one will build on the server eventually:)

I think there is still something wrong. The builds are fine now but recalculation/publication does still not work.

It does work, but it takes forever ..... However, we solved a number of IO bottlenecks (network and disk) this morning. It seems it has improved the situation, but I suspect it will take until tomorrow until we see the full effect. The caches on various workers and servers need to get filled again. But we have a trend change in any case, the steady increasing number of to repositories which needs to be recalculate is decreasing now again (currently already <4000 repos on x86_64 home:* projects). -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org