[Bug 496363] New: OBS packages aren't pushed to rsync.opensuse.org
http://bugzilla.novell.com/show_bug.cgi?id=496363 Summary: OBS packages aren't pushed to rsync.opensuse.org Classification: openSUSE Product: openSUSE.org Version: unspecified Platform: All OS/Version: All Status: NEW Severity: Major Priority: P5 - None Component: BuildService AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: services@mindtouch.com QAContact: adrian@novell.com Found By: --- User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko/2009032711 Ubuntu/8.10 (intrepid) Firefox/3.0.8 It's been roughly a week that packages built from my home project aren't published (ie pushed to the rsync server). The package publishing options are enabled and triggering a rebuild doesn't help. Package link : https://build.opensuse.org/package/show?package=DekiWiki_9.02.1&project=home%3Amindtouch OBS repository : http://download.opensuse.org/repositories/home://mindtouch/CentOS_5/noarch/ RSYNC repository : http://rsync.opensuse.org/repositories/home:/mindtouch/CentOS_5/noarch/ Please check the rsync server setup. Reproducible: Always Steps to Reproduce: 1. trigger a build in OBS 2. check that the packages exist in the OBS repository 3. wait for the push to rsync.opensuse.org Actual Results: packages not pushed Expected Results: packages published in the rsync repository -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=496363 Adrian Schröter <adrian@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|bnc-team-screening@forge.pr |poeml@novell.com |ovo.novell.com | -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=496363 User poeml@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=496363#c1 --- Comment #1 from Peter Poeml <poeml@novell.com> 2009-04-21 04:10:53 MDT --- The repository pushing to mirrors might be lossy. The log shows that pushing of "home:mindtouch" basically ceased after the 26th of March. Tue, 24 Mar 2009 09:53:24 +0100 run for home:mindtouch to sourceshare.org Tue, 24 Mar 2009 09:53:41 +0100 run for home:mindtouch to susestudio.com Tue, 24 Mar 2009 09:53:58 +0100 run for home:mindtouch to unixheads.net Tue, 24 Mar 2009 09:54:14 +0100 run for home:mindtouch to widehat.opensuse.org Tue, 24 Mar 2009 09:54:37 +0100 run for home:mindtouch to anorien.csc.warwick.ac.uk Wed, 25 Mar 2009 23:02:41 +0100 run for home:mindtouch to dispenser.yandex.net Wed, 25 Mar 2009 23:05:30 +0100 run for home:mindtouch to widehat.opensuse.org Wed, 25 Mar 2009 23:26:49 +0100 run for home:mindtouch to anorien.csc.warwick.ac.uk Wed, 25 Mar 2009 23:27:46 +0100 run for home:mindtouch to ftp5.gwdg.de Wed, 25 Mar 2009 23:28:25 +0100 run for home:mindtouch to metalink_hashes_update_job Wed, 25 Mar 2009 23:28:44 +0100 run for home:mindtouch to mirror.leaseweb.com Wed, 25 Mar 2009 23:38:49 +0100 run for home:mindtouch to df.lth.se Wed, 25 Mar 2009 23:49:29 +0100 run for home:mindtouch to unixheads.net Wed, 25 Mar 2009 23:52:42 +0100 run for home:mindtouch to sourceshare.org Wed, 25 Mar 2009 23:57:24 +0100 run for home:mindtouch to mirror.z-gamez.ru Thu, 26 Mar 2009 00:02:02 +0100 run for home:mindtouch to susestudio.com Thu, 26 Mar 2009 20:14:38 +0100 run for home:mindtouch to metalink_hashes_update_job Thu, 26 Mar 2009 20:30:33 +0100 run for home:mindtouch to mirror.leaseweb.com Thu, 26 Mar 2009 20:52:15 +0100 run for home:mindtouch to dispenser.yandex.net Thu, 26 Mar 2009 21:17:06 +0100 run for home:mindtouch to mirror.z-gamez.ru Thu, 26 Mar 2009 21:19:16 +0100 run for home:mindtouch to df.lth.se Thu, 26 Mar 2009 21:39:06 +0100 run for home:mindtouch to widehat.opensuse.org Thu, 26 Mar 2009 21:40:37 +0100 run for home:mindtouch to ftp5.gwdg.de Thu, 26 Mar 2009 21:42:13 +0100 run for home:mindtouch to sourceshare.org Thu, 26 Mar 2009 21:42:55 +0100 run for home:mindtouch to unixheads.net Thu, 26 Mar 2009 21:43:45 +0100 run for home:mindtouch to anorien.csc.warwick.ac.uk Thu, 26 Mar 2009 23:31:14 +0100 run for home:mindtouch to susestudio.com Fri, 03 Apr 2009 14:20:49 +0200 ftp.twaren.net: start syncing home:mindtouch Fri, 03 Apr 2009 15:41:48 +0200 ftp.twaren.net: start syncing home:mindtouch Tue, 14 Apr 2009 20:17:44 +0200 metalink_hashes_update_job: start syncing home:mindtouch Tue, 14 Apr 2009 21:27:41 +0200 mirror.leaseweb.com: start syncing home:mindtouch Tue, 21 Apr 2009 10:29:05 +0200 metalink_hashes_update_job: start syncing home:mindtouch The fact that it is found on ftp5.gwdg.de is not because it is pushed there; that mirror pull-syncs also, which is why it has the newer packages. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=496363 User poeml@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=496363#c2 --- Comment #2 from Peter Poeml <poeml@novell.com> 2009-04-21 08:00:38 MDT --- I'm not sure yet about the reason why the push didn't happen. I believe that it is due to the combination of two facts: * the push lags behind in general * the push handles repositories with higher priority if they are first in the alphabet I changed the latter scheme today, to first handle those projects first which have been built the longest time ago. The number of projects pending for being pushed is right now: 310 anorien.csc.warwick.ac.uk 132 df.lth.se 118 dispenser.yandex.net 166 ftp5.gwdg.de 1816 ftp.iasi.roedu.net 1938 ftp.nux.ipb.pt 1028 ftp.twaren.net 83 metalink_hashes_update_job 21 mirror.leaseweb.com 41 mirror.z-gamez.ru 1188 sourceshare.org 643 susestudio.com 241 unixheads.net 213 widehat.opensuse.org I observed widehat.opensuse.org and the number has been decreasing since my change from about 450 to 213 now. Thus, the push is catching up at the moment. I'll have to add some more reporting and logging to be better able to find out if something's wrong, and what. home:mindtouch will take a further while, because it was rebuilt today, so it is quite at the end of the queue right now. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=496363 User poeml@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=496363#c3 --- Comment #3 from Peter Poeml <poeml@novell.com> 2009-04-22 07:23:19 MDT --- See http://lists.opensuse.org/archive/opensuse-buildservice/2009-04/msg00184.htm... for a further status update. In short, * widehat.o.o (aka rsync.o.o) has caught up; * some mirrors are too slowly synced to really catch up, * most mirrors have caught up and are fully in sync now. * the reason for the bug lag that was seen in the last weeks is unclear. I made the lag easier to observe now, so let's see how it works now. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=496363 User poeml@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=496363#c4 --- Comment #4 from Peter Poeml <poeml@novell.com> 2009-07-14 04:54:56 MDT --- After a lot of investigation and work, the push is now fast enough to keep up with build service builds. I excluded more content from pushing also. The queue looks good now, usually: /srv/bs/bin/needed_push | cut -d: -f1 | uniq -c 1 df.lth.se 1 dispenser.yandex.net 1 ftp5.gwdg.de 1 ftp.nux.ipb.pt 202 ftp.twaren.net 1 metalink_hashes_update_job 1 mirror.leaseweb.com 1 mirror.z-gamez.ru 1 susestudio.com 1 unixheads.net 1 widehat.opensuse.org (The Japanese mirror cannot be synced faster because it is so far away.) There is a remaining bug though, which causes the sync script to slow down considerably after about two days. It needs to be restarted then, to catch up. I suspect this is a bash bug, because there's nothing than a "while true do ... done" loop which shouldn't become slower after a while. No memleak or otherwise recognizable bottleneck. I have started working on a new sync mechanism that should replace the current one. will take a further while though. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=496363 User poeml@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=496363#c5 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |CLOSED CC| |poeml@cmdline.net Resolution| |FIXED --- Comment #5 from Peter Poeml <poeml@novell.com> 2009-10-07 15:04:21 MDT --- Checking this again, the queue size remains in good shape nowadays. No further "losses" (missing packages) have been reported. There's still the problem that the bash "while true; do ... done" loop-type script slows down to a crawl after a few days, but we can work around that easily with daily restarts (that don't harm otherwise). However, the new sync mechanism I mentioned above will remain a project for long winter nights I guess. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com