[opensuse-buildservice] Re: [opensuse] rsync.opensuse.org and download.opensuse.org not in sync?!?!?
Hi there, I didn't receive any feedback about this on opensuse@opensuse.org, maybe the buildservice list is more appropriate. I just looked again at the output from the rsync command below and it's still the same :-( TIA, cheers. l8er manfred On Mon, 30 Mar 2009, 15:58:02 +0200, Manfred Hollstein wrote:
Hi there,
I thought that download.opensuse.org/repositories gets filled from rsync.opensuse.org, but that doesn't seem to be the case (at least ATM):
if you look at download.opensuse.org/repositories which version of MozillaFirefox is available for openSUSE-10.3, you'll find
http://download.opensuse.org/repositories/mozilla/openSUSE_10.3/i586/Mozilla...
however, looking at the output from running
rsync -n -av rsync.opensuse.org::buildservice-repos/mozilla/openSUSE_10.3/i586 . | grep MozillaFirefox
you'll see
i586/MozillaFirefox-3.0.7-1.4.i586.rpm
There must be something wrong with rsync.opensuse.org, no? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
I looked into it, but so far I can only see that it is a problem with our repository pushing script. It seems to ignore the mozilla project and doesn't push it (to any mirror). I don't know if it affects other projects as well; could be. Peter On Wed, Apr 01, 2009 at 07:27:52PM +0200, Manfred Hollstein wrote:
Hi there,
I didn't receive any feedback about this on opensuse@opensuse.org, maybe the buildservice list is more appropriate. I just looked again at the output from the rsync command below and it's still the same :-(
TIA, cheers.
l8er manfred
On Mon, 30 Mar 2009, 15:58:02 +0200, Manfred Hollstein wrote:
Hi there,
I thought that download.opensuse.org/repositories gets filled from rsync.opensuse.org, but that doesn't seem to be the case (at least ATM):
if you look at download.opensuse.org/repositories which version of MozillaFirefox is available for openSUSE-10.3, you'll find
http://download.opensuse.org/repositories/mozilla/openSUSE_10.3/i586/Mozilla...
however, looking at the output from running
rsync -n -av rsync.opensuse.org::buildservice-repos/mozilla/openSUSE_10.3/i586 . | grep MozillaFirefox
you'll see
i586/MozillaFirefox-3.0.7-1.4.i586.rpm
There must be something wrong with rsync.opensuse.org, no? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
I do not know if it is the same issue, or if it is a problem at all. But the x86_64 packages in http://download.opensuse.org/repositories/KDE:/KDE4:/Community/openSUSE_11.1... are not updated to the builds for KDE 4.2.2. This seems to fit for all x86_64 packages in KDE:KDE4:Community. The i586 packages are updated. Best greetings Buschmann On Thursday 02 April 2009 12:47:23 Peter Poeml wrote:
I looked into it, but so far I can only see that it is a problem with our repository pushing script. It seems to ignore the mozilla project and doesn't push it (to any mirror). I don't know if it affects other projects as well; could be.
Peter
On Wed, Apr 01, 2009 at 07:27:52PM +0200, Manfred Hollstein wrote:
Hi there,
I didn't receive any feedback about this on opensuse@opensuse.org, maybe the buildservice list is more appropriate. I just looked again at the output from the rsync command below and it's still the same :-(
TIA, cheers.
l8er manfred
On Mon, 30 Mar 2009, 15:58:02 +0200, Manfred Hollstein wrote:
Hi there,
I thought that download.opensuse.org/repositories gets filled from rsync.opensuse.org, but that doesn't seem to be the case (at least ATM):
if you look at download.opensuse.org/repositories which version of MozillaFirefox is available for openSUSE-10.3, you'll find
<http://download.opensuse.org/repositories/mozilla/openSUSE_10.3/i586/M ozillaFirefox-3.0.8-1.1.i586.rpm>
however, looking at the output from running
rsync -n -av rsync.opensuse.org::buildservice-repos/mozilla/openSUSE_10.3/i586 . | grep MozillaFirefox
you'll see
i586/MozillaFirefox-3.0.7-1.4.i586.rpm
There must be something wrong with rsync.opensuse.org, no?
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
---------------------------------------------------------------------------------- openSUSE Member - de.opensuse.org Sys-Op http://en.opensuse.org/User:Buschmann23 http://en.opensuse.org/How_to_Participate http://en.opensuse.org/Geeko_wants_you!
On Thu, Apr 02, 2009 at 03:26:35PM +0200, Matthias Fehring wrote:
I do not know if it is the same issue, or if it is a problem at all.
But the x86_64 packages in http://download.opensuse.org/repositories/KDE:/KDE4:/Community/openSUSE_11.1... are not updated to the builds for KDE 4.2.2. This seems to fit for all x86_64 packages in KDE:KDE4:Community. The i586 packages are updated.
Best greetings Buschmann
If there's a problem, it's a different one. (I only see a million packages that don't mean much to me, so I don't know if something is wrong. The project seems to be built successfully, thus it should be published on download.o.o if publishing is not disabled.) If there's something missing, it would be a problem between buildservice and download.opensuse.org - different problem. Thank! Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development
On Thursday 02 April 2009 21:18:37 Peter Poeml wrote:
On Thu, Apr 02, 2009 at 03:26:35PM +0200, Matthias Fehring wrote:
I do not know if it is the same issue, or if it is a problem at all.
But the x86_64 packages in http://download.opensuse.org/repositories/KDE:/KDE4:/Community/openSUSE_1 1.1_KDE4_Factory_Desktop/x86_64/ are not updated to the builds for KDE 4.2.2. This seems to fit for all x86_64 packages in KDE:KDE4:Community. The i586 packages are updated.
Best greetings Buschmann
If there's a problem, it's a different one.
(I only see a million packages that don't mean much to me, so I don't know if something is wrong. The project seems to be built successfully, thus it should be published on download.o.o if publishing is not disabled.)
If there's something missing, it would be a problem between buildservice and download.opensuse.org - different problem.
Thank! Peter
It is gone, one hour later or so, the packages were on download.o.o. So, it was a different "problem". Now it works. Best greetings Buschmann ---------------------------------------------------------------------------------- openSUSE Member - de.opensuse.org Sys-Op http://en.opensuse.org/User:Buschmann23 http://en.opensuse.org/How_to_Participate http://en.opensuse.org/Geeko_wants_you!
Hi, the sync scheduling to mirrors should be a little faster now. I removed some unnecessary delays. I believe this could have been the main cause, I'm not sure. There could be another reason why the mozilla project might have been ignored. I had to work on the output and logging quite a lot to fix some bugs and make the output more useful, and in the cause I also synced the mozilla project; I'm not entirely sure why it was ignored. Most likely it was too late in the alphabet. We will probably need to prioritize the scheduling by download popularity. This will have to be revisited later. Peter On Thu, Apr 02, 2009 at 12:47:23PM +0200, Peter Poeml wrote:
I looked into it, but so far I can only see that it is a problem with our repository pushing script. It seems to ignore the mozilla project and doesn't push it (to any mirror). I don't know if it affects other projects as well; could be.
Peter
On Wed, Apr 01, 2009 at 07:27:52PM +0200, Manfred Hollstein wrote:
Hi there,
I didn't receive any feedback about this on opensuse@opensuse.org, maybe the buildservice list is more appropriate. I just looked again at the output from the rsync command below and it's still the same :-(
TIA, cheers.
l8er manfred
On Mon, 30 Mar 2009, 15:58:02 +0200, Manfred Hollstein wrote:
Hi there,
I thought that download.opensuse.org/repositories gets filled from rsync.opensuse.org, but that doesn't seem to be the case (at least ATM):
if you look at download.opensuse.org/repositories which version of MozillaFirefox is available for openSUSE-10.3, you'll find
http://download.opensuse.org/repositories/mozilla/openSUSE_10.3/i586/Mozilla...
however, looking at the output from running
rsync -n -av rsync.opensuse.org::buildservice-repos/mozilla/openSUSE_10.3/i586 . | grep MozillaFirefox
you'll see
i586/MozillaFirefox-3.0.7-1.4.i586.rpm
There must be something wrong with rsync.opensuse.org, no? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development
Hi Peter, On Fri, 03 Apr 2009, 16:23:29 +0200, Peter Poeml wrote:
Hi,
the sync scheduling to mirrors should be a little faster now. I removed some unnecessary delays. I believe this could have been the main cause, I'm not sure. There could be another reason why the mozilla project might have been ignored. I had to work on the output and logging quite a lot to fix some bugs and make the output more useful, and in the cause I also synced the mozilla project; I'm not entirely sure why it was ignored. Most likely it was too late in the alphabet. We will probably need to prioritize the scheduling by download popularity. This will have to be revisited later.
first of all, many thanks for your analysis and work to resurrect the issue. It appears that your assumption is correct as I just found out, that the "vdr" repository is also completely out of sync (which is even later in the alphabet than "mozilla").
Peter
Cheers. l8er manfred -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Apr 03, 2009 at 05:35:19PM +0200, Manfred Hollstein wrote:
the sync scheduling to mirrors should be a little faster now. I removed some unnecessary delays. I believe this could have been the main cause, I'm not sure. There could be another reason why the mozilla project might have been ignored. I had to work on the output and logging quite a lot to fix some bugs and make the output more useful, and in the cause I also synced the mozilla project; I'm not entirely sure why it was ignored. Most likely it was too late in the alphabet. We will probably need to prioritize the scheduling by download popularity. This will have to be revisited later.
first of all, many thanks for your analysis and work to resurrect the issue. It appears that your assumption is correct as I just found out, that the "vdr" repository is also completely out of sync (which is even later in the alphabet than "mozilla").
We use already most of the available bandwidth for syncing the BS repositories; there is a little bit of room for opimization. Anyhow, what we see is that for repositories like mozilla or vdr, it affects the users because it slows downloads for them, but the packages and functionality is there, even though there's no mirror to redirect to. An additional issue is that one of the mirrors that the packages are pushed to is widehat.opensuse.org, which is a.k.a. rsync.opensuse.org. Thus, anyone waiting for packages on rsync.o.o (or another mirror pulling from there) can see major delays. The connection to rsync.o.o could be much faster in practice than it is; we plan to improve on that in the future (GBit link). It would be good if at least rsync.o. becomes synced in a timely fashion. In addition, when KDE:* is affected by a sync delay, then we quickly get a problem because it doesn't take too long until there are enough downloads of the packages directly from download.o.o which really blow up the server (much more bandwidth needed than exists). This has hit us at least twice during the last days. Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development
On Fri, Apr 03, 2009 at 05:35:19PM +0200, Manfred Hollstein wrote:
On Fri, 03 Apr 2009, 16:23:29 +0200, Peter Poeml wrote:
Hi,
the sync scheduling to mirrors should be a little faster now. I removed some unnecessary delays. I believe this could have been the main cause, I'm not sure. There could be another reason why the mozilla project might have been ignored. I had to work on the output and logging quite a lot to fix some bugs and make the output more useful, and in the cause I also synced the mozilla project; I'm not entirely sure why it was ignored. Most likely it was too late in the alphabet. We will probably need to prioritize the scheduling by download popularity. This will have to be revisited later.
first of all, many thanks for your analysis and work to resurrect the issue. It appears that your assumption is correct as I just found out, that the "vdr" repository is also completely out of sync (which is even later in the alphabet than "mozilla").
I made another little change which now prioritizes the projects in a way that home:* is handled after other projects. This will make mozilla, multimedia:audio, and similar projects go to the mirrors first. The (sometimes) highly active but small home:* projects now have a worse chance to be mirrored but that affects much fewer users. There are 2260 home projects, and 335 non-home projects. I eliminated a sync push pause of 2x1 hour a day that existed only for one particular mirrror. This should also help. Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development
participants (3)
-
Manfred Hollstein -
Matthias Fehring -
Peter Poeml