https://bugzilla.novell.com/show_bug.cgi?id=411409 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=411409#c1 --- Comment #1 from Peter Poeml 2008-07-25 09:20:46 MDT --- I fixed the first issue. The sync will now first pull the new packages, and then the metadata. Thus, the update tree should be in near-perfect shape now. Of course it can still happen that metadata is replaced while a client is downloading it, but there is no way around this, and the time window is pretty small because the sync which pulls the metadata is very fast now, due to the exclusion of the rpm subdirectories. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=411409 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=411409#c3 --- Comment #3 from Peter Poeml 2008-07-25 10:04:52 MDT ---

im not an http expert, but wasnt there an http replycode that said something like server busy for now, or please revisit at a later time again. meaning to delay the http fetching of the metadata file.

The above fix pretty much avoids the need for this from the start. I am rather sure that the rest is neglectable. I'd rather spend time on fixing the bigger problems with Factory and BS next. :)

if i understand the whole mirror concept properly, this metadata comes from only one source: your opensuse download server. everbody gets their metadata from there and only gets redirected (if applicable) filewise to mirrors and such.

Yes, that's right.

so during the time period of updating the metadata file on this one masterserver, the httpd itself could actually block the delivering of the currently being updated metadta file with a http-reply that the visting client (yast, zypper, whatever) should revisit in like 10seconds or half a minute or whatever the intervals would fit.

Yes, that would be possible. I suggested something similar in the past for Factory updates. Since we control the client pretty much, this could be implemented. However, for the update tree we also need to take into account that there are other clients than libzypp which access the tree, so it shouldn't disturb them. People sometimes have self-written scripts that check for updates and such stuff. A 5xx code as such wouldn't be handled by them (neither would it by libzypp in its present form). For Factory it would be fine I assume.

something like that? maybe that would be the safest way to handle the metadataupdates.

503 service unavailable? and maybe implement some delay to re-try/re-fetch the metadata again when 503 is being given.

I believe this is best to be handled at the application level. After all, libzypp might talk to an entire separate server (a mirror, a local mirror, or a proxy cache) where it could encounter a similar situation, without provision for return codes with special meaning (if we did that). If libzypp handles it on the application level (in an "oops, something's wrong, let's just check this again" manner), it could work around these edge cases under various circumstances. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=411409 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=411409#c5 --- Comment #5 from Peter Poeml 2008-09-17 04:53:46 MDT --- A push sync could be an alternative. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=411409 User ro@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=411409#c6 Ruediger Oertel changed: What |Removed |Added ---------------------------------------------------------------------------- Info Provider|ro@novell.com |poeml@novell.com --- Comment #6 from Ruediger Oertel 2008-11-05 09:10:36 MST --- touching a lockfile somewhere inside the tree is of course doable. I'd prefer "opensuse-ftp/pub/opensuse" however since that's the base path for the this ftp-server and I would not have to play too many tricks. I'd be using lockfiles like "putpatch.running" and "putpatch-publictest.running" but before I add code ... can you explain that race condition more closely ?

The first sync (packages only) ran while putpatch had not started (and the packages were not there yet); and the second sync (metadata only) ran when putpatch was completely done. Thus, download.o.o pulled only the metadata.

are you saying there is a double sync running where the second one only looks at metadata ? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.