[opensuse-project] Repository switch delayed
We didn't do the switch to the new repositories today, as was announced, because Marc and I have been working on how to properly transfer the repositories from our OBS instance to the primary server all night, but given the extreme complexity of the matter, we didn't manage to get done with it. We will continue to work on it tonight, and as we managed to at least find the right concept, it should be working very soon, hopefully tonight (CET). JFYI, in essence, the issue is properly synchronizing OBS pushing with rsync and mirrors pulling from there with rsync. And now that you think that you have an easy solution, stop before replying and think about it again ;) The plan, right now, is to * have our OBS rsync diffs to a staging directory, instead of into the real package tree directly (with rsync's very, very neat --compare-dest feature) * a cron job on the main server will, on a regular basis: * disable/block rsync for our OBS * disable/block rsync for the mirrors * wait for any running rsyncs to finish * rsync locally from the staging tree into the real tree * verify and re-sign the packages with the key that is used to sign the repository metadata (to avoid the issues people have had for so long with the signature verification on our packages) * re-enable rsync for OBS and mirrors The issue, basically, is to have a mechanism that gives a window of exclusive access to the file trees to that cron job on the main server, to avoid having incomplete and corrupt repositories (because e.g. our OBS is pushing files while mirrors are pulling from the same directory). If anyone else has an idea how to safely and atomically lock/sync processes across the network (with only ssh), please let me know ;) (and yes, we did try *many* things before giving up and choosing to let the cron job cut off rsync for our OBS instance, including many attempts at ssh'ing with flock etc...). cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf
participants (1)
-
Pascal Bleser