Atomicity of zypper repository update
Hello! (From "New Tumbleweed snapshot 20220102 released!":
On Mon, Jan 03, 2022 at 01:00:42PM +0000, Dominique Leuenberger <dimstar@suse.de> wrote:
Please note that this mail was generated by a script.
I'm referencing this thread since it's time-related but the topic is independent of this particular update.) I can see that the files listed in [1] now have timestamps 03-Jan-2022 13:03, the timestamp in the repomd.xml (hopefully) clarifies the timezone
date -d @1641215029 Mon 03 Jan 2022 02:03:49 PM CET
Around the same time a Github CI failed due to failed download:
Building repository 'repo-oss' cache [....done] File './repodata/f4fba37aa004f93b3b9f72917e4621a69964f902184252d9cfce7f49399cb2fe-primary.xml.gz' not found on medium 'http://download.opensuse.org/update/tumbleweed/' Abort, retry, ignore? [a/r/i/...? shows all options] (a): a Error building the cache: [repo-update|http://download.opensuse.org/update/tumbleweed/] Valid metadata not found at specified URL
subprocess.CalledProcessError: Command '['zypper', '--root', '/var/tmp/mkosi-q7daafdg/root', '--gpg-auto-import-keys', 'install', '-y', '--no-recommends', '--download-in-advance', 'acl', [...] ]' returned non-zero exit status 106.
AFAICT from the Github log, the command ran at
Mon, 03 Jan 2022 13:06:52 GMT
i.e. some 183 seconds after the repo timestamp (assuming synced clocks). Interestingly, a later re-run of the CI passed just fine (13:46:35 GMT). The Github CI may run (i.e. `zypper in`) relatively often (various PRs, force-pushes) therefore it doesn't look like a rare occasion that it can race with the TW repo update (that is rolling frequently on the other end). The suspected result of the race is a flaky CI due to failed repo download. Is the update of the zypper repository atomic, i.e. repomd.xml won't reference an invalid <location/> at any time? Or am I just chasing ghosts and the error means something else and time coincidence is just a coincidence? (Please keep me CC'd to the thread.) Thanks for advice, Michal [1] http://download.opensuse.org/update/tumbleweed/repodata/ [2] https://github.com/systemd/systemd/runs/4691185759?check_suite_focus=true#st... [3] https://github.com/systemd/systemd/actions/runs/1649357573
On Monday 2022-01-03 16:45, Michal Koutný wrote:
Building repository 'repo-oss' cache [....done] File './repodata/f4fba37aa004f93b3b9f72917e4621a69964f902184252d9cfce7f49399cb2fe-primary.xml.gz' not found on medium 'http://download.opensuse.org/update/tumbleweed/'
Is the update of the zypper repository atomic, i.e. repomd.xml won't reference an invalid <location/> at any time?
I guess it's possible. Even though createrepo creates repomd.xml last, rsyncing the repodata/ directory to mirrors could happen in arbitrary order, therefore sending repomd.xml before f4fba..-primary.xml.gz.
On Tue, Jan 18, 2022 at 09:50:44PM +0100, Jan Engelhardt <jengelh@inai.de> wrote:
I guess it's possible. Even though createrepo creates repomd.xml last, rsyncing the repodata/ directory to mirrors could happen in arbitrary order, therefore sending repomd.xml before f4fba..-primary.xml.gz.
I was pointed to the mirror hypothesis (in a personal conversation) too. So createrepo is likely safe but there's a missing "mirrorrepo". I've found these instructions [1] and they suggest a "plain" rsync call. Would it make sense to adjust the command to include `--delay-updates`? (First idea after reading rsync(1) section "Sorted transfer order". This won't close the race window but hopefully narrows it substantially. Second would be the explicit file list with referencer coming after referencees.) Michal [1] https://en.opensuse.org/openSUSE:Mirror_howto#Walk-through
On Wednesday 2022-01-19 08:45, Michal Koutný wrote:
On Tue, Jan 18, 2022 at 09:50:44PM +0100, Jan Engelhardt <jengelh@inai.de> wrote:
I guess it's possible. Even though createrepo creates repomd.xml last, rsyncing the repodata/ directory to mirrors could happen in arbitrary order, therefore sending repomd.xml before f4fba..-primary.xml.gz.
Would it make sense to adjust the command to include `--delay-updates`?
As the manpage says, that still isn't atomic (and that is a bug of rsync). To get real atomic updates, rsync on the receive side would have to do the equivalent of `cp -al` of the entire hierarchy to a temp dir first, then perform the synchronization as usual, and then it could do an atomic rename of the temp directory with the renameat2(RENAME_EXCHANGE) syscall). Let's see if they will ever do it.
Engelhardt wrote:
As the manpage says, that still isn't atomic (and that is a bug of rsync). To get real atomic updates, rsync on the receive side would have to do the equivalent of `cp -al` of the entire hierarchy to a temp dir first, then perform the synchronization as usual, and then it could do an atomic rename of the temp directory with the renameat2(RENAME_EXCHANGE) syscall). Let's see if they will ever do it. The --link-dest trick in this script may help to make rsync atomic:
https://opensource.apple.com/source/rsync/rsync-55/rsync/support/atomic-rsyn... Greetings, Björn
participants (3)
-
Bjoern Voigt -
Jan Engelhardt -
Michal Koutný