On Wed, 2020-03-18 at 11:11 +0100, Per Jessen wrote:
Yes, finally we go this fixed...
I was looking at it yesterday, but not being familiar with the process, I didn't get very far. What was the problem?
I hope I manage to explain the setup - as that is crucial to understand (if you wish, I can also have a call with you on the topic - as you are the one mostly taking care of the mirror infra, this would be reasonable) So, as an overview of what happens while we get a new snapshot/build out (same for TW as for Leap): * We build in openSUSE * ToTestManager 'flags' for test ('release to :ToTest) * openQA syncs the data from OBS to openQA Servers * As part of this, a full changelog dump from the DVD being synced is extracted and stored in the openQA backend. * When openQA finished the testing and things are marked 'ok', ToTestManager triggers a 'publish' on OBS (publish flag in :ToTest's metadata) * OBS passes its scheduler code, preparing the direcrory structure on OBS infra * For Leap and TW, there is an extra script on OBS publisher to 'ask openQA Servers (apache/pythonapp) for the diff for the current snapshot and tells that same openQA hosted script 'snapshot XY.Z has been published. The openQA hosted scripts move a 'current' pointer to what is marked for publishing now (like this we can diff against the 'last published' - not only the 'last built' snapshot) * OBS RSyncs (push) this to download.o.o in the stage area * Some Mirrors have access to sync from Stage - so we get some mirrors ready before the snapshot is 'really there' * About two hours later, we sync from ftp-stage to ftp This should give a quick overview how things move around - and in this specific case, the publish script on OBS had some issues and no longer queried the openQA-hosted scripts. Cheers, Dominique