Re: [heroes] Did someone/something touch gcc.opensuse.org?
Hello Lars, On Wed, 18 Mar 2020, Lars Vogdt wrote:
While... on the other hand... now that we have a prove of concept that we can upgrade tumbleweed machines... If you don't mind, I want to put this machine on a list of "run zypper dup once a month". If we check right afterward that everything still works as expected (=> Monitoring: just tell me what we should check) we should be ready to go. Worst thing would be that we might loose a few hours, if we notice early enough that something is broken and we need to revert to a former snapshot state.
The two important "services" on that would be a simple HTTPS check on some random URL, let's say https://gcc.opensuse.org/ (better would be https://gcc.opensuse.org/gcc-old/SPEC/CINT/sb-czerny-head-64-2006/recent.htm... but the 'gcc-old' part at least changed once over the last 15 years, so it's only semi-stable), and a check that would verify rsync via ssh access (that is what got broken, the webserver continued to work), from inside the SUSE network. I don't know if you have the capability to do such checking. (I could add a ssh key for that purpose). Ciao, Michael. -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Am March 19, 2020 2:07:03 PM UTC schrieb Michael Matz <matz@suse.de>:
The two important "services" on that would be a simple HTTPS check on some random URL, let's say https://gcc.opensuse.org/ (better would be https://gcc.opensuse.org/gcc-old/SPEC/CINT/sb-czerny-head-64-2006/recent.htm...
Ok: done.
and a check that would verify rsync via ssh access (that is what got broken, the webserver continued to work), from inside the SUSE network. I don't know if you have the capability to do such checking. (I could add a ssh key for that purpose).
I think I don't need to be inside the SUSE network to test of a rsync via ssh to this machine works: shouldn't it be enough to test this from the monitoring server? I would suggest to create a local user for this - and test transfers as this user. This should not be too intrusive (the user can be very limited) but in the other side allow to see if the service works in general. I have to admit that this of course does not cover network issues to the SUSE side - but these might be another check. You could also consider to sync a file containing a timestamp in your script. Let's say: https://gcc.opensuse.org/.last_sync - containing something like "2020-03-19 22:02:03". Our monitoring could download the file, parse it's content, and alert if the date is older than a day (or whenever your sync normally runs). What do you think? [ ] test ssh (sync) with local, separate user [ ] test freshness of delivered content via special file Regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Hello, On Thu, 19 Mar 2020, Lars Vogdt wrote:
broken, the webserver continued to work), from inside the SUSE network. I don't know if you have the capability to do such checking. (I could add a ssh key for that purpose).
I think I don't need to be inside the SUSE network to test of a rsync via ssh to this machine works: shouldn't it be enough to test this from the monitoring server?
(I was unsure if ssh access to gate.opensuse.org:2271 was or wasn't allowed from the outside. It once was internal-only many years ago.)
You could also consider to sync a file containing a timestamp in your script. Let's say: https://gcc.opensuse.org/.last_sync - containing something like "2020-03-19 22:02:03". Our monitoring could download the file, parse it's content, and alert if the date is older than a day (or whenever your sync normally runs).
This is actually a great idea, thanks! The sync job does this now, and there is: % curl https://gcc.opensuse.org/gcc-old/last_synced 2020-03-20T00:14:41+0100 (i.e. you could directly pass this into 'date --date=$abovestring +%s' to calculate seconds difference to a reference seconds-since-epoch; should not be more than 36 hours normally) Ciao, Michael. -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Hello, Am Freitag, 20. März 2020, 00:26:28 CET schrieb Michael Matz:
On Thu, 19 Mar 2020, Lars Vogdt wrote:
You could also consider to sync a file containing a timestamp in your script. Let's say: https://gcc.opensuse.org/.last_sync - containing something like "2020-03-19 22:02:03". Our monitoring could download the file, parse it's content, and alert if the date is older than a day (or whenever your sync normally runs).
This is actually a great idea, thanks! The sync job does this now, and there is:
% curl https://gcc.opensuse.org/gcc-old/last_synced 2020-03-20T00:14:41+0100
(i.e. you could directly pass this into 'date --date=$abovestring +%s' to calculate seconds difference to a reference seconds-since-epoch; should not be more than 36 hours normally)
More boring idea - maybe just check the file timestamp, like we already do for logfiles (and warn if they are older than x hours)? Regards, Christian Boltz --
Herr K hat seine Seele verkauft. Nein, hochgeladen, zu Leuten, die damit verantwortungsvoll umgehen können. [> Nicole Britz und Kristian Köhntopp in https://plus.google.com/+KristianKöhntopp/posts/2rvcxyr3RVR]
-- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
On Sat, 21 Mar 2020 13:46:30 +0100 Christian Boltz <opensuse@cboltz.de> wrote:
% curl https://gcc.opensuse.org/gcc-old/last_synced 2020-03-20T00:14:41+0100
(i.e. you could directly pass this into 'date --date=$abovestring +%s' to calculate seconds difference to a reference seconds-since-epoch; should not be more than 36 hours normally)
Luckily we had such kind of tests already for the release-notes and mirrors.opensuse.org pages :-) => done.
More boring idea - maybe just check the file timestamp, like we already do for logfiles (and warn if they are older than x hours)?
Never trust a time stamp given by a web server ;-) Regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
participants (3)
-
Christian Boltz -
Lars Vogdt -
Michael Matz