[heroes] Did someone/something touch gcc.opensuse.org?
Hello, because something changed on Friday, and now the libraries are in an inconsistent state, such that rsync isn't working anymore leading to a cron job of mine breaking. % ssh -p2271 gcc@gate.opensuse.org Last login: Wed Jun 27 13:30:30 2018 from 192.168.47.252 Have a lot of fun... sed: relocation error: /lib64/libacl.so.1: symbol getxattr, version ATTR_1.0 not defined in file libattr.so.1 with link time reference sed: relocation error: /lib64/libacl.so.1: symbol getxattr, version ATTR_1.0 not defined in file libattr.so.1 with link time reference gcc@gcc-stats:~> rsync --help rsync: relocation error: /lib64/libacl.so.1: symbol getxattr, version ATTR_1.0 not defined in file libattr.so.1 with link time reference gcc@gcc-stats:~> last gcc pts/0 192.168.47.252 Mon Mar 16 13:58 still logged in root pts/0 192.168.253.202 Fri Mar 13 19:28 - 19:31 (00:02) root pts/0 192.168.47.7 Fri Mar 13 19:25 - 19:27 (00:01) root pts/0 192.168.253.202 Thu Jan 2 23:22 - 23:24 (00:02) I will fix it by hand eventually (by hand because also rpm isn't working anymore), but I'd like to find out the cause for this. From 'last' it would seem something (semi-)automatic, and whatever it was it's failure mode isn't graceful. Ciao, Michael. -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Am March 16, 2020 1:52:07 PM UTC schrieb Michael Matz <matz@suse.de>:
because something changed on Friday, and now the libraries are in an inconsistent state, such that rsync isn't working anymore leading to a cron job of mine breaking.
Auutsch! That was me, resp. my scripts to blame. JFYI: I tried to catch as many machines with the current kernel update - looks like I 'catched' too much. Your machine is running tumbleweed - that is something my script was not prepared for. :-( Lessons learned: check /etc/os-release before running zypper commands - and skip tumbleweed for now. But, I'm sorry, I need to ask the following to avoid problems in the future also with other tumbleweed installs: how often should productive tumbleweed machines see a zypper dup? From my personal view, I don't think multiple times every week is needed. But for me it would be good to have some time frame like a month(?) to get the latest updates installed. If we could agree on something like "the first Thursday every month" or "after every kernel version update", that would help a lot to keep our "Fuhrpark" clean.
I will fix it by hand eventually (by hand because also rpm isn't working anymore), but I'd like to find out the cause for this.
Hey: I broke it, I will fix it. Not today any more (sorry, too late for me), but I promise to get it back to live tomorrow. We should even still have a snapshot of the machine from last week available, if that would be ok for you - and we don't loose important data on the machine (as the snapshot will restore exactly the state the week before). SORRY! Lars -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Hello, On Mon, 16 Mar 2020, Lars Vogdt wrote:
because something changed on Friday, and now the libraries are in an inconsistent state, such that rsync isn't working anymore leading to a cron job of mine breaking.
Auutsch! That was me, resp. my scripts to blame. JFYI: I tried to catch as many machines with the current kernel update - looks like I 'catched' too much. Your machine is running tumbleweed - that is something my script was not prepared for. :-(
Lessons learned: check /etc/os-release before running zypper commands - and skip tumbleweed for now.
But, I'm sorry, I need to ask the following to avoid problems in the future also with other tumbleweed installs: how often should productive tumbleweed machines see a zypper dup?
I would be fine with "never". It's serving static web pages without any interactive content.
From my personal view, I don't think multiple times every week is needed. But for me it would be good to have some time frame like a month(?) to get the latest updates installed. If we could agree on something like "the first Thursday every month" or "after every kernel version update", that would help a lot to keep our "Fuhrpark" clean.
Would work for me as well, as long as it can continue receiving rsync stuff and pushing out web pages.
I will fix it by hand eventually (by hand because also rpm isn't working anymore), but I'd like to find out the cause for this.
Hey: I broke it, I will fix it. Not today any more (sorry, too late for me), but I promise to get it back to live tomorrow. We should even still have a snapshot of the machine from last week available, if that would be ok for you - and we don't loose important data on the machine (as the snapshot will restore exactly the state the week before).
That's okay. The master data sits somewhere else, just the rsync will shuffle more data. Ciao, Michael. -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
On Tue, 17 Mar 2020 13:43:39 +0000 (UTC) Michael Matz <matz@suse.de> wrote:
But, I'm sorry, I need to ask the following to avoid problems in the future also with other tumbleweed installs: how often should productive tumbleweed machines see a zypper dup?
I would be fine with "never". It's serving static web pages without any interactive content.
"Never" is a bad option for anything that has connection to the bad world which is called "internet". If this machine is just serving static pages, why do we need a machine at all? We already have a cluster of machines doing that for a couple of other openSUSE pages (like static.o.o, fontinfo.o.o, shop.o.o, people.o.o) ... It would make much more sense and produce less work if static pages could be consolidated on that cluster.
From my personal view, I don't think multiple times every week is needed. But for me it would be good to have some time frame like a month(?) to get the latest updates installed. If we could agree on something like "the first Thursday every month" or "after every kernel version update", that would help a lot to keep our "Fuhrpark" clean.
Would work for me as well, as long as it can continue receiving rsync stuff and pushing out web pages.
Of course: the main service of a machine should be up and running. even after an up*grade*.
Hey: I broke it, I will fix it. Not today any more (sorry, too late for me), but I promise to get it back to live tomorrow. We should even still have a snapshot of the machine from last week available, if that would be ok for you - and we don't loose important data on the machine (as the snapshot will restore exactly the state the week before).
That's okay. The master data sits somewhere else, just the rsync will shuffle more data.
OK: I learned that we have currently just snapshots 3 days back :-( I upgraded the machine now from a Tumbleweed 2017 installation to the current 20200314 one - and activated chronyd and remote syslog. A first look around did not show any real problems (only your / is at 87.0%) - but you know for sure better than me if there are some issues. Regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Hello, On Tue, 17 Mar 2020, Lars Vogdt wrote:
But, I'm sorry, I need to ask the following to avoid problems in the future also with other tumbleweed installs: how often should productive tumbleweed machines see a zypper dup?
I would be fine with "never". It's serving static web pages without any interactive content.
"Never" is a bad option for anything that has connection to the bad world which is called "internet".
Sure, I get that.
If this machine is just serving static pages, why do we need a machine at all? We already have a cluster of machines doing that for a couple of other openSUSE pages (like static.o.o, fontinfo.o.o, shop.o.o, people.o.o) ... It would make much more sense and produce less work if static pages could be consolidated on that cluster.
History. It also does some local processing of data to generate some pages (meanwhile most of that went to some other machine). But also the data it serves (which aren't just HTML pages) is overall 22GiB. And eventually the processing it does should move back from lnt.opensuse.org to gcc.opensuse.org, whenever we find time to do that. So, for now, a separate VM is just fine.
Hey: I broke it, I will fix it. Not today any more (sorry, too late for me), but I promise to get it back to live tomorrow. We should even still have a snapshot of the machine from last week available, if that would be ok for you - and we don't loose important data on the machine (as the snapshot will restore exactly the state the week before).
That's okay. The master data sits somewhere else, just the rsync will shuffle more data.
OK: I learned that we have currently just snapshots 3 days back :-(
I upgraded the machine now from a Tumbleweed 2017 installation to the current 20200314 one - and activated chronyd and remote syslog. A first look around did not show any real problems (only your / is at 87.0%) - but you know for sure better than me if there are some issues.
Thank you. Works again. The filling of / is currently no problem (we could remove more packages, but there's not pressing need). Ciao, Michael. -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
On Wed, 18 Mar 2020 15:20:21 +0000 (UTC) Michael Matz <matz@suse.de> wrote:
History. It also does some local processing of data to generate some pages (meanwhile most of that went to some other machine). But also the data it serves (which aren't just HTML pages) is overall 22GiB.
And eventually the processing it does should move back from lnt.opensuse.org to gcc.opensuse.org, whenever we find time to do that. So, for now, a separate VM is just fine.
Fine with me. I just want to avoid extra work if it's not needed. Admins are lazy ;-)
Thank you. Works again. The filling of / is currently no problem (we could remove more packages, but there's not pressing need).
Perfect, thanks! Sorry for the breakage in the first run. I adjusted my script now. While... on the other hand... now that we have a prove of concept that we can upgrade tumbleweed machines... If you don't mind, I want to put this machine on a list of "run zypper dup once a month". If we check right afterward that everything still works as expected (=> Monitoring: just tell me what we should check) we should be ready to go. Worst thing would be that we might loose a few hours, if we notice early enough that something is broken and we need to revert to a former snapshot state. With kind regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
participants (2)
-
Lars Vogdt
-
Michael Matz