Forums (and possibly other services) offline?
![](https://seccdn.libravatar.org/avatar/008a8db3f6a813af5f8064f2be96e100.jpg?s=120&d=mm&r=g)
Was just headed to the forums to check something, and got no response from the server. I had also checked the wiki's availability, but it's also nonresponsive. Is there a known issue/outage? Jim
![](https://seccdn.libravatar.org/avatar/77cb4da5f72bc176182dcc33f03a18f3.jpg?s=120&d=mm&r=g)
On 10/10/2021 01.14, Jim Henderson wrote:
There is some confirmation on IRC, but the log is incomplete. 2021-10-09T22:49:49 <a-865k> It's not just you! forums.opensuse.org is down. But no previous message in hours, so no idea with whom he was talking to. And no message afterwards either. -- Cheers / Saludos, Carlos E. R. (from oS Leap 15.2 x86_64 (Minas Tirith))
![](https://seccdn.libravatar.org/avatar/a836ff90f492078f494adcf0c6059fc6.jpg?s=120&d=mm&r=g)
Carlos E. R. composed on 2021-10-10 01:38 (UTC+0200):
On 10/10/2021 01.14, Jim Henderson wrote:
Is there a known issue/outage?
There is some confirmation on IRC, but the log is incomplete.
2021-10-09T22:49:49 <a-865k> It's not just you! forums.opensuse.org is down.
But no previous message in hours, so no idea with whom he was talking to. And no message afterwards either.
That was me. Last successful forum page load was almost 2 hours ago. https://downforeveryoneorjustme.com/forums.opensuse.org says down. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
![](https://seccdn.libravatar.org/avatar/c1a90587ddfde63b222f10157a11b950.jpg?s=120&d=mm&r=g)
Am 10. Oktober 2021 00:37:51 UTC schrieb Felix Miata <mrmazda@earthlink.net>:
Forums and other Apps went down because (at least) one of the hypervisors (which are running the database cluster) went down or became at least unreachable. Bernhard (SUSE-IT) thankfully restarted a lot of nodes on the remaining hypervisors. Sadly, two out of three DB nodes were affected, leaving the cluster with just a single node. This lead to a "no quorum" situation, resulting in the left-over node refusing connections for the safety of the stored data. A resync of the other DB nodes with the remaining one helped: all services should meanwhile be back. The reason, why the hypervisors became unreachable, has to be checked and clarified. But I guess this is something for next week, once SUSE-IT is back at work. Kudos to Bernhard, who noticed the problem and stepped in! Regards, Lars
![](https://seccdn.libravatar.org/avatar/3af26b63955b9162f06c0f82c86231ac.jpg?s=120&d=mm&r=g)
On 10/10/2021 01.14, Jim Henderson wrote:
Thanks for the report. I typed up a small post-mortem https://etherpad.opensuse.org/p/postmortem-2021-10-10
participants (5)
-
Bernhard M. Wiedemann
-
Carlos E. R.
-
Felix Miata
-
Jim Henderson
-
Lars Vogdt