Mailinglist Archive: opensuse-buildservice (75 mails)

< Previous Next >
Re: [opensuse-buildservice] Post-mortem: Downtime on March 23, 2018
Hi Jan,

thanks for the feedback.

The full crash of the application also caused a crash of the delayed
jobs workers, which we use for asynchronous jobs. This was not
discovered until Monday morning which caused e.g. that the notification
system was not working over the weekend and delayed job needed to
process several thousand jobs.

We added the information to the postmortem report.

http://openbuildservice.org/2018/03/23/post-mortem-6/

Christian


On 03/26/2018 06:07 PM, Jan Engelhardt wrote:
On Monday 2018-03-26 17:59, Christian Bruckmayer wrote:
as some of you might noticed, we had a downtime of ~25 minutes last Friday.
For details what happened exactly, please have a look at our post mortem
post:
http://openbuildservice.org/2018/03/23/post-mortem-6/
Certainly an entertaining read.

Do you also have some notes on the failure of the notification system over
the weekend, or is that, in fact, connected to that deployment problem?


Your Open Build Service Tea
This typo makes me want to suddenly have that kind of branded merchandise. It
would be the next generation after the famous openSUSE beer. ;-)

--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-buildservice+owner@xxxxxxxxxxxx

< Previous Next >