Comment # 22 on bug 1192457 from Martin Wilck

(In reply to Michal Suchanek from comment #20)
> (In reply to Dominique Leuenberger from comment #17)

> > > Everyone understands that if unsurmountable problems occur, the project
> > > fails. But what happened here is that we pulled back on the first occurence
> > > of an issue. The side effect of this is that real-world testing effectively
> > > doesn't happen any more. So possible other issues will not be found, and the
> > > known ones will be harder to solve.
> > 
> > It was not 'pulled back' - but moved to the 'devel area' instead of the
> > product. OBS is powerful enough to let us work out kinks outside of the
> > actual product build as long as we need to. And only put it inside the
> > product when we feel ready for it.
> 
> Then you probably want to link Kernel:HEAD rather than Kernel:stable

That's the point - factory kernels come from Kernel:stable, and by reverting to
xz in Kernel:stable, the feature was effectively "pulled back" as far as the
factory kernel is concerned.

Maybe we should create a temporary branch that's equivalent to Kernel:stable
but uses zstd?

> > > It was expected that some issues would occur. It was known beforehand that
> > > package size would increase.
> > 
> > So why did nobody look at the product deliverables and verify that they can
> > be produced? Throw over the fence and let Release Managers pick up the
> > broken glass is not the way to go.
> 
> And how do we look at them and ensure that they are buildable?
> 
> That's what we have release managers for. To know the deliverables.

Ack. This was in Jira for some time. IMO this is part of the reason why we have
Jira. There are product managers and release engineers involved who should have
this on their radar. You can't expect *kernel* people to anticipate issues in
every area.

> > Be my guest: branch it, fix it, submit it. 
> 
> That's not so easy. Last time I tried to branch a release package it would
> not build because it requires some complex project setup which is not
> documented.

Image building requires special knowledge that only a few people have. I
certainly don't. Also, I'm pretty sure that you wouldn't be pleased if Michal
or myself started throwing things out of the Live image to our own taste. We
need to do this together somehow.

(In reply to Dominique Leuenberger from comment #17)

@Dominique, I agree that the burden shouldn't be on _you_. The question is how
it can be shared.

> There are 'rough edged' and 'unbuildable product deliverables'. Rough edges
> can be ok, rebuildable deliverables are not ok. If the lives ar enot being
> built, there is no Tumbleweed snapshot being produced which in turn means:
> no user gets the change to test anyway. So what do you win? Nada.

I wasn't aware that building live images was a prerequisite for releasing
snapshots. My expectation was that few people are using these images. I thought
they were mainly for newcomers checking out openSUSE. I don't think I ever used
one.

> >  And this is what happens with "factory first" - effectively
> > we beta-test on a distribution that others use in production. It's not the
> > first time something breaks in factory, and not a unique incident by any
> > means.
> 
> Guess what: there were plenty of cases where a snapshot is not published
> because of broken submissions. We have QA, we block snapshots if they are
> too broken. Developers are still meant to do their tasks and deliver
> functioning pieces.

The pieces do function. It's malfunctioning at an entirely different level,
image creation, which is outside the scope of kernel developers.

I believe that releasing a snapshot without a live CD image would have close to
zero impact on regular TW users, unlike e.g. the move to glibc 2.34.

> We don't give up - we move back to the drawing board with the findings and
> work out the issues.

Good, thank you very much. 

As Michal suggested, you may want to link your test project
to Kernel:HEAD. That means you'll also get another kernel, but that shouldn't
matter much, hopefully.