On Wed, Dec 11, 2019 at 03:39:24PM +0100, Stefan Hundhammer wrote:

The Problem of the Day ======================

In the last few days we were reviewing and merging Rodion's changes to the libyui packages for the REST API and better testability.

The Recurring Problem =====================

We found out (again) that it's a very painful and error-prone process to get a green build from Travis and to submit all those libyui packages; the library SO version had to be bumped from 10 to 11 to ensure binary compatibility, so Rodion had to touch every single one of all those libyui packages and create pull requests for each one:

- libyui - libyui-rest-api - libyui-qt - libyui-qt-rest-api - libyui-qt-pkg - libyui-graph - libyui-ncurses - libyui-ncurses-rest-api - libyui-ncurses-pkg

...and later even the externally maintained ones:

- libyui-gtk - libyui-gtk-pkg - ... (?)

yast2-ycp-ui-bindings defines %yui_so in the spec file which must also be updated.

The Annoyance =============

This is error-prone and tedious; there were build problems in the in-between stages all the time, and they are not easy to resolve, and we had one pretty unhappy Dimstar who saw failing things all around him.

Only for the very first of those PRs there is a realistic chance for a green Travis build; for all subsequent ones, there is only the (now outdated) docker image that also includes those libyui packages. That image needs to be rebuilt, but that fails because now there is a libyui with a higher SO number, but no matching libyui-something packages.

The Pragmatic Brute-Force Fix =============================

So the only way out is to brute-force merge despite red Travis and hope for the best. And to do this a couple of times until it gets green in Jenkins and in OBS.

This is a mess. There must be a better way; a way without very unhappy release managers that are confronted with half-ready UI stuff that breaks all kinds of other packages.

The Future ==========

So, what can we do?

Is it really a wise idea to have all those UI packages fragmented into so many different source repositories?

Unify the Fragments? =====================

Would it be better to have one large libyui repo that contains the base libyui, and also most of the others (except libyui-gtk* ?) and create all the binary packages that we have now from that single source repo?

That would give us a chance to do atomic changes; a transaction with a well-defined "before" and "after" state and no in-between mess.

We'd have ONE pull request for all the different subpackages. We could review that as a whole and not get into a PR frenzy when things need to happen quickly to avoid undefined in-between states.

Would there still be several source packages with each having a tarball or would there just be one source package generating all RPMs? In the latter case people could complain about more freqently rebuilds of e.g. libyui-qt due to a new libzypp.

Of course we could still work separately in the different UI parts (Qt, NCurses) with different people. We could still do PRs for those parts separately if we want to.

But we could (and should) have one central place where things like the UI so number is defined. Not sure, but maybe we should also have one single consistent package version number for all the subpackages (libyui, libyui-qt, libyui-ncurses, ...).

What do you think? Does anybody have a better idea? Or is there some killer argument against this?

Not being able to update the system to work on libyui for a whole day is for me a killer argument against keeping it like it is now. ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Software Solutions Germany GmbH Maxfeldstraße 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

On Wed, 11 Dec 2019 15:39:24 +0100 Stefan Hundhammer <shundhammer@suse.de> wrote:

The Problem of the Day ======================

In the last few days we were reviewing and merging Rodion's changes to the libyui packages for the REST API and better testability.

The Recurring Problem =====================

We found out (again) that it's a very painful and error-prone process to get a green build from Travis and to submit all those libyui packages; the library SO version had to be bumped from 10 to 11 to ensure binary compatibility, so Rodion had to touch every single one of all those libyui packages and create pull requests for each one:

- libyui - libyui-rest-api - libyui-qt - libyui-qt-rest-api - libyui-qt-pkg - libyui-graph - libyui-ncurses - libyui-ncurses-rest-api - libyui-ncurses-pkg

...and later even the externally maintained ones:

- libyui-gtk - libyui-gtk-pkg - ... (?)

The Annoyance =============

This is error-prone and tedious; there were build problems in the in-between stages all the time, and they are not easy to resolve, and we had one pretty unhappy Dimstar who saw failing things all around him.

Only for the very first of those PRs there is a realistic chance for a green Travis build; for all subsequent ones, there is only the (now outdated) docker image that also includes those libyui packages. That image needs to be rebuilt, but that fails because now there is a libyui with a higher SO number, but no matching libyui-something packages.

The Pragmatic Brute-Force Fix =============================

So the only way out is to brute-force merge despite red Travis and hope for the best. And to do this a couple of times until it gets green in Jenkins and in OBS.

This is a mess. There must be a better way; a way without very unhappy release managers that are confronted with half-ready UI stuff that breaks all kinds of other packages.

The Future ==========

So, what can we do?

Is it really a wise idea to have all those UI packages fragmented into so many different source repositories?

Unify the Fragments? =====================

Would it be better to have one large libyui repo that contains the base libyui, and also most of the others (except libyui-gtk* ?) and create all the binary packages that we have now from that single source repo?

That would give us a chance to do atomic changes; a transaction with a well-defined "before" and "after" state and no in-between mess.

We'd have ONE pull request for all the different subpackages. We could review that as a whole and not get into a PR frenzy when things need to happen quickly to avoid undefined in-between states.

Of course we could still work separately in the different UI parts (Qt, NCurses) with different people. We could still do PRs for those parts separately if we want to.

But we could (and should) have one central place where things like the UI so number is defined. Not sure, but maybe we should also have one single consistent package version number for all the subpackages (libyui, libyui-qt, libyui-ncurses, ...).

What do you think?

Hi, my question is how other projects do it? I really believe we are not only library with consumers. And I also believe that they do not put everything to one place.

Does anybody have a better idea?

I definitively would like to not reinvent wheel, but I think at least small research how other libraries with plugins do it would be good ( at least we can document why we do not do it same way ). I think in linux/FOSS worls is definitively bigger projects then libyui that have to deal with similar challenges and we can at least inspire how they handle it.

Or is there some killer argument against this?

I do not have any "killer" one, but how it will then go with gtk one? and I agree with stasiek that having separated pkg makes also some sense for me. And what to do with another plugins like qt-graph? And another code that we maybe use like libyui-bindings? Also another potential issue is that if we have everything in one repo, that travis, jenkins, etc will take much longer to build. Also risk of collisions is bigger. I am also not sure how other users distro use libyui. If they take tarball from github as upstream release, then I am not sure how it will work when we have everything in one repo. Josef

Kind regards

-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

Dne 11. 12. 19 v 15:39 Stefan Hundhammer napsal(a):

Only for the very first of those PRs there is a realistic chance for a green Travis build; for all subsequent ones, there is only the (now outdated) docker image that also includes those libyui packages. That image needs to be rebuilt, but that fails because now there is a libyui with a higher SO number, but no matching libyui-something packages.

That's partly caused by the circular dependency libyui <-> libyui-<something>...

So the only way out is to brute-force merge despite red Travis and hope for the best. And to do this a couple of times until it gets green in Jenkins and in OBS.

Yes, that's ugly...

Would it be better to have one large libyui repo that contains the base libyui, and also most of the others (except libyui-gtk* ?) and create all the binary packages that we have now from that single source repo?

We could include all plugins if we allow building only some parts, e.g. we would skip the Gtk plugins, some other distribution could skip the *-pkg (libzypp) plugins, etc... If you could easily configure which parts you want to build then it should be OK I think.

Or is there some killer argument against this?

For me it looks like a quite good idea. We just need to do some research or maybe try it to find the drawbacks. And as Josef pointed out, we should check how the other projects solve this, we do not need to reinvent the wheel... -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org