[opensuse-factory] Replacing libzypp stack with libdnf stack
From what I can tell, the only things that need any further integration work besides just switching out the package manager would be the YaST package manager module (which iirc also covers installation) and transactional-update. Everything else (even stuff
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE. In condensed list form, my complaints are: - zypp using external binaries for tasks (rpm, repo2solv) causes updates that change those binaries to be be particularly unstable - zypp upgrading is unstable when curl is being updated (from what I can remember this is because it uses private libcurl APIs that don't have any stability guarantees) - lack of parallel downloading of packages and repository metadata (even libalpm has this now) - PackageKit backend is extremely subpar (it's the only backend out of ten or so that doesn't support cancelling transactions) - nonexistence of a useful and well-documented plugin interface for zypper - difficult to use from a variety of programming languages as most bindings are broken and unsupported - the C++ API itself is a mess, exposing a ton of implementation details such as the XML parser in public API Unfortunately, the lack of a developer ecosystem around and using libzypp gives me extremely little hope that any of these architectural issues will be fixed. The DNF stack already maintained as part of openSUSE has none of these issues, resulting in a noticeably better user experience when switched in, especially for less technically competent users that rely on friendly utilities like GNOME Software or Plasma Discover which rely on PackageKit, where dnf provides a much better backend than zypp. Plus, the larger ecosystem of developers and users of the DNF stack gives me more confidence that it'll remain well maintained and any issues that arise will be fixed. like snapshot integration) that's needed for a cohesive package management experience with the libdnf stack already exists in openSUSE and I already see plenty of users setting up the libdnf stack to skip out on all the issues of the default libzypp stack. And speaking with an upstream (KDE dev) hat on, complaints from openSUSE users re. Discover "not working" are about as useful as bugs with stack traces from Arch, which doesn't ship debug symbols in any capacity at all, due to the abysmal quality of the PackageKit backend. For a lot of other KDE developers I've converted to openSUSE, the abysmal quality of the libzypp PackageKit backend was a near dealbreaker due to not being able to work on Discover with distro packages until I shared how to set up the DNF PackageKit backend. If I see someone complaining about Discover straight up not functioning, it's almost certainly an openSUSE user. I wouldn't be surprised if our friends at GNOME had a similar situation with GNOME Software. Any thoughts/objections regarding the possibility of replacing the libzypp stack in openSUSE with the libdnf stack? -- Carson Black [ jan Pontajosi, Jan Blackquill ] -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
W dniu 07.07.2020 o 05:45, Carson Black pisze:
Any thoughts/objections regarding the possibility of replacing the libzypp stack in openSUSE with the libdnf stack?
One of first things I always do with freshly installed openSUSE is to remove PackageKit completely. YaST and zypper were always more reliable interfaces for package management. If you have something that can improve this situation, go kick ass! https://web.archive.org/web/20130106180239/http://blog.hennevogel.de/kick-as...
Hi all, (the following is my personal opinion/view) Carson Black <uhhadd@gmail.com> writes:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.
In condensed list form, my complaints are: - zypp using external binaries for tasks (rpm, repo2solv) causes updates that change those binaries to be be particularly unstable
Especially zypper subprocessing to call rpm is downright dangerous and caused a few issues when the database format of rpmdb changes (does not happen often, but we've had a recent one from BDB to NDB).
- zypp upgrading is unstable when curl is being updated (from what I can remember this is because it uses private libcurl APIs that don't have any stability guarantees) - lack of parallel downloading of packages and repository metadata (even libalpm has this now) - PackageKit backend is extremely subpar (it's the only backend out of ten or so that doesn't support cancelling transactions) - nonexistence of a useful and well-documented plugin interface for zypper - difficult to use from a variety of programming languages as most bindings are broken and unsupported - the C++ API itself is a mess, exposing a ton of implementation details such as the XML parser in public API
I do agree with all of these. Also zypper lacks an equivalent of `dnf repoquery` which I use so frequently, that I have switched to using dnf 99% of the time (the remaining 1% is occasional kernel updates, see below).
*snip*
Any thoughts/objections regarding the possibility of replacing the libzypp stack in openSUSE with the libdnf stack?
I see currently one technical blocker: zypper has some special handling built in when it comes to kernel updates, that dnf does not have. This combined with the kernel's self-obsoletion [1] causes occasional issues on upgrades with dnf (dnf will not be able to update the kernel). However, this should be fixable by removing the self-obsoletion from the kernel's spec file. Well, and then there's the elephant in the room: dnf is a Fedora (and thus a RedHat) product, while zypper is SUSE's child. Just because of that it's going to be a rocky road and it will probably take ages or be downright impossible to push dnf as the default to SLE (zypper has a lot of SLE specific handling, for example packagehub integration). While I _personally_ would love to have dnf as the default on Tumbleweed, I'm afraid it's going to be very hard to even push it to become an equal to zypper. That does not mean that we shouldn't try though. Cheers, Dan Footnotes: [1] see https://bugzilla.opensuse.org/show_bug.cgi?id=1170232 and https://bugzilla.redhat.com/show_bug.cgi?id=1824102 if you are interested in further details. -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On Tue, Jul 7, 2020 at 10:09, Dan Čermák <dcermak@suse.com> wrote:
Well, and then there's the elephant in the room: dnf is a Fedora (and thus a RedHat) product, while zypper is SUSE's child. Just because of that it's going to be a rocky road and it will probably take ages or be downright impossible to push dnf as the default to SLE (zypper has a lot of SLE specific handling, for example packagehub integration).
Looking at SUSE support articles, Package Hub stuff is added with YaST and SUSEConnect, so zypp/er don't really have much to do with that Kinda related There were some plans to make an OBS plugin to make it easier to interact with OBS, but that was kinda abandoned, since OBS api requires authentication for every single action. https://github.com/rpm-software-management/dnf-plugins-extras/issues/64 Integrations like that aren't too hard to do, as long as there is an api of some kind LCP [Stasiek] https://lcp.world -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Stasiek Michalski <hellcp@opensuse.org> writes:
On Tue, Jul 7, 2020 at 10:09, Dan Čermák <dcermak@suse.com> wrote:
Well, and then there's the elephant in the room: dnf is a Fedora (and thus a RedHat) product, while zypper is SUSE's child. Just because of that it's going to be a rocky road and it will probably take ages or be downright impossible to push dnf as the default to SLE (zypper has a lot of SLE specific handling, for example packagehub integration).
Looking at SUSE support articles, Package Hub stuff is added with YaST and SUSEConnect, so zypp/er don't really have much to do with that
Kinda related There were some plans to make an OBS plugin to make it easier to interact with OBS, but that was kinda abandoned, since OBS api requires authentication for every single action. https://github.com/rpm-software-management/dnf-plugins-extras/issues/64
This is probably not going to get any simpler given the recent DOS "attacks" (not sure if they were attacks, but OBS' public routes got hammered by web crawlers), see https://github.com/openSUSE/open-build-service/issues/9876
Integrations like that aren't too hard to do, as long as there is an api of some kind
OBS has a pretty extensive API, but unfortunately (in this case) all of it requires authentication. However, I don't really see a reason why an interested user couldn't create an authentication token and put it into a config file? The UX wouldn't be as great as with the copr plugin, but better than nothing? Cheers, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On Tue, Jul 7, 2020 at 10:44, Dan Čermák <dcermak@suse.com> wrote:
This is probably not going to get any simpler given the recent DOS "attacks" (not sure if they were attacks, but OBS' public routes got hammered by web crawlers), see https://github.com/openSUSE/open-build-service/issues/9876
It's probably all the people using HTML instead of the API because it doesn't require authentication LCP [Stasiek] https://lcp.world -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am 07.07.20 um 10:53 schrieb Stasiek Michalski:
On Tue, Jul 7, 2020 at 10:44, Dan Čermák <dcermak@suse.com> wrote:
This is probably not going to get any simpler given the recent DOS "attacks" (not sure if they were attacks, but OBS' public routes got hammered by web crawlers), see https://github.com/openSUSE/open-build-service/issues/9876
It's probably all the people using HTML instead of the API because it doesn't require authentication
At least stable released OBS versions (no idea what is running on build.o.o) do not need authentication for most stuff, just go the /public route: seife:~> curl https://build.opensuse.org/build/home:seife <status code="anonymous_user"> <summary>Anonymous user is not allowed here - please login</summary> </status> seife:~> curl https://build.opensuse.org/public/build/home:seife <directory> <entry name="openSUSE_Factory"/> <entry name="SLE_11_SP1"/> <entry name="SLE_10SP3"/> <entry name="Fedora_24"/> </directory> -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tuesday 2020-07-07 13:42, Stefan Seyfried wrote:
Am 07.07.20 um 10:53 schrieb Stasiek Michalski:
On Tue, Jul 7, 2020 at 10:44, Dan Čermák <dcermak@suse.com> wrote:
This is probably not going to get any simpler given the recent DOS "attacks" (not sure if they were attacks, but OBS' public routes got hammered by web crawlers), see https://github.com/openSUSE/open-build-service/issues/9876
It's probably all the people using HTML instead of the API because it doesn't require authentication
At least stable released OBS versions (no idea what is running on build.o.o) do not need authentication for most stuff, just go the /public route:
And because "/public/" does not look much different from "/", I had already filed a bug related to that :-/ https://github.com/openSUSE/open-build-service/issues/9880
seife:~> curl https://build.opensuse.org/build/home:seife <status code="anonymous_user"> <summary>Anonymous user is not allowed here - please login</summary> </status> seife:~> curl https://build.opensuse.org/public/build/home:seife <directory> <entry name="openSUSE_Factory"/> <entry name="SLE_11_SP1"/> <entry name="SLE_10SP3"/> <entry name="Fedora_24"/> </directory> -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Stefan Seyfried <stefan.seyfried@googlemail.com> writes:
Am 07.07.20 um 10:53 schrieb Stasiek Michalski:
On Tue, Jul 7, 2020 at 10:44, Dan Čermák <dcermak@suse.com> wrote:
This is probably not going to get any simpler given the recent DOS "attacks" (not sure if they were attacks, but OBS' public routes got hammered by web crawlers), see https://github.com/openSUSE/open-build-service/issues/9876
It's probably all the people using HTML instead of the API because it doesn't require authentication
At least stable released OBS versions (no idea what is running on build.o.o) do not need authentication for most stuff, just go the /public route:
As far as I know nothing under /public is part of the official & guaranteed API (ironically one tends to refer to this as the *public* API), so code relying on these routes can break at any time.
seife:~> curl https://build.opensuse.org/build/home:seife <status code="anonymous_user"> <summary>Anonymous user is not allowed here - please login</summary> </status> seife:~> curl https://build.opensuse.org/public/build/home:seife <directory> <entry name="openSUSE_Factory"/> <entry name="SLE_11_SP1"/> <entry name="SLE_10SP3"/> <entry name="Fedora_24"/> </directory> -- Stefan Seyfried
"For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
-- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Am 07.07.20 um 13:51 schrieb Dan Čermák:
As far as I know nothing under /public is part of the official & guaranteed API (ironically one tends to refer to this as the *public* API), so code relying on these routes can break at any time. My experience with OBS interfaces is "everything can break any time and will not necessarily be fixed, because it was broken on purpose". And "backporting stuff to stable releases is a rare event, because OBS developers only run unstable code anyway".
Not a pleasant thing to use, I agree. The OBS developer community's attitude has improved (depends on who gets to process your github issue) but still the experience is -- ahem -- "interesting", especially compared to friendly communities like, for example, kiwi, but no real alternatives to OBS exist for my use case. So my way of interfacing with OBS is "use what's available, and when they kill it, just look for an alternative, or hack some replacement in" ;-) -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mittwoch, 8. Juli 2020, 09:53:14 CEST wrote Stefan Seyfried:
Am 07.07.20 um 13:51 schrieb Dan Čermák:
As far as I know nothing under /public is part of the official & guaranteed API (ironically one tends to refer to this as the *public* API), so code relying on these routes can break at any time.
It is explicit stated that it is not official. It is only for OBS interconnect: https://github.com/openSUSE/open-build-service/blob/master/docs/api/api/api....
My experience with OBS interfaces is "everything can break any time and will not necessarily be fixed, because it was broken on purpose". And "backporting stuff to stable releases is a rare event, because OBS developers only run unstable code anyway".
hey, you can count the numbers we changed incompatible on your hands. And they are mentioned in ReleaseNotes always. IMHO we a very good job in keeping a 15 years old api compatible and only extend it. There are not many doing that for that period of time... -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, Jul 8, 2020 at 16:15, Adrian Schröter <adrian@suse.de> wrote:
IMHO we a very good job in keeping a 15 years old api compatible and only extend it. There are not many doing that for that period of time...
I assume that's why most APIs have a version number in the URL, because we assume nothing is forever, and OBS is the proof of that ;) LCP [Stasiek] https://lcp.world -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mittwoch, 8. Juli 2020, 17:35:52 CEST wrote Stasiek Michalski:
On Wed, Jul 8, 2020 at 16:15, Adrian Schröter <adrian@suse.de> wrote:
IMHO we a very good job in keeping a 15 years old api compatible and only extend it. There are not many doing that for that period of time...
I assume that's why most APIs have a version number in the URL, because we assume nothing is forever, and OBS is the proof of that ;)
we could have a version, but it means you need to maintain old and new versions then. And keep maintaining the old ones for the new content in OBS. Therefore we do not do that by default. However, we have of course added a second way sometimes if the old way did not cover new use cases. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 4:45 AM Dan Čermák <dcermak@suse.com> wrote:
Stasiek Michalski <hellcp@opensuse.org> writes:
On Tue, Jul 7, 2020 at 10:09, Dan Čermák <dcermak@suse.com> wrote:
Well, and then there's the elephant in the room: dnf is a Fedora (and thus a RedHat) product, while zypper is SUSE's child. Just because of that it's going to be a rocky road and it will probably take ages or be downright impossible to push dnf as the default to SLE (zypper has a lot of SLE specific handling, for example packagehub integration).
Looking at SUSE support articles, Package Hub stuff is added with YaST and SUSEConnect, so zypp/er don't really have much to do with that
Kinda related There were some plans to make an OBS plugin to make it easier to interact with OBS, but that was kinda abandoned, since OBS api requires authentication for every single action. https://github.com/rpm-software-management/dnf-plugins-extras/issues/64
This is probably not going to get any simpler given the recent DOS "attacks" (not sure if they were attacks, but OBS' public routes got hammered by web crawlers), see https://github.com/openSUSE/open-build-service/issues/9876
Integrations like that aren't too hard to do, as long as there is an api of some kind
OBS has a pretty extensive API, but unfortunately (in this case) all of it requires authentication. However, I don't really see a reason why an interested user couldn't create an authentication token and put it into a config file? The UX wouldn't be as great as with the copr plugin, but better than nothing?
The problem is that OBS has no API token concept, so I can't even do that. I *could* write the plugin to use the ~/.config/oscrc that exists on disk, though. If I can get some time for this again, I may implement a "blind" plugin with obs:// URIs and add configuration for mapping obs:// URIs to download URLs. It's not great, but at least it's better than nothing, and essentially makes it the same as zypper's handling of obs:// URIs for adding OBS repos. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 07, 2020 at 10:09:52AM +0200, Dan Čermák wrote:
(the following is my personal opinion/view)
Carson Black <uhhadd@gmail.com> writes:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.
In condensed list form, my complaints are: - zypp using external binaries for tasks (rpm, repo2solv) causes updates that change those binaries to be be particularly unstable
Especially zypper subprocessing to call rpm is downright dangerous and caused a few issues when the database format of rpmdb changes (does not happen often, but we've had a recent one from BDB to NDB).
Ooh, be careful with that argument. Database format changes are not really supported by rpm/dnf at all. It took me quite some time to get it in a somewhat working state, but the pull request was never merged upstream. [snip]
Well, and then there's the elephant in the room: dnf is a Fedora (and thus a RedHat) product, while zypper is SUSE's child. Just because of that it's going to be a rocky road and it will probably take ages or be downright impossible to push dnf as the default to SLE (zypper has a lot of SLE specific handling, for example packagehub integration).
There's another elephant: dnf is currently getting completely rewritten. You shouldn't even think about changing to it unless that has been completed. Cheers, Michael. -- Michael Schroeder SUSE Software Solutions Germany GmbH mls@suse.de GF: Felix Imendoerffer HRB 36809, AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 5:04 AM Michael Schroeder <mls@suse.de> wrote:
On Tue, Jul 07, 2020 at 10:09:52AM +0200, Dan Čermák wrote:
(the following is my personal opinion/view)
Carson Black <uhhadd@gmail.com> writes:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.
In condensed list form, my complaints are: - zypp using external binaries for tasks (rpm, repo2solv) causes updates that change those binaries to be be particularly unstable
Especially zypper subprocessing to call rpm is downright dangerous and caused a few issues when the database format of rpmdb changes (does not happen often, but we've had a recent one from BDB to NDB).
Ooh, be careful with that argument. Database format changes are not really supported by rpm/dnf at all. It took me quite some time to get it in a somewhat working state, but the pull request was never merged upstream.
[snip]
What are you talking about? Database format changes are supported just fine for more than a year. We've gone through this exercise in Fedora long before pulling the trigger on switching to SQLite for the rpmdb. In fact, I was waiting longer to have Zypper prepared in Fedora for it than DNF on openSUSE. Yes, some optimizations were broken because they assumed BDB file locations, but those were ripped out entirely. The version that is shipped in Leap 15.2 has those patched to work, even though they aren't needed. For openSUSE Tumbleweed, I've just made SRs to update the stack to the latest version, which does not require my patch.
Well, and then there's the elephant in the room: dnf is a Fedora (and thus a RedHat) product, while zypper is SUSE's child. Just because of that it's going to be a rocky road and it will probably take ages or be downright impossible to push dnf as the default to SLE (zypper has a lot of SLE specific handling, for example packagehub integration).
There's another elephant: dnf is currently getting completely rewritten. You shouldn't even think about changing to it unless that has been completed.
This is probably the right time to have that discussion so we can have openSUSE-specific gaps factored into the upstream development. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 07, 2020 at 08:00:20AM -0400, Neal Gompa wrote:
What are you talking about? Database format changes are supported just fine for more than a year. We've gone through this exercise in Fedora long before pulling the trigger on switching to SQLite for the rpmdb. In fact, I was waiting longer to have Zypper prepared in Fedora for it than DNF on openSUSE.
I am talking about switching to a new rpm version that can no longer write the old format. See my pull request in rpm upstream. That's exactly what we had in Tumbleweed: the new rpm can no longer write berkeley db. Cheers, Michael. -- Michael Schroeder SUSE Software Solutions Germany GmbH mls@suse.de GF: Felix Imendoerffer HRB 36809, AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tuesday 2020-07-07 05:45, Carson Black wrote:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.[...] From what I can tell, the only things that need any further integration work besides just switching out the package manager would be[...]
* "product" support * downgrade support (zypper dup) No plugins -> no strange bugreports :-D -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Jan Engelhardt <jengelh@inai.de> writes:
On Tuesday 2020-07-07 05:45, Carson Black wrote:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.[...] From what I can tell, the only things that need any further integration work besides just switching out the package manager would be[...]
* "product" support * downgrade support (zypper dup)
Downgrades are supported via `dnf distro-sync --allowerasing` (allowerasing is necessary sometimes when packages get dropped from Factory). Cheers, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On Tue, Jul 07, 2020 at 10:52:31AM +0200, Dan Čermák wrote:
Jan Engelhardt <jengelh@inai.de> writes:
On Tuesday 2020-07-07 05:45, Carson Black wrote:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.[...] From what I can tell, the only things that need any further integration work besides just switching out the package manager would be[...]
* "product" support * downgrade support (zypper dup)
Downgrades are supported via `dnf distro-sync --allowerasing` (allowerasing is necessary sometimes when packages get dropped from Factory).
Be careful with allowerasing, it can completly wreck your system. The difference between dnf and libzypp is actually the "keeporphans" option: dnf sets it to true telling libsolv that it must keep all orphaned packages (packages that are not backed by a repository) installed. Cheers, Michael. -- Michael Schroeder SUSE Software Solutions Germany GmbH mls@suse.de GF: Felix Imendoerffer HRB 36809, AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 5:07 AM Michael Schroeder <mls@suse.de> wrote:
On Tue, Jul 07, 2020 at 10:52:31AM +0200, Dan Čermák wrote:
Jan Engelhardt <jengelh@inai.de> writes:
On Tuesday 2020-07-07 05:45, Carson Black wrote:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.[...] From what I can tell, the only things that need any further integration work besides just switching out the package manager would be[...]
* "product" support * downgrade support (zypper dup)
Downgrades are supported via `dnf distro-sync --allowerasing` (allowerasing is necessary sometimes when packages get dropped from Factory).
Be careful with allowerasing, it can completly wreck your system.
The difference between dnf and libzypp is actually the "keeporphans" option: dnf sets it to true telling libsolv that it must keep all orphaned packages (packages that are not backed by a repository) installed.
Yes, so if you want to remove orphans, you can use "dnf autoremove". Note that this relies on the DNF history database, so I don't think it'd do anything if you don't have much in there. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tuesday 2020-07-07 12:33, Neal Gompa wrote:
* downgrade support (zypper dup)
Downgrades are supported via `dnf distro-sync --allowerasing`
The difference between dnf and libzypp is actually the "keeporphans" option
Yes, so if you want to remove orphans, you can use "dnf autoremove". Note that this relies on the DNF history database[..]
Ah yeah, speaking of it: weakremover() support in zypper. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 12:50, Jan Engelhardt <jengelh@inai.de> wrote:
Ah yeah, speaking of it: weakremover() support in zypper.
Very happy to let package manager specific provides tags go away at last. If you need it, add it to rpm instead please. LCP [Stasiek] https://lcp.world -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tuesday 2020-07-07 05:45, Carson Black wrote:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.[...] From what I can tell, the only things that need any further integration work besides just switching out the package manager would be[...]
Be careful with this argument: I have spent time with both (used to work for Red Hat), and I have to say I prefer zypper hugely, both for its speed and features. Some things are not so easy (I cannot do `sudo zypper in /usr/bin/name`), but some things are better. And from what I know about dnf, than it was mostly rather poor rewrite of libzypp, which was so bad, they are rewriting it again. I don't think we should attach ourselves to something which has been effectively abandoned. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Never ascribe to malice that which is adequately explained by stupidity. -- Napoleon Bonaparte (or many other people to whom this quote is ascribed) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Matěj Cepl <mcepl@cepl.eu> writes:
On Tuesday 2020-07-07 05:45, Carson Black wrote:
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.[...] From what I can tell, the only things that need any further integration work besides just switching out the package manager would be[...]
Be careful with this argument: I have spent time with both (used to work for Red Hat), and I have to say I prefer zypper hugely, both for its speed and features. Some things are not so easy (I cannot do `sudo zypper in /usr/bin/name`), but some things are better.
I am honestly curious here, which parts of zypper do you miss from dnf?
And from what I know about dnf, than it was mostly rather poor rewrite of libzypp, which was so bad, they are rewriting it again. I don't think we should attach ourselves to something which has been effectively abandoned.
I am not sure, but it might be related to them integrating modularity a bit tighter, as the dnf team took over modularity from the old modularity team. Cheers, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On Tuesday 2020-07-07 13:55, Dan Čermák wrote:
Well, with all the talk about how great dnf is, I shall try it (on openSUSE).
I am honestly curious here, which parts of zypper do you miss from dnf?
* parallel download of metadata seems to be not happening * `dnf update` is much more verbose, akin to `zypper -v up` and I do not know of a way to make dnf give terse output. 14:07 a4:~ # zypper up The following item is locked and will not be changed by any action: Installed: wicked-service The following 42 package updates will NOT be installed: binutils binutils-gold coreutils coreutils-doc cpp cpp10 cpp9 cross-sparc64-binutils gcc gcc-PIE gcc-c++ gcc-info gcc10 gcc10-c++ gcc10-debugsource gcc10-info gcc9 gcc9-c++ gdb gsoap-devel libasan5 libasan6 libatomic1 libctf-nobfd0 libctf0 libgcc_s1 libgcc_s1-debuginfo libgfortran4 libgomp1 libitm1 liblsan0 libobjc4 libpcap1 libquadmath0 libstdc++-devel libstdc++6 libstdc++6-debuginfo libstdc++6-devel-gcc10 libstdc++6-devel-gcc9 libtsan0 libubsan1 wicked-service The following 3 packages are going to be upgraded: libstdc++6-pp-gcc11 mingw64-libwinpthread1 mingw64-winpthreads-devel 3 packages to upgrade. Overall download size: 162.9 KiB. Already cached: 0 B. No additional space will be used or freed after the operation. » dnf update ================================================================================ Package Arch Version Repo Size ================================================================================ Upgrading: binutils x86_64 2.34-386.1 gcc 5.8 M binutils-gold x86_64 2.34-386.1 gcc 1.3 M * dnf by default proposes vendor changes, zypper doesn't * dnf allows for vendor changes to sneak in: The "repo" column may tell me where the update is coming from, but does not tell me if that's gcc->gcc or tumbleweed->gcc. * dnf help | grep -i lock where is the package lock feature? * `dnf in --setopt=install_weak_deps=False blah` is quite a mouthful over `zypper in --no-r blah` * dnf is slower in some regard - e.g. single package install # time zypper in -y cdecl Loading repository data... Reading installed packages... Resolving package dependencies... The following NEW package is going to be installed: cdecl 1 new package to install. Overall download size: 128.5 KiB. Already cached: 0 B. After the operation, additional 328.5 KiB will be used. Continue? [y/n/v/...? shows all options] (y): y Retrieving package cdecl-7.3-1.1.x86_64 (1/1), 128.5 KiB (328.5 KiB unpacked) Retrieving: cdecl-7.3-1.1.x86_64.rpm .....................................[done] Checking for file conflicts: .............................................[done] (1/1) Installing: cdecl-7.3-1.1.x86_64 ...................................[done] real 0m1.335s user 0m1.095s sys 0m0.203s # time dnf -y in cdecl Repository 'base' is missing name in configuration, using id. Repository 'debug' is missing name in configuration, using id. Repository 'src' is missing name in configuration, using id. Repository 'gcc' is missing name in configuration, using id. Repository 'kopano' is missing name in configuration, using id. Repository 'mingw' is missing name in configuration, using id. Repository 'jeng' is missing name in configuration, using id. Repository 'packman' is missing name in configuration, using id. Last metadata expiration check: 0:12:14 ago on Tue Jul 7 14:05:26 2020. Dependencies resolved. ================================================================================ Package Architecture Version Repository Size ================================================================================ Installing: cdecl x86_64 7.3-1.1 base 128 k Transaction Summary ================================================================================ Install 1 Package Total download size: 128 k Installed size: 328 k Downloading Packages: cdecl-7.3-1.1.x86_64.rpm 6.9 MB/s | 128 kB 00:00 -------------------------------------------------------------------------------- Total 6.6 MB/s | 128 kB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : cdecl-7.3-1.1.x86_64 1/1 Verifying : cdecl-7.3-1.1.x86_64 1/1 Installed: cdecl-7.3-1.1.x86_64 Complete! real 0m2.653s user 0m2.484s sys 0m0.125s That's all I have for the time being. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hello Carson, On Tue, Jul 7, 2020 at 11:46 AM Carson Black <uhhadd@gmail.com> wrote:
Any thoughts/objections regarding the possibility of replacing the libzypp stack in openSUSE with the libdnf stack?
-- Carson Black [ jan Pontajosi, Jan Blackquill ]
Mine is a feedback as a Tumbleweed user, and switching to dnf was very good for me: 1) Parallel download is a blessing because mirrors in China aren't always very fast. 2) I don't need to taboo packages I don't want to reinstall after each dup anymore because from what I can tell dnf remembers them unlike zypper. 3) Aliases already in place for dnf to mimic zypper are for the most part enough for my user case. Plus i'm discovering additional dnf features that zypper is missing such as removing orphan packages. I don't have the experience to go in depth, but as a user the above was enough to make dnf a permanent switch. Best Regards, Maurizio -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, 7 Jul 2020 19:25:44 +0800, Maurizio Galli <mauriziogalli@opensuse.org> wrote:
Hello Carson,
On Tue, Jul 7, 2020 at 11:46 AM Carson Black <uhhadd@gmail.com> wrote:
Any thoughts/objections regarding the possibility of replacing the libzypp stack in openSUSE with the libdnf stack?
-- Carson Black [ jan Pontajosi, Jan Blackquill ]
Mine is a feedback as a Tumbleweed user, and switching to dnf was very good for me:
1) Parallel download is a blessing because mirrors in China aren't always very fast. 2) I don't need to taboo packages I don't want to reinstall after each dup anymore because from what I can tell dnf remembers them unlike zypper. 3) Aliases already in place for dnf to mimic zypper are for the most part enough for my user case. Plus i'm discovering additional dnf features that zypper is missing such as removing orphan packages.
Is there a list of "matching" options yet? zypper clean --all dnf clean all zypper dup dnf upgrade zypper in dnf install zypper lr dnf repolist zypper ls dnf repolist zypper lu dnf check-update --cacheonly zypper lp dnf updateinfo --cacheonly --security zypper patch dnf update-minimal --cacheonly --security zypper ref dnf makecache zypper rm dnf erase zypper se dnf search zypper up dnf update Oh, and every time I need to use dnf (which I remember as acronym for Did Not Finish) I curse the fact that it doesn't know abbreviations like zypper does. I already entered "dnf se pattern" and "dnf in foo" a zillion times on CentOS. I also dislike the always automatic refresh that yum and dnf do, I want to refresh once and than set --cacheonly for all subsequent commands. (see below for zypper) And "dnf clean all" mostoften *also* requires a "rm -rf /var/cache/yum" Having to use both on many systems on an almost daily basis, I prefer zypper over dnf by far.
I don't have the experience to go in depth, but as a user the above was enough to make dnf a permanent switch. Best Regards,
I have a "y2" command that "bundles" the YaST2 and zypper commands to a default that matches my daily work, and it has above list builtin, so I can use "y2" on CentOS too for the simple stuff. "y2" also adds "--no-refresh" to lu/lp/patch/up/ and dup and -l to patch/up/dup. # y2 se libfoo now works everywhere (on the systems where I put y2 in my bin/)
Maurizio
-- H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/ using perl5.00307 .. 5.31 porting perl5 on HP-UX, AIX, and Linux https://useplaintext.email https://tux.nl http://www.test-smoke.org http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 14:09, H.Merijn Brand <linux@tux.freedom.nl> wrote:
Oh, and every time I need to use dnf (which I remember as acronym for Did Not Finish) I curse the fact that it doesn't know abbreviations like zypper does. I already entered "dnf se pattern" and "dnf in foo" a zillion times on CentOS.
That was fixed, dnf in TW has abbreviated commands.
I also dislike the always automatic refresh that yum and dnf do, I want to refresh once and than set --cacheonly for all subsequent commands. (see below for zypper)
The refresh timer can be (and usually is) set in repo files. For Fedora repos I believe it's set to a couple of days, for Tumbleweed it would probably be daily for the sake of dupping. For Leap couple of days should be more than enough. LCP [Stasiek] https://lcp.world -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 14:15, Stasiek Michalski <hellcp@opensuse.org> wrote:
On Tue, Jul 7, 2020 at 14:09, H.Merijn Brand <linux@tux.freedom.nl> wrote: The refresh timer can be (and usually is) set in repo files. For Fedora repos I believe it's set to a couple of days, for Tumbleweed it would probably be daily for the sake of dupping. For Leap couple of days should be more than enough.
I looked into the documentation, and that reminded me that since zypp has the awful habit of rewriting every repo file it reads, if you use both at the same time, you will have your metadata refreshed in dnf after every time you used zypp/yast. Yes, it's the worst. Just use dnf ;) LCP [Stasiek] https://lcp.world -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 8:10 AM H.Merijn Brand <linux@tux.freedom.nl> wrote:
On Tue, 7 Jul 2020 19:25:44 +0800, Maurizio Galli <mauriziogalli@opensuse.org> wrote:
Hello Carson,
On Tue, Jul 7, 2020 at 11:46 AM Carson Black <uhhadd@gmail.com> wrote:
Any thoughts/objections regarding the possibility of replacing the libzypp stack in openSUSE with the libdnf stack?
-- Carson Black [ jan Pontajosi, Jan Blackquill ]
Mine is a feedback as a Tumbleweed user, and switching to dnf was very good for me:
1) Parallel download is a blessing because mirrors in China aren't always very fast. 2) I don't need to taboo packages I don't want to reinstall after each dup anymore because from what I can tell dnf remembers them unlike zypper. 3) Aliases already in place for dnf to mimic zypper are for the most part enough for my user case. Plus i'm discovering additional dnf features that zypper is missing such as removing orphan packages.
Is there a list of "matching" options yet?
zypper clean --all dnf clean all zypper dup dnf upgrade zypper in dnf install zypper lr dnf repolist zypper ls dnf repolist zypper lu dnf check-update --cacheonly zypper lp dnf updateinfo --cacheonly --security zypper patch dnf update-minimal --cacheonly --security zypper ref dnf makecache zypper rm dnf erase zypper se dnf search zypper up dnf update
Oh, and every time I need to use dnf (which I remember as acronym for Did Not Finish) I curse the fact that it doesn't know abbreviations like zypper does. I already entered "dnf se pattern" and "dnf in foo" a zillion times on CentOS. I also dislike the always automatic refresh that yum and dnf do, I want to refresh once and than set --cacheonly for all subsequent commands. (see below for zypper)
For reasons that I still don't get, RHEL and Fedora don't install the Zypper compatibility aliases by default. That said, some abbreviations are supported now as of CentOS 8.2, just not the zypper alias ones (dup, ref, etc.). The aliases that map to DNF commands cleanly are installed and activated by default on SUSE distributions. You can also make your own aliases and install them into /etc/dnf/aliases.d, or use the dnf alias command to set them up.
And "dnf clean all" mostoften *also* requires a "rm -rf /var/cache/yum"
This should not be necessary at all. Also, DNF does not write into /var/cache/yum. :) -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Jul 7, 2020 at 8:10 PM H.Merijn Brand <linux@tux.freedom.nl> wrote:
Is there a list of "matching" options yet?
zypper clean --all dnf clean all zypper dup dnf upgrade zypper in dnf install zypper lr dnf repolist zypper ls dnf repolist zypper lu dnf check-update --cacheonly zypper lp dnf updateinfo --cacheonly --security zypper patch dnf update-minimal --cacheonly --security zypper ref dnf makecache zypper rm dnf erase zypper se dnf search zypper up dnf update
On top of my head some of the similar commands are: zypper dup > dnf dup zypper in > dnf in zypper ref > dnf ref zypper se > dnf se zypper rm > dnf rm zypper up > dnf up I'm probably missing others. Best, Maurizio -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am 07.07.20 um 14:09 schrieb H.Merijn Brand:
Is there a list of "matching" options yet?
zypper clean --all dnf clean all
https://wiki.archlinux.org/index.php/Pacman/Rosetta -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Wed, Jul 8, 2020 at 9:10 AM Bernhard M. Wiedemann <bernhardout@lsmod.de> wrote:
Am 07.07.20 um 14:09 schrieb H.Merijn Brand:
Is there a list of "matching" options yet?
zypper clean --all dnf clean all
That's a pretty good comparison, however I'll note that there is a "dnf shell" subcommand to give you interactive shell. :) -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Lately, I've become somewhat displeased by a lot of shortcomings of the libzypp stack in regards to how it serves openSUSE.
In condensed list form, my complaints are: - zypp using external binaries for tasks (rpm, repo2solv) causes updates that change those binaries to be be particularly unstable We rarely have bug reports about this, so I'm not really sure what you are referring to here. Especially updates to librpm are messy if you link against it directly. That's one of the reasons
- zypp upgrading is unstable when curl is being updated (from what I can remember this is because it uses private libcurl APIs that don't have any stability guarantees) AFAIK we do not use private libcurl APIs do you have a example about
- lack of parallel downloading of packages and repository metadata (even libalpm has this now) We are currently working on exactly that. First step is parallel
On 7/7/20 5:45 AM, Carson Black wrote: libzypp uses rpm as a external binary. that? If that is the case I'd look into changing it. metadata download, next one is parallel downloads of rpms. Since the libzypp APIs is not at all fit for concurrency we need to jump through some hoops to make it work.
- PackageKit backend is extremely subpar (it's the only backend out of ten or so that doesn't support cancelling transactions) Can't say much about that since the packagekit backend is not maintained by the zypp team though. But I have to admit I have it disabled because it interferes with me using zypper directly. - nonexistence of a useful and well-documented plugin interface for zypper There is a plugin interface https://doc.opensuse.org/projects/libzypp/HEAD/zypp-plugins.html though. - difficult to use from a variety of programming languages as most bindings are broken and unsupported True, currently only C++ would be supported. - the C++ API itself is a mess, exposing a ton of implementation details such as the XML parser in public API
Unfortunately, the lack of a developer ecosystem around and using libzypp gives me extremely little hope that any of these architectural issues will be fixed. We are actively working on fixing many of those issues, but since we are only a few ppl working on it full time we need to balance it with bug fixing etc. Of course contributors would be welcome :)
-- Benjamin Zeller <bzeller@suse.de> Systems Programmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuremberg, Germany Tel: +49-911-74053-0; Fax: +49-911-7417755; https://www.suse.com/ (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (14)
-
Adam Mizerski -
Adrian Schröter -
Benjamin Zeller -
Bernhard M. Wiedemann -
Carson Black -
Dan Čermák -
H.Merijn Brand -
Jan Engelhardt -
Matěj Cepl -
Maurizio Galli -
Michael Schroeder -
Neal Gompa -
Stasiek Michalski -
Stefan Seyfried