what zypper should adopt from pacman
- starting with the biggest downloads, psychologically therapeutic - showing the total done so far - showing the total time remaing -- "The true measure of a man's character, is how he treats those who can do nothing for him". Seen in a forum by OzDozer
On 9/12/24 5:56 PM, bent fender wrote:
- starting with the biggest downloads, psychologically therapeutic
- showing the total done so far
- showing the total time remaing
Ummm...., This slices the differences pretty thin. Zypper shows pkg/total instead of bytes/total, but otherwise the information is pretty similar. The biggest difference is that pacman (configurable) will start 5 package downloads in parallel (default). While I haven't had issues with Arch mirrors to the same extent I have with the US openSUSE mirrors (or lack thereof - some repos are .de only), the parallel download may help with the slow mirror issue. What I see with Tumbleweed updates is sometimes download speeds are okay, other times (like last night) the download struggles between 0KiB/s and 100KiB/s causing some relatively small packages to take tens of minutes to download. This renders the time remaining number meaningless. When mirrors slow down, the time remaining grows wildly, when they speed back up, the reverse occurs. That doesn't mean total remaining time shouldn't be shown, that just means it is rarely very accurate at any given moment in time. Of course you can come up with a better time remaining based on average download speed over the course of the update, or some moving-average or sliding-window -- but -- if the download speed changes significantly (like when you get to openSUSE packman packages), even the average goes out the window. The one observation I would add to your list is that the zypper output is quite busy looking compared to Arch, and the resulting 2-line per-package output of zypper makes things scroll quickly when a bunch of small packages are downloaded. It may benefit zypper to remove, instead of adding, to the output :) -- David C. Rankin, J.D.,P.E.
Thu, 12 Sep 2024 18:33:16 -0500 "David C. Rankin" <drankinatty@gmail.com> :
On 9/12/24 5:56 PM, bent fender wrote:
- starting with the biggest downloads, psychologically therapeutic
- showing the total done so far
- showing the total time remaing
Ummm....,
This slices the differences pretty thin. Zypper shows pkg/total instead of bytes/total, but otherwise the information is pretty similar.
The biggest difference is that pacman (configurable) will start 5 package downloads in parallel (default). While I haven't had issues with Arch mirrors to the same extent I have with the US openSUSE mirrors (or lack thereof - some repos are .de only), the parallel download may help with the slow mirror issue.
What I see with Tumbleweed updates is sometimes download speeds are okay, other times (like last night) the download struggles between 0KiB/s and 100KiB/s causing some relatively small packages to take tens of minutes to download.
This renders the time remaining number meaningless. When mirrors slow down, the time remaining grows wildly, when they speed back up, the reverse occurs. That doesn't mean total remaining time shouldn't be shown, that just means it is rarely very accurate at any given moment in time.
Of course you can come up with a better time remaining based on average download speed over the course of the update, or some moving-average or sliding-window -- but -- if the download speed changes significantly (like when you get to openSUSE packman packages), even the average goes out the window.
The one observation I would add to your list is that the zypper output is quite busy looking compared to Arch, and the resulting 2-line per-package output of zypper makes things scroll quickly when a bunch of small packages are downloaded. It may benefit zypper to remove, instead of adding, to the output :)
1 I really should have said pacman defaults as used in Artix and zypper defaults as used in Tumbleweed. 2 What I'm citing is neither the speed nor the dual file downloads but the therapeutic 'illusion' of improving speed and the 'totals' line always at the bottom. Summing up I feel better doing it in Artix than in TW :-) TRY IT IF YOU GET A CHANCE. pacman in Artix always starts the downloads with the biggest file (few exceptions) so it seems slow at first when the patience vial is still full but starts to seemingly really fly as patience wears thin. The totals line keeps me better informed, after all, among the things I want to know are definitely how much time is left, where we are in the list, what the current DL rate is ...as changing as these might be dynamically. https://paste.opensuse.org/pastes/a19315bb4d6b
On 9/13/24 2:52 PM, bent fender wrote:
pacman in Artix always starts the downloads with the biggest file (few exceptions) so it seems slow at first when the patience vial is still full but starts to seemingly really fly as patience wears thin. The totals line keeps me better informed, after all, among the things I want to know are definitely how much time is left, where we are in the list, what the current DL rate is ...as changing as these might be dynamically.
Glad you posted the link! Haven't tried Artix, will have to look and see what it is, but the download in descending order of size is on a Per-Repo basis, so for normal Arch, that is a minimum of core + extra repos and you can see the result in: https://paste.opensuse.org/pastes/9ac4387de789 I share your observation that the pacman progress output is much easier on the eyes to understand at a glance. I don't have any complaints about the zypper output, it's just busier to follow along during the update. Anyway, I like your underlying point, that if there were to be changes to modernize the zypper output, looking at what Arch does is always a good place to gather idea. Who know, the zypper devs may find a better way and then we could make the suggestion on the Arch list to take a look at the way SUSE is doing it with zypper. pacman 7 (a major update) is out shortly, within the next day or so, and it will be interesting to see if there are any visual update (doubtful). -- David C. Rankin, J.D.,P.E.
Fri, 13 Sep 2024 18:21:41 -0500 "David C. Rankin" <drankinatty@gmail.com> :
On 9/13/24 2:52 PM, bent fender wrote:
pacman in Artix always starts the downloads with the biggest file (few exceptions) so it seems slow at first when the patience vial is still full but starts to seemingly really fly as patience wears thin. The totals line keeps me better informed, after all, among the things I want to know are definitely how much time is left, where we are in the list, what the current DL rate is ...as changing as these might be dynamically.
Glad you posted the link!
Haven't tried Artix, will have to look and see what it is
You can take the kde iso install it and bingo, different challenges from those with TW, that's all. If you're knowlegable you can build from lower levels on up. I found the forum regulars very helpful, as those of most distros are.
in descending order of size is on a Per-Repo basis, so for normal Arch, that is a minimum of core + extra repos and you can see the result in:
https://paste.opensuse.org/pastes/9ac4387de789
I share your observation that the pacman progress output is much easier on the eyes to understand at a glance. I don't have any complaints about the zypper output, it's just busier to follow along during the update.
Anyway, I like your underlying point, that if there were to be changes to modernize the zypper output, looking at what Arch does is always a good place to gather idea. Who know, the zypper devs may find a better way and then we could make the suggestion on the Arch list to take a look at the way SUSE is doing it with zypper.
Sure thing, I mean I didn't post this to complain, everyone should always see how others deal with the same tasks, that's MY way of seeing it. Difference for only the sake of difference is bullshit. A Chinese rep was visiting a local garage and when asked about 'pieface knockoffs' he said "well, we just analyse all we can find and then we improve on it" ...that's more than a bit of a stretch but the idea is there :-)
On 13.09.2024 kl. 00.56 bent fender wrote:
I have never used pacman but when talking about zypper: Parallel operation: Selecting download after dependencies. So as soon as a package (including all necessary dependencies) is downloaded, it could start installing that - while continuing to download for the next batch. But I can foresee that it's a huge pile of worms to keep track on. So while I think it could be cool, maybe the time necessary for such, could be better used elsewhere :-)
- starting with the biggest downloads, psychologically therapeutic
- showing the total done so far
- showing the total time remaing my 5©
-- Klaus
On 9/13/24 16:56, Klaus Vink Slott via openSUSE Users wrote:
On 13.09.2024 kl. 00.56 bent fender wrote:
I have never used pacman but when talking about zypper:
Parallel operation: Selecting download after dependencies. So as soon as a package (including all necessary dependencies) is downloaded, it could start installing that - while continuing to download for the next batch.
This is not possible, librpm wants all packages for a transaction on disk, or at least all package headers. So we need to predownload all packages then start the installation. What we will do is doing everything in parallel until that point... download, checksums, signatures, copying etc... However the code as it is currently has two major problems: - It has global state all over the place ( which is a problem for another feature we need ) - It is completely written with serial downloads in mind, not only the logic also the libzypp APIs, so there is no simple way to "just" parallelize things, we need a full source refactoring and this is what I spent most of my time on.
But I can foresee that it's a huge pile of worms to keep track on. So while I think it could be cool, maybe the time necessary for such, could be better used elsewhere :-)
- starting with the biggest downloads, psychologically therapeutic
- showing the total done so far
- showing the total time remaing my 5©
-- Benjamin Zeller <bzeller@suse.de> Systems Programmer SUSE Software Solutions Germany GmbH Frankenstraße 146 90461 Nürnberg Germany Geschäftsführer: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman (HRB 36809, AG Nürnberg)
On 13.09.2024 kl. 17.03 Benjamin Zeller wrote:
On 9/13/24 16:56, Klaus Vink Slott via openSUSE Users wrote:
Parallel operation: Selecting download after dependencies. So as soon as a package (including all necessary dependencies) is downloaded, it could start installing that - while continuing to download for the next batch.
This is not possible, librpm wants all packages for a transaction on disk, or at least all package headers. So we need to predownload all packages then start the installation. Thanks for enlightening me. So this is a limitation in librpm and not really in zypper. Anyway as I stated in the first place, for me this clearly a "cool to have" and not a real need.
What we will do is doing everything in parallel until that point... download, checksums, signatures, copying etc... That IS cool!
However the code as it is currently has two major problems:
- It has global state all over the place ( which is a problem for another feature we need ) - It is completely written with serial downloads in mind, not only the logic also the libzypp APIs, so there is no simple way to "just" parallelize things, we need a full source refactoring and this is what I spent most of my time on. Thanks. I have a little experience in how the playground changes when parallelizing a lot of interlocking tasks. Your work is much appreciated.
-- Regards Klaus
participants (4)
-
Benjamin Zeller -
bent fender -
David C. Rankin -
Klaus Vink Slott