[Bug 537870] New: download speed seems grossly underestimated
http://bugzilla.novell.com/show_bug.cgi?id=537870 User poeml@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=537870#c2 Summary: download speed seems grossly underestimated Classification: openSUSE Product: openSUSE 11.2 Version: Factory Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: libzypp AssignedTo: zypp-maintainers@forge.provo.novell.com ReportedBy: poeml@novell.com QAContact: qa@suse.de Found By: --- When downloading packages, zypper seems to report a much lower speed than what's actually the case. I was noticing this when discussing too slow downloads and non-saturated links. When comparing the wall clock time that it takes to download a large package, the download via zypper isn't actually much slower than the speed I get with a manual aria2c call. There is a slight difference, but not one that large as the number printed by zypper would account for. root@linux-7q6o:~ # rm kernel-source-2.6.31-7.1.noarch.rpm* root@linux-7q6o:~ # time aria2c --follow-metalink=mem --check-integrity=true --file-allocation=none --server-stat-of=/var/cache/zypp/aria2.stats --server-stat-if=/var/cache/zypp/aria2.stats --uri-selector=adaptive --use-head=false --max-tries=5 --max-concurrent-downloads=2 --header="X-ZYpp-AnonymousId: da93627e-e5ca-42ba-8ebc-122fe15c9a8d" --header="X-ZYpp-DistributionFlavor: ftp" --header=Pragma: --connect-timeout=180 http://download.opensuse.org/factory/repo/oss/suse/noarch/kernel-source-2.6.... 2009-09-08 19:17:00.633052 NOTICE - ServerStat file /var/cache/zypp/aria2.stats loaded successfully. 2009-09-08 19:17:00.777625 NOTICE - Download complete: /root/kernel-source-2.6.31-7.1.noarch.rpm.metalink 2009-09-08 19:17:00.860744 ERROR - Checksum error detected. file=/root/kernel-source-2.6.31-7.1.noarch.rpm [#2 SIZE:66.2MiB/67.2MiB(98%) CN:5 SPD:1.5MiBs] 2009-09-08 19:17:28.774711 NOTICE - Download complete: /root/kernel-source-2.6.31-7.1.noarch.rpm 2009-09-08 19:17:28.802228 NOTICE - ServerStat file /var/cache/zypp/aria2.stats saved successfully. Download Results: gid|stat|avg speed |path/URI ===+====+===========+=========================================================== 1| OK| n/a|[MEMORY]kernel-source-2.6.31-7.1.noarch.rpm.metalink 2| OK| 2.4MiB/s|/root/kernel-source-2.6.31-7.1.noarch.rpm Status Legend: (OK):download completed. aria2c --follow-metalink=mem --check-integrity=true --file-allocation=none 0.46s user 23.55s system 84% cpu 28.312 total aria2c shows 1.5 MB/s being transferred, and is done after 30 seconds. root@linux-7q6o:~ # rm -r /var/cache/zypp/packages/factory root@linux-7q6o:~ # time zypper in -f -d kernel-source Loading repository data... Reading installed packages... Resolving package dependencies... The following NEW package is going to be installed: kernel-source 1 new package to install. Overall download size: 67.3 MiB. After the operation, additional 334.1 MiB will be used. Continue? [y/n/?] (y): y Retrieving package kernel-source-2.6.31-7.1.noarch (1/1), 67.3 MiB (334.1 MiB unpacked) Retrieving: kernel-source-2.6.31-7.1.noarch.rpm [done (16.5 KiB/s)] zypper in -f -d kernel-source 1.32s user 32.19s system 75% cpu 44.538 total zypper shows 16.5 kb/s being transferred, and is done after 44 seconds (some the time is zypper startup time). -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 User ma@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=537870#c1 Michael Andres <ma@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 - None |P3 - Medium CC| |ma@novell.com --- Comment #1 from Michael Andres <ma@novell.com> 2009-11-12 06:07:09 MST --- We must check MediaAria2c::getFileCopy. Average speed computation is for sure wrong if aria changes the reporting unit (KiB/MiB) during download. Zypp provides a ByteCount class to do such computation with changing units easily: ByteCount bc( 15, K ); bc += ByteCount( 5, M ); -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 User jkupec@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=537870#c2 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jkupec@novell.com --- Comment #2 from Ján Kupec <jkupec@novell.com> 2009-11-12 06:12:16 MST --- IIRC, i anticipated this the last time i was fixing parsing of aria's output. The question is whether it stayed the same since then (or whether i made a mistake). If it changed, that's too bad and upstream should consider providing more solid output for machine consumption. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 User jkupec@novell.com added comment http://bugzilla.novell.com/show_bug.cgi?id=537870#c3 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koenig@linux.de --- Comment #3 from Ján Kupec <jkupec@novell.com> 2009-11-12 06:15:43 MST --- *** Bug 554236 has been marked as a duplicate of this bug. *** http://bugzilla.novell.com/show_bug.cgi?id=554236 -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c5 Ladislav Slezak <lslezak@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |johannesobermayr@gmx.de --- Comment #5 from Ladislav Slezak <lslezak@novell.com> 2009-11-24 15:57:58 UTC --- *** Bug 551647 has been marked as a duplicate of this bug. *** http://bugzilla.novell.com/show_bug.cgi?id=551647 -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c6 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P3 - Medium |P2 - High Status|NEW |ASSIGNED AssignedTo|zypp-maintainers@forge.prov |jkupec@novell.com |o.novell.com | --- Comment #6 from Ján Kupec <jkupec@novell.com> 2009-11-24 17:11:11 UTC --- Oops. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c7 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |NEEDINFO Info Provider| |tatsuhiro.t@gmail.com --- Comment #7 from Ján Kupec <jkupec@novell.com> 2009-11-25 15:07:28 UTC --- OK, my mistake. I also used bad constant to get bytes from megabytes :O( Also, currently the code only expects KiBs and MiBs in aria's output, but also Bs (and probably also the higher multiples) can be reported. Tatsuhiro, it would be most useful for machine use, if aria had an option to switch to machine-friendly output, where e.g. only plain, easily parsable bytes would be reported instead of all the B/K/M/G/T-bytes. Does aria already have this? If not, could you please consider adding something like that? -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c8 --- Comment #8 from Ján Kupec <jkupec@novell.com> 2009-11-25 15:28:27 UTC --- @ma: ByteCount does not have a floating point constructor - "3.9MiBs" would loose its 0.9 megs if i used ByteCount(3.9, ByteCount::M). Should i add a constructor taking float or double? -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c9 Tatsuhiro Tsujikawa <tatsuhiro.t@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tatsuhiro.t@gmail.com --- Comment #9 from Tatsuhiro Tsujikawa <tatsuhiro.t@gmail.com> 2009-11-26 12:44:01 UTC --- Currently, aria2 outputs speed and size in MiB, KiB or B. I think machine friendly output is good idea. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c10 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |ASSIGNED Info Provider|tatsuhiro.t@gmail.com | Status Whiteboard| |in-git --- Comment #10 from Ján Kupec <jkupec@novell.com> 2009-11-26 15:19:59 UTC --- Thanx for the info! So for now zypp (6.24.1) will look for MiBs, KiBs or Bs. If in the future aria will get the machine friendly output, we can adapt the code then. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c11 --- Comment #11 from Michael Andres <ma@novell.com> 2009-11-26 15:32:02 UTC --- (In reply to comment #8)
@ma: ByteCount does not have a floating point constructor - "3.9MiBs" would loose its 0.9 megs if i used ByteCount(3.9, ByteCount::M)
You can do ByteCount( 3.9 * ByteCount::M.factor() ). -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c12 --- Comment #12 from Ján Kupec <jkupec@novell.com> 2009-11-26 15:47:48 UTC --- True. Nevertheless, using ByteCount won't help this particular code too much, so i'd just leave it as it is now. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c13 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED Status Whiteboard|in-git | --- Comment #13 from Ján Kupec <jkupec@novell.com> 2009-12-03 14:17:45 UTC --- fixed in 6.24.1 to be released in SLE11-SP1 and openSUSE 11.2 maintenance update. closing. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c14 --- Comment #14 from Johannes Obermayr <johannesobermayr@gmx.de> 2010-01-03 13:42:09 UTC --- (In reply to comment #13)
fixed in 6.24.1 to be released in SLE11-SP1 and openSUSE 11.2 maintenance update. closing.
When will it appear in openSUSE:11.2:Update? -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=537870 http://bugzilla.novell.com/show_bug.cgi?id=537870#c15 Swamp Workflow Management <swamp@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status Whiteboard| |maint:released:11.2:32913 --- Comment #15 from Swamp Workflow Management <swamp@suse.com> 2010-05-06 15:04:39 UTC --- Update released for: PackageKit, PackageKit-debuginfo, PackageKit-debugsource, PackageKit-devel, PackageKit-doc, PackageKit-lang, libpackagekit-glib12, libpackagekit-glib12-debuginfo, libpackagekit-glib12-devel, libpackagekit-qt12, libpackagekit-qt12-debuginfo, libpackagekit-qt12-devel, libsatsolver, libsatsolver-debugsource, libsatsolver-demo, libsatsolver-demo-debuginfo, libsatsolver-devel, libsatsolver-devel-debuginfo, libzypp, libzypp-bindings, libzypp-debuginfo, libzypp-debugsource, libzypp-devel, perl-satsolver, perl-satsolver-debuginfo, perl-zypp, python-satsolver, python-satsolver-debuginfo, python-zypp, ruby-satsolver, ruby-satsolver-debuginfo, ruby-zypp, ruck, satsolver-tools, satsolver-tools-debuginfo, yast2-ncurses, yast2-ncurses-debuginfo, yast2-ncurses-debugsource, yast2-ncurses-devel, yast2-ncurses-pkg, yast2-ncurses-pkg-debuginfo, yast2-ncurses-pkg-debugsource, yast2-qt-pkg, yast2-qt-pkg-debuginfo, yast2-qt-pkg-debugsource, zypp-testsuite-tools, zypper, zypper-debuginfo, zypper-debugsource Products: openSUSE 11.2 (debug, i586, x86_64) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com