[Bug 564816] New: when aria2 fails to download, zypper misinterprets the code and acts weird
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c0 Summary: when aria2 fails to download, zypper misinterprets the code and acts weird Classification: openSUSE Product: openSUSE 11.2 Version: Final Platform: Other OS/Version: Other Status: NEW Severity: Major Priority: P5 - None Component: libzypp AssignedTo: zypp-maintainers@forge.provo.novell.com ReportedBy: jnelson-suse@jamponi.net QAContact: qa@suse.de Found By: --- Blocker: --- User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5) Gecko/20091103 SUSE/3.5.5-1.1.2 Firefox/3.5.5 Today zypper tried to download (using aria2) some packages. The download appeard to fail (but should not have, keep reading) but zypper for some reason tried to go ahead anyway and use a package that didn't download, which results in some unexpected errors. This violates the principle of least surprise. zypper should use the exit code of aria2 to determine success or failure rather than trying to parse the output of aria2, which is not by any stretch of the imagination fixed in stone. Example: Continue? [y/n/?] (y): committing Retrieving package pam-modules-32bit-11.2-4.2.1.x86_64 (1/6), 38.0 KiB (87.0 KiB unpacked) Retrieving delta: /rpm/x86_64/pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm, 11.0 KiB Retrieving: http://download.opensuse.org/update/11.2/rpm/x86_64/pam-modules-32bit-11.2-4... [startingRetrieving: http://download.opensuse.org/update/11.2/rpm/x86_64/pam-modules-32bit-11.2-4... [0% (0 B/Retrieving: http://download.opensuse.org/update/11.2/rpm/x86_64/pam-modules-32bit-11.2-4... [done (0 B/s)] [openSUSE-11.2-Update|http://download.opensuse.org/update/11.2/] Can't provide file './rpm/x86_64/pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm' from repository 'openSUSE-11.2-Update' History: - Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update - Can't provide ./rpm/x86_64/pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm : Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update Retrieving: http://download.opensuse.org/update/11.2/rpm/x86_64/pam-modules-32bit-11.2-4... [done (0 B/s)] Failed to provide Package pam-modules-32bit-11.2-4.2.1. Do you want to retry retrieval? [openSUSE-11.2-Update|http://download.opensuse.org/update/11.2/] Can't provide file './rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm' from repository 'openSUSE-11.2-Update' History: - Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update - Can't provide ./rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm : Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update Abort, retry, ignore? [a/r/i] (a): part of zypper.log: 2009-12-15 09:15:09 <1> worklaptop(23739) [zypp:fetcher] Fetcher.cc(provideFromCache):348 start fetcher with 1 cache directories. 2009-12-15 09:15:09 <1> worklaptop(23739) [zypp:fetcher] Fetcher.cc(provideToDest):546 Not found in cache, downloading 2009-12-15 09:15:09 <1> worklaptop(23739) [zypp++] MediaSetAccess.cc(provide):201 Going to try to provide file /rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm from media number 1 2009-12-15 09:15:09 <1> worklaptop(23739) [zypp++] MediaManager.cc(checkDesired):112 checkDesired(1): desired (cached) 2009-12-15 09:15:09 <1> worklaptop(23739) [zypp++] ExternalProgram.cc(start_program):221 Executing 'aria2c' '--user-agent=ZYpp 6.21.2 (aria2 version 1.5.2) openSUSE-11.2-x86_64' '--summary-interval= 1' '--follow-metalink=mem' '--check-integrity=true' '--file-allocation=none' '--server-stat-of=/var/cache/zypp/aria2.stats' '--server-stat-if=/var/cache/zypp/aria2.stats' '--uri-selector=adaptive' ' --use-head=false' '--max-tries=5' '--max-concurrent-downloads=2' '--header=X-ZYpp-AnonymousId: 871b8bec-1cc4-42e9-bb3c-6ad8c58505b4' '--header=X-ZYpp-DistributionFlavor: ftp' '--header=Pragma:' '--c onnect-timeout=180' '--dir=/var/adm/mount/AP_0x0000002b/rpm/x86_64' 'http://download.opensuse.org/update/11.2/rpm/x86_64/pam-modules-32bit-11.2-4...' 2009-12-15 09:15:09 <1> worklaptop(23739) [zypp++] ExternalProgram.cc(start_program):373 pid 23897 launched 2009-12-15 09:15:17 <3> worklaptop(23739) [zypp] MediaAria2c.cc(getFileCopy):386 Can't parse progress from '[#1 SIZE:0B/0B CN:1 SPD:0Bs]' 2009-12-15 09:15:17 <1> worklaptop(23739) [zypp++] ExternalProgram.cc(checkStatus):431 Pid 23897 successfully completed 2009-12-15 09:15:17 <1> worklaptop(23739) [zypp++] MediaHandler.cc(provideFile):1004 provideFile(./rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm) 2009-12-15 09:15:17 <1> worklaptop(23739) [zypp] PathInfo.cc(hardlinkCopy):833 hardlinkCopy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm -> /var/cache/zypp/packages/openSUSE-11.2-Update/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm 2009-12-15 09:15:17 <2> worklaptop(23739) [zypp] PathInfo.cc(__Log_Result):299 FAILED: (2)No such file or directory 2009-12-15 09:15:17 <3> worklaptop(23739) [zypp:fetcher] Fetcher.cc(provideToDest):560 /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm does not exist 2009-12-15 09:15:17 <1> worklaptop(23739) [zypp++] MediaSetAccess.cc(releaseFile):85 Going to release file /rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm from media number 1 2009-12-15 09:15:17 <5> worklaptop(23739) [zypp] Exception.cc(log):133 Fetcher.cc(provideToDest):565 THROW: Fetcher.cc(provideToDest):565: Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update 2009-12-15 09:15:17 <5> worklaptop(23739) [zypp] Exception.cc(log):133 Fetcher.cc(provideToDest):572 CAUGHT: Fetcher.cc(provideToDest):565: Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update 2009-12-15 09:15:17 <5> worklaptop(23739) [zypp] Exception.cc(log):133 Fetcher.cc(provideToDest):582 RETHROW: Fetcher.cc(provideToDest):565: Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update 2009-12-15 09:15:17 <5> worklaptop(23739) [zypp] Exception.cc(log):133 RepoProvideFile.cc(provideFile):309 CAUGHT: Fetcher.cc(provideToDest):582: Can't hardlink/copy /var/adm/mount/AP_0x0000002b/rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm to /var/cache/zypp/packages/openSUSE-11.2-Update 2009-12-15 09:15:17 <2> worklaptop(23739) [zypp] RepoProvideFile.cc(provideFile):313 Trying next url 2009-12-15 09:15:17 <5> worklaptop(23739) [zypp] Exception.cc(log):133 RepoProvideFile.cc(provideFile):318 THROW: RepoProvideFile.cc(provideFile):318: [openSUSE-11.2-Update|http://download.opensuse.org/update/11.2/] Can't provide file './rpm/x86_64/pam-modules-32bit-11.2-4.2.1.x86_64.rpm' from repository 'openSUSE-11.2-Update' 2009-12-15 09:15:17 <3> worklaptop(23739) [zypp] PackageProvider.cc(providePackage):115 Failed to provide Package (37080)pam-modules-32bit-11.2-4.2.1.x86_64(openSUSE-11.2-Update) 2009-12-15 09:15:17 <1> worklaptop(23739) [zypper++] repo.h(problem):186 error report When I execute the same commandline: aria2c --user-agent='ZYpp 6.21.2 (aria2 version 1.5.2) openSUSE-11.2-x86_64' --summary-interval=1 --follow-metalink=mem --check-integrity=true --file-allocation=none --server-stat-of=/var/cache/zypp/aria2.stats --server-stat-if=/var/cache/zypp/aria2.stats --uri-selector=adaptive --use-head=false --max-tries=5 --max-concurrent-downloads=2 --header='X-ZYpp-DistributionFlavor: ftp' --header='Pragma:' --connect-timeout=180 --dir=/var/adm/mount/AP_0x0000002b/rpm/x86_64 http://download.opensuse.org/update/11.2/rpm/x86_64/pam-modules-32bit-11.2-4... it does download correctly, and exits with exit code 0. I set the severity to major because this appears to be preventing zypper from downloading updates. Reproducible: Always Steps to Reproduce: 1. 2. 3. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c1 Michael Andres <ma@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 - None |P2 - High CC| |ma@novell.com --- Comment #1 from Michael Andres <ma@novell.com> 2009-12-16 17:34:33 UTC --- Looks like the problem occurs after the download, when moving the file to the cachedir. The download itself was successful. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c2 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO Info Provider| |jnelson-suse@jamponi.net --- Comment #2 from Ján Kupec <jkupec@novell.com> 2010-02-10 13:29:35 UTC --- Wow, i'm staring at the code but i can't see how could there be no file after successful download... I see no code that could remove the file after aria finishes in MediaAria2c.cc. The program then continues in MediaHandler/MediaSetAccess (no clue there either), and ends in Fetcher doing: if ( filesystem::hardlinkCopy( tmp_file, dest_full_path ) != 0 ) { if ( ! PathInfo(tmp_file).isExist() ) ERR << tmp_file << " does not exist" << endl; .. ZYPP_THROW( Exception("Can't hardlink/copy " + tmp_file.asString() + " to " + dest_dir.asString())); } How come? Jon, if you still can reproduce this, can you help us hunt it down? Either using gdb, or by inserting some checks into the code. Also, i never seen such error so far except of this report. Could it be something specific with your aria2c setup? Perhaps your aria is deleting the file after downloading :O) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|when aria2 fails to |file not found after |download, zypper |successful download by |misinterprets the code and |aria2c |acts weird | -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c3 Michael Andres <ma@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Info Provider|jnelson-suse@jamponi.net |tatsuhiro.t@gmail.com --- Comment #3 from Michael Andres <ma@novell.com> 2010-02-11 17:08:20 UTC --- @Ttatsuhiro: There is a similar report in bug #567993. There aria2c reports '[Netrc.cc:103] Netrc:parse error. %s encounterd where 'machine' or 'default'. Nevertheless it exits with 0, but there seems to be no file being no file downloaded. May it be the aria2c return code is not reliable? @Jano: It won't hurt, if we test for the file being present even if aria returned 0, so we at least get a proper error report to the user. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c4 --- Comment #4 from Ján Kupec <jkupec@novell.com> 2010-02-11 17:11:22 UTC --- (In reply to comment #3)
@Jano: It won't hurt, if we test for the file being present even if aria returned 0, so we at least get a proper error report to the user.
Yup, i thought about that, too. I'll add the check. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c5 Tatsuhiro Tsujikawa <tatsuhiro.t@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tatsuhiro.t@gmail.com --- Comment #5 from Tatsuhiro Tsujikawa <tatsuhiro.t@gmail.com> 2010-02-12 13:07:03 UTC --- bug #567993 occurred before download begins. It turns out that aria2 cannot handle return code well if error occurred before download begins. This is a bug of aria2. In this issue, aria2 seemed to begin downloading because the log said: MediaAria2c.cc(getFileCopy):386 Can't parse progress from '[#1 SIZE:0B/0B CN:1 SPD:0Bs]' So this is not the same issue. If you can check aria2 log, it tells you what aria2 actually downloaded. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c6 Michael Andres <ma@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |RESOLVED Info Provider|tatsuhiro.t@gmail.com | Resolution| |FIXED --- Comment #6 from Michael Andres <ma@novell.com> 2010-02-23 16:36:54 UTC --- Fixed libzypp-6.30.3 to check whether a file was actually downloaded. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=564816 http://bugzilla.novell.com/show_bug.cgi?id=564816#c7 Swamp Workflow Management <swamp@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status Whiteboard| |maint:released:11.2:32913 --- Comment #7 from Swamp Workflow Management <swamp@suse.com> 2010-05-06 15:05:29 UTC --- Update released for: PackageKit, PackageKit-debuginfo, PackageKit-debugsource, PackageKit-devel, PackageKit-doc, PackageKit-lang, libpackagekit-glib12, libpackagekit-glib12-debuginfo, libpackagekit-glib12-devel, libpackagekit-qt12, libpackagekit-qt12-debuginfo, libpackagekit-qt12-devel, libsatsolver, libsatsolver-debugsource, libsatsolver-demo, libsatsolver-demo-debuginfo, libsatsolver-devel, libsatsolver-devel-debuginfo, libzypp, libzypp-bindings, libzypp-debuginfo, libzypp-debugsource, libzypp-devel, perl-satsolver, perl-satsolver-debuginfo, perl-zypp, python-satsolver, python-satsolver-debuginfo, python-zypp, ruby-satsolver, ruby-satsolver-debuginfo, ruby-zypp, ruck, satsolver-tools, satsolver-tools-debuginfo, yast2-ncurses, yast2-ncurses-debuginfo, yast2-ncurses-debugsource, yast2-ncurses-devel, yast2-ncurses-pkg, yast2-ncurses-pkg-debuginfo, yast2-ncurses-pkg-debugsource, yast2-qt-pkg, yast2-qt-pkg-debuginfo, yast2-qt-pkg-debugsource, zypp-testsuite-tools, zypper, zypper-debuginfo, zypper-debugsource Products: openSUSE 11.2 (debug, i586, x86_64) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com