[Bug 476430] New: Aria2c goes haywire when connection is refused due to too many connections
https://bugzilla.novell.com/show_bug.cgi?id=476430 Summary: Aria2c goes haywire when connection is refused due to too many connections Classification: openSUSE Product: openSUSE 11.2 Version: Alpha 0 Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: libzypp AssignedTo: zypp-maintainers@forge.provo.novell.com ReportedBy: dave.plater@yahoo.co.uk QAContact: qa@suse.de Found By: --- Created an attachment (id=273247) --> (https://bugzilla.novell.com/attachment.cgi?id=273247) aria2c log and y2log User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.5) Gecko/2008121300 SUSE/3.0.5-1.1 Firefox/3.0.5 I started yast2 software manager updating many packages from factory and on return I had a login refused message for ftp:ftp.is.co.za/mirror/opensuse/factory/repo/non-oss most probably due to too many connections but aria2c was still merrily downloading even after I clicked ignore and yast had closed. The package concerned was netbeans. I had to kill aria2c to stop it. Reproducible: Always Steps to Reproduce: 2. 3. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c1 --- Comment #1 from Peter Poeml <poeml@novell.com> 2009-02-17 06:31:50 MST --- I see this for the netbeans request: HTTP/1.0 401 Unauthorized^M Server: squid/2.7.STABLE5^M Date: Tue, 17 Feb 2009 11:42:35 GMT^M Content-Type: text/html^M Content-Length: 1356^M X-Squid-Error: ERR_FTP_FORBIDDEN 0^M X-Cache: MISS from Arbuthnot.plater^M X-Cache-Lookup: MISS from Arbuthnot.plater:3128^M Via: 1.0 Arbuthnot.plater:3128 (squid/2.7.STABLE5)^M Connection: close 2009-02-17 13:42:35.435906 DEBUG - CUID#11 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:42:35.436223 ERROR - CUID#11 - Download aborted. URI=ftp://ftp.is.co.za/mirror/opensuse/factory/repo/non-oss/suse/x86_64/netbeans-6.0-80.28.x86_64.rpm Exception: Authorization failed. After that, tons of "socket: read:1" log lines. I wonder, what did it download from where? Did it actually download something, or just get stuck in some kind of loop, trying to download from the proxy? 2009-02-17 13:47:13.275462 DEBUG - CUID#6 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.309701 DEBUG - CUID#10 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.344926 DEBUG - CUID#10 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.379168 DEBUG - CUID#6 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.413427 DEBUG - CUID#6 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.448657 DEBUG - CUID#6 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.482899 DEBUG - CUID#6 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.517136 DEBUG - CUID#6 - socket: read:1, write:0, hup:0, err:0 2009-02-17 13:47:13.552865 DEBUG - CUID#8 - socket: read:1, write:0, hup:0, err:0 But it seems to download really: 2009-02-17 13:47:13.552962 INFO - CUID#8 - The download for one segment completed successfully. 2009-02-17 13:47:13.553005 DEBUG - usedPieces.size()=4 2009-02-17 13:47:13.553023 DEBUG - Attach segment#65 to CUID#8. 2009-02-17 13:47:13.553037 DEBUG - index=65, length=1048576, segmentLength=1048576, writtenLength=0 And finally: 2009-02-17 13:47:25.961701 INFO - Saving the segment file /var/adm/mount/AP_0x00000004/suse/x86_64/netbeans-6.0-80.28.x86_64.rpm.aria2 2009-02-17 13:47:25.961967 INFO - The segment file was saved successfully. The file is extremely large -- 126MB, so it is at least logical that aria2c is busy with it for quite a while: -rw-r--r-- 2 mirror stage 126M Feb 13 23:10 /srv/ftp/pub/opensuse/factory/repo/non-oss/suse/x86_64/netbeans-6.0-80.28.x86_64.rpm So it looks to me as if - aria2c successfully downloaded the file - the one FTP error was spurious (happened only in one connection / connection attempt) and didn't hinder the download in the end - quitting yast2 didn't quit aria2c What do you think? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c2 --- Comment #2 from Peter Poeml <poeml@novell.com> 2009-02-17 06:33:56 MST --- Slight correction, after reading the log until the end -- which shows how aria2c is killed, and how yast2 already had removed the tempdir before where aria2c is downloading: - aria2c didn't complete the download, but was busy with it until the end, saving segment after segment to disk. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c3 --- Comment #3 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-17 06:57:03 MST --- I searched /var/ and all subdirectories for a file named netb* and found nothing. I don't know the size of the installation but ktraffic analyzer stats say I've downloaded 569 MB today and the installation was the only large download operation. /tmp has nothing bigger than 9.2MB. There is no trace of any file of significant size so it's a mystery where it went. I have 120.6 MB of files in /tmp so maybe they are all segments. I have 427.7 MB of files with todays date in my package cache and subtracted from ktraffic stats it leaves 141.3 MB which is about the right size for netbeans. It must be fragmented in /tmp -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c4 --- Comment #4 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-17 07:01:18 MST --- Sorry it can't be in /tmp the file dates are all too old. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c5 --- Comment #5 from Peter Poeml <poeml@novell.com> 2009-02-17 07:08:07 MST --- I don't expect that aria2c put anything in /tmp. aria2c normally creates an empty file of the size of the intended download, and starts to "fill" it with the downloaded segments. This happens in a temporary directory that yast2/libzypp creates. Once yast2/libzypp are terminated, they clean up this directory for good. This is also what aria2c said in it's logfile (/var/adm/something disappeared). So this is to be expected when yast2/libzypp are terminated, and aria2c with them. It just seems to me as if yast2 should have terminated aria2c, when it was terminated itself (when you quit it while the download happened). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c6 --- Comment #6 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-17 07:38:15 MST --- One problem is aria2c not exiting when it fails to write the segment file but the other one is libzypp not stopping aria2c when it detects an error which aria2c doesn't see as an error. If libzypp had ignored the login failed exception the file download would have completed. This is what happens if I download manually :- # aria2c ftp://anonymous@ftp.is.co.za/mirror/opensuse/factory/repo/non-oss/suse/x86_64/netbeans-6.0-80.28.x86_64.rpm 2009-02-17 16:23:10.370677 WARN - You may encounter the certificate verification error with HTTPS server. See --ca-certificate and --check-certificate option. [#1 SIZE:240.0KiB/125.8MiB(0%) CN:5 SPD:34.03KiB/s ETA:1h02m58s] 2009-02-17 16:23:18.771136 ERROR - CUID#11 - Download aborted. URI=ftp://anonymous@ftp.is.co.za/mirror/opensuse/factory/repo/non-oss/suse/x86_64/netbeans-6.0-80.28.x86_64.rpm Exception: Authorization failed. [#1 SIZE:1.0MiB/125.8MiB(0%) CN:4 SPD:47.01KiB/s ETA:45m19s]^C 2009-02-17 16:23:40.937001 NOTICE - Shutdown sequence commencing... Press Ctrl-C again for emergency shutdown. Download Results: gid|stat|avg speed |path/URI ===+====+===========+========================================================== 1|INPR| 44.0KiB/s|ftp://anonymous@ftp.is.co.za/mirror/opensuse/factory/repo/non-oss/suse/x86_64/netbeans-6.0-80.28.x86_64.rpm (4more) Status Legend: (INPR):download in-progress. aria2 will resume download if the transfer is restarted. If there are any errors, then see the log file. See '-l' option in help/man page for details. Aria2c gives the exeption but by tailing the log I can still see it downloading and netbeans has miraculously appeared in /var/cache/zypp/packages/factory/suse/x86_64/ with a size of 128855K which is the same as the remote package but it fails to install with Bad file descriptor. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c7 --- Comment #7 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-17 07:46:36 MST --- I tried aria2c http://download.opensuse.org/factory/repo/non-oss/suse/x86_64/netbeans-6.0-8... in the same directory as the existing file and in the absence of exceptions the download completed but the file still fails to install so it must have been corrupted at some stage. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c8 --- Comment #8 from Peter Poeml <poeml@novell.com> 2009-02-17 11:55:59 MST --- Hm, so I understand it like this: libzypp is reading aria2c's output, and regards something in it as error -- even though the output of aria2c is not supposed to indicate an error? Does this happen only when the proxy is in between? (Anyway, the metalink for that file (http://download.opensuse.org/factory/repo/non-oss/suse/x86_64/netbeans-6.0-8...) contains all the checksums, so it can virtually be expected that aria2c successfully downloads the file under any circumstances.) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c9 --- Comment #9 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-18 01:27:22 MST --- Created an attachment (id=273467) --> (https://bugzilla.novell.com/attachment.cgi?id=273467) zypper.log, tcpdump.txt, access.log (squid) and aria2c.log I performed a non proxy zypper in netbeans and then a proxy one. This problem doesn't occur with http only ftp, ftp.is.co.za only allows 4 connections from one ip. If I use http://download.opensuse.org the problem doesn't occur due to many mirrors being used. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c10 --- Comment #10 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-18 01:31:20 MST --- problem only occurs with proxy on -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c11 --- Comment #11 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-18 02:17:18 MST --- I see aria2c receives a 530 error from ftp.is.co.za without squid and handles it gracefully whereas with squid it receives HTTP/1.0 401 Unauthorized and confuses libzypp. I would think that aria2c should treat any error in one of multiple download threads the same and simply shut down the thread as it does when there is no proxy? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c12 --- Comment #12 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-19 23:23:35 MST --- Libzypp shouldn't show an error when aria2c is still downloading only when aria2c fails completely and outputs "Status Legend: (ERR):error occurred." -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c13 --- Comment #13 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-20 02:00:34 MST --- *** Bug 476427 has been marked as a duplicate of this bug. *** https://bugzilla.novell.com/show_bug.cgi?id=476427 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c14 --- Comment #14 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-20 02:46:33 MST --- Created an attachment (id=274219) --> (https://bugzilla.novell.com/attachment.cgi?id=274219) aria2c log and zypper log -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dave.plater@yahoo.co.uk added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c15 --- Comment #15 from Dave Plater <dave.plater@yahoo.co.uk> 2009-02-20 03:02:17 MST --- The Comment #14 attachment contains logs for the following zypper session which I think reinforces Comment #12 The following is the actual session where aria2c finds no package at one url but switches to a mirror and finds the package but libzypp stops on the first error. Fortunately aria2c had finished downloading by the time I pressed retry and the transaction completed :- Overall download size: 943.0 K. After the operation, additional 2.3 M will be used. Continue? [YES/no]: committing Retrieving package yast2-schema-2.18.0-1.1.noarch (1/3), 64.0 K (794.0 K unpacked) Retrieving: oss [done] Retrieving: media [done] Media source 'http://download.opensuse.org/factory/repo/oss/' does not contain the desired medium Abort, retry, ignore? [A/r/i]: r Retrieving: oss [done] Retrieving: yast2-schema-2.18.0-1.1.noarch.rpm [done] Installing: yast2-schema-2.18.0-1.1 [done] Retrieving package autoyast2-installation-2.18.1-1.1.noarch (2/3), 245.0 K (1.3 M unpacked) Retrieving: oss [done] Retrieving: autoyast2-installation-2.18.1-1.1.noarch.rpm [done] Installing: autoyast2-installation-2.18.1-1.1 [done] Retrieving package autoyast2-2.18.1-1.1.noarch (3/3), 634.0 K (1.5 M unpacked) Retrieving: oss [done] Retrieving: autoyast2-2.18.1-1.1.noarch.rpm [done] Installing: autoyast2-2.18.1-1.1 [done] committingCommitResult 3 (errors 0, remaining 0, srcremaining 0) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 Dave Plater <dave.plater@yahoo.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Aria2c goes haywire when |Libzypp doesn't handle |connection is refused due |aria2c errors correctly |to too many connections | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 Michael Andres <ma@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ma@novell.com AssignedTo|zypp-maintainers@forge.prov |dmacvicar@novell.com |o.novell.com | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=476430 User dmacvicar@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=476430#c16 Duncan Mac-Vicar <dmacvicar@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |DUPLICATE --- Comment #16 from Duncan Mac-Vicar <dmacvicar@novell.com> 2009-03-16 04:42:51 MST --- duplicate *** This bug has been marked as a duplicate of bug 481115 *** https://bugzilla.novell.com/show_bug.cgi?id=481115 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com