[Bug 602434] New: Broken handling of metalink files for files without mirrors
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c0 Summary: Broken handling of metalink files for files without mirrors Classification: openSUSE Product: openSUSE.org Version: unspecified Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Download Infrastructure AssignedTo: pascal.bleser@opensuse.org ReportedBy: mrueckert@novell.com QAContact: adrian@novell.com CC: lrupp@novell.com, mls@novell.com, pascal.bleser@opensuse.org, andrea@opensuse.org, poeml@cmdline.net Found By: --- Blocker: --- [[[ Retrieving repository 'openSUSE-11.2-Update' metadata [error] Repository 'openSUSE-11.2-Update' is invalid. Can't provide /repodata/1ef1cd51fee27c4731999eb4710431f99d1814b2-susedata.xml.gz : Can't hardlink/copy /var/adm/mount/AP_0x00000001/repodata/1ef1cd51fee27c4731999eb4710431f99d1814b2-susedata.xml.gz to /var/cache/zypp/raw/repo-updateG2UwkO Please check if the URIs defined for this repository are pointing to a valid repository. Skipping repository 'openSUSE-11.2-Update' because of the above error. Some of the repositories have not been refreshed because of an error. ]]] After some investigations 1. If the mirrordb has no mirrors for the file the <resources> section is empty. I wonder if mod_mirrorbrain should add its own instance there. (just in the case of mirror_cnt == 0 maybe) 2. aria2c handles that situation in an unexpected way: a) it downloads metalink file b) sees empty resources section c) exits with returncode 0 d) yast thinks the file is downloaded successfully. e) fails on hardlinking/copying I think the natural expectation here would be: If the file isnt downloaded properly dont return with 0. I think Pascal wins this bug for the aria fix. :) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c1 --- Comment #1 from Peter Poeml <poeml@cmdline.net> 2010-05-04 15:37:17 UTC --- (In reply to comment #0)
After some investigations 1. If the mirrordb has no mirrors for the file the <resources> section is empty.
Not in your case. That's the case *only* if a client tries to access a metalink by appending .metalink to an URL (which aria2c does not), *and* there are no fallback mirrors configured (which you could do, if you'd care). aria2c & MirrorBrain negotiats the type of the response transparently, which means that (if there are no mirrors) aria2c simply receives the actual content instead of a metalink. (That's how it's supposed to work anyway; if you see something different, please let me know!)
2. aria2c handles that situation in an unexpected way:
If you are really sure about that (are you?): A metalink without resources might be a good reason to reject it. Or just to ignore it. On the other hand, MirrorBrain could return a 404, maybe. Anyway, it seems that you need to debug a different problem. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c2 --- Comment #2 from Michael Schröder <mls@novell.com> 2010-05-04 15:52:31 UTC --- Argh, too bad that I didn't save the strace. We're seeing something different, that's why we're letting you know with this bug report. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c3 --- Comment #3 from Michael Schröder <mls@novell.com> 2010-05-04 17:41:42 UTC --- This is about mirrorbrain, thus directed at Peter. Pascal, please ignore ;-) The error log contains some suspicious lines: [mod_mirrorbrain] could not chose a server. Shouldn't have happened. (please fix the typo, it should be "choose") Looking at the code it seems to me that some mirror was found, but it didn't match any the 'if' statements, and thus could not be classified. As the comment says that should not happen. ;-) Specifying a fallback would not help in this (very rare) case. (Btw, it's strange that a curl <filename>.metalink returns the file content if no mirror is found. You should handle this somewhat different.) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c4 Michael Schröder <mls@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Normal |Major --- Comment #4 from Michael Schröder <mls@novell.com> 2010-05-04 17:45:19 UTC --- (Uh, it doesn't seem to be so rare. This really needs to be fixed ASAP, as it breaks zypper.) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c5 Michael Schröder <mls@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Major |Normal --- Comment #5 from Michael Schröder <mls@novell.com> 2010-05-05 14:20:28 UTC --- (Simple workaround added, deployed to pontifex. So it's no longer a critical issue.) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c6 --- Comment #6 from Peter Poeml <poeml@cmdline.net> 2010-05-06 11:46:00 UTC --- This report doesn't contain many details about what's going on. The zypper error message could mean anything -- I can't interprete that at all. So frankly, I don't have much of an idea what you are seeing and what could be wrong. Can you provide details how to reproduce the problem? And could you please attach the debug log for a request? Debug logging is enabled by setting MirrorBrainDebug On for a directory. (It's not recommended to do this for the whole server, because it creates a flood of logs. Use it on a subdirectory only.) -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c Peter Poeml <poeml@cmdline.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |mls@novell.com -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c7 Michael Schröder <mls@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|mls@novell.com | --- Comment #7 from Michael Schröder <mls@novell.com> 2010-05-06 14:06:09 UTC --- Just look at your code. You get a list of mirrors from the database and then classify them into different arrays (one large "if" cascade). There's no final 'else' clause (and there's also the possibility that the mirror is deemed unusable), so there's a chance that all of the classification arrays stay empty. In that case you'll return an empty resource list. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c8 Peter Poeml <poeml@cmdline.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |mls@novell.com --- Comment #8 from Peter Poeml <poeml@cmdline.net> 2010-05-06 14:26:27 UTC --- The question is why that happens. To find out, I need details. Could you provide the requested information? -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c9 Michael Schröder <mls@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|mls@novell.com | --- Comment #9 from Michael Schröder <mls@novell.com> 2010-05-06 14:56:58 UTC --- Nope, I can't. Sorry. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c10 --- Comment #10 from Marcus Rückert <mrueckert@novell.com> 2010-05-06 15:06:42 UTC --- steps to reproduce: 1. add a new file to the tree 2. add the file to the DB 3. file has no mirrors yet 4. try to download the metalink file for it 5. you will see the <resources> section is empty. does that help? we cant really reproduce it on download.o.o as it is kind of a race situation and i dont want to fiddle around with that. but the steps should help to reproduce it. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=602434 http://bugzilla.novell.com/show_bug.cgi?id=602434#c11 --- Comment #11 from Michael Schröder <mls@novell.com> 2010-05-06 15:18:15 UTC --- No, that doesn't reproduce it. You need to have a mirror in the database that doesn't qualify for the request, e.g. where region_only is set. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=602434 https://bugzilla.novell.com/show_bug.cgi?id=602434#c12 Peter Poeml <poeml@cmdline.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED --- Comment #12 from Peter Poeml <poeml@cmdline.net> 2010-09-01 15:19:54 UTC --- I see. This can indeed happen when there is only a region_only mirror from a different region, and the resource is negotiated to be represented by a metalink. (Not for a metalink requested explicitely by .meta4 or .metalink extension.) I can easily reproduce that. Thanks for the details, and for the workaround you added. Since r8011 (and later changed in r8086), this case is handled: case META4: case METALINK: if (meta_negotiated) { debugLog(r, cfg, "would have to send empty metalink... -> deliver directly"); return DECLINED; } else { debugLog(r, cfg, "would have to send empty metalink... -> 404"); ap_log_rerror(APLOG_MARK, APLOG_WARNING, 0, r, "[mod_mirrorbrain] Can't send metalink for %s (no mirrors)", filename); return HTTP_NOT_FOUND; } Other representations (e.g., triggered by appending .sha1 extension, or torrents) don't depend on mirrors, so they can be generated regardless. The fix will be in 2.13.0. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com