http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c0
Summary: Broken handling of metalink files for files without mirrors Classification: openSUSE Product: openSUSE.org Version: unspecified Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Download Infrastructure AssignedTo: pascal.bleser@opensuse.org ReportedBy: mrueckert@novell.com QAContact: adrian@novell.com CC: lrupp@novell.com, mls@novell.com, pascal.bleser@opensuse.org, andrea@opensuse.org, poeml@cmdline.net Found By: --- Blocker: ---
[[[ Retrieving repository 'openSUSE-11.2-Update' metadata [error] Repository 'openSUSE-11.2-Update' is invalid. Can't provide /repodata/1ef1cd51fee27c4731999eb4710431f99d1814b2-susedata.xml.gz : Can't hardlink/copy /var/adm/mount/AP_0x00000001/repodata/1ef1cd51fee27c4731999eb4710431f99d1814b2-susedata.xml.gz to /var/cache/zypp/raw/repo-updateG2UwkO Please check if the URIs defined for this repository are pointing to a valid repository. Skipping repository 'openSUSE-11.2-Update' because of the above error. Some of the repositories have not been refreshed because of an error. ]]]
After some investigations 1. If the mirrordb has no mirrors for the file the <resources> section is empty.
I wonder if mod_mirrorbrain should add its own instance there. (just in the case of mirror_cnt == 0 maybe)
2. aria2c handles that situation in an unexpected way: a) it downloads metalink file b) sees empty resources section c) exits with returncode 0 d) yast thinks the file is downloaded successfully. e) fails on hardlinking/copying
I think the natural expectation here would be: If the file isnt downloaded properly dont return with 0.
I think Pascal wins this bug for the aria fix. :)
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c1
--- Comment #1 from Peter Poeml poeml@cmdline.net 2010-05-04 15:37:17 UTC --- (In reply to comment #0)
After some investigations
- If the mirrordb has no mirrors for the file the <resources> section is
empty.
Not in your case. That's the case *only* if a client tries to access a metalink by appending .metalink to an URL (which aria2c does not), *and* there are no fallback mirrors configured (which you could do, if you'd care).
aria2c & MirrorBrain negotiats the type of the response transparently, which means that (if there are no mirrors) aria2c simply receives the actual content instead of a metalink.
(That's how it's supposed to work anyway; if you see something different, please let me know!)
- aria2c handles that situation in an unexpected way:
If you are really sure about that (are you?): A metalink without resources might be a good reason to reject it. Or just to ignore it. On the other hand, MirrorBrain could return a 404, maybe.
Anyway, it seems that you need to debug a different problem.
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c2
--- Comment #2 from Michael Schröder mls@novell.com 2010-05-04 15:52:31 UTC --- Argh, too bad that I didn't save the strace. We're seeing something different, that's why we're letting you know with this bug report.
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c3
--- Comment #3 from Michael Schröder mls@novell.com 2010-05-04 17:41:42 UTC --- This is about mirrorbrain, thus directed at Peter. Pascal, please ignore ;-)
The error log contains some suspicious lines:
[mod_mirrorbrain] could not chose a server. Shouldn't have happened.
(please fix the typo, it should be "choose")
Looking at the code it seems to me that some mirror was found, but it didn't match any the 'if' statements, and thus could not be classified.
As the comment says that should not happen. ;-) Specifying a fallback would not help in this (very rare) case.
(Btw, it's strange that a curl <filename>.metalink returns the file content if no mirror is found. You should handle this somewhat different.)
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c4
Michael Schröder mls@novell.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Severity|Normal |Major
--- Comment #4 from Michael Schröder mls@novell.com 2010-05-04 17:45:19 UTC --- (Uh, it doesn't seem to be so rare. This really needs to be fixed ASAP, as it breaks zypper.)
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c5
Michael Schröder mls@novell.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Severity|Major |Normal
--- Comment #5 from Michael Schröder mls@novell.com 2010-05-05 14:20:28 UTC --- (Simple workaround added, deployed to pontifex. So it's no longer a critical issue.)
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c6
--- Comment #6 from Peter Poeml poeml@cmdline.net 2010-05-06 11:46:00 UTC --- This report doesn't contain many details about what's going on. The zypper error message could mean anything -- I can't interprete that at all. So frankly, I don't have much of an idea what you are seeing and what could be wrong.
Can you provide details how to reproduce the problem?
And could you please attach the debug log for a request? Debug logging is enabled by setting MirrorBrainDebug On for a directory. (It's not recommended to do this for the whole server, because it creates a flood of logs. Use it on a subdirectory only.)
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c
Peter Poeml poeml@cmdline.net changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |mls@novell.com
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c7
Michael Schröder mls@novell.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|mls@novell.com |
--- Comment #7 from Michael Schröder mls@novell.com 2010-05-06 14:06:09 UTC --- Just look at your code. You get a list of mirrors from the database and then classify them into different arrays (one large "if" cascade). There's no final 'else' clause (and there's also the possibility that the mirror is deemed unusable), so there's a chance that all of the classification arrays stay empty. In that case you'll return an empty resource list.
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c8
Peter Poeml poeml@cmdline.net changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |mls@novell.com
--- Comment #8 from Peter Poeml poeml@cmdline.net 2010-05-06 14:26:27 UTC --- The question is why that happens. To find out, I need details. Could you provide the requested information?
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c9
Michael Schröder mls@novell.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|mls@novell.com |
--- Comment #9 from Michael Schröder mls@novell.com 2010-05-06 14:56:58 UTC --- Nope, I can't. Sorry.
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c10
--- Comment #10 from Marcus Rückert mrueckert@novell.com 2010-05-06 15:06:42 UTC --- steps to reproduce: 1. add a new file to the tree 2. add the file to the DB 3. file has no mirrors yet 4. try to download the metalink file for it 5. you will see the <resources> section is empty.
does that help? we cant really reproduce it on download.o.o as it is kind of a race situation and i dont want to fiddle around with that. but the steps should help to reproduce it.
http://bugzilla.novell.com/show_bug.cgi?id=602434
http://bugzilla.novell.com/show_bug.cgi?id=602434#c11
--- Comment #11 from Michael Schröder mls@novell.com 2010-05-06 15:18:15 UTC --- No, that doesn't reproduce it. You need to have a mirror in the database that doesn't qualify for the request, e.g. where region_only is set.
https://bugzilla.novell.com/show_bug.cgi?id=602434
https://bugzilla.novell.com/show_bug.cgi?id=602434#c12
Peter Poeml poeml@cmdline.net changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED
--- Comment #12 from Peter Poeml poeml@cmdline.net 2010-09-01 15:19:54 UTC --- I see. This can indeed happen when there is only a region_only mirror from a different region, and the resource is negotiated to be represented by a metalink. (Not for a metalink requested explicitely by .meta4 or .metalink extension.)
I can easily reproduce that. Thanks for the details, and for the workaround you added.
Since r8011 (and later changed in r8086), this case is handled:
case META4: case METALINK: if (meta_negotiated) { debugLog(r, cfg, "would have to send empty metalink... -> deliver directly"); return DECLINED; } else { debugLog(r, cfg, "would have to send empty metalink... -> 404"); ap_log_rerror(APLOG_MARK, APLOG_WARNING, 0, r, "[mod_mirrorbrain] Can't send metalink for %s (no mirrors)", filename); return HTTP_NOT_FOUND; }
Other representations (e.g., triggered by appending .sha1 extension, or torrents) don't depend on mirrors, so they can be generated regardless.
The fix will be in 2.13.0.
-
bugzilla_noreply＠novell.com