https://bugzilla.novell.com/show_bug.cgi?id=483189 User abittner@stud.fh-heilbronn.de added comment https://bugzilla.novell.com/show_bug.cgi?id=483189#c1 --- Comment #1 from andreas bittner <abittner@stud.fh-heilbronn.de> 2009-03-07 12:38:52 MST --- additional info: i am trying the netinst isos to install current factory (x86) i am using: http://download.opensuse.org/factory/iso/openSUSE-Factory-NET-i586-Build0026... downloading from repository via http through proxy. i am experiencing the same problems at various packages: Sat Mar 7 20:32:11 2009 280 192.168.150.47 TCP_MISS/302 933 GET http://download.opensuse.org/factory/repo/oss/suse/noarch/bundle-lang-kde-en... - DIRECT/195.135.221.130 text/html Sat Mar 7 20:32:11 2009 102 192.168.150.47 TCP_MISS/404 1420 GET http://ftp5.gwdg.de/pub/opensuse/factory/repo/oss/suse/noarch/bundle-lang-kd... - DIRECT/134.76.12.5 text/html and i thought i was remembering that the mirrorbrain infrastructure somehow kept track of the status of the mirrors, of their reliability, and what files are present on what servers and what files need to be served directly from the suse/novell infrastructure and so forth... either there is some serious bug in the mirrorbrain today or just for factory, or the mirrors are somewhat corrupted. if you ask me, the whole download mechanism (zypper?) or whatever component is involved need to get serious enhancements, needs to get more fault tolerant, more foolproof, failsafe measures or simple retries a few times before bringing up the error dialog boxes for a thousand rpm packages that are missing on various incomplete mirror servers. factory is pretty much uninstallable if you dont plan spending all day in front of your installation and interactively taking part and having fun with dialogboxes. thanks. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=483189 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=483189#c3 Peter Poeml <poeml@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dmacvicar@novell.com, | |poeml@novell.com AssignedTo|bnc-team-screening@forge.pr |poeml@novell.com |ovo.novell.com | --- Comment #3 from Peter Poeml <poeml@novell.com> 2009-03-08 15:53:06 MST --- Hi Andreas. With 11.1, you are using the "plain, old, dumb" zypper that will always have problems downloading stuff because whenever it encounters a problem it'll not deal with it. That will not change (in 11.1), but in 11.2 we plan to offer a download client that works remarkably better. See http://en.opensuse.org/Libzypp/Failover for an overview. Actually, if you update to Factory, you will be one of the first who get the benefit of the new technology. Of course, we cannot give much guarentee yet that you won't also run into severe bugs. Anyhow, testing will mature this, and it is worth it. Of course, to get there, you will have to use 11.1's libzypp to update to the current state at all, which could be a little painful. However, you could actually look at http://en.opensuse.org/Libzypp/Failover which describes how to activate the hidden prototype that shipped with 11.1 -- or how to update some packages from zypp:Head and go from there. Anyway, during the last two days we faced an additional little hurdle; there was a severe bug in the new libzypp download client, but this bug was not "published" to Factory yet, and we found it in time by testing. Now, I wanted to make sure that the fixed libzypp/ download client arrives in the published Factory tree the same time as the bug, but not later, because it would have meant some hassle for our users (manual intervention would have been needed). In order to prevent the broken change going out I stopped the publishing of Factory, while mirrors were still being updated. Thus, what you might have seen during the last 48 hours is that mirrors had a slightly different state than Factory on download.opensuse.org. Note that wouldn matter if you'd already use the new download client, ant that it would not happen this way if I hadn't intervened here (for a reason). Meanwhile, the bug fix is there and sync is active again. Note that download.opensuse.org _will_, for some time, redirect to mirrors that give a 404 for Factory(!) (not elsewhere). This is normal for now and is due to the way that Factory is synced to mirrors. This is planned to be solved by implementing the Factory in a way that integrates with the mirror database (so it always knows, what mirrors have, and doesn't find out about it only later). In addition, with the new download client it doesn't matter at all, because it is robust against that. The "new" Factory sync will take some more time to be implemented though. Maybe you have read my recent posting here http://lists.opensuse.org/opensuse-project/2009-02/msg00110.html about the status of Factory publishing. So, what you see is not a serious bug in MirrorBrain. We just sync Factory in a way that is a bit incompatible with it and the old failure-prone zypper. What you see is the reverse coin of: "http://download.opensuse.org/factory/ keeps the previous package version available for a grace period." This allows you to continue updating while all is in flux... While download.opensuse.org keeps the packages, they are quickly deleted on mirrors. There is not much to do about it, except what I mentioned above - full integration of the sync and the database update needs to be implemented in the future for Factory. * I got your mail to info@mirrorbrain.org, and it's fine to write there; about openSUSE relevant issues, the bugzilla at novell.com is appropriate. There is actually a product "openSUSE.org" and component "download infrastructure", which you can use. The issue at hand is specific to Factory. Either place to report things is fine in the end, as long as it reaches me ;-) which it usually does quite quickly. * You are fully correct with the assumption that if a "bad" redirect, or a mirror failure, or a failed transfer, et cetera, happens for one of the *initial* files that are loaded in a network install, like for instance the initial ramdisk. I can tell from experience that I have found whole broken mirrors just because of this single issue, because a simple "retry" doesn't help here -- or rather, a "retry" comprises a complete new boot and try again to load the instsys, which is rather cumbersome if one runs into problems during that. My hope is that the new powerful download client (aria2c) will be in the instsys, so it is used during installation, and I expect this to be the case. The loading of the initial files would remain the last heel of Achilles. It might be conveivable to implement a minimal metalink client in the instsys, which uses the provided metalink to verify the download and retry listed mirrors in order (which would be easy to implement). * About your very last question: all download behaviour is part of libzypp, which is then used by both YaST and zypper. There is a bugzill component libzypp which is the correct component for all issues around libzypp downloads, regardless the medium. *** I hope this somehow answers your questions, even if not all may be to your satisfaction so far. I am doing an unscheduled mirror scan to clean up old factory redirects, to improve the chances of a reasonable update to Factory for the next users to come. Thanks for your report - for your time and effort, Andreas! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=483189 User poeml@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=483189#c5 --- Comment #5 from Peter Poeml <poeml@novell.com> 2009-03-08 21:00:49 MST --- Check features.opensuse.org, there is a suggestion about downloading in parallel to installing being tracked already. https://features.opensuse.org/120340 for instance. And maybe there is another one. Good news, over the last hours I have implemented a subdirectory scan that actually deletes files from the database. So far, our scanner could do either complete scans, or partial scans without deletions. Since I reimplemented the whole mirror file database in a different way, it is structured in a way that it is possible to do efficient substring matches on path names. This means that I can do cleanup scans (which remove stale files from the database, thus preventing bad redirects) much more efficiently. Now it doesn't require an expensive full scan anymore, but just a subdirectory scan which I can do every few hours. This will greatly alleviate the issue reported here. I have an idea in the back of my head for parallel download; we could provide a "precompiled" metalink with all the files, and libzypp could download that and use it to download the referenced files from it that it needs. Would be perfect for installation. Just an idea for now though, not thought through. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.