[opensuse-factory] The frustration of updating

Robert Kaiser

26 Jan 2009 26 Jan '09

01:21

Hi, After a few weeks of staying with one state of the Factory system, I decided to update to a current state once again a few hours ago. This requires about 5 GB of download and a few hours to run, so it's not something I'm doing lightly and often. And once again, as so often, YaST started to spit out errors right in the middle of the process telling me the Factory repo doesn't contain some of the packages it wanted to install - probably it's been updated underneath my install and the packages had been replaced with newer versions. I canceled and tried again and then I ran into the same problem with the KDE4 Desktop OBS repo halfway in the install. Can't we have some system that doesn't frustrate you by pulling out files you need right while you're installing them? Couldn't OBS keep files, say, 12-24h after they have been removed from the repo index? This is frustrating enough that I'm only updating this system very rarely nowadays, and I think updating often and testing newer stuff is actually what Factory should be about. The problems I should deal with (if any at all) are ones that arise due to new code, not due to the repo info not matching the available files at the time I'm actually trying to install them. Is there any solution for this? Robert Kaiser -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Show replies by date

Josef Reidinger

26 Jan 26 Jan

07:42

Robert Kaiser wrote:

...

Hi,

After a few weeks of staying with one state of the Factory system, I decided to update to a current state once again a few hours ago. This requires about 5 GB of download and a few hours to run, so it's not something I'm doing lightly and often.

And once again, as so often, YaST started to spit out errors right in the middle of the process telling me the Factory repo doesn't contain some of the packages it wanted to install - probably it's been updated underneath my install and the packages had been replaced with newer versions. I canceled and tried again and then I ran into the same problem with the KDE4 Desktop OBS repo halfway in the install.

Can't we have some system that doesn't frustrate you by pulling out files you need right while you're installing them? Couldn't OBS keep files, say, 12-24h after they have been removed from the repo index?

This is frustrating enough that I'm only updating this system very rarely nowadays, and I think updating often and testing newer stuff is actually what Factory should be about. The problems I should deal with (if any at all) are ones that arise due to new code, not due to the repo info not matching the available files at the time I'm actually trying to install them.

Is there any solution for this?

Robert Kaiser

This should help you, important is that you must must disable autoupdate, so you do refresh and after download of rpm you install it. http://lizards.opensuse.org/2008/10/30/how-survive-zypper-dup-on-system-with... JR -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Carlos E. R.

12:02

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2009-01-26 at 08:42 +0100, Josef Reidinger wrote:

...

...
Is there any solution for this?

This should help you, important is that you must must disable autoupdate, so you do refresh and after download of rpm you install it. http://lizards.opensuse.org/2008/10/30/how-survive-zypper-dup-on-system-with...

Yes, caching the files locally avoids doing a half-install. But we still need to download those 1 or 5 gigs complete before starting, and it is downloading them what is a problem because they are changed before we can finish the download. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkl9pl8ACgkQtTMYHG2NR9UuhQCcC6wYyzZJFNcOIe3RkseHFswO OSgAnRtvSl0IsAH/33JXTo+2ex+LMCuO =XiV5 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Josef Reidinger

16:27

Carlos E. R. wrote:

...

On Monday, 2009-01-26 at 08:42 +0100, Josef Reidinger wrote:

...
...
Is there any solution for this?

This should help you, important is that you must must disable autoupdate, so you do refresh and after download of rpm you install it. http://lizards.opensuse.org/2008/10/30/how-survive-zypper-dup-on-system-with...

Yes, caching the files locally avoids doing a half-install. But we still need to download those 1 or 5 gigs complete before starting, and it is downloading them what is a problem because they are changed before we can finish the download.

-- Cheers, Carlos E. R.

But not all files changes, so you next time download only updated files ;) and this should help JR -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Carlos E. R.

20:06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2009-01-26 at 17:27 +0100, Josef Reidinger wrote:

...

Carlos E. R. wrote:

...
Yes, caching the files locally avoids doing a half-install. But we still need to download those 1 or 5 gigs complete before starting, and it is downloading them what is a problem because they are changed before we can finish the download.

But not all files changes, so you next time download only updated files ;) and this should help

Are you sure? I admit that I haven't tried recently, but usually all files change, even if it is only to get a new version number (or release or whatever). I can not try for the moment... - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkl+F7UACgkQtTMYHG2NR9Wn9gCgkbVCyJF2vxQ9WhiZSWEeIrKR ZI0An0tWKMyq1hmkL3BPZ5ohQM4JeT0u =ncA1 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Hans-Peter Holler

21:12

Am Montag 26 Januar 2009 schrieb Carlos E. R.:

...

On Monday, 2009-01-26 at 17:27 +0100, Josef Reidinger wrote:

...
Carlos E. R. wrote:

...
Yes, caching the files locally avoids doing a half-install. But we still need to download those 1 or 5 gigs complete before starting, and it is downloading them what is a problem because they are changed before we can finish the download. This is the way how Factory works: you never know when an update will happen. If the the guys in Nuremberg decide to push an update, this can be at night in Germany. There you can expect, everyone's sleeping.

But not all files changes, so you next time download only updated files ;) and this should help So, when will be "next time"? if you're in the middle of an update and the content changes, you receive a message like "... not found". Good chance to break this time and do a "zypper ref; zypper dup" or "yast update"? Yes and no. If packages with dependecies changed in this time frame you have a good chance to mess up. [further up next quote]

Are you sure? I admit that I haven't tried recently, but usually all files change, even if it is only to get a new version number (or release or whatever). Yes, not all changes are important, only why this "whatever".number after the dot changes. But: if _YOU_ decide to update/dist-upgrade your "whatever" kind of "your" installation from Factory, _YOU_ are the one, who's responsible for doing this. The way, the openSUSE guys are treating this, is their decision and all you can do is blaming. And where? at bugzilla.novell.com.

...

I can not try for the moment...

-- Cheers, Carlos E. R.

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Eberhard Moenkeberg

21:25

Hi, On Mon, 26 Jan 2009, Hans-Peter Holler wrote:

...

Am Montag 26 Januar 2009 schrieb Carlos E. R.:

...
On Monday, 2009-01-26 at 17:27 +0100, Josef Reidinger wrote:

...
Carlos E. R. wrote:

...

...
...
...
Yes, caching the files locally avoids doing a half-install. But we still need to download those 1 or 5 gigs complete before starting, and it is downloading them what is a problem because they are changed before we can finish the download.

...

This is the way how Factory works: you never know when an update will happen. If the the guys in Nuremberg decide to push an update, this can be at night in Germany. There you can expect, everyone's sleeping.

...
...
But not all files changes, so you next time download only updated files ;) and this should help

...

So, when will be "next time"? if you're in the middle of an update and the content changes, you receive a message like "... not found". Good chance to break this time and do a "zypper ref; zypper dup" or "yast update"? Yes and no. If packages with dependecies changed in this time frame you have a good chance to mess up. [further up next quote]

...
Are you sure? I admit that I haven't tried recently, but usually all files change, even if it is only to get a new version number (or release or whatever).

...

Yes, not all changes are important, only why this "whatever".number after the dot changes. But: if _YOU_ decide to update/dist-upgrade your "whatever" kind of "your" installation from Factory, _YOU_ are the one, who's responsible for doing this. The way, the openSUSE guys are treating this, is their decision and all you can do is blaming. And where? at bugzilla.novell.com.

If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed. On ftp5.gwdg.de, last night one update lasted from 01:11 h to 02:58 h MET. All others where finished within one minute. So what... Viele Grüße Eberhard Mönkeberg (emoenke@gwdg.de, em@kki.org) -- Eberhard Mönkeberg Arbeitsgruppe IT-Infrastruktur E-Mail: emoenke@gwdg.de Tel.: +49 (0)551 201-1551 ------------------------------------------------------------------------- Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Fassberg 11, 37077 Göttingen URL: http://www.gwdg.de E-Mail: gwdg@gwdg.de Tel.: +49 (0)551 201-1510 Fax: +49 (0)551 201-2150 Geschäftsführer: Prof. Dr. Bernhard Neumair Aufsichtsratsvorsitzender: Dipl.-Kfm. Markus Hoppe Sitz der Gesellschaft: Göttingen Registergericht: Göttingen Handelsregister-Nr. B 598 -------------------------------------------------------------------------

Robert Kaiser

22:05

Eberhard Moenkeberg wrote:

...

If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed.

Well, it every time takes me a few hours, as it tends to be here, then doing it three times is not really fun. I managed to do it with two times for the main Factory update and then doing about 200 button clicks for the Packages from KDE4:UNSTABLE (after needing a click for every one of them to step over during the install run already) and updating them from the software installation module again.

...

On ftp5.gwdg.de, last night one update lasted from 01:11 h to 02:58 h MET.

Well, I did run my update from about 10-11pm until 3-4am MET that night. I posted the mail when the second Factory update run started to complain on the missing UNSTABLE KDE4 packages. This was just one of my problems with that update run, see http://home.kairo.at/blog/2009-01/living_on_the_edge for more fun. ;-) Robert Kaiser -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Eberhard Moenkeberg

22:16

Hi, On Mon, 26 Jan 2009, Robert Kaiser wrote:

...

Eberhard Moenkeberg wrote:

...

...
If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed.

Well, it every time takes me a few hours, as it tends to be here, then doing it three times is not really fun. I managed to do it with two times for the main Factory update and then doing about 200 button clicks for the Packages from KDE4:UNSTABLE (after needing a click for every one of them to step over during the install run already) and updating them from the software installation module again.

...
On ftp5.gwdg.de, last night one update lasted from 01:11 h to 02:58 h MET.

Well, I did run my update from about 10-11pm until 3-4am MET that night. I posted the mail when the second Factory update run started to complain on the missing UNSTABLE KDE4 packages.

This was just one of my problems with that update run, see http://home.kairo.at/blog/2009-01/living_on_the_edge for more fun. ;-)

So it seems you got it to manage the factory hassle the usual way: just restart, maybe more than once, but in the end you will succeed. So what... Viele Grüße Eberhard Mönkeberg (emoenke@gwdg.de, em@kki.org) -- Eberhard Mönkeberg Arbeitsgruppe IT-Infrastruktur E-Mail: emoenke@gwdg.de Tel.: +49 (0)551 201-1551 ------------------------------------------------------------------------- Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Fassberg 11, 37077 Göttingen URL: http://www.gwdg.de E-Mail: gwdg@gwdg.de Tel.: +49 (0)551 201-1510 Fax: +49 (0)551 201-2150 Geschäftsführer: Prof. Dr. Bernhard Neumair Aufsichtsratsvorsitzender: Dipl.-Kfm. Markus Hoppe Sitz der Gesellschaft: Göttingen Registergericht: Göttingen Handelsregister-Nr. B 598 -------------------------------------------------------------------------

Peter Poeml

17 Feb 17 Feb

17:40

Robert, On Mon, Jan 26, 2009 at 11:05:13PM +0100, Robert Kaiser wrote:

...

Eberhard Moenkeberg wrote:

...
If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed.

Well, it every time takes me a few hours, as it tends to be here, then doing it three times is not really fun. I managed to do it with two times for the main Factory update and then doing about 200 button clicks for the Packages from KDE4:UNSTABLE (after needing a click for every one of them to step over during the install run already) and updating them from the software installation module again.

...
On ftp5.gwdg.de, last night one update lasted from 01:11 h to 02:58 h MET.

Well, I did run my update from about 10-11pm until 3-4am MET that night. I posted the mail when the second Factory update run started to complain on the missing UNSTABLE KDE4 packages.

This was just one of my problems with that update run, see http://home.kairo.at/blog/2009-01/living_on_the_edge for more fun. ;-)

Robert Kaiser

http://en.opensuse.org/Libzypp/Failover will help you for many of the scenarios of errors that you could run into when updating Factory. The top of the page describes how to switch on the "download failover" behaviour. I suggest that anyone tries this out, to give it as much testing as possible, so it can hopefully be the standard behaviour on 11.2. Just note that Factory as published right now does not work in this regard; you'd need stuff from zypp:Head for now, or wait until later Factory version. So it a little inconvenient at this time to suggest this experimentation. You can watch http://en.opensuse.org/Libzypp/Failover, which I will update when this works again in Factory. Then you just need to switch it on and watch it downloading away, and have less risk of breakage by installing an unreleased libzypp from zypp:Head. But feel feel to try it if you feel comfortable replacing your zypp package. But seriously, this prevents exactly the problems you experienced. As detailed elsewhere it can't help for the case where a package isn't available on download.opensuse.org anymore, but all those "retry other mirror" cases will be handled automatically. Peter -- Contact: admin@opensuse.org (a.k.a. ftpadmin@suse.com) #opensuse-mirrors on freenode.net Info: http://en.opensuse.org/Mirror_Infrastructure SUSE LINUX Products GmbH Research & Development

Carlos E. R.

27 Jan 27 Jan

01:20

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2009-01-26 at 22:25 +0100, Eberhard Moenkeberg wrote:

...

If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed.

This may work for you with your kind of network, but not with mine. An update attempt may take as "little" as four hours. If I trigger it at 2 am, to try catch a period of developers resting, and I go to bed, chances are it will not finish. A restart on the morning does not work, because the repo has changed, so I have to wait till the next night. I may have to try for a week before I finally get it done. So I don't. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkl+YXAACgkQtTMYHG2NR9VAdQCghf7qUMuf3BeGNj7v8v2gz6vB 2uMAnjtHgeL8zsqkrKq9sJ93FlAFk2m8 =jFyh -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Eberhard Moenkeberg

01:33

Hi, On Tue, 27 Jan 2009, Carlos E. R. wrote:

...

On Monday, 2009-01-26 at 22:25 +0100, Eberhard Moenkeberg wrote:

...
If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed.

This may work for you with your kind of network, but not with mine. An update attempt may take as "little" as four hours. If I trigger it at 2 am, to try catch a period of developers resting, and I go to bed, chances are it will not finish. A restart on the morning does not work, because the repo has changed, so I have to wait till the next night.

No. ftp5 is updating factory every 4 hours.

...

I may have to try for a week before I finally get it done.

Never.

...

So I don't.

Incredible. 1 or 2 hours later again, and you would have succeded. Why the hell are you complaining about a renewing process which is running _THIS_ fast? It must be something else what is boring you, so please explain or shut up. Interrupting the update process at night because you need to sleep is generally a good thing. After wakeup, the chance to fulfill is 100%. Viele Grüße Eberhard Mönkeberg (emoenke@gwdg.de, em@kki.org) -- Eberhard Mönkeberg Arbeitsgruppe IT-Infrastruktur E-Mail: emoenke@gwdg.de Tel.: +49 (0)551 201-1551 ------------------------------------------------------------------------- Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Fassberg 11, 37077 Göttingen URL: http://www.gwdg.de E-Mail: gwdg@gwdg.de Tel.: +49 (0)551 201-1510 Fax: +49 (0)551 201-2150 Geschäftsführer: Prof. Dr. Bernhard Neumair Aufsichtsratsvorsitzender: Dipl.-Kfm. Markus Hoppe Sitz der Gesellschaft: Göttingen Registergericht: Göttingen Handelsregister-Nr. B 598 -------------------------------------------------------------------------

Carlos E. R.

02:48

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tuesday, 2009-01-27 at 02:33 +0100, Eberhard Moenkeberg wrote:

...

Hi,

On Tue, 27 Jan 2009, Carlos E. R. wrote:

...
On Monday, 2009-01-26 at 22:25 +0100, Eberhard Moenkeberg wrote:

...
If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed.

This may work for you with your kind of network, but not with mine. An update attempt may take as "little" as four hours. If I trigger it at 2 am, to try catch a period of developers resting, and I go to bed, chances are it will not finish. A restart on the morning does not work, because the repo has changed, so I have to wait till the next night.

No. ftp5 is updating factory every 4 hours.

So what? That's part of the problem.

...

...
I may have to try for a week before I finally get it done.

Never.

Always.

...

...
So I don't.

Incredible. 1 or 2 hours later again, and you would have succeded.

I have tried. Does not work.

...

Why the hell are you complaining about a renewing process which is running _THIS_ fast?

Fast? FAST? You gotta be out of your mind.

...

It must be something else what is boring you, so please explain or shut up.

Don't tell me to shut up. :-/

...

Interrupting the update process at night because you need to sleep is generally a good thing. After wakeup, the chance to fulfill is 100%.

I don't interrupt the process. I leave it running. It almost always fails on its own. And no, when I come back a retry does not work, because factory has changed again and has to restart again. I try again, and again it fails. I try again and again it fails. I have to retry a dozen times, and each attempt takes several hours. What is it you do not understand? Simply downloading the files takes several hours! I simply can not download the thing before the repo changes and I have to restart the download, again and again and again. You can do it because you have the entire network of the university or wherever your server is at your disposal. I don't have that class of bandwidth. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkl+dhIACgkQtTMYHG2NR9UjPgCfSIX004gY2iJ79dJGOYtqtMoD fsIAn31uZ86LjdQEjOxTWpisYXQvf8Hm =1Jg0 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Eberhard Moenkeberg

10:33

Hi, On Tue, 27 Jan 2009, Carlos E. R. wrote:

...

On Tuesday, 2009-01-27 at 02:33 +0100, Eberhard Moenkeberg wrote:

...
On Tue, 27 Jan 2009, Carlos E. R. wrote:

...
On Monday, 2009-01-26 at 22:25 +0100, Eberhard Moenkeberg wrote:

...
If an update aborts due to recent factory changes, you can just restart. Maybe you need that a second and a third time, but usually an update needs only a limited time, so if you continue you will succeed.

This may work for you with your kind of network, but not with mine. An update attempt may take as "little" as four hours. If I trigger it at 2 am, to try catch a period of developers resting, and I go to bed, chances are it will not finish. A restart on the morning does not work, because the repo has changed, so I have to wait till the next night.

No. ftp5 is updating factory every 4 hours.

So what? That's part of the problem.

...
...
I may have to try for a week before I finally get it done.

Never.

Always.

...
...
So I don't.

Incredible. 1 or 2 hours later again, and you would have succeded.

I have tried. Does not work.

...
Why the hell are you complaining about a renewing process which is running _THIS_ fast?

Fast? FAST? You gotta be out of your mind.

...
It must be something else what is boring you, so please explain or shut up.

Don't tell me to shut up. :-/

...
Interrupting the update process at night because you need to sleep is generally a good thing. After wakeup, the chance to fulfill is 100%.

I don't interrupt the process. I leave it running. It almost always fails on its own. And no, when I come back a retry does not work, because factory has changed again and has to restart again. I try again, and again it fails. I try again and again it fails. I have to retry a dozen times, and each attempt takes several hours.

What is it you do not understand? Simply downloading the files takes several hours! I simply can not download the thing before the repo changes and I have to restart the download, again and again and again.

You can do it because you have the entire network of the university or wherever your server is at your disposal. I don't have that class of bandwidth.

Yesterday was only one update, and on ftp5 it lasted from 0:11 h to 2:58 h MET. So any update starting 2:59 h or later should have run smooth. Viele Grüße Eberhard Mönkeberg (emoenke@gwdg.de, em@kki.org) -- Eberhard Mönkeberg Arbeitsgruppe IT-Infrastruktur E-Mail: emoenke@gwdg.de Tel.: +49 (0)551 201-1551 ------------------------------------------------------------------------- Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Fassberg 11, 37077 Göttingen URL: http://www.gwdg.de E-Mail: gwdg@gwdg.de Tel.: +49 (0)551 201-1510 Fax: +49 (0)551 201-2150 Geschäftsführer: Prof. Dr. Bernhard Neumair Aufsichtsratsvorsitzender: Dipl.-Kfm. Markus Hoppe Sitz der Gesellschaft: Göttingen Registergericht: Göttingen Handelsregister-Nr. B 598 -------------------------------------------------------------------------

Stephan Kulow

09:49

Am Montag 26 Januar 2009 schrieb Hans-Peter Holler:

...

Am Montag 26 Januar 2009 schrieb Carlos E. R.:

...
On Monday, 2009-01-26 at 17:27 +0100, Josef Reidinger wrote:

...
Carlos E. R. wrote:

...
Yes, caching the files locally avoids doing a half-install. But we still need to download those 1 or 5 gigs complete before starting, and it is downloading them what is a problem because they are changed before we can finish the download.

This is the way how Factory works: you never know when an update will happen. If the the guys in Nuremberg decide to push an update, this can be at night in Germany. There you can expect, everyone's sleeping.

Noone decides to push. We decide to accept submitrequests and whenever all packages built, they are pushed. This can be at any time. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Patrick Shanahan

14:49

* Stephan Kulow [01-27-09 04:51]:

...

Noone decides to push. We decide to accept submitrequests and whenever all packages built, they are pushed. This can be at any time.

Could a flag be set on the wiki indicating the start of a build and maybe another indicating the end? Then consulting the wiki page would provide an indication of an optimal timeframe for local updating. -- Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 @ http://counter.li.org -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Marcus Meissner

15:11

On Tue, Jan 27, 2009 at 09:49:22AM -0500, Patrick Shanahan wrote:

...

* Stephan Kulow [01-27-09 04:51]:

...
Noone decides to push. We decide to accept submitrequests and whenever all packages built, they are pushed. This can be at any time.

Could a flag be set on the wiki indicating the start of a build and maybe another indicating the end? Then consulting the wiki page would provide an indication of an optimal timeframe for local updating.

Hmm, What about having two factory directories that are flipped every night. factory-yesterday factory-current that way you always get a stable factory snapshot. Ciao, Marcus -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Eberhard Moenkeberg

15:21

Hi, On Tue, 27 Jan 2009, Marcus Meissner wrote:

...

On Tue, Jan 27, 2009 at 09:49:22AM -0500, Patrick Shanahan wrote:

...
* Stephan Kulow [01-27-09 04:51]:

...
Noone decides to push. We decide to accept submitrequests and whenever all packages built, they are pushed. This can be at any time.

Could a flag be set on the wiki indicating the start of a build and maybe another indicating the end? Then consulting the wiki page would provide an indication of an optimal timeframe for local updating.

Hmm,

What about having two factory directories that are flipped every night.

factory-yesterday factory-current

that way you always get a stable factory snapshot.

It should be done a) with hard links b) below /pub/opensuse/factory/ so that mirrors which rsync factory separately can benefit from -H. Keep it simple - f.e. just add a directory /pub/opensuse/factory/old/ which gets filled with hardlinks to ../repo/... before repo/ gets changed. Viele Grüße Eberhard Mönkeberg (emoenke@gwdg.de, em@kki.org) -- Eberhard Mönkeberg Arbeitsgruppe IT-Infrastruktur E-Mail: emoenke@gwdg.de Tel.: +49 (0)551 201-1551 ------------------------------------------------------------------------- Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Fassberg 11, 37077 Göttingen URL: http://www.gwdg.de E-Mail: gwdg@gwdg.de Tel.: +49 (0)551 201-1510 Fax: +49 (0)551 201-2150 Geschäftsführer: Prof. Dr. Bernhard Neumair Aufsichtsratsvorsitzender: Dipl.-Kfm. Markus Hoppe Sitz der Gesellschaft: Göttingen Registergericht: Göttingen Handelsregister-Nr. B 598 -------------------------------------------------------------------------

Richard Guenther

16:48

On Tue, 27 Jan 2009, Marcus Meissner wrote:

...

On Tue, Jan 27, 2009 at 09:49:22AM -0500, Patrick Shanahan wrote:

...
* Stephan Kulow [01-27-09 04:51]:

...
Noone decides to push. We decide to accept submitrequests and whenever all packages built, they are pushed. This can be at any time.

Could a flag be set on the wiki indicating the start of a build and maybe another indicating the end? Then consulting the wiki page would provide an indication of an optimal timeframe for local updating.

Hmm,

What about having two factory directories that are flipped every night.

factory-yesterday factory-current

that way you always get a stable factory snapshot.

I assume files have "full" names including version and build. So why not just keep old files around so that old meta-data does not get invalidated by partial updates? Richard. -- Richard Guenther Novell / SUSE Labs SUSE LINUX Products GmbH - Nuernberg - AG Nuernberg - HRB 16746 - GF: Markus Rex -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

28 Jan 28 Jan

09:06

Am Dienstag 27 Januar 2009 schrieb Richard Guenther:

...

I assume files have "full" names including version and build. So why not just keep old files around so that old meta-data does not get invalidated by partial updates?

"Why not just"? It's 14G without source and debuginfo. How many of those copies do you think our mirrors would prefer to have? Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Daniel Fuhrmann

09:57

I had never problems with updating factory since 2 years, and i have only a DSL 2000 connection. Probabely I'm lucky. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

10:14

Am Mittwoch 28 Januar 2009 schrieb Daniel Fuhrmann:

...

I had never problems with updating factory since 2 years, and i have only a DSL 2000 connection. Probabely I'm lucky.

Someone tells he's frustrated and you tell him you never had his problems - you really know how to comfort :) Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Daniel Fuhrmann

12:34

Am Mittwoch, 28. Januar 2009 11:14:54 schrieb Stephan Kulow:

...

you really know how to comfort :)

Sorry :( There are only few who complain about inconsistent updates because of timeout. Is it another reason? Or really bad timing? When will the delta updates for factory be available? -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

wormey＠eskimo.com

29 Jan 29 Jan

06:53

On Jan 28, 1:34pm, Daniel Fuhrmann wrote: } Subject: Re: [opensuse-factory] The frustration of updating

...

Am Mittwoch, 28. Januar 2009 11:14:54 schrieb Stephan Kulow:

...
you really know how to comfort :)

Sorry :(

There are only few who complain about inconsistent updates because of timeout.

Just to jump on the dogpile here... I have over 9400 packages on my system. A major update will refresh nearly half of those, and I end up downloading about 4GB. Fortunately, major updates happen at most once a week. Though I'm on cable, it still takes 8-10 hours to do all the downloads, and I frequently find that things have changed in the meantime (sometimes even when I'm only updating several hundred packages). I use a package manager that downloads everything before installing (smart). When it didn't get all the packages because they've changed, it's a simple matter to rescan the repositories, redo the update list, then start the downloads again. Sometimes I have to repeat the process before getting everything needed to install, but each time it's an order of magnitude less packages that have to be downloaded than the time before. And it seldom takes more than a couple of days. I'm sure that most peoples' systems are not nearly so heavily laden as mine, so surely it's not an insurmountable task for those on slower links to enjoy at least a weekly update. Best, ~Steve -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Basil Chupin

07:47

Space Case wrote:

...

On Jan 28, 1:34pm, Daniel Fuhrmann wrote: } Subject: Re: [opensuse-factory] The frustration of updating

...
Am Mittwoch, 28. Januar 2009 11:14:54 schrieb Stephan Kulow:

...
you really know how to comfort :)

Sorry :(

There are only few who complain about inconsistent updates because of timeout.

Just to jump on the dogpile here... I have over 9400 packages on my system. A major update will refresh nearly half of those, and I end up downloading about 4GB. Fortunately, major updates happen at most once a week. Though I'm on cable, it still takes 8-10 hours to do all the downloads, and I frequently find that things have changed in the meantime (sometimes even when I'm only updating several hundred packages).

I use a package manager that downloads everything before installing (smart). When it didn't get all the packages because they've changed, it's a simple matter to rescan the repositories, redo the update list, then start the downloads again.

Which is why I have been using smart for some time - and today installed it on my "working" system (and the first thing it did was find more updates which the default openSUSE showed weren't in existence). But before I did this this morning I started an update of 11.1 (with KDE4.2) using zypper. Some 15 minutes after I did this I had to go out to do some shopping with my wife and when I came back I found that zypper had been sitting for an hour or so waiting for a manual input from me to confirm that that an Error had occurred and then an input from me about I wanted to do about the error. Responding to the error message finally finished the download of the rest of the RPMs (after some 30+ minutes). *ONE*, repeat *1*, RPM from http://download.opensuse.org/repositories/.... held up the whole of the download process in this update "cycle"! What a waste of damn time and resources! smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed. Intelligence at work.

...

Sometimes I have to repeat the process before getting everything needed to install, but each time it's an order of magnitude less packages that have to be downloaded than the time before. And it seldom takes more than a couple of days.

I'm sure that most peoples' systems are not nearly so heavily laden as mine, so surely it's not an insurmountable task for those on slower links to enjoy at least a weekly update.

Those who have put zypper/zypp/whatever together have done so without having any practical experience of using a system which relies on real-life internet connections with ISPs through either dial-up connection or even broadband. All they know is what they experience while working with a computer sitting a few metres from them on the same LAN cable. Sad. Ciao. -- "I do not instruct the uninterested; I do not help those who fail to try. If I mention one corner of a subject and the pupil does not deduce therefrom the other three, I drop him." Confucius -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dominique Leuenberger

08:25

...

...
...
On 1/29/2009 at 8:47 AM, Basil Chupin wrote: Those who have put zypper/zypp/whatever together have done so without having any practical experience of using a system which relies on real-life internet connections with ISPs through either dial-up connection or even broadband. All they know is what they experience while working with a computer sitting a few metres from them on the same LAN cable. Sad.

I think the SAD part here is that the devs behind this great tool have to be insulted in such ways. This is in absolutely no way constructive. Sorry: please keep on topic with the discussion and keep insults out. The motivation of a dev reading the thread might be much higher. Dominique -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Eberhard Moenkeberg

09:00

Hi, On Thu, 29 Jan 2009, Dominique Leuenberger wrote:

...

...
...
...
On 1/29/2009 at 8:47 AM, Basil Chupin wrote:

...

...
Those who have put zypper/zypp/whatever together have done so without having any practical experience of using a system which relies on real-life internet connections with ISPs through either dial-up connection or even broadband. All they know is what they experience while working with a computer sitting a few metres from them on the same LAN cable. Sad.

I think the SAD part here is that the devs behind this great tool have to be insulted in such ways. This is in absolutely no way constructive.

Sorry: please keep on topic with the discussion and keep insults out. The motivation of a dev reading the thread might be much higher.

But don't miss the major aspect he has pointed to: Waiting/stopping in the middle of the update process is bad design. Viele Grüße Eberhard Mönkeberg (emoenke@gwdg.de, em@kki.org) -- Eberhard Mönkeberg Arbeitsgruppe IT-Infrastruktur E-Mail: emoenke@gwdg.de Tel.: +49 (0)551 201-1551 ------------------------------------------------------------------------- Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Fassberg 11, 37077 Göttingen URL: http://www.gwdg.de E-Mail: gwdg@gwdg.de Tel.: +49 (0)551 201-1510 Fax: +49 (0)551 201-2150 Geschäftsführer: Prof. Dr. Bernhard Neumair Aufsichtsratsvorsitzender: Dipl.-Kfm. Markus Hoppe Sitz der Gesellschaft: Göttingen Registergericht: Göttingen Handelsregister-Nr. B 598 -------------------------------------------------------------------------

Dominique Leuenberger

09:13

...

...
...
On 1/29/2009 at 10:00 AM, Eberhard Moenkeberg wrote: But don't miss the major aspect he has pointed to:

Waiting/stopping in the middle of the update process is bad design.

No arguing there... but like you just did: it has to stay on topic, without insults. On the other hand if an rpm can't be downloaded, I'm not sure if zypper should just 'skip' this one, finish the rest of it's task and install what it downloaded. This could render your system very very unusable (think of libzypp changing so version, thus a new zypper links against it. zypper is downloaded, libzypp fails. zypper being installed and libzypp skipped. you'll not be able to start zypper anymore.). So at best the 'remainings' can be tried to be downloaded, but I would for sure not like the PM trying to install the packages which could be downloaded. Sure, make the PM very smart to skip a whole bunch of packages that depend on each other. Which would in fact mean that after every skipped package you have to run the solver. which might be as broken an idea as nothing else: repodata might have changed on the server. New conflicts appear, existing conflicts that were solved by the user invalidated.... you name them. I think the approach with more potential that was mentioned in the list is in fact to find a solution to keep packages that get replaced for 24 hours longer on the mirror. This would help against change of repodata in the middle. Dominique -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dave Plater

09:50

Dominique Leuenberger wrote:

...

...
...
...
On 1/29/2009 at 10:00 AM, Eberhard Moenkeberg wrote:

But don't miss the major aspect he has pointed to:

Waiting/stopping in the middle of the update process is bad design.

No arguing there... but like you just did: it has to stay on topic, without insults.

On the other hand if an rpm can't be downloaded, I'm not sure if zypper should just 'skip' this one, finish the rest of it's task and install what it downloaded. This could render your system very very unusable (think of libzypp changing so version, thus a new zypper links against it. zypper is downloaded, libzypp fails. zypper being installed and libzypp skipped. you'll not be able to start zypper anymore.).

Whether it skips or aborts, there are still packages that have already been installed or removed and that is enough to cause system problems so you may as well complete the download and save time. See https://bugzilla.novell.com/show_bug.cgi?id=379480 and https://bugzilla.novell.com/show_bug.cgi?id=431854 although the first bug is not really relevant and the second one is for yast they are both examples of why download first and install on completion is the way to go and the developers are already working on it for 11.2 I believe. Regards Dave P

...

So at best the 'remainings' can be tried to be downloaded, but I would for sure not like the PM trying to install the packages which could be downloaded. Sure, make the PM very smart to skip a whole bunch of packages that depend on each other. Which would in fact mean that after every skipped package you have to run the solver. which might be as broken an idea as nothing else: repodata might have changed on the server. New conflicts appear, existing conflicts that were solved by the user invalidated.... you name them.

I think the approach with more potential that was mentioned in the list is in fact to find a solution to keep packages that get replaced for 24 hours longer on the mirror. This would help against change of repodata in the middle.

Dominique

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stanislav Visnovsky

10:00

On Štvrtok 29 Január 2009 10:50:31 Dave Plater wrote:

...

Dominique Leuenberger wrote:

...
...
...
...
On 1/29/2009 at 10:00 AM, Eberhard Moenkeberg wrote:

But don't miss the major aspect he has pointed to:

Waiting/stopping in the middle of the update process is bad design.

No arguing there... but like you just did: it has to stay on topic, without insults.

On the other hand if an rpm can't be downloaded, I'm not sure if zypper should just 'skip' this one, finish the rest of it's task and install what it downloaded. This could render your system very very unusable (think of libzypp changing so version, thus a new zypper links against it. zypper is downloaded, libzypp fails. zypper being installed and libzypp skipped. you'll not be able to start zypper anymore.).

Whether it skips or aborts, there are still packages that have already been installed or removed and that is enough to cause system problems so you may as well complete the download and save time. See https://bugzilla.novell.com/show_bug.cgi?id=379480 and https://bugzilla.novell.com/show_bug.cgi?id=431854 although the first bug is not really relevant and the second one is for yast they are both examples of why download first and install on completion is the way to go and the developers are already working on it for 11.2 I believe.

Yes, we are looking into this for 11.2. Stano -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stanislav Visnovsky

12:08

On Štvrtok 29 Január 2009 08:47:49 Basil Chupin wrote:

...

Space Case wrote:

...
On Jan 28, 1:34pm, Daniel Fuhrmann wrote: } Subject: Re: [opensuse-factory] The frustration of updating

...
Am Mittwoch, 28. Januar 2009 11:14:54 schrieb Stephan Kulow:

...
you really know how to comfort :)

Sorry :(

There are only few who complain about inconsistent updates because of timeout.

Just to jump on the dogpile here... I have over 9400 packages on my system. A major update will refresh nearly half of those, and I end up downloading about 4GB. Fortunately, major updates happen at most once a week. Though I'm on cable, it still takes 8-10 hours to do all the downloads, and I frequently find that things have changed in the meantime (sometimes even when I'm only updating several hundred packages).

I use a package manager that downloads everything before installing (smart). When it didn't get all the packages because they've changed, it's a simple matter to rescan the repositories, redo the update list, then start the downloads again.

Which is why I have been using smart for some time - and today installed it on my "working" system (and the first thing it did was find more updates which the default openSUSE showed weren't in existence).

But before I did this this morning I started an update of 11.1 (with KDE4.2) using zypper.

Some 15 minutes after I did this I had to go out to do some shopping with my wife and when I came back I found that zypper had been sitting for an hour or so waiting for a manual input from me to confirm that that an Error had occurred and then an input from me about I wanted to do about the error.

Responding to the error message finally finished the download of the rest of the RPMs (after some 30+ minutes).

*ONE*, repeat *1*, RPM from http://download.opensuse.org/repositories/.... held up the whole of the download process in this update "cycle"!

What a waste of damn time and resources!

smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed.

I've added your suggestion to feature https://features.opensuse.org/305624 Stano -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Basil Chupin

30 Jan 30 Jan

12:56

Stanislav Visnovsky wrote:

...

On Å tvrtok 29 JanuÃ¡r 2009 08:47:49 Basil Chupin wrote:

...
Space Case wrote:

...
On Jan 28, 1:34pm, Daniel Fuhrmann wrote: } Subject: Re: [opensuse-factory] The frustration of updating

...
Am Mittwoch, 28. Januar 2009 11:14:54 schrieb Stephan Kulow:

...
you really know how to comfort :)

Sorry :(

There are only few who complain about inconsistent updates because of timeout.

Just to jump on the dogpile here... I have over 9400 packages on my system. A major update will refresh nearly half of those, and I end up downloading about 4GB. Fortunately, major updates happen at most once a week. Though I'm on cable, it still takes 8-10 hours to do all the downloads, and I frequently find that things have changed in the meantime (sometimes even when I'm only updating several hundred packages).

I use a package manager that downloads everything before installing (smart). When it didn't get all the packages because they've changed, it's a simple matter to rescan the repositories, redo the update list, then start the downloads again.

Which is why I have been using smart for some time - and today installed it on my "working" system (and the first thing it did was find more updates which the default openSUSE showed weren't in existence).

But before I did this this morning I started an update of 11.1 (with KDE4.2) using zypper.

Some 15 minutes after I did this I had to go out to do some shopping with my wife and when I came back I found that zypper had been sitting for an hour or so waiting for a manual input from me to confirm that that an Error had occurred and then an input from me about I wanted to do about the error.

Responding to the error message finally finished the download of the rest of the RPMs (after some 30+ minutes).

*ONE*, repeat *1*, RPM from http://download.opensuse.org/repositories/.... held up the whole of the download process in this update "cycle"!

What a waste of damn time and resources!

smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed.

I've added your suggestion to feature https://features.opensuse.org/305624

Stano

Thank you. I raised this issue some 3 years ago and got howled down because the idea of doing what I suggested had no merit. Ciao. -- "I do not instruct the uninterested; I do not help those who fail to try. If I mention one corner of a subject and the pupil does not deduce therefrom the other three, I drop him." Confucius -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Rajko M.

31 Jan 31 Jan

00:40

On Friday 30 January 2009 06:56:41 am Basil Chupin wrote:

...

...
I've added your suggestion to feature https://features.opensuse.org/305624

Stano

Thank you.

I raised this issue some 3 years ago and got howled down because the idea of doing what I suggested had no merit.

At that time majority was screaming about using lesser resources, specifically lesser RAM, for package management, so it was done to download and install package after package. IMHO, both ways should be available as options, because some people have a lot of resources, some not, some have one resource (RAM, hard disk, speedy Internet) and not the other. -- Regards, Rajko -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stanislav Visnovsky

07:37

On Sobota 31 Január 2009 01:40:28 Rajko M. wrote:

...

On Friday 30 January 2009 06:56:41 am Basil Chupin wrote:

...
...
I've added your suggestion to feature https://features.opensuse.org/305624

Stano

Thank you.

I raised this issue some 3 years ago and got howled down because the idea of doing what I suggested had no merit.

At that time majority was screaming about using lesser resources, specifically lesser RAM, for package management, so it was done to download and install package after package.

IMHO, both ways should be available as options, because some people have a lot of resources, some not, some have one resource (RAM, hard disk, speedy Internet) and not the other.

I agree. Stano -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Duncan Mac-Vicar Prett

2 Feb 2 Feb

17:39

Basil Chupin wrote:

...

Thank you.

I raised this issue some 3 years ago and got howled down because the idea of doing what I suggested had no merit.

Ciao.

3 years ago the ZYpp package management stack was in a state where you may guess in which order it downloaded the packages was pretty much unimportant. Duncan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Patrick Shanahan

29 Jan 29 Jan

14:43

* Basil Chupin [01-29-09 03:21]:

...

smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed.

Another approach would be to continue on with the other packages and retry the errant operation after the others have completed their download. Perhaps a retry counter and only fail aftery all others have completed and/or failed > 2 times. -- Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 @ http://counter.li.org -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Rajko M.

30 Jan 30 Jan

00:51

On Thursday 29 January 2009 08:43:26 am Patrick Shanahan wrote:

...

* Basil Chupin [01-29-09 03:21]:

...
smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed.

Another approach would be to continue on with the other packages and retry the errant operation after the others have completed their download. Perhaps a retry counter and only fail aftery all others have completed and/or failed > 2 times.

The response of package management should depend on type of error. If server was responding and all of the sudden can't be found retry DNS resolution again. If server is below common network speed try another one. This would require some application that will establish what is average network connection speed. -- Regards, Rajko -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

wormey＠eskimo.com

02:44

On Jan 29, 6:51pm, "Rajko M." wrote: } Subject: Re: [opensuse-factory] The frustration of updating

...

On Thursday 29 January 2009 08:43:26 am Patrick Shanahan wrote:

...
* Basil Chupin [01-29-09 03:21]:

...
smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed.

Another approach would be to continue on with the other packages and retry the errant operation after the others have completed their download. Perhaps a retry counter and only fail aftery all others have completed and/or failed > 2 times.

The response of package management should depend on type of error.

If server was responding and all of the sudden can't be found retry DNS resolution again.

If server is below common network speed try another one. This would require some application that will establish what is average network connection speed.

Here's some of my experiences with smart, and imo, is what zypp should be aspiring to: smart will download multiple threads. A couple from each repository, and however many repositories you're downloading from in parallel. This really speeds up the download process. I have had smart fail to download for various reasons: Failure to resolve repository host name. This happens to me even after many packages have been downloaded from the host. A restart of the download will usually fix that. Packages no longer available. Happens frequently to me. Rescan the repositories, rerun the solver, and download again. Most of the packages are already downloaded, and just a few dozen (or worst case, in the event of a 4000 package update, a few hundred) need to be downloaded to finish up. Stalls. About every second or third time I update, I get one or more stalls. Restarting the download will pick up where it got stalled at. Unlike the resolve failure or no package failure, these threads don't time out and go on. It's not hard to notice stalled downloads and restart the download process. And if I abort the download process so I can restart it and get the threads going again, any packages that were in the process of being downloaded will carry on from the point where they were interrupted when the download process gets around to them again. ~Steve -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Basil Chupin

13:14

Patrick Shanahan wrote:

...

* Basil Chupin [01-29-09 03:21]:

...
smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed.

Another approach would be to continue on with the other packages and retry the errant operation after the others have completed their download. Perhaps a retry counter and only fail aftery all others have completed and/or failed > 2 times.

Good point. It would also be most useful if the user had control over 2 things: the number of retries and the delay time between retries. Only last week someone gave me what had to be done to control the delay before zypper issues the error message that the attempted download has timed out - and I have now set it to 20 seconds (prior to this you had time to mow the lawn, have a cup or two of tea, have an argument with the wife, make up and still had time to spare before zypper came up with the ERROR message). Reason for my setting zypper's response time is that my ISP has OFF PEAK and PEAK periods, and I try and do my updates/upgrades during the off peak period. Often zypper will "hang" close to the end of downloading often 100+ files, the off peak period is coming to the end and one has to sit and wait for b****** zypper to come up with the ERROR message before the rest of the downloads can be completed. GRRRRR! Ciao. -- "I do not instruct the uninterested; I do not help those who fail to try. If I mention one corner of a subject and the pupil does not deduce therefrom the other three, I drop him." Confucius -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Duncan Mac-Vicar Prett

2 Feb 2 Feb

17:33

Basil Chupin wrote:

...

*ONE*, repeat *1*, RPM from http://download.opensuse.org/repositories/.... held up the whole of the download process in this update "cycle"!

What a waste of damn time and resources!

smart on the other hand simply downloads whatever RPM/s are downloadable and if it/they are not downloadable smart skips it/them and continues with downloading the rest. You then restart smart to download the missed RPMs when, for example, you come back from shopping or wake up in the morning and see that the task hasn't been completed.

Intelligence at work.

https://features.opensuse.org/305624 We need to implement this, and we know. It has been postponed over and over due to priorities, but for 11.2 dist-upgrade seems to be high priority. However, is also a waste of time and resources to read such comments making sarcastic fun of our products. Duncan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Richard Guenther

28 Jan 28 Jan

12:05

On Wed, 28 Jan 2009, Stephan Kulow wrote:

...

Am Dienstag 27 Januar 2009 schrieb Richard Guenther:

...
I assume files have "full" names including version and build. So why not just keep old files around so that old meta-data does not get invalidated by partial updates?

"Why not just"? It's 14G without source and debuginfo. How many of those copies do you think our mirrors would prefer to have?

I hope it will be a lot less if we improve on not syncing out identical packages. But, indeed - "just". It's useful to be able to revert a package to a previously known good state as well. Richard. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Peter Poeml

17 Feb 17 Feb

17:46

On Wed, Jan 28, 2009 at 01:05:09PM +0100, Richard Guenther wrote:

...

On Wed, 28 Jan 2009, Stephan Kulow wrote:

...
Am Dienstag 27 Januar 2009 schrieb Richard Guenther:

...
I assume files have "full" names including version and build. So why not just keep old files around so that old meta-data does not get invalidated by partial updates?

"Why not just"? It's 14G without source and debuginfo. How many of those copies do you think our mirrors would prefer to have?

I hope it will be a lot less if we improve on not syncing out identical packages. But, indeed - "just". It's useful to be able to revert ^^^^^^^^^^^^^^^^^ a package to a previously known good state as well. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

That's a good point indeed. I would be nice to arrange delayed deletion of packages in a way that there's always one previous package version. Would be highly convenient for manual repair of breakage. Peter -- Contact: admin@opensuse.org (a.k.a. ftpadmin@suse.com) #opensuse-mirrors on freenode.net Info: http://en.opensuse.org/Mirror_Infrastructure SUSE LINUX Products GmbH Research & Development

Stephan Kulow

27 Jan 27 Jan

09:46

Am Montag 26 Januar 2009 schrieb Carlos E. R.:

...

On Monday, 2009-01-26 at 17:27 +0100, Josef Reidinger wrote:

...
Carlos E. R. wrote:

...
Yes, caching the files locally avoids doing a half-install. But we still need to download those 1 or 5 gigs complete before starting, and it is downloading them what is a problem because they are changed before we can finish the download.

But not all files changes, so you next time download only updated files ;) and this should help

Are you sure? I admit that I haven't tried recently, but usually all files change, even if it is only to get a new version number (or release or whatever).

WIth a full rebuild, all files change. But we're not every factory update is a full rebuild. Usuually mondays contain the full rebuild and that's what happened yesterday :) Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Robert Kaiser

26 Jan 26 Jan

20:18

Josef Reidinger wrote:

...

This should help you, important is that you must must disable autoupdate, so you do refresh and after download of rpm you install it. http://lizards.opensuse.org/2008/10/30/how-survive-zypper-dup-on-system-with...

Thanks for the pointer, this is an interesting workaround - but it still is one, and I'd rather use YaST with the FACTORY-update module because it allows me to resolve package conflicts and let me select specific versions of packages, etc. with the power of the GUI. I feel the problem should be solved in a way that we don't need workarounds like the one you describe - probably the easiest solution would be on the server side with delayed deletion of obsolete packages. Robert Kaiser -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stanislav Visnovsky

21:24

On Pondelok 26 Január 2009 21:18:12 Robert Kaiser wrote:

...

Josef Reidinger wrote:

...
This should help you, important is that you must must disable autoupdate, so you do refresh and after download of rpm you install it. http://lizards.opensuse.org/2008/10/30/how-survive-zypper-dup-on-system-w ith-bad-internet-connection/

Thanks for the pointer, this is an interesting workaround - but it still is one, and I'd rather use YaST with the FACTORY-update module because it allows me to resolve package conflicts and let me select specific versions of packages, etc. with the power of the GUI. I feel the problem should be solved in a way that we don't need workarounds like the one you describe - probably the easiest solution would be on the server side with delayed deletion of obsolete packages.

Can I suggest to test our new tool for this - yast2-wagon? It should be the GUI tool for distribution upgrade for future. Stano, still trying to find time to blog about it -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

09:43

Am Montag 26 Januar 2009 02:21:21 schrieb Robert Kaiser:

...

Hi,

After a few weeks of staying with one state of the Factory system, I decided to update to a current state once again a few hours ago. This requires about 5 GB of download and a few hours to run, so it's not something I'm doing lightly and often.

And once again, as so often, YaST started to spit out errors right in the middle of the process telling me the Factory repo doesn't contain some of the packages it wanted to install - probably it's been updated underneath my install and the packages had been replaced with newer versions. I canceled and tried again and then I ran into the same problem with the KDE4 Desktop OBS repo halfway in the install.

Can't we have some system that doesn't frustrate you by pulling out files you need right while you're installing them? Couldn't OBS keep files, say, 12-24h after they have been removed from the repo index?

This is frustrating enough that I'm only updating this system very rarely nowadays, and I think updating often and testing newer stuff is actually what Factory should be about. The problems I should deal with (if any at all) are ones that arise due to new code, not due to the repo info not matching the available files at the time I'm actually trying to install them.

Is there any solution for this?

5GB? What kind of installation is that? The solution to your problem contains two parts (and both are worked on): - Factory will push out smaller deltas. So you only need to download let's say 400MB instead of 5GB. - zypper will download these files first before starting installation. Keeping files for 24 hours sounds like an interesting idea, but I have no idea how to implement this technically. If a new factory has built, it's pushed out by a script. That script first uploades the new files and then deletes the old ones. Hmm, it could keep an index to see when it uploaded specific files and delete from there. Need to think about it. But 5GB? I have pretty much a full installation with all kind of development stuff and still my factory updates are 1.2G. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Carlos E. R.

12:09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2009-01-26 at 10:43 +0100, Stephan Kulow wrote:

...

...
Is there any solution for this?

5GB? What kind of installation is that? The solution to your problem contains two parts (and both are worked on):

- Factory will push out smaller deltas. So you only need to download let's say 400MB instead of 5GB.

That would be nice.

...

- zypper will download these files first before starting installation.

Keeping files for 24 hours sounds like an interesting idea, but I have no idea how to implement this technically. If a new factory has built, it's pushed out by a script. That script first uploades the new files and then deletes the old ones. Hmm, it could keep an index to see when it uploaded specific files and delete from there. Need to think about it.

For instance, having two repos: one that changes as soon as new packakes are available, and another that keeps a photo made on fixed, known dates. Or by having two or more metadata structures, and the data (the packages) remaining there at least three days. This would perhaps need support by zypper.

...

But 5GB? I have pretty much a full installation with all kind of development stuff and still my factory updates are 1.2G.

Well, mine is about 1 GiB, but because I force myself to have only one desktop (gnome in my case). I could remove OOo, perhaps. But even with "only" 1 GiB, I have problems completing factory upgrades. It takes me sometimes several retries, during a week, to get it done. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkl9qBEACgkQtTMYHG2NR9UHJwCggijlvwot6n31j3YOHd1oIN1b PRgAnjngNDr/GZCyoAExCkQoe97GKZnq =ENBM -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Rob OpenSuSE

13:00

2009/1/26 Stephan Kulow :

...

Am Montag 26 Januar 2009 02:21:21 schrieb Robert Kaiser:

...

Keeping files for 24 hours sounds like an interesting idea, but I have no idea how to implement this technically. If a new factory has built, it's pushed out by a script. That script first uploades the new files and then deletes the old ones.

My trick to do this, is simply to queue the deletions in a batch job, to be run when is comparatively lightly loaded, rather than carry them out immediately. I prefer null-terminated string lists, and using xargs -0 rm -f on them. In short term you do require more disk space, but soon a new equilibrium is acheived. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Robert Kaiser

16:10

Stephan Kulow wrote:

...

Am Montag 26 Januar 2009 02:21:21 schrieb Robert Kaiser: 5GB? What kind of installation is that?

Well, 5GB is what the FACTORY-Update module shows it needs to install - not sure if that's download (i.e. before installation) or installed size, could be the latter. In that case, might be a good idea to have that displayed somewhere ;-)

...

The solution to your problem contains two parts (and both are worked on):

- Factory will push out smaller deltas. So you only need to download let's say 400MB instead of 5GB.

- zypper will download these files first before starting installation.

Both sound interesting and could surely help. OTOH, I tend to have the feeling that applying deltas takes long enough that a download of the full package is often faster.

...

Keeping files for 24 hours sounds like an interesting idea, but I have no idea how to implement this technically. If a new factory has built, it's pushed out by a script. That script first uploades the new files and then deletes the old ones. Hmm, it could keep an index to see when it uploaded specific files and delete from there. Need to think about it.

The script could keep an index of when files became obsolete (instead of deleting them right away) and only delete those that have that date more than 24 hours in the past. Robert Kaiser -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Carlos E. R.

16:24

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2009-01-26 at 17:10 +0100, Robert Kaiser wrote: ...

...

The script could keep an index of when files became obsolete (instead of deleting them right away) and only delete those that have that date more than 24 hours in the past.

Use the "access" timestamp of linux filesystems? :-? - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkl946kACgkQtTMYHG2NR9XH0QCghXAhT5xcAdiSDjpu6KzQIgvT AEsAoJC4+8P2ZdoTQu83/DDc68+1qLR9 =mIr+ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Robert Kaiser

20:13

Carlos E. R. wrote:

...

On Monday, 2009-01-26 at 17:10 +0100, Robert Kaiser wrote:

...
The script could keep an index of when files became obsolete (instead of deleting them right away) and only delete those that have that date more than 24 hours in the past.

Use the "access" timestamp of linux filesystems? :-?

I think that's a bit unreliable, as it can mean too many different things, even that someone read it for downloading (?) - but then, I don't care too much about the actual implementation, I guess that's best to leave to the admins of the OBS. Robert Kaiser -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Peter Poeml

17 Feb 17 Feb

17:29

Hi, On Mon, Jan 26, 2009 at 05:10:54PM +0100, Robert Kaiser wrote:

...

Stephan Kulow wrote:

...
The solution to your problem contains two parts (and both are worked on):

- Factory will push out smaller deltas. So you only need to download let's say 400MB instead of 5GB.

- zypper will download these files first before starting installation.

Both sound interesting and could surely help. OTOH, I tend to have the feeling that applying deltas takes long enough that a download of the full package is often faster.

...
Keeping files for 24 hours sounds like an interesting idea, but I have no idea how to implement this technically. If a new factory has built, it's pushed out by a script. That script first uploades the new files and then deletes the old ones. Hmm, it could keep an index to see when it uploaded specific files and delete from there. Need to think about it.

The script could keep an index of when files became obsolete (instead of deleting them right away) and only delete those that have that date more than 24 hours in the past.

Good idea; I have been ventilating this for some time. A simple time-based deletion scheme, which came to mind first, might not be enough, though -- tracking whether files have actually been referenced by metadata in some timeframe before deletion could be better. An additional aspect is the decision whether these duplicate (or more) files should be present on Factory mirrors, which effectively doubles the storage need for Factory (and hardlinks won't change this) -- or if it is sufficient if download.opensuse.org is offering them for some time, and knows places where they still exist which are possibly closer to the user (mirrors that are not yet up to date). All this will be pretty fragile without Factory having "libzypp download failover" (http://en.opensuse.org/Libzypp/Failover) enabled. I hope this will be enabled in the near future. If it is enabled, it will make sure that outdated URLs don't matter. So it is quite critical for us. A different, server-side way to make sure that clients don't end up on a mirror that has *just* deleted a file (because transfer is in progress) would be very hard to implement, without tight integration of rsync transfer and mirror database (and rsync deletes on the receiver side are not actually seen (logged) on the sender side; they could only be implied. It is a bit more feasible when we sync the factory/ tree exclusively per push sync (rsync originating from our side, connecting to the mirrors), because we are then in more control and know when the mirror tree changes in which way. Push sync makes this slightly more feasible than an ssh-triggered pull sync (which is also conceivable, in order to sync Factory out in a more organized fashion). A push sync is the way to go anyway, and we have good field experience with it for the repositories/ tree. Still experimenting. The outdatedness of many Factory mirrors during the last 10 days is fallout from my work on this; I could not continue working on it for a while though; sorry folks. Current proposal: 1) come up with a push syncing solution and set it up for some mirrors, and delayed deletion on download.o.o 2) implement libzypp download failover Both play together. 2) alone does not fix this particular problem; it requires 1) because download.opensuse.org cannot suggest mirrors for files that are unknown to it. And 1) requires 2) because 100% mirror control is not feasible, especially if mirrors pull sync.

...

Robert Kaiser

Peter -- Contact: admin@opensuse.org (a.k.a. ftpadmin@suse.com) #opensuse-mirrors on freenode.net Info: http://en.opensuse.org/Mirror_Infrastructure SUSE LINUX Products GmbH Research & Development

5546

Age (days ago)

5568

Last active (days ago)

List overview

Download

51 comments

20 participants

participants (20)

Basil Chupin
Carlos E. R.
Carlos E. R.
Daniel Fuhrmann
Dave Plater
Dominique Leuenberger
Duncan Mac-Vicar Prett
Eberhard Moenkeberg
Hans-Peter Holler
Josef Reidinger
Marcus Meissner
Patrick Shanahan
Peter Poeml
Rajko M.
Richard Guenther
Rob OpenSuSE
Robert Kaiser
Stanislav Visnovsky
Stephan Kulow
wormey＠eskimo.com