-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Saturday 2008-04-05 at 15:50 +0200, Stephan Kulow wrote:

Am Samstag, 5. April 2008 schrieb Carlos E. R.:

...

Hi,

Each week I do a "zypper dup" of factory it means downloading about 1.6 GiB, meaning a minimum of 4 hours, probably 8. Would it be possible to reduce the size somehow? Deltas, perhaps?

Yeah, we have some ideas what to do, but no time to do it. One thing we did already was switching to lzma, which saves some of the download size. Deltas are another, but that needs some effort to be done correctly.

Another improvemennt would be separating the download and the install on diferent threads or whatever, so that the next rpm is been downloaded while another is beeng installed. Higher compression also means longer time decompressing, thus longer time not downloading and perhaps longer overall time. Mmmm! That would be curious to measure! - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFH95XstTMYHG2NR9URAgfRAJ9+Dyh0BgA6FPokNmliHXSk0ATOUgCfUlkx /M7wTMN7+5HsoAq4qoAWBZQ= =X2Ds -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On 05/04/2008, Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

The Saturday 2008-04-05 at 15:50 +0200, Stephan Kulow wrote:

...

Am Samstag, 5. April 2008 schrieb Carlos E. R.:

...

Hi,

Each week I do a "zypper dup" of factory it means downloading about 1.6 GiB, meaning a minimum of 4 hours, probably 8. Would it be possible to reduce the size somehow? Deltas, perhaps?

Yeah, we have some ideas what to do, but no time to do it. One thing we did already was switching to lzma, which saves some of the download size. Deltas are another, but that needs some effort to be done correctly.

Another improvemennt would be separating the download and the install on diferent threads or whatever, so that the next rpm is been downloaded while another is beeng installed.

This was something proposed for ZYpp, but I'm not sure quite what happened. Parallel downloads in addition to this (like Smart) would help, too.

Higher compression also means longer time decompressing, thus longer time not downloading and perhaps longer overall time. Mmmm! That would be curious to measure!

See http://www.kdedevelopers.org/node/3326 for some figures and explanation. Higher compression doesn't have to necessitate either longer time decompressing or even longer time compressing -- it depends on the efficiency of your algorithm. In this case, LZMA gives considerable benefits for both the size and the speed of decompression in comparison to bzip2. Kind thoughts, -- Francis Giannaros http://francis.giannaros.org --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Saturday 2008-04-05 at 18:33 +0200, Monkey 9 wrote:

...

This was something proposed for ZYpp, but I'm not sure quite what happened. Parallel downloads in addition to this (like Smart) would help, too.

Where will the downloads be stored when simultaneous downloads will be integrated? To be prepared i already made a /temp partition about 5GB. ;)

I guess zypper would have to define a maximum size reserved for that use. Remember that another improvement that some of us have asked for is keeping the downloaded rpms somewhere in the filesystem, not deleting them, as they can be used to update another system, for instance. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFH98GDtTMYHG2NR9URAgw9AKCTU1V8z5kuLgDSQ9Ahc/X9gP5GvwCfSHgb BYEtk7z5rYYL2aiyshn7JDM= =+Xne -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Carlos E. R. schreef:

...

The Saturday 2008-04-05 at 18:33 +0200, Monkey 9 wrote:

...

...

This was something proposed for ZYpp, but I'm not sure quite what happened. Parallel downloads in addition to this (like Smart) would help, too.

Where will the downloads be stored when simultaneous downloads will be integrated? To be prepared i already made a /temp partition about 5GB. ;)

I guess zypper would have to define a maximum size reserved for that use. Remember that another improvement that some of us have asked for is keeping the downloaded rpms somewhere in the filesystem, not deleting them, as they can be used to update another system, for instance.

-- Cheers, Carlos E. R.

I anticipated that, that's why it is that big.. ;) "Libzypp has had the ability to keep downloaded packages for a while now. "Keep packages behavior is enabled/disabled by repository. Add 'keeppackages=1' to .repo file." Above quote from zypper-devel list 14th Feb. Simply set keeppackages in

Monkey 9 wrote: the appropriate .repo file in /etc/zypp and rpms stay in /var/cache/zypp/packages in the repo's cache directory. regards Dave --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Saturday 2008-04-05 at 17:12 +0100, Francis Giannaros wrote:

...

Another improvemennt would be separating the download and the install on diferent threads or whatever, so that the next rpm is been downloaded while another is beeng installed.

This was something proposed for ZYpp, but I'm not sure quite what happened. Parallel downloads in addition to this (like Smart) would help, too.

It would be nice. I have been 5 hours downloading things... a part of that time is the installation time.

...

Higher compression also means longer time decompressing, thus longer time not downloading and perhaps longer overall time. Mmmm! That would be curious to measure!

See http://www.kdedevelopers.org/node/3326 for some figures and explanation.

[...] Interesting! So he did measure it. :-) Ok, er... how about measuring with slow machines? :-) (ducking)

Higher compression doesn't have to necessitate either longer time decompressing or even longer time compressing -- it depends on the efficiency of your algorithm.

In this case, LZMA gives considerable benefits for both the size and the speed of decompression in comparison to bzip2.

Could be. Byt default the logs are compressed as bzip2 nowdays; when I have to search in them using zgrep it takes way longer, so that I'm considering forcing gzip again.... done. "apropos lzma" returns nothing. What utility is used? - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFH98DEtTMYHG2NR9URAn6uAKCE2cn7/UI1uhP8Nds33xPw0hpX0gCcCD22 YsuvmokvkKAo0Ta82J97EEU= =6n3t -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Juan Erbes schreef:

2008/4/5, Carlos E. R. :

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Hi,

Each week I do a "zypper dup" of factory it means downloading about 1.6 GiB, meaning a minimum of 4 hours, probably 8. Would it be possible to reduce the size somehow? Deltas, perhaps?

I tried to update yesterday, but some packages were not yet synced to the mirrors, thus instead of updating overnight I have to do it on Saturday, loosing one test day of the the weekend.

Welcome to factory!

Many times, when the download of about 2 GB was at 95% with smart, the files in the mirrors changed, and then appears dependency problems. The only way to use the 95% of the 2 GB of packages, was, installing it by hand and forcing dependencys, and later making a new update, to complete it. After 8 or 9 "forced" upgrades, appears rpmdb corruptions.

Cheers

Yes, not nice at all. I noticed that with smaller updates, some 600MB. When the wrong pkgs change, or pkgs move from unstable to stable, you have no updater at all. I had to reinstall from a3KDE cd yesterday for that reason. Still very much probs from that.. ;( But it seems to progresss now. QT was wrong version when update stopped, so yast not working.. But luckily this time zypper does, last time nothing worked, no rpm-data base, no pkgmngr... Last days very many server time outs.. this is killing the complete os..;( The downloads are incredibly fast now :) (if only there were no server-timeouts.... -- Nice continuance of your life to you... M9. Besturingssysteem: Linux 2.6.25-rc7-git2-11-default x86_64 Current user: monkey9@AMD64x2-sfn1 System: openSUSE 11.0 (x86_64) Alpha3 KDE: 4.00.67 (KDE 4.0.67 >= 20080325) "release 3.2" --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Felix Miata schreef:

On 2008/04/05 17:44 (GMT+0200) Monkey 9 apparently typed:

...

I noticed that with smaller updates, some 600MB. When the wrong pkgs change, or pkgs move from unstable to stable, you have no updater at all. I had to reinstall from a3KDE cd yesterday for that reason. Still very much probs from that.. ;( But it seems to progresss now. QT was wrong version when update stopped, so yast not working.. But luckily this time zypper does, last time nothing worked, no rpm-data base, no pkgmngr... Last days very many server time outs.. this is killing the complete os..;(

I recently started Factory Smart updating with a script containing: smart upgrade --yes rpm smart upgrade --yes smart zypper glibc perl-Bootloader mkinitrd yast2-bootloader

urpmi sort of does the same thing automatically.

The probs with rpmdatabase could not be solved, rebuilding did not work, there was just no access... I thought smart would be smart enough to fix a problem: The list of unsolved deps was long, fix all resulted in no action at all.... Or i do not have the right repo's installed, or the missing pkgs are not present. One in particular irritates me: lipzypp.so.406 (64bit), it is needed by yast2-qt-pkg, but not found, and yast-gui needs plugin qt_pkg, which will not install without that: 'UI Syntax error: couldn't load plugin qt_pkg, check the log file!' -- Nice continuance of your Day/Night... M9. OS: Linux 2.6.25-rc5-git3-6-default x86_64 Current user: monkey9@AMD64x2-sfn1 System: openSUSE 11.0 (x86_64) Alpha3 KDE: 4.0.2 (KDE 4.0.2) "release 8" Checking relations... All checked packages have correct relations. Checking relations... Unsatisfied dependency: CASA_auth_token_client-1.7.1414-113@i586 requires libgssapi.so.2 Unsatisfied dependency: CASA_auth_token_client-1.7.1414-113@i586 requires libgssapi.so.2(libgssapi_CITI_2) Unsatisfied dependency: NetworkManager-novellvpn-0.7.0-33@i586 requires novell-nortelplugins >= 0.1 Unsatisfied dependency: NetworkManager-novellvpn-0.7.0-33@x86_64 requires novell-nortelplugins >= 0.1 Unsatisfied dependency: PackageKit-0.1.9.2-6@i586 requires libzypp.so.406 Unsatisfied dependency: PackageKit-0.1.9.2-6@x86_64 requires libzypp.so.406()(64bit) Unsatisfied dependency: amavisd-new-2.5.1-81@i586 requires unace Unsatisfied dependency: amavisd-new-2.5.1-81@x86_64 requires unace Unsatisfied dependency: antivir-avguard-2.1.10.15-31@i586 requires antivir = 2.1.10.15 Unsatisfied dependency: antivir-gui-2.1.10.15-49@i586 requires antivir Unsatisfied dependency: audit-libs-python-1.6.8-2@i586 requires python = 2.5 Unsatisfied dependency: audit-libs-python-1.6.8-2@x86_64 requires python = 2.5 Unsatisfied dependency: autotest-0.1.1208-17@i586 requires /usr/bin/python2.4 Unsatisfied dependency: autotest-0.1.1208-17@x86_64 requires /usr/bin/python2.4 Unsatisfied dependency: eclipse-platform-3.3-125@noarch requires jre1.6.x Unsatisfied dependency: gnumeric-1.8.2-3@i586 requires agfa-fonts Unsatisfied dependency: gnumeric-1.8.2-3@x86_64 requires agfa-fonts Unsatisfied dependency: graphviz-tcl-2.18-5@i586 requires /usr/bin/php Unsatisfied dependency: graphviz-tcl-2.18-5@i586 requires /usr/bin/lua Unsatisfied dependency: graphviz-tcl-2.18-5@x86_64 requires /usr/bin/php Unsatisfied dependency: graphviz-tcl-2.18-5@x86_64 requires /usr/bin/lua Unsatisfied dependency: iceWMCP-addons-3.2-87@noarch requires java-sun Unsatisfied dependency: javagroups-2.1.1-233@noarch requires jms Unsatisfied dependency: libnotify1-32bit-0.4.4-113@x86_64 requires libatk-1_0-0-32bit = 0.4.4 Unsatisfied dependency: qpid-mono-0.1-8@i586 requires mono(nunit.framework) = 2.2.8.0 Unsatisfied dependency: qpid-mono-0.1-8@x86_64 requires mono(nunit.framework) = 2.2.8.0 Unsatisfied dependency: ruby-RMagick-1.15.8-80@x86_64 requires libWand.so.10()(64bit) Unsatisfied dependency: ruby-RMagick-1.15.8-80@x86_64 requires libMagick.so.10()(64bit) Unsatisfied dependency: susedoc-4.2_20070830-1@noarch requires agfa-fonts Unsatisfied dependency: virtualbox-ose-guest-tools-1.5.4-16@i586 requires virtualbox-kmp Unsatisfied dependency: virtualbox-ose-guest-tools-1.5.4-16@x86_64 requires virtualbox-kmp Unsatisfied dependency: yast2-qt-pkg-2.16.29-10@x86_64 requires libzypp.so.406()(64bit)

Carlos E. R. schreef:

The Saturday 2008-04-05 at 20:19 +0200, Monkey 9 wrote:

...

One in particular irritates me: lipzypp.so.406 (64bit), it is needed by yast2-qt-pkg, but not found,

I had that yesterday. I postponed the update till today, tried again, worked fine. Guess I had to wait for mirror sync.

But I use zypper.

-- Cheers, Carlos E. R.

It is not about zypper or yast, the apps should work, that is all...;) I am testing to ensure this is a fact. Will try today. Now i have, since install from CD, that my seconds in clock don't work anymore... (had the updates already, but made no difference..) -- Enjoy your time around, Oddball (Now or never...) OS: Linux 2.6.25-rc8-12-default x86_64 Current user: oddball@AMD64x2-sfn1 System: openSUSE 11.0 (x86_64) Alpha3 KDE: 4.0.3 (KDE 4.0.3) "release 3.2" --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Saturday 2008-04-05 at 12:18 -0300, Juan Erbes wrote:

Welcome to factory!

Many times, when the download of about 2 GB was at 95% with smart, the files in the mirrors changed, and then appears dependency problems. The only way to use the 95% of the 2 GB of packages, was, installing it by hand and forcing dependencys, and later making a new update, to complete it. After 8 or 9 "forced" upgrades, appears rpmdb corruptions.

Well, that's why I update on Friday night or Saturday, hopping that developers will be resting and leaving the mirrors alone to catch up ;-) And I make from inside the 10.3 stable partition - else I'd might be stuck with an unusable machine for hours. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFH98JQtTMYHG2NR9URAuT0AJ9k4BX86KoFSltdr3Z8H4bVfftKjACfcjvG qY1VFJuc2a+nMyLXD488o9c= =Tz1W -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org