[zypp-devel] yum vs zypp performance

older
[zypp-devel] unable to run zypper...

Zhu, Peter J

10 Sep 2009 10 Sep '09

08:51

Hi, I did a yum vs zypp performance evaluation against a yum repository. Just like what http://duncan.mac-vicar.com/blog/archives/309 suggested, zypper is better than yum if I just stop after yum/zypper ask user if do a really installation/update. Summary: yum make dbcache 14.985s zypper ref -B 9.435s yum install make 6.935s zypper in make: 1.247s yum upgrade 17.486s zypper update: 1.795s But if I did a real installation/update with 57 packages in one time, zypper performance is worse than yum. Yum just needs 5m38s but zypper needs 6m40s. I can see some difference is that yum would try download all the packages at first and then install them in local system while zypper would do an iteration of retrieve/install one by one that should be the root cause. Any comments/ Thanks, Peter -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Show replies by date

Michael Schroeder

10 Sep 10 Sep

09:00

On Thu, Sep 10, 2009 at 04:51:54PM +0800, Zhu, Peter J wrote:

...

But if I did a real installation/update with 57 packages in one time, zypper performance is worse than yum. Yum just needs 5m38s but zypper needs 6m40s. I can see some difference is that yum would try download all the packages at first and then install them in local system while zypper would do an iteration of retrieve/install one by one that should be the root cause.

Hard to tell. Could also be a different mirror. You should create a copy of the repository an a local disk and update with that repository to rule out network issues. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Zhu, Peter J

09:15

Michael, Michael Schroeder wrote on 2009-09-10:

...

...
But if I did a real installation/update with 57 packages in one time, zypper

On Thu, Sep 10, 2009 at 04:51:54PM +0800, Zhu, Peter J wrote: performance is worse than yum. Yum just needs 5m38s but zypper needs 6m40s. I can see some difference is that yum would try download all the packages at first and then install them in local system while zypper would do an iteration of retrieve/install one by one that should be the root cause.

Hard to tell. Could also be a different mirror. You should create a copy of the repository an a local disk and update with that repository to rule out network issues.

But users would work with network repo mostly. Anyway, seems start a single call to curl to download all packages at first should be a better idea than making 57 calls to curl. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Duncan Mac-Vicar Prett

11:05

On Thursday 10 September 2009 10:51:54 Zhu, Peter J wrote:

...

Hi,

I did a yum vs zypp performance evaluation against a yum repository. Just like what http://duncan.mac-vicar.com/blog/archives/309 suggested, zypper is better than yum if I just stop after yum/zypper ask user if do a really installation/update.

Summary: yum make dbcache 14.985s zypper ref -B 9.435s yum install make 6.935s zypper in make: 1.247s yum upgrade 17.486s zypper update: 1.795s

But if I did a real installation/update with 57 packages in one time, zypper performance is worse than yum. Yum just needs 5m38s but zypper needs 6m40s. I can see some difference is that yum would try download all the packages at first and then install them in local system while zypper would do an iteration of retrieve/install one by one that should be the root cause.

ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first. -- Duncan Mac-Vicar P. - Engineering Manager, YaST SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Klaus Kaempf

11:18

* Duncan Mac-Vicar Prett [Sep 10. 2009 13:05]:

...

ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

Great ! But how does it download ? All in 'one go' or with a separate http connection for each package ? iirc, establishing the http connection can take considerable time. Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Michael Matz

11:28

Hi, On Thu, 10 Sep 2009, Klaus Kaempf wrote:

...

* Duncan Mac-Vicar Prett [Sep 10. 2009 13:05]:

...
ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

Great ! But how does it download ? All in 'one go' or with a separate http connection for each package ?

iirc, establishing the http connection can take considerable time.

If used correctly curl does all of this for you. If the server allows persistent connections (and most of them do) curl uses that. Don't know what the aria lib is doing. (and that feature might or might not interact with the relocation that download.o.o is doing). Ciao, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Jon Nelson

12:27

On Thu, Sep 10, 2009 at 6:28 AM, Michael Matz wrote:

...

Hi,

On Thu, 10 Sep 2009, Klaus Kaempf wrote:

...
* Duncan Mac-Vicar Prett [Sep 10. 2009 13:05]:

...
ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

Great ! But how does it download ? All in 'one go' or with a separate http connection for each package ?

iirc, establishing the http connection can take considerable time.

If used correctly curl does all of this for you. If the server allows persistent connections (and most of them do) curl uses that. Don't know what the aria lib is doing. (and that feature might or might not interact with the relocation that download.o.o is doing).

The TCP and HTTP overhead itself is so small as to be nearly free. The presence or absence of persistent (keepalive) connections helps but only at the very extreme - over thousands of downloads worth less than a few seconds in most cases. (That is not to say that it's not a very good idea to use it whenever possible for lots of *other* reasons). Even modest hardware is more than capable of thousands of requests per second - therefore, any latency observed when downloading much be assigned to some other aspect - perhaps the process by which the http connections are initiated (a process fork and exec, or library initialization, etc...) can be blamed. Still, even modest hardware should be capable of forking off and execing wget (as an example) a few dozen times a second. A person experienced in analysis could probably use 'strace -f -tt -T' (and other flags) to get a real good idea as to where zypper/yum are spending their time. My off-the-cuff guess is that the redirection that download.o.o is doing is at least partially to blame. Taking a look at a single request, it appears that d.o.o takes approx 0.29s to issue the redirection. 0.29 times 60 is 18 seconds. My guess is that 0.29 is too low as an average, especially during periods of high load. In the case of downloading a *small* .rpm (approx 18K or so) fulfilling the request actually took about 0.03s whereas the redirect was 10 times that. Perhaps it would not be unreasonable to consider looking at the 302 redirection mechanism that is in place and try to find some way to improve the response time. -- Jon -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Peter Poeml

14:41

On Thu, Sep 10, 2009 at 07:27:17AM -0500, Jon Nelson wrote:

...

On Thu, Sep 10, 2009 at 6:28 AM, Michael Matz wrote:

...
Hi,

On Thu, 10 Sep 2009, Klaus Kaempf wrote:

...
* Duncan Mac-Vicar Prett [Sep 10. 2009 13:05]:

...
ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

Great ! But how does it download ? All in 'one go' or with a separate http connection for each package ?

iirc, establishing the http connection can take considerable time.

If used correctly curl does all of this for you. If the server allows persistent connections (and most of them do) curl uses that. Don't know what the aria lib is doing. (and that feature might or might not interact with the relocation that download.o.o is doing).

The TCP and HTTP overhead itself is so small as to be nearly free. The presence or absence of persistent (keepalive) connections helps but only at the very extreme - over thousands of downloads worth less than a few seconds in most cases. (That is not to say that it's not a very good idea to use it whenever possible for lots of *other* reasons).

You underestimate the effects of introduced latency. While your picture is correct for near, low-latency connections, it looks completely different for long-distance networks. The latency can be considerable, and is incurred for each protocol step. Take 500 ms for a cross-atlantic latency, for instance, and you do see remarkable wall clock delays for traffic. Keepalive makes a huge difference there. It's quite normal to see 50% loss of speed when not using HTTP Keepalive. [...]

...

My off-the-cuff guess is that the redirection that download.o.o is doing is at least partially to blame. Taking a look at a single request, it appears that d.o.o takes approx 0.29s to issue the redirection. 0.29 times 60 is 18 seconds. My guess is that 0.29 is too low as an average, especially during periods of high load.

No, what you see is latencies. d.o.o needs merely 0.3 ms to completely respond to your request. Thus, it's a thousand times faster than it seems to you; the rest of time is spent in shuffling the little bits forth and back over the network.

...

In the case of downloading a *small* .rpm (approx 18K or so) fulfilling the request actually took about 0.03s whereas the redirect was 10 times that.

That's why we don't redirect for tiny files (< 2K). They are just sent directly.

...

Perhaps it would not be unreasonable to consider looking at the 302 redirection mechanism that is in place and try to find some way to improve the response time.

Where are you? In the US? That causes about those 0.3s of response time that you see. The only way to make this faster is to set up an additional server in the US. The concept is easy: put a server on each continent, and use GeoDNS to make the clients use the one that's closest to them. Alas, the openSUSE project lacks the resources to implement that. Thus, we live with a single server in Europe, which is slower to reach from other places. That's one reason why I have often suggested to do as few HTTP requests as possible, and avoid unneeded ones, because the cost is high (and the effect on users considerable). And Keepalive to d.o.o _does_ help, which is what yum is doing (I think). aria2c is spawned per request, which doesn't help in this regard. I think here is room for improvement. Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development

Duncan Mac-Vicar Prett

13:20

On Thursday 10 September 2009 13:18:44 Klaus Kaempf wrote:

...

* Duncan Mac-Vicar Prett [Sep 10. 2009 13:05]:

...
ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

Great ! But how does it download ? All in 'one go' or with a separate http connection for each package ?

This would help if we get parallel downloads afterwards, so the download part is done multiple times in parallel, and only for very small packages. I don't think keeping the connection open is making any difference because we have to ask download.opensuse.org for every package and get redirected to the real mirror. -- Duncan Mac-Vicar P. - Engineering Manager, YaST SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Michael Andres

13:15

On Thursday 10 September 2009 13:05:37 Duncan Mac-Vicar Prett wrote:

...

ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

zypper-1.2.3 provides just --download-only. I suppose Jano will add the options for the other policies soon. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Jano Kupec

14:18

On 09/10/2009 03:15 PM, Michael Andres wrote:

...

On Thursday 10 September 2009 13:05:37 Duncan Mac-Vicar Prett wrote:

...
ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

zypper-1.2.3 provides just --download-only. I suppose Jano will add the options for the other policies soon.

Yes, i'd like to. Michael, are these policies already configurable in zypp.conf? That would be the first place i would expect it. The command line options would be available to override the conf file ones. Any ideas how to name the command line options? Or should it be just one option taking an argument? -- cheers, jano Ján Kupec YaST team ---------------------------------------------------------(PGP)--- Key ID: 637EE901 Fingerprint: 93B9 C79B 2D20 51C3 800B E09B 8048 46A6 637E E901 ---------------------------------------------------------(IRC)--- Server: irc.freenode.net Nick: jniq Channels: #zypp #yast #suse #susecz ---------------------------------------------------------(EOF)---

Michael Andres

15:07

On Thursday 10 September 2009 16:18:09 Jano Kupec wrote:

...

...
zypper-1.2.3 provides just --download-only. I suppose Jano will add the options for the other policies soon.

Yes, i'd like to.

Michael, are these policies already configurable in zypp.conf? That would be the first place i would expect it.

Not yet.

...

The command line options would be available to override the conf file ones.

Any ideas how to name the command line options? Or should it be just one option taking an argument?

I'd prefer: --download (only|in-advance|in-heaps|as-needed)|list --download list : lists available policies. maybe --download-foo as alias for '--download foo' -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Zhu, Peter J

13:24

Duncan, Duncan Mac-Vicar Prett wrote on 2009-09-10:

...

On Thursday 10 September 2009 10:51:54 Zhu, Peter J wrote:

...
Hi,

I did a yum vs zypp performance evaluation against a yum repository. Just like what http://duncan.mac-vicar.com/blog/archives/309 suggested, zypper is better than yum if I just stop after yum/zypper ask user if do a really installation/update.

Summary: yum make dbcache 14.985s zypper ref -B 9.435s yum install make 6.935s zypper in make: 1.247s yum upgrade 17.486s zypper update: 1.795s

But if I did a real installation/update with 57 packages in one time, zypper performance is worse than yum. Yum just needs 5m38s but zypper needs 6m40s. I can see some difference is that yum would try download all the packages at first and then install them in local system while zypper would do an iteration of retrieve/install one by one that should be the root cause.

ZYpp head (what will be present in 11.2 and SLE SP1) has a policy for this, so it can be switched to download first.

Cool. Does libzypp-6.14.2 in factory support this feature? -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

5344

Age (days ago)

5344

Last active (days ago)

List overview

Download

12 comments

9 participants

participants (9)

Duncan Mac-Vicar Prett
Jano Kupec
Jon Nelson
Klaus Kaempf
Michael Andres
Michael Matz
Michael Schroeder
Peter Poeml
Zhu, Peter J