[opensuse-buildservice] Deltarpm generation and size limits
I was wondering why I was getting so few deltarpms for big package updates (like libreoffice), and I went and checked the source[0] It turns out that in line 32 it removes the drpm if its size is more than half the non-delta size. Thing is, this criterion is really ill-suited for big RPMs (which is where deltas save the most bandwidth), since a DRPM on libreoffice could be more than half the non-delta size, but still save considerable bandwidth (hundreds of MB). So... why not use a different threshold? I propose 80% for packages bigger than 200k, and 50% for smaller packages. Pseudo-patch: @ -32,1 +32,5 - let drpmsize=$drpmsize+$drpmsize + if (( $drpmsize/200 < 204800 )) ; then + let ndrpmsize=$drpmsize+$drpmsize + else + let ndrpmsize=$drpmsize+$drpmsize/5 + fi [0] https://github.com/openSUSE/open-build-service/blob/master/src/backend/worke... -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Sat, Aug 11, 2012 at 07:04:12PM -0300, Claudio Freire wrote:
I was wondering why I was getting so few deltarpms for big package updates (like libreoffice), and I went and checked the source[0]
It turns out that in line 32 it removes the drpm if its size is more than half the non-delta size.
Thing is, this criterion is really ill-suited for big RPMs (which is where deltas save the most bandwidth), since a DRPM on libreoffice could be more than half the non-delta size, but still save considerable bandwidth (hundreds of MB).
So... why not use a different threshold?
I propose 80% for packages bigger than 200k, and 50% for smaller packages.
Yes, we could do that. The drawback is that it also takes a lot of time to reassemble the rpm from the deltarpm (the payload has to be compressed again, an operation that isn't cheap), so I used a fixed percentage. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Mon, Aug 13, 2012 at 6:37 AM, Michael Schroeder <mls@suse.de> wrote:
So... why not use a different threshold?
I propose 80% for packages bigger than 200k, and 50% for smaller packages.
Yes, we could do that. The drawback is that it also takes a lot of time to reassemble the rpm from the deltarpm (the payload has to be compressed again, an operation that isn't cheap), so I used a fixed percentage.
I remember the discussion. But unless you have a download bandwidth greater than the recompression bandwidth, it should work out fine. And recompression bandwidth is quite big (MB/s). But, more importantly, bandwidth depends on the user, not the server. OBS should generate the deltas, and zypper should be told not to use them unless it's worth it (which is a simple function of known bandwidths). The server cannot make that decision. In my case, with a 1Mbps connection, recompression is quite a bit faster than download.
From the server's perspective, saving bandwidth should be top priority.
Just MHO. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (2)
-
Claudio Freire -
Michael Schroeder