[opensuse-buildservice] osc ci downloads large amounts of data prior to check in where does it go?
Hi, I'm a disk space disadvantaged packager and I've found that when I use osc ci from local copy that osc is downloading a large quantity of data, +-300MB, prior to the diff screen coming up. Where does all this data go and what is it for? This particular package home:plater kicad had a 188MB doc.tar.bz2 in it at the time, due to having bzr info in it. I've noticed this behaviour with other packages with large tarballs as well, is osc possibly downloading the tarballs to compare them? If so couldn't this comparison be done in another way like an md5 sum for instance? Regards Dave P -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Mittwoch, 26. Mai 2010, 10:18:23 schrieb Dave Plater:
Hi, I'm a disk space disadvantaged packager and I've found that when I use osc ci from local copy that osc is downloading a large quantity of data, +-300MB, prior to the diff screen coming up. Where does all this data go and what is it for? This particular package home:plater kicad had a 188MB doc.tar.bz2 in it at the time, due to having bzr info in it. I've noticed this behaviour with other packages with large tarballs as well, is osc possibly downloading the tarballs to compare them? If so couldn't this comparison be done in another way like an md5 sum for instance?
Seriously, when you deal with such large files, you should not complain about disk usage. You can use osc co -l $SIZE option with latest osc to ignore files larger than the defined value. And with osc add http://.... you can even avoid to download it at all to your system when submitting a new version. It still happens when doing a local build of course. bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
On 05/26/2010 10:40 AM, Adrian Schröter wrote:
Am Mittwoch, 26. Mai 2010, 10:18:23 schrieb Dave Plater:
Hi, I'm a disk space disadvantaged packager and I've found that when I
use osc ci from local copy that osc is downloading a large quantity of
data, +-300MB, prior to the diff screen coming up. Where does all this
data go and what is it for? This particular package home:plater kicad
had a 188MB doc.tar.bz2 in it at the time, due to having bzr info in it.
I've noticed this behaviour with other packages with large tarballs as
well, is osc possibly downloading the tarballs to compare them? If so
couldn't this comparison be done in another way like an md5 sum for
instance?
Seriously, when you deal with such large files, you should not complain about
disk usage.
You can use
osc co -l $SIZE
option with latest osc to ignore files larger than the defined value.
And with
osc add http://....
you can even avoid to download it at all to your system when submitting
a new version. It still happens when doing a local build of course.
bye
adrian
You still haven't answered the question about what osc was downloading prior to uploading the checked in packages. The amount of download was almost as if osc downloads the files on the build service server before checking in the new files. This must be a problem for the openoffice maintainer for instance. Regards Dave P -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 2010-05-26 10:18:23 +0200, Dave Plater wrote:
Hi, I'm a disk space disadvantaged packager and I've found that when I use osc ci from local copy that osc is downloading a large quantity of data, +-300MB, prior to the diff screen coming up. Where does all this data go and what is it for? This particular package home:plater kicad had a 188MB doc.tar.bz2 in it at the time, due to having bzr info in it. I've noticed this behaviour with other packages with large tarballs as well, is osc possibly downloading the tarballs to compare them? If so couldn't this comparison be done in another way like an md5 sum for instance? You've hit bug #599186. The solution is a "rewrite" of osc's update/working copy handling. It's already on my todo but it definitely won't make it into the next osc release (because it's a bigger change and the release will be soon and I'm a bit busy atm).
Marcus -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 05/27/2010 12:04 AM, Marcus Hüwe wrote:
On 2010-05-26 10:18:23 +0200, Dave Plater wrote:
Hi, I'm a disk space disadvantaged packager and I've found that when I use osc ci from local copy that osc is downloading a large quantity of data, +-300MB, prior to the diff screen coming up. Where does all this data go and what is it for? This particular package home:plater kicad had a 188MB doc.tar.bz2 in it at the time, due to having bzr info in it. I've noticed this behaviour with other packages with large tarballs as well, is osc possibly downloading the tarballs to compare them? If so couldn't this comparison be done in another way like an md5 sum for instance?
You've hit bug #599186. The solution is a "rewrite" of osc's update/working copy handling. It's already on my todo but it definitely won't make it into the next osc release (because it's a bigger change and the release will be soon and I'm a bit busy atm).
Marcus
Thanks for the explanation, I maintain two packages with 65+MB doc files, the one I refer to above was large because of a mistake on my part. I know now that I should work around the bug by using the web ui to remove and add the large files, I've also discovered xz compression. At least the build service works well with a mix of web ui and local work now, it used to cause me big headaches with spec file conflicts. I've cc'ed myself in the bug. I've also discovered that quilt leaves large files in /temp. Thanks Dave P -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Thu, May 27, 2010 at 08:08:19AM +0200, Dave Plater wrote:
I've also discovered that quilt leaves large files in /temp.
Please file a separate new bug for the quilt issue if you're able to reporoduce it. As we're using it on many different openSUSE and SUSE Linux Enterprise at least with kernel and Samba packages I'm quite curious what's causing your trouble. Lars -- Lars Müller [ˈlaː(r)z ˈmʏlɐ] Samba Team SUSE Linux, Maxfeldstraße 5, 90409 Nürnberg, Germany
participants (4)
-
Adrian Schröter -
Dave Plater -
Lars Müller -
Marcus Hüwe