Mailinglist Archive: opensuse-buildservice (83 mails)

< Previous Next >
Re: [opensuse-buildservice] "quick" way to checkout a repo
On 2014-04-16 07:24:16 +0200, Stephan Kulow wrote:
Am 15.04.2014 19:02, schrieb Marcus Hüwe:
On 2014-04-15 12:23:27 -0400, Darin Perusich wrote:
Is there is a "quick" way to checkout a repository, like mirror
w/rsync, aria2c, or something like that? I need to checkout the
devel:languages:R:CRAN-packages repository, or at least a very large
number of packages, which has some 3400+ packages and checking out
w/osc is going to take a long time.

I assume you want to checkout the source "project" and not the (binary)
repository. AFAIK, there is no way to checkout the sources except doing
a http request for each single file.

I think we could speed up osc's checkout process, if the backend has
support for retrieving the source files of a package in a single
cpio archive. Adding such a feature shouldn't be too hard (the workers
already retrieve the sources this way).

Any opinions?

Actually in my experience with checking out large directories, the http
transfer is not the only problem with osc. osc also parses and recreates
huge XML files all the time.

What do you mean with a large directory? A project with many packages or
a package with many files? If it's the latter, do you have an example?

You notice this especially when updating existant checkouts - when
almost no files are transferred.

Hmm I would still suspect that this is due to the http requests: even
if no files changed, osc does at least one (or two, if the package is
an expanded link) http request.

But yes, speeding up source downloads ven try would help too (actually,
I'm afraid we don't even support compression for these http transfers.

Ok. I'll have a look at it.


Marcus
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-buildservice+owner@xxxxxxxxxxxx

< Previous Next >