Mailinglist Archive: opensuse-buildservice (348 mails)

< Previous Next >
Re: [opensuse-buildservice] osc co/up hangs and eventually corrupts working copy
  • From: Olaf Hering <olaf@xxxxxxxxx>
  • Date: Tue, 27 Apr 2010 18:53:07 +0200
  • Message-id: <20100427165307.GA22142@xxxxxxxxx>
On Tue, Apr 27, Adrian Schröter wrote:

Once I ran the same osc command for my project, I experienced the same
hangs. Most of the time 2MB of a file were downloaded, then osc hung.
If interrupted with ctrl c, osc may eventually be able to continue with
'osc up'. I tried it many times. osc downloads a block of 1048576 or
2097152 bytes, then it stops.
This morning I was finally able to finish a fresh checkout after a few
attempts.

Can you please retry with "osc -d ..." and look from where these packages
are comming ?

-d shows all files coming from https://api.opensuse.org/source/home...

It might be an api problem or a mirror problem, because osc tries first
to use download.opensuse.org redirector.

download.opensuse.org is not listed in the local named log, only
api.opensuse.org is.

But the thing is: once more packages get added via the webinterface an
'osc up' inside the local working copy will hang as well. If interrupted
it will corrupt the working copy by adding a new incomplete directory.
Several 'osc up' runs will stop with 'directory XY does already exist'.
Removing that XY directory does seldom help.

Hm, do you use some proxy or anything else what makes your setup special ?
I am not aware that anyone else has such problems atm.

I looked at it briefly. Its the way things are downloaded.
It probably goes like this:

get package_list
foreach pkg in package_list
mkdir pkg
get filelist > pkg/.osc/_files
foreach file in filelist
get file > pkg/file
update .osc/_packages

To make the ctrl c during checkout or update robust it may be like this:

foreach pkg in package_list
get filelist > ${tmp}/_files
update .osc/_packages
mkdir pkg
cp ${tmp}/_files > pkg/.osc/_files
foreach file in filelist
get file > pkg/file

If I update the .osc/_packages file manually, osc knows about an
incomplete package and tries to update it.

The osc meta data look very simple and allow edit with a texteditor, so
its possible to fix things up manually.

Is there a way to fix the local working copy from such breakage?
Doing a fresh checkout is appearently no option, as it starts to hang
again.

Perhaps the servers should return some sort of EBUSY right away when
they cant service the 'checkout/update' request. This would prevent the
local corruption and it probably reduces the load on the servers,
because repeated checkout attempts may put even more pressure on them.

right, local corruption should not happen in any case.

Its the way files and metadata are downloaded.
If they are downloaded and stored in the correct order, osc can be more
robust and handle such situations.

So, this can not be a load problem IMHO, but maybe we have a bug somewhere.
I just wonder what is triggering this and how we can reproduce it.

I can trigger it most of the time.
According to the 'time osc -v -v -d -H up --unexpand-link' output, it
can hang in any of the GET requests:

PRJ/PKG/?rev=latest
PRJ/PKG/_meta
PRJ/PKG/filename

But it can be of course also just a broken router on your side ...

Maybe, but my workload has not shown any issues so far with the network
connection.


Olaf
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-buildservice+help@xxxxxxxxxxxx

< Previous Next >
Follow Ups