Mailinglist Archive: opensuse-buildservice (262 mails)

< Previous Next >
Re: [opensuse-buildservice] Re: Notifications (was: Re: [opensuse-packaging] New packages for factory)
  • From: Adrian Schröter <adrian@xxxxxxx>
  • Date: Tue, 11 Aug 2009 18:57:17 +0200
  • Message-id: <200908111857.17508.adrian@xxxxxxx>
Am Dienstag, 11. August 2009 16:39:43 schrieb Johannes Meixner:
Hello,

On Aug 11 15:03 Adrian Schröter wrote (shortened):
Am Dienstag, 11. August 2009 12:45:57 schrieb Johannes Meixner:
On Aug 11 10:50 Adrian Schröter wrote (shortened):
Am Dienstag, 11. August 2009 10:34:58 schrieb Johannes Meixner:
One needs transaction semantics when working on packages:
1. Begin of transaction
2. Change it
3. End of transaction

This isn't there by intention. Basically no scm implements it because
it blocks other people and slow downs development. Usually first
submitter wins and followers need to adapt.

I can not imagine any system right now, which solves social problems,
in the end people need always to talk to each other if they work on
same code.

A weak synchronization point preferably with a kind of weak locking
before someone starts to change it (so that all others who also
work on it are at least informed) and a strong synchronization point
preferably with a kind of three way merge when changes are committed.

I do not understand how a weak synchronization point before
blocks other people and slow downs development.

The problem with the current build service is not the
optimistic concurrency control policy at commit time,
the problem is that nobody knows about others who may also work
on a package until it is too late to start collaborating.

Remember Coolo's mass-changes of packages.
All others notice what Coolo did only when it was already done.
If there was a "begin of transaction" the others could get
at least informed before.

I fail to see how a package locking would help here.

I never requested hard locks.
A "Begin of transaction" does not require locks.

Transaction semantics is "all or nothing", i.e. either all
changest between "Begin of transaction" and "End of transaction"
are committed or nthing at all.

Transaction semantics works well with an optimistic concurrency
control policy (i.e. first commit wins).

Transaction semantics with optimistic concurrency control would do:


1. Begin of transaction for package foo
Remember the current write time stamps for all files of package foo

2. Read whatever files of package foo are needed
and change a local copy of them.

3. End of transaction for package foo
3a)
Do a hard lock for all files of package foo on the server
3b)
Compare if a write time stamp for one of the files
which was locally changed did already change on the server and
if yes do not change anythjing on the server
if no write the locally changed files onto the server
and finally release the lock.

all this is there IMHO, just that usually md5sums are used instead of
timestamps.


It would use by the
script, so it would briefly lock the package and than submit it. Not much
time in between.

Yes.
The hard lock happens only while a transaction is
at the end finally in it commit procedure.

No need for a lock, we just do it atomar already. And another commit fails if
it is not based on the former revision.

Therefore we have the source links and devel projects, one get the
changes (either automatically or by manually solving via repairlink).

But as you wrote - and as other threads here show - mutual information
is not really within the scope of the build service but is left to the
users of the build service to do it somehow on their own beside the
build service via manual mails.

We can discuss to have something like a "I am currently working on this
package" flag, but I strongly mind to block something.

Exactly what I ask for.
I just want to be informed who else works on a package
while I am working on it.

What I have in mind is a kind of "current workers list" on the server
which contains who currently works on a package.

1. Begin of transaction
1a) Lock the current workers list (for read and write)
1b) Test if current workers list is empty
1b1) if empty, continue with step 1c
1b2) if not empty, show me the current workers list
and ask me if I like to work on it additionally
and wait (on the server) up to 1 minute for my response
1b2a) if response is "no" or if timeout,
unlock the current workers list
and cancel the transaction
1b2b) if response is "yes", continue with step 1c
1c) Add myself to the current workers list
1d) Send the current workers list to all current workers
1d) Unlock the current workers list
1f) Create local copy (i.e. "Read")
2. Modify local copy
3. End of transaction
3a) Lock
3b) Test if modifications do not conflict (i.e. "Validate")
3b1) if no conflict, write modifications (i.e. "Write")
3b2) if conflict, ignore modifications (loss of work)
3c) Unlock
3d) Remove myself from the current workers list
3e) Send the current workers list to me and all current workers

What is the timeframe between 1 and 3 ?

3. itself runs already atomar, so I do not see a reason for locking.

Do you really thing this is needed, why can other developers with other scms
survive without ?

I mean, we even have source links and submit requests which allows parallel
development, so what makes us so special that we need such a system ?

The two kind of "send the current workers list" messages should
be processed by hermes (i.e. one can suppress them).

Perferably each worker could provide an optional notification
which is also stored in the "current workers list"
what the reason is why he works on a package like
"fixing bnc#12345".

A worker is someone else have write access in your project or package, right ?

IMHO a submit as often as possible approach would have already this effect,
and you can already subscribe to source changes in your package in hermes.



...
In contrast it is crucial to have meningful "osc log" messages
when I query whatever package on the server to find out which
revisions are of particular interest regarding a particular issue.

what do you miss exactly ?

I do not mean the request messages but my "osc commit" messsages.

I would like to have the diff of the changes file as the minimum
of meaningful "osc log" messages.

"osc help log" reads "Shows the commit log of a package" but
currently it seems "osc log" does not show a log regarding
what has changed inside a package (i.e. its changelog)
but a log regarding what happened to a package in the obs
i.e. whereto it was submited and if it was accepted and so on
but according to my understanding this is not the commit log
but a submit log.


Okay, the changelog handling is on our todo list since the begin of OBS. But
it is a quite complex task.

However, what is important to understand is that the commit log is target for
the developer, which the changes file is target for the end-user. So the
content of them might complete different. Nevertheless there should be of
course a connection to avoid double work.

The use case which I have in mind is:

End-user jane runs out there whatever package from the obs.
I get a bug report from her.
I let her send me "rpm -q --changelog" and therein I find
a suspicious entry:

* Wed Jan 14 2009 johndoe@somewhere
- removed various obsolete stuff

Now I want to know exactly what johndoe did at that time.

My first problem is to find out the osc revision which matches
to this RPM changelog entry. I don't know a direct way to get it.

You even have more problems, you don't know from which project (or even
package) this is build from and you don't know which code version it is
exactly (there might be multiple versions with this last changelog entry).
Last but not least, you don't know if it is really from this build service
instance.

We have introduced therefore the disturl tag sicne SLE 11. So please ask your
users for the output of it:

# rpm -q --qf '%{DISTURL}\n' osc
obs://build.opensuse.org/openSUSE:Tools/openSUSE_Factory/8f3b8001f49bc9e991b80cf0d7cfb5c4-
osc

you can check out the code now via

# osc co openSUSE:Tools osc -r 8f3b8001f49bc9e991b80cf0d7cfb5c4

or call rdiff to see the changes since then.

bye
adrian

--

Adrian Schroeter
SUSE Linux Products GmbH
email: adrian@xxxxxxx

--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-buildservice+help@xxxxxxxxxxxx

< Previous Next >