[Sorry, I broke cross posting: still awaiting caffeination impact..] Am Dienstag, 11. August 2009 schrieb Dominique Leuenberger:

...

...

...

On 8/10/2009 at 19:54, Peter Poeml wrote:

But the largest blocker for me is that I don't get any commit diffs at all. (Not implemented.) How can I track the work of others, and review it, and how can they follow on what I do, without? So far, I have to rely on good commit messages, or dig out the changes in an elaborate manual process.

Well, the commit diff in a mail would be a nice addition, but you can always use osc rq show <id> --diff

This normally serves quite well. Or did you mean something else?

Well, the question is, if it is that easy, what does it take to append this to the notification mail? If something looks obviously fishy, you spare one terminal/MUA turnaround cycle (one less need to carry the <id> around, etc...) Even from a sr POV, this diff would be nice (when accepted), as I was responsible for a python-qt4 build glitch lately, and this message would have given me a clue, at which point the issue creeped in. Pete -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

Am Dienstag, 11. August 2009 10:34:58 schrieb Johannes Meixner:

Hello,

On Aug 10 19:54 Peter Poeml wrote (shortened):

...

But the largest blocker for me is that I don't get any commit diffs at all. (Not implemented.)

How can I track the work of others, and review it, and how can they follow on what I do, without? So far, I have to rely on good commit messages, or dig out the changes in an elaborate manual process.

Any kind of commit diffs is too late because then the other one had already finished his work and all you can do is to accept or reject his work but one cannot do collaboration from the beginning.

One needs an information when someone starts to work on a package to have real concurrency control and to be able to do collaboration from the beginning.

One needs transaction semantics when working on packages: 1. Begin of transaction 2. Change it 3. End of transaction

This isn't there by intention. Basically no scm implements it because it blocks other people and slow downs development. Usually first submitter wins and followers need to adapt. I can not imagine any system right now, which solves social problems, in the end people need always to talk to each other if they work on same code.

A weak synchronization point preferably with a kind of weak locking before someone starts to change it (so that all others who also work on it are at least informed) and a strong synchronization point preferably with a kind of three way merge when changes are committed.

Unfortunately - as far as I know - the build service does currently not implement concurrency control.

by intention as written above.

As far as I know the current build service does even not implement real revision control (like e.g. SVN).

it does, check "osc log". bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

...

...

On 8/11/2009 at 10:50, Adrian Schröter wrote: As far as I know the current build service does even not implement real revision control (like e.g. SVN).

it does, check "osc log".

I think the osc log is getting a bit to few attention. Also, when somebody accepts an SR, there is no log entry created (by default). I think if somebody accepts an SR, this should be marked in the osc log: osc rq accept 12345 (assuming it was an SR against GNOME:Factory/gnome-commander) then I think it might be good if: osc log GNOME:Factory/gnome-commander after shows me in the log: 'vuntz accepted SR 12345 by dimstar', followed by any other commit message 'vuntz' in this case might have added using -m. This info get's currently completely lost. (is similar to the git merge log entries) Dominique -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

* Johannes Meixner (jsmeix@suse.de) [20090811 10:37]:

One needs transaction semantics when working on packages: 1. Begin of transaction 2. Change it 3. End of transaction

Impossible to solve by technical means! You have to rely on others with check-in rights communicating with you. No software can help there. Philipp -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

Le mardi 11 août 2009, à 12:32 +0200, Philipp Thomas a écrit :

* Johannes Meixner (jsmeix@suse.de) [20090811 10:37]:

...

One needs transaction semantics when working on packages: 1. Begin of transaction 2. Change it 3. End of transaction

Impossible to solve by technical means! You have to rely on others with check-in rights communicating with you. No software can help there.

Well, software can still help... In the GNOME team, we use the "osc gnome" plugin (which you can now use for any project that is related to oS:F, not just GNOME:Factory), and there's a reservation feature in the plugin that is quite helpful. If you have any doubt, just ask anybody in the team :-) The typical example is when a new GNOME upstream release is out, we have a lot of work to do, and we can quickly see when someone is working on a package, so we just take another one. And by all means, it's a weak reservation system: you can ignore the reservation, and reservation automatically expires after 36 hours if it wasn't removed before. Vincent -- Les gens heureux ne sont pas pressés. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

Am Dienstag, 11. August 2009 12:45:57 schrieb Johannes Meixner:

Hello,

On Aug 11 10:50 Adrian Schröter wrote (shortened):

...

Am Dienstag, 11. August 2009 10:34:58 schrieb Johannes Meixner:

...

One needs transaction semantics when working on packages: 1. Begin of transaction 2. Change it 3. End of transaction

This isn't there by intention. Basically no scm implements it because it blocks other people and slow downs development. Usually first submitter wins and followers need to adapt.

I can not imagine any system right now, which solves social problems, in the end people need always to talk to each other if they work on same code.

...

A weak synchronization point preferably with a kind of weak locking before someone starts to change it (so that all others who also work on it are at least informed) and a strong synchronization point preferably with a kind of three way merge when changes are committed.

I do not understand how a weak synchronization point before blocks other people and slow downs development.

The problem with the current build service is not the optimistic concurrency control policy at commit time, the problem is that nobody knows about others who may also work on a package until it is too late to start collaborating.

Remember Coolo's mass-changes of packages. All others notice what Coolo did only when it was already done. If there was a "begin of transaction" the others could get at least informed before.

I fail to see how a package locking would help here. It would use by the script, so it would briefly lock the package and than submit it. Not much time in between. Therefore we have the source links and devel projects, one get the changes (either automatically or by manually solving via repairlink).

But as you wrote - and as other threads here show - mutual information is not really within the scope of the build service but is left to the users of the build service to do it somehow on their own beside the build service via manual mails.

We can discuss to have something like a "I am currently working on this package" flag, but I strongly mind to block something.

In short: The build service is just what it is called, a plain build service but nothing more.

This is o.k. for me - I did just misinterpret what it actually is.

...

...

As far as I know the current build service does even not implement real revision control (like e.g. SVN).

it does, check "osc log".

This is mostly useless for me:

jsmeix@nelson:/obs $ osc log openSUSE:Factory cups

shows all entries with "unknown" and "<no message>" but I know that I explicitely provided messages while I did "osc commit".

osc commit messages go only to your project. openSUSE:Factory does currently not use the build service accept mechanism unfortunatly. So the special Rudi tools would need to get extended to add "request accept" comment. Sounds important ...

In contrast in my own working copy directory

jsmeix@nelson:/obs/Printing/cups $ osc log

shows more info but still not my "osc commit" messages. In my own working copy directory I do not need such information very much because in my own working copy directory I know well what I did there.

huh ? # osc log Printing ---------------------------------------------------------------------------- r170 | autobuild | 2009-07-31 19:04:44 | 8c3f68c5fe8d9e82ec6fb6aa07a8971e | unknown checked in ---------------------------------------------------------------------------- r169 | jsmeix | 2009-07-31 15:22:11 | 5df4c71fb22b1a8ddd1d76ed399d9b79 | unknown Copy from home:jsmeix:branches:Printing/cups via accept of submit request 16211 Request was accepted with message: Fixed major bug bnc#526847 ---------------------------------------------------------------------------- r168 | autobuild | 2009-07-29 17:20:30 | 8c3f68c5fe8d9e82ec6fb6aa07a8971e | unknown checked in

In contrast it is crucial to have meningful "osc log" messages when I query whatever package on the server to find out which revisions are of particular interest regarding a particular issue.

what do you miss exactly ? bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

Hello, On Aug 11 15:03 Adrian Schröter wrote (shortened):

Am Dienstag, 11. August 2009 12:45:57 schrieb Johannes Meixner:

...

On Aug 11 10:50 Adrian Schröter wrote (shortened):

...

Am Dienstag, 11. August 2009 10:34:58 schrieb Johannes Meixner:

...

One needs transaction semantics when working on packages: 1. Begin of transaction 2. Change it 3. End of transaction

This isn't there by intention. Basically no scm implements it because it blocks other people and slow downs development. Usually first submitter wins and followers need to adapt.

I can not imagine any system right now, which solves social problems, in the end people need always to talk to each other if they work on same code.

...

A weak synchronization point preferably with a kind of weak locking before someone starts to change it (so that all others who also work on it are at least informed) and a strong synchronization point preferably with a kind of three way merge when changes are committed.

I do not understand how a weak synchronization point before blocks other people and slow downs development.

The problem with the current build service is not the optimistic concurrency control policy at commit time, the problem is that nobody knows about others who may also work on a package until it is too late to start collaborating.

Remember Coolo's mass-changes of packages. All others notice what Coolo did only when it was already done. If there was a "begin of transaction" the others could get at least informed before.

I fail to see how a package locking would help here.

I never requested hard locks. A "Begin of transaction" does not require locks. Transaction semantics is "all or nothing", i.e. either all changest between "Begin of transaction" and "End of transaction" are committed or nthing at all. Transaction semantics works well with an optimistic concurrency control policy (i.e. first commit wins). Transaction semantics with optimistic concurrency control would do: 1. Begin of transaction for package foo Remember the current write time stamps for all files of package foo 2. Read whatever files of package foo are needed and change a local copy of them. 3. End of transaction for package foo 3a) Do a hard lock for all files of package foo on the server 3b) Compare if a write time stamp for one of the files which was locally changed did already change on the server and if yes do not change anythjing on the server if no write the locally changed files onto the server and finally release the lock.

It would use by the script, so it would briefly lock the package and than submit it. Not much time in between.

Yes. The hard lock happens only while a transaction is at the end finally in it commit procedure.

Therefore we have the source links and devel projects, one get the changes (either automatically or by manually solving via repairlink).

...

But as you wrote - and as other threads here show - mutual information is not really within the scope of the build service but is left to the users of the build service to do it somehow on their own beside the build service via manual mails.

We can discuss to have something like a "I am currently working on this package" flag, but I strongly mind to block something.

Exactly what I ask for. I just want to be informed who else works on a package while I am working on it. What I have in mind is a kind of "current workers list" on the server which contains who currently works on a package. 1. Begin of transaction 1a) Lock the current workers list (for read and write) 1b) Test if current workers list is empty 1b1) if empty, continue with step 1c 1b2) if not empty, show me the current workers list and ask me if I like to work on it additionally and wait (on the server) up to 1 minute for my response 1b2a) if response is "no" or if timeout, unlock the current workers list and cancel the transaction 1b2b) if response is "yes", continue with step 1c 1c) Add myself to the current workers list 1d) Send the current workers list to all current workers 1d) Unlock the current workers list 1f) Create local copy (i.e. "Read") 2. Modify local copy 3. End of transaction 3a) Lock 3b) Test if modifications do not conflict (i.e. "Validate") 3b1) if no conflict, write modifications (i.e. "Write") 3b2) if conflict, ignore modifications (loss of work) 3c) Unlock 3d) Remove myself from the current workers list 3e) Send the current workers list to me and all current workers The two kind of "send the current workers list" messages should be processed by hermes (i.e. one can suppress them). Perferably each worker could provide an optional notification which is also stored in the "current workers list" what the reason is why he works on a package like "fixing bnc#12345".

...

In contrast in my own working copy directory

jsmeix@nelson:/obs/Printing/cups $ osc log

shows more info but still not my "osc commit" messages. In my own working copy directory I do not need such information very much because in my own working copy directory I know well what I did there.

huh ?

When I do "osc commit", there is a commit message which defaults to the differences in the changes file. I leave at least the default but often I add something more.

# osc log Printing ---------------------------------------------------------------------------- r170 | autobuild | 2009-07-31 19:04:44 | 8c3f68c5fe8d9e82ec6fb6aa07a8971e | unknown

checked in ---------------------------------------------------------------------------- r169 | jsmeix | 2009-07-31 15:22:11 | 5df4c71fb22b1a8ddd1d76ed399d9b79 | unknown

Copy from home:jsmeix:branches:Printing/cups via accept of submit request 16211 Request was accepted with message: Fixed major bug bnc#526847 ---------------------------------------------------------------------------- r168 | autobuild | 2009-07-29 17:20:30 | 8c3f68c5fe8d9e82ec6fb6aa07a8971e | unknown

checked in

...

In contrast it is crucial to have meningful "osc log" messages when I query whatever package on the server to find out which revisions are of particular interest regarding a particular issue.

what do you miss exactly ?

I do not mean the request messages but my "osc commit" messsages. I would like to have the diff of the changes file as the minimum of meaningful "osc log" messages. "osc help log" reads "Shows the commit log of a package" but currently it seems "osc log" does not show a log regarding what has changed inside a package (i.e. its changelog) but a log regarding what happened to a package in the obs i.e. whereto it was submited and if it was accepted and so on but according to my understanding this is not the commit log but a submit log. The use case which I have in mind is: End-user jane runs out there whatever package from the obs. I get a bug report from her. I let her send me "rpm -q --changelog" and therein I find a suspicious entry: * Wed Jan 14 2009 johndoe@somewhere - removed various obsolete stuff Now I want to know exactly what johndoe did at that time. My first problem is to find out the osc revision which matches to this RPM changelog entry. I don't know a direct way to get it. If I had the RPM changelog differences as the absolute minimum in the "osc log" output, I could do an appropriate "osc rdiff" but currently I have no idea - except dumb trial and error - how to get what johndoe did at that time. Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany AG Nuernberg, HRB 16746, GF: Markus Rex

On Tue, Aug 11, 2009 at 04:40:25 +0200, Klaas Freitag wrote:

...

I get commit mails for some projects (Subversion), and not for some others (devel:languages:python). The "_mypackages" filter in hermes doesn't work correctly, I gues

Hurrah for Peter because he brought me on the right track with this

Where the actual comment was (made on IRC): "looking at _mypackages, I have the impression that I get mails for packages that I maintain _if_ (and only if) I was the creator of the package."

comment. Indeed the _mypackages filter had a bug: It was just considering the people directly from the _package_ meta file, not these who inherit the permissions through the _project_ meta file. That happened because in one point of time the api stopped to combine this information in the pack meta file :-(

I fixed that accordingly and deployed already. Please keep your eyes open if the behaviour is now more logical, predictable etc. and what you expect.

Sorry,

Klaas

Thank you very much. Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development

Am Dienstag, 11. August 2009 16:39:43 schrieb Johannes Meixner:

Hello,

On Aug 11 15:03 Adrian Schröter wrote (shortened):

...

Am Dienstag, 11. August 2009 12:45:57 schrieb Johannes Meixner:

...

On Aug 11 10:50 Adrian Schröter wrote (shortened):

...

Am Dienstag, 11. August 2009 10:34:58 schrieb Johannes Meixner:

...

One needs transaction semantics when working on packages: 1. Begin of transaction 2. Change it 3. End of transaction

This isn't there by intention. Basically no scm implements it because it blocks other people and slow downs development. Usually first submitter wins and followers need to adapt.

I can not imagine any system right now, which solves social problems, in the end people need always to talk to each other if they work on same code.

...

A weak synchronization point preferably with a kind of weak locking before someone starts to change it (so that all others who also work on it are at least informed) and a strong synchronization point preferably with a kind of three way merge when changes are committed.

I do not understand how a weak synchronization point before blocks other people and slow downs development.

The problem with the current build service is not the optimistic concurrency control policy at commit time, the problem is that nobody knows about others who may also work on a package until it is too late to start collaborating.

Remember Coolo's mass-changes of packages. All others notice what Coolo did only when it was already done. If there was a "begin of transaction" the others could get at least informed before.

I fail to see how a package locking would help here.

I never requested hard locks. A "Begin of transaction" does not require locks.

Transaction semantics is "all or nothing", i.e. either all changest between "Begin of transaction" and "End of transaction" are committed or nthing at all.

Transaction semantics works well with an optimistic concurrency control policy (i.e. first commit wins).

Transaction semantics with optimistic concurrency control would do:

1. Begin of transaction for package foo Remember the current write time stamps for all files of package foo

2. Read whatever files of package foo are needed and change a local copy of them.

3. End of transaction for package foo 3a) Do a hard lock for all files of package foo on the server 3b) Compare if a write time stamp for one of the files which was locally changed did already change on the server and if yes do not change anythjing on the server if no write the locally changed files onto the server and finally release the lock.

all this is there IMHO, just that usually md5sums are used instead of timestamps.

...

It would use by the script, so it would briefly lock the package and than submit it. Not much time in between.

Yes. The hard lock happens only while a transaction is at the end finally in it commit procedure.

No need for a lock, we just do it atomar already. And another commit fails if it is not based on the former revision.

...

Therefore we have the source links and devel projects, one get the changes (either automatically or by manually solving via repairlink).

...

But as you wrote - and as other threads here show - mutual information is not really within the scope of the build service but is left to the users of the build service to do it somehow on their own beside the build service via manual mails.

We can discuss to have something like a "I am currently working on this package" flag, but I strongly mind to block something.

Exactly what I ask for. I just want to be informed who else works on a package while I am working on it.

What I have in mind is a kind of "current workers list" on the server which contains who currently works on a package.

1. Begin of transaction 1a) Lock the current workers list (for read and write) 1b) Test if current workers list is empty 1b1) if empty, continue with step 1c 1b2) if not empty, show me the current workers list and ask me if I like to work on it additionally and wait (on the server) up to 1 minute for my response 1b2a) if response is "no" or if timeout, unlock the current workers list and cancel the transaction 1b2b) if response is "yes", continue with step 1c 1c) Add myself to the current workers list 1d) Send the current workers list to all current workers 1d) Unlock the current workers list 1f) Create local copy (i.e. "Read") 2. Modify local copy 3. End of transaction 3a) Lock 3b) Test if modifications do not conflict (i.e. "Validate") 3b1) if no conflict, write modifications (i.e. "Write") 3b2) if conflict, ignore modifications (loss of work) 3c) Unlock 3d) Remove myself from the current workers list 3e) Send the current workers list to me and all current workers

What is the timeframe between 1 and 3 ? 3. itself runs already atomar, so I do not see a reason for locking. Do you really thing this is needed, why can other developers with other scms survive without ? I mean, we even have source links and submit requests which allows parallel development, so what makes us so special that we need such a system ?

The two kind of "send the current workers list" messages should be processed by hermes (i.e. one can suppress them).

Perferably each worker could provide an optional notification which is also stored in the "current workers list" what the reason is why he works on a package like "fixing bnc#12345".

A worker is someone else have write access in your project or package, right ? IMHO a submit as often as possible approach would have already this effect, and you can already subscribe to source changes in your package in hermes.

...

...

...

In contrast it is crucial to have meningful "osc log" messages when I query whatever package on the server to find out which revisions are of particular interest regarding a particular issue.

what do you miss exactly ?

I do not mean the request messages but my "osc commit" messsages.

I would like to have the diff of the changes file as the minimum of meaningful "osc log" messages.

"osc help log" reads "Shows the commit log of a package" but currently it seems "osc log" does not show a log regarding what has changed inside a package (i.e. its changelog) but a log regarding what happened to a package in the obs i.e. whereto it was submited and if it was accepted and so on but according to my understanding this is not the commit log but a submit log.

Okay, the changelog handling is on our todo list since the begin of OBS. But it is a quite complex task. However, what is important to understand is that the commit log is target for the developer, which the changes file is target for the end-user. So the content of them might complete different. Nevertheless there should be of course a connection to avoid double work.

The use case which I have in mind is:

End-user jane runs out there whatever package from the obs. I get a bug report from her. I let her send me "rpm -q --changelog" and therein I find a suspicious entry:

* Wed Jan 14 2009 johndoe@somewhere - removed various obsolete stuff

Now I want to know exactly what johndoe did at that time.

My first problem is to find out the osc revision which matches to this RPM changelog entry. I don't know a direct way to get it.

You even have more problems, you don't know from which project (or even package) this is build from and you don't know which code version it is exactly (there might be multiple versions with this last changelog entry). Last but not least, you don't know if it is really from this build service instance. We have introduced therefore the disturl tag sicne SLE 11. So please ask your users for the output of it: # rpm -q --qf '%{DISTURL}\n' osc obs://build.opensuse.org/openSUSE:Tools/openSUSE_Factory/8f3b8001f49bc9e991b80cf0d7cfb5c4- osc you can check out the code now via # osc co openSUSE:Tools osc -r 8f3b8001f49bc9e991b80cf0d7cfb5c4 or call rdiff to see the changes since then. bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

Hello, On Aug 11 18:57 Adrian Schröter wrote (shortened):

Do you really thing this is needed, why can other developers with other scms survive without ?

Of course I survive without it. Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany AG Nuernberg, HRB 16746, GF: Markus Rex