The repository name is not sufficient, as the repository settings can change over time. Either we need to log those changes as well or we can use a full URL for the package in the log file.

Each repository can have more than one url. And I am not sure if commit knows which url was used at the end. Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Jan Kupec [Jul 31. 2008 14:43]:

I've got a mail day, so here's another one :O) We need to decide some things regarding the $subject.

DATA

According to my research, we need to log these data:

datetime (e.g. YYYY-MM-DD HH:mm:ss format) action (install | remove | upgrade | downgrade | reinstall)

- We should stick to 'update' for package updates. - What's 'reinstall' supposed to be ? + repo (see below)

reqby (auto | user) // meaning requested by - user or dependencies

It should be the real username, remember we want to be role-based in the future. Probably with a '@hostname' to cope for remote administration.

pkgname evr arch vendor repo ??? // this is requested, but do we really want it? The vendor // should be sufficient IMO

The vendor can be faked. IMHO, the repository url is needed here. (There is no need to repeat the complete repo url for hundreds of packages. A single "repo #3 is http://...." log entry and referencing the repo as '#3' is less verbose.

oldevr // relevant for upgrades/downgrades oldarch

oldvendor Only if the vendor changed

oldrepo ??? No, since the original repo is in the log.

comments welcome

FORMAT

It should be - ascii - line based (easy to view) - human readable (to some extend; i.e. not XML, but also not too verbose) - parseable (by a computer program)

i suggest a CSV format, one entry per line, with values separated by whatever char, e.g. a pipe, example:

Agreed.

2008-10-12 12:55:03|upgrade|user||0.12.0|i386|suse|dvd11|0.11.9|i386|unknown|somerepo 2008-10-12 12:55:10|remove|user|zypper|0.11.9|i386|unknown|dvd11|||| 2008-10-12 12:55:03|install|user|zypper|0.12.0|i386|suse|dvd11||||

Some suggestions: - make the action a fixed length, its easier to read - move the 'user' towards the end, the package is more important information. - don't split name, evr and arch into separate columns. name-evr.arch is a quite common notation. - have a flexible number of columns, improves readability - How about using the '@' notation also for packages and their repos ? - 'auto' could be eliminated, everything without a user is 'auto' - We could include dependency information by putting a package into the 'user' column. Don't know how useful this actually is Example: 2008-10-12 12:55:03|repo|1|dvd11 2008-10-12 12:55:03|update |libzypp-1.42-39.i586|libzypp-1.27-63.i586@1|root@localhost 2008-10-12 12:55:10|remove |zypper-0.11.9.i386|pope@vatican 2008-10-12 12:55:03|install|libzypp-1.43.0.i586@1 Alternative: 2008-10-12 12:55:03|install|libzypp-1.43.0.i586@1|zypper-0.12.0.i586@1 2008-10-12 12:55:03|install|zypper-0.12.0.i586@1|root@remote.host

This would be machine-readable and still quite human readable.

Agreed. Just prepare for value between the |'s having |'s inside ;-)

OTHER ISSUES

The request is that the package history should be human-readable as well as machine-readable. It could as well be two separate logs, one for humans, one for machines.

Well, the 'human readable' should be 'admin readable'. I don't see the need for a 'natural language' but a nicely structured, line based format in a single(!) log file.

@Michael (ma): i see you're working on refactoring the zypp commit (?). Please keep in mind that, in case of upgrades/downgrades, we will need also the data of the old solvable from which we upgrade/downgrade there.

Klaus -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Duncan Mac-Vicar Prett [Jul 31. 2008 15:09]:

...

The repository name is not sufficient, as the repository settings can change over time. Either we need to log those changes as well or we can use a full URL for the package in the log file.

Each repository can have more than one url. And I am not sure if commit knows which url was used at the end.

And it doesn't really matter for mirrors. That said, a package checksum might be needed to fully reconstruct the origin of a package. Klaus -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Klaus Kaempf wrote:

* Duncan Mac-Vicar Prett [Jul 31. 2008 15:09]:

...

...

The repository name is not sufficient, as the repository settings can change over time.

Yes, but not on the client, also on the servers!

...

...

Either we need to log those changes as well or we can use a full URL for the package in the log file.

Each repository can have more than one url. And I am not sure if commit knows which url was used at the end.

And it doesn't really matter for mirrors.

That said, a package checksum might be needed to fully reconstruct the origin of a package.

Yes, this is really a tricky part. While i understand Stano's and Duncan's points, i don't understand Klaus' :O) But first things first: if we are going to log the origin of packages (vendor, repo, etc..). We should first clarify what will be the *purpose* of this 'origin' data. What do we want to accomplish with this. This is fundamental, because if we just need a hint about the origin of the package and we will not care about somebody faking that info, we can happily use user-defined repo aliases, or names, or the vendor. If we need to do something more serious with these data, we must specify WHAT EXACTLY, and then try to find the answer to this. cheers, jano PS: Just a side implementation question, will/do we have access to Solvables in commit, and they are linked with RepoInfos somehow, right? -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

* Jan Kupec [Jul 31. 2008 15:44]:

Klaus Kaempf wrote:

...

* Duncan Mac-Vicar Prett [Jul 31. 2008 15:09]:

...

...

The repository name is not sufficient, as the repository settings can change over time.

Yes, but not on the client, also on the servers!

...

...

...

Either we need to log those changes as well or we can use a full URL for the package in the log file.

Each repository can have more than one url. And I am not sure if commit knows which url was used at the end.

And it doesn't really matter for mirrors.

That said, a package checksum might be needed to fully reconstruct the origin of a package.

Yes, this is really a tricky part. While i understand Stano's and Duncan's points, i don't understand Klaus' :O)

But first things first: if we are going to log the origin of packages (vendor, repo, etc..). We should first clarify what will be the *purpose* of this 'origin' data. What do we want to accomplish with this.

To unambiguously identify which package was installed/updated/removed. It should also serve as a 'rollback' information, if one must revert an update.

PS: Just a side implementation question, will/do we have access to Solvables in commit, and they are linked with RepoInfos somehow, right?

Yes, without it the commit would be unable to download the rpm. Klaus -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

Klaus Kaempf wrote:
> * Jan Kupec  [Jul 31. 2008 14:43]:
>> datetime (e.g. YYYY-MM-DD HH:mm:ss format)
>> action   (install | remove | upgrade | downgrade | reinstall)
> - What's 'reinstall' supposed to be ?
zypper in --force (rpm -i --force i guess). I guess this should be 
logged a separate action (?).

>> reqby    (auto | user)  // meaning requested by - user or dependencies
> 
> It should be the real username, remember we want to be role-based in
> the future. Probably with a '@hostname' to cope for remote
> administration.

OK, makes sense.

>> repo ??? // this is requested, but do we really want it? The vendor
>>          // should be sufficient IMO
> 
> The vendor can be faked. IMHO, the repository url is needed here.
> 
> (There is no need to repeat the complete repo url for hundreds of
> packages. A single "repo #3 is http://...." log entry and referencing
> the repo as '#3' is less verbose.

OK, makes sense, too. Let's see what comes out of the 'package origin' 
discussion.

> Some suggestions:
> 
> - make the action a fixed length, its easier to read

OK

> - move the 'user' towards the end, the package is more important
>   information.

OK

> - don't split name, evr and arch into separate columns. name-evr.arch
>   is a quite common notation.

I meant this to be more easily parsable. It's harder to parse 
name-evr.arch, than have these already separated by the line reader.

> - have a flexible number of columns, improves readability

OK, each action will have it own well-defined set of columns.

> - 'auto' could be eliminated, everything without a user is 'auto'

I see no problem with that. Just i side note, any package installed
by external tools and not logged will be considered as installed by
(unknown) user; for whatever purpose like auto-removal of unused packages.

> - We could include dependency information by putting a package into
>   the 'user' column. Don't know how useful this actually is

Neither do i :O) But it might be a problem to distinguish the user name 
from a pacakge-evr.a, or not (i don't know what's the policy about unix 
user names).

> Example:
> 
> 2008-10-12 12:55:03|repo|1|dvd11
> 2008-10-12 12:55:03|update |libzypp-1.42-39.i586|libzypp-1.27-63.i586@1|root@localhost
> 2008-10-12 12:55:10|remove |zypper-0.11.9.i386|pope@vatican
> 2008-10-12 12:55:03|install|libzypp-1.43.0.i586@1
> Alternative: 2008-10-12 12:55:03|install|libzypp-1.43.0.i586@1|zypper-0.12.0.i586@1
> 2008-10-12 12:55:03|install|zypper-0.12.0.i586@1|root@remote.host

>> OTHER ISSUES
>>
>> The request is that the package history should be human-readable as well as 
>> machine-readable.  It could as well be two separate logs, one for humans, 
>> one for machines.
> 
> Well, the 'human readable' should be 'admin readable'. I don't see the
> need for a 'natural language' but a nicely structured, line based
> format in a single(!) log file.

agreed

-- 
To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org
For additional commands, e-mail: zypp-devel+help@opensuse.org

* Jan Kupec  [Jul 31. 2008 16:24]:
> Klaus Kaempf wrote:
>> * Jan Kupec  [Jul 31. 2008 14:43]:
>>> datetime (e.g. YYYY-MM-DD HH:mm:ss format)
>>> action   (install | remove | upgrade | downgrade | reinstall)
>> - What's 'reinstall' supposed to be ?
> zypper in --force (rpm -i --force i guess). I guess this should be logged a 
> separate action (?).

I do not think this is needed. In the end, this is just a package
install.

>> - don't split name, evr and arch into separate columns. name-evr.arch
>>   is a quite common notation.
>
> I meant this to be more easily parsable. It's harder to parse 
> name-evr.arch, than have these already separated by the line reader.
>

Agreed. However, reading and parsing name-evr.arch is well established.

>> - 'auto' could be eliminated, everything without a user is 'auto'
>
> I see no problem with that. Just i side note, any package installed
> by external tools and not logged will be considered as installed by
> (unknown) user; for whatever purpose like auto-removal of unused packages.
>

Well, all changes to the rpm db can be logged since @system.solv gives
you the last 'valid' state. External package installs/updates will
miss the repo and user, though.

>> - We could include dependency information by putting a package into
>>   the 'user' column. Don't know how useful this actually is
>
> Neither do i :O) But it might be a problem to distinguish the user name 
> from a pacakge-evr.a, or not (i don't know what's the policy about unix 
> user names).

Stano brought up a good point on why dependency information is useful:
Product and pattern removals. If a package is installed due to a
pattern or product, keeping this relation information helps in
removing the product or pattern.

The yast2-users maintainer should know the policy about unix user
names ;-)

Klaus
-- 
To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org
For additional commands, e-mail: zypp-devel+help@opensuse.org

Dne Thursday 31 of July 2008 16:49:11 Klaus Kaempf napsal(a):

* Jan Kupec [Jul 31. 2008 16:24]:

...

Klaus Kaempf wrote:

...

* Jan Kupec [Jul 31. 2008 14:43]:

...

datetime (e.g. YYYY-MM-DD HH:mm:ss format) action (install | remove | upgrade | downgrade | reinstall)

- What's 'reinstall' supposed to be ?

zypper in --force (rpm -i --force i guess). I guess this should be logged a separate action (?).

I do not think this is needed. In the end, this is just a package install.

Well, but then when you do a rollback, you cannot distinguish between zypper in foo; rpm -e foo; zypper in foo and zypper in foo; zypper in --force foo Don't know if this is an issue, since you you will want to rollback the last zypper in and in these cases the expected result is different :-/ Anyway, this case is probably not an issue. Jiri -- Regards, Jiri Srain YaST Team Leader --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: jsrain@suse.cz Lihovarska 1060/12 tel: +420 284 028 959 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz

On Thu, Jul 31, Klaus Kaempf wrote:

Well, I believe you can. After the 'rpm -e', zypper (resp. libzypp) will not that the rpm database is not in sync with @system.solv and log the differences.

But whatever we do, it's guessing unless rpm itself provides a reliable history. In that case any commit could simply grep the last entry, and later check if something happened thereafter. What if zypp dies or is aborted by user, in or after commit, before re-rebuilding the @system.solv. The changes are already logged in the history, but not reflected by the solv file. We would re-log them as foreign. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

On Fri, Aug 01, Klaus Kaempf wrote:

Otoh, I haven't seen zypp crashing for a very long time ;-)

I can change this, if you like ;) -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org

On Thu, Jul 31, 2008 at 03:11:13PM +0200, Klaus Kaempf wrote:

* Duncan Mac-Vicar Prett [Jul 31. 2008 15:09]:

...

...

The repository name is not sufficient, as the repository settings can change over time. Either we need to log those changes as well or we can use a full URL for the package in the log file.

Each repository can have more than one url. And I am not sure if commit knows which url was used at the end.

And it doesn't really matter for mirrors.

It would be *very* helpful for us if we log the origin of a package (URL), especially the "last" URL which the client ends up using when following a series of HTTP redirects.

That said, a package checksum might be needed to fully reconstruct the origin of a package.

Klaus

Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development