Mailinglist Archive: opensuse-buildservice (351 mails)

< Previous Next >
Re: [opensuse-buildservice] openSUSE Build Service Trust concept
  • From: Dirk Mueller <dmueller@xxxxxxx>
  • Date: Wed, 9 Jul 2008 00:27:07 +0200
  • Message-id: <200807090027.08379.dmueller@xxxxxxx>
On Monday 07 July 2008, Marko Jung wrote:

http://en.opensuse.org/Build_Service/Concepts/Trust

[quote rpmlint]
A first implementation should only check whether any test failed and possibly
how many problems have been found. As a subsequent improvement, I would
suggest to add a parser to identify the failed tests and to further add a
severity measure for the identified problems.
[/quote]

We already have such a system with rpmlint: each check has an associated
"badness" score that is cumulated. However, it is a weak indication of trust:
checks can be filtered, and if one chooses to trust a package that has certain
"fatal" checks filtered depends on quite some factors that one probably can not
easily judge.

[quality checks]
I think the most obvious ones here are missing: no implicit file conflicts, no
"evil" reverse dependencies (supplements, enhances), no conflicts or obsoletes
on core packages. nonstandardized %pre/%post scripts are also something that
should lower trust. Number of services that are installed at boot, firewall
exceptions etc.

[Reverse build dependencies]

I think the paragraph is the wrong way around: a package doesn't need a high
trust level if it is a leaf - package. if it is a core package with many other
things depending on it, we don't want to have "newbie changes" in there. Note
that this is not only about buildtime but also about runtime dependencies.

[action based ratings]
I agree that actions are the best indicator for trust - an accepted submission
request should grow trust between the two parties, permanently declined
requests lower it. submission "dialogs" however again indicate an ongoing
relationship.

[package history / version numbers]
The easiest way is probably comparing timestamps of the files in the source
tarball versus timestamp of the spec file. or something like that. In any case
I think this is not really useful as an indicator of trust: if the time
between upstream and package is really short, then this either indicates a
script at work or some spoofing going on. if the time is long, it does not mean
much either: it could be that the previous packager dropped dead and somebody
else finally came around updating the package. this situation should increase
the trust in the package instead of lowering it.

Overall, I think the conceptual description is not very clear and is lacking
some fundamental decisions, which make evaluation of the proposed concept
difficult:

a) is trust an attribute of an identity, of modifications, projects, packages
or binaries? I think this is the most important question to answer. according
to the introduction, trust appears to be a relationship between identities.
identities are however only loosely connected to projects or packages. the
proposed implementation suggests that the trust is related to a project
(lowest trustlevel of all maintainers). The implementation proposal is btw
flawed - trust has to be sticky, otherwise I can trojan a project and
afterwards remove myself from the package maintainership list, and the package
would immediately gain high trust level again.
I suggest to split and clarify the definition of trust as a relationship
between and as an entity of modifications (submit requests or OBS commits). It
is not generally true that a brillant packager of package foo is also good
when he touches package bar.


b) the novell/opensuse employee exceptions in the trust scheme do not make it
trustworthy to outsiders. an opensuse employee is not inherently less
vulnerable to screwing something up. it should be possible to reduce those
exceptions to a bare minimum, or make them symmetric so that any outside group
of people can gain the same privileges vice versa.

c) the quoted definition for trust is quite good: A trusted party is presumed
to seek to fulfill ethical codes, law and policies. Therefore it would be
useful to propose a list of policies (law's) that allow, when being fulfilled,
being an indicator of trust. One obvious example: the source of tarballs
should be verifyable, be it URL, gpg signature of the packager or some other
way defined by the upstream project. Other indicators are easy to come up with.
for each of those, the impact on the trust should be elaborated: how bad is it
if a tarball is updated but the signature is not? how bad is it if a signature
is removed upon a version upgrade? how good is it if somebody adds a signature
or if somebody verifies the tarball? Note that trusting somebody doesn't mean
that some weird datamining determined him to be trustworthy. it means that, a
particular person seeked to fulfill previous promises, ethical standards and
policies. A clear prerequirement here is: the list of promises and policies to
be defined from the start.

d) category ratings&Acceptance: This is a difficult part. how much impact has a
rating from a total stranger? how do you define trust for a rating from
somebody to whom you have no trust relationship with? Also, are ratings
related to an identity, to a package, to a particular package version? can
people grant cookies (virtual beer) to people who fix their bugreports in the
package? can they express distrust to a package when a version upgrade, when
being installed, hoses their system? and if they do so, is the distrust
bouncing back to the packager, to the lastest change, to the package..? if
they rate a package, does that increase trust in the packager, in the one
doing the latest change, in the package..?

e) we want to reward people for complying to policies and for being active. we
don't necessarily want to reward them for screwing up, but as time goes by,
shit happens. the stickyness of bad behaviour in trust systems like e.g. ebay
cause people to change their identity quite frequently. we would like to avoid
that. I would propose therefore to track badness of packages independently of
"badness" of identities. it should be possible to get rid of "mistrust" by
dropping a really bad package (perhaps it was all upstreams fault, really),
and to gain trust by being more active and interacting with more people. A
person who seeks relationship is more trustworthy than one who does not. A
reward system might be a good indicator and motivational factor for people to
participate in the trust scheme. an important factor for that is to define
privileges or advantages and to make the rules for the trust as simple and as
fair as possible.


Greetings,
Dirk
---------------------------------------------------------------------
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-buildservice+help@xxxxxxxxxxxx

< Previous Next >
References