On Mon, 7 Jul 2008, Marko Jung wrote:

after a lot of inspiring conversations with co-workers at SUSE and friends from the open source community I finally published a first abstract draft how we could add some metrics to openSUSE build service to create a trust system.

http://en.opensuse.org/Build_Service/Concepts/Trust

This article does not go much into particular neither explains concrete implementation details. Main target of this proposal is to have a basis for discussion. The article even goes beyond the focus on trust and proposes some additional metrics to support openSUSE users in their decision from which repositories they should obtain their software packages.

Within the next weeks I would like to discuss this with you. So if you are interested in this topic please contact me. Any suggestions, feedback or critics are highly appreciated.

Some comments. -snip- Proven identity A user that identifies himself in a trusted way, by sending e.g. his work/living address, contact details or a copy of his id, in a trusted way to a trusted entity makes himself much more responsible for his work. A SUSE/Novell employee has done this identification by signing his work contract, but others shall be able to get on the same level by sending their id to a review board or an adequate standardised process. Here are several options possible. For example, we could adopt the assurance process of CAcert.org, where a new user has to get approved by several existing ones, before he is able to create certificates. Another possibility is to request a user to submit his public GNU Privacy Guard key, which has to be signed by a given amount of well known openSUSE build service users. -snip- Although my name is my real-world-name, my network identity has very few crossings with my life in the outside world. Also my work-indentity and reputation is not very closely linked to my networking myself. Thus such an proven identity would prove I am I, but would not tell you anything more about the fact if I'm trustworthy or not. There have been and there are many people who trust me in case of software development, who have never ever seen me, talked to me in person or even had a phone call to me. Nevertheless they trust me. If I would use a pseuddnum, the facts would be the same, only it would be impossible for me to have an "proven identity".

From my side of view openSUSE build service should start with some internal metrics first:

a) How long is the user member of the community b) What is his activity level c) How does he handle bug reports d) rpmlint/failures/package qualities and stuff like that. Next would be ratings and human opinion specifics a) What do others thing about the user b) Involved in upstream c) Involved in bug fixes -snip- Availability for several architectures and products A good packager takes care of his packages for all shipped architectures (like i586, amd64, s390, ppc) as well as older (not discontinued) products (e.g. openSUSE 10.2, openSUSE 10.3). In addition to the history described above we should enhance the version and build history to observe all platforms and products. -snip- I don't think this can be used as rating. Lots of new packages don't build with older products and also the discussion to keep old stuff available seems an never ending story. I don't think this qualifies as trust indicator. -snip- Package history / version numbers Some users have a high demand for bleeding edge software. On the other hand, many users just want to get (security) bugs fixed without any further changes or feature additions. -snip- Although it's a nice idea, it first requires a way to notice the packager about new versions. I maintain a lot of packages and cannot have a look at every package everday to see if there is a new release. Also I cannot subscribe to all the mailingslists and info channels to be notified about news (I already get too much mail, wont have to check more). Ciao -- http://www.dstoecker.eu/ (PGP key available) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

On Monday 07 July 2008, Marko Jung wrote:

http://en.opensuse.org/Build_Service/Concepts/Trust

[quote rpmlint] A first implementation should only check whether any test failed and possibly how many problems have been found. As a subsequent improvement, I would suggest to add a parser to identify the failed tests and to further add a severity measure for the identified problems. [/quote] We already have such a system with rpmlint: each check has an associated "badness" score that is cumulated. However, it is a weak indication of trust: checks can be filtered, and if one chooses to trust a package that has certain "fatal" checks filtered depends on quite some factors that one probably can not easily judge. [quality checks] I think the most obvious ones here are missing: no implicit file conflicts, no "evil" reverse dependencies (supplements, enhances), no conflicts or obsoletes on core packages. nonstandardized %pre/%post scripts are also something that should lower trust. Number of services that are installed at boot, firewall exceptions etc. [Reverse build dependencies] I think the paragraph is the wrong way around: a package doesn't need a high trust level if it is a leaf - package. if it is a core package with many other things depending on it, we don't want to have "newbie changes" in there. Note that this is not only about buildtime but also about runtime dependencies. [action based ratings] I agree that actions are the best indicator for trust - an accepted submission request should grow trust between the two parties, permanently declined requests lower it. submission "dialogs" however again indicate an ongoing relationship. [package history / version numbers] The easiest way is probably comparing timestamps of the files in the source tarball versus timestamp of the spec file. or something like that. In any case I think this is not really useful as an indicator of trust: if the time between upstream and package is really short, then this either indicates a script at work or some spoofing going on. if the time is long, it does not mean much either: it could be that the previous packager dropped dead and somebody else finally came around updating the package. this situation should increase the trust in the package instead of lowering it. Overall, I think the conceptual description is not very clear and is lacking some fundamental decisions, which make evaluation of the proposed concept difficult: a) is trust an attribute of an identity, of modifications, projects, packages or binaries? I think this is the most important question to answer. according to the introduction, trust appears to be a relationship between identities. identities are however only loosely connected to projects or packages. the proposed implementation suggests that the trust is related to a project (lowest trustlevel of all maintainers). The implementation proposal is btw flawed - trust has to be sticky, otherwise I can trojan a project and afterwards remove myself from the package maintainership list, and the package would immediately gain high trust level again. I suggest to split and clarify the definition of trust as a relationship between and as an entity of modifications (submit requests or OBS commits). It is not generally true that a brillant packager of package foo is also good when he touches package bar. b) the novell/opensuse employee exceptions in the trust scheme do not make it trustworthy to outsiders. an opensuse employee is not inherently less vulnerable to screwing something up. it should be possible to reduce those exceptions to a bare minimum, or make them symmetric so that any outside group of people can gain the same privileges vice versa. c) the quoted definition for trust is quite good: A trusted party is presumed to seek to fulfill ethical codes, law and policies. Therefore it would be useful to propose a list of policies (law's) that allow, when being fulfilled, being an indicator of trust. One obvious example: the source of tarballs should be verifyable, be it URL, gpg signature of the packager or some other way defined by the upstream project. Other indicators are easy to come up with. for each of those, the impact on the trust should be elaborated: how bad is it if a tarball is updated but the signature is not? how bad is it if a signature is removed upon a version upgrade? how good is it if somebody adds a signature or if somebody verifies the tarball? Note that trusting somebody doesn't mean that some weird datamining determined him to be trustworthy. it means that, a particular person seeked to fulfill previous promises, ethical standards and policies. A clear prerequirement here is: the list of promises and policies to be defined from the start. d) category ratings&Acceptance: This is a difficult part. how much impact has a rating from a total stranger? how do you define trust for a rating from somebody to whom you have no trust relationship with? Also, are ratings related to an identity, to a package, to a particular package version? can people grant cookies (virtual beer) to people who fix their bugreports in the package? can they express distrust to a package when a version upgrade, when being installed, hoses their system? and if they do so, is the distrust bouncing back to the packager, to the lastest change, to the package..? if they rate a package, does that increase trust in the packager, in the one doing the latest change, in the package..? e) we want to reward people for complying to policies and for being active. we don't necessarily want to reward them for screwing up, but as time goes by, shit happens. the stickyness of bad behaviour in trust systems like e.g. ebay cause people to change their identity quite frequently. we would like to avoid that. I would propose therefore to track badness of packages independently of "badness" of identities. it should be possible to get rid of "mistrust" by dropping a really bad package (perhaps it was all upstreams fault, really), and to gain trust by being more active and interacting with more people. A person who seeks relationship is more trustworthy than one who does not. A reward system might be a good indicator and motivational factor for people to participate in the trust scheme. an important factor for that is to define privileges or advantages and to make the rules for the trust as simple and as fair as possible. Greetings, Dirk --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org