[opensuse-factory] Announcing Tumbleweed snapshot review site
Following up on my prior announcement [1] of Tumbleweed Snapshots, introducing a snapshot review site [2]. By utilizing a variety of sources of feedback pertaining to snapshots a stability score is estimated. The goal is to err on the side of caution and to allow users to avoid troublesome releases. Obviously, there are many enthusiasts who enjoy encountering issues and working to resolve them, but others are looking for a relatively stable experience. Releases with a low score will continue to impact future release scores with a gradual trail-off. Given that issues generally are not fixed immediately in the next release this assumes the next few releases may still be affected. If the issue persists and is severe it will likely be mentioned again in the mailing list and the score again reduced. Major system components that are either release candidates or low minor releases are also considered to be risky. For example, recent Mesa release candidates caused white/black screens for many users which is not-trivial to recover from for less-technical users. Such issues come around from time to time since openQA will not catch everything. Release stability is considered to be pending for the first week after release to allow time for reports to surface. This of course depends on enthusiasts who update often, encounter, and report problems. The scoring is likely to be tweaked over time to reflect observations. It may also make sense to add a manual override feature to aid scoring when something critical is encountered. Integrating the scoring data into the tumbleweed-cli would allow users to pick a minimum stability level or score and only update to those releases. Such a mechanism can be vital for systems run by family members, servers, or the wave of gamers looking for the latest OSS graphics stack. For more details see the code behind the site [3]. Currently, you can see the very low scores for the releases laden with shader cache issues and those therafter. This is the first iteration of the site so nothing too fancy and the score is fairly basic. The site also provides a machine readable (YAML) version of the data [4]. As a side-node, Tumbleweed Snapshots are limited to 50 snapshots due to a hosting restriction [5], but that should generally be over two months worth. Hopefully others find this useful, enjoy! [1] https://lists.opensuse.org/opensuse-factory/2017-11/msg00591.html [2] http://review.tumbleweed.boombatower.com/ [3] https://github.com/boombatower/tumbleweed-review [4] http://review.tumbleweed.boombatower.com/data.html [5] http://release-tools.opensuse.org/2018/02/09/w05-06.html -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 2018-02-26T16:22:09, Jimmy Berry <jimmy@boombatower.com> wrote: Hey Jimmy, this is a quite interesting development. I like how you're aggregating these various metrics; not just prior to release, but also by incorporating pre-release discussions. Since I use TW as a rolling release (meaning I blindly zypper dup and then fix whatever breaks ;-), the current trend is most interesting to me.
Integrating the scoring data into the tumbleweed-cli would allow users to pick a minimum stability level or score and only update to those releases. Such a mechanism can be vital for systems run by family members, servers, or the wave of gamers looking for the latest OSS graphics stack.
That, or as an overall "health trend" for the distribution, and how well the release processes, openQA are working out. Graphing this would be quite helpful to get a quick overview. e.g., also using your review score, combined with open incidents (Major and up?) at any given time, etc. Thanks for putting in this effort, it is appreciated! Regards, Lars -- SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) "Experience is the name everyone gives to their mistakes." -- Oscar Wilde -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Feb 27, 2018 at 1:16 PM, Lars Marowsky-Bree <lmb@suse.com> wrote:
That, or as an overall "health trend" for the distribution, and how well the release processes, openQA are working out.
Graphing this would be quite helpful to get a quick overview. e.g., also using your review score, combined with open incidents (Major and up?) at any given time, etc.
I had considered this and will likely pull the scoring data into metrics.opensuse.org which would provide a graph of scores and discussions overtime. I appreciate the feedback. -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
As referenced in 2018w09-10 release tools highlights [1] the relevant data from TW review site is now available on metrics.o.o [2]. Enjoy! [1] http://release-tools.opensuse.org/2018/03/09/w09-10.html [2] https://metrics.opensuse.org/d/osrt_release/osrt-release -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (2)
-
Jimmy Berry
-
Lars Marowsky-Bree