Mailinglist Archive: opensuse-buildservice (123 mails)

< Previous Next >
[opensuse-buildservice] openSUSE staging workflow metrics path forward
  • From: Jimmy Berry <jberry@xxxxxxxx>
  • Date: Fri, 22 Sep 2017 15:25:40 -0500
  • Message-id: <10179294.a4QbsB8Zvf@boomba.local>
A while back you may remember a request for read access to a few tables or a
snapshot of them from which one could work on extracting metrics of interest.
That request never went anywhere so when revisiting this subject recently I
went for a completely external approach using the API and some processing
code. Since InfluxDB and Grafana were mentioned as the tools which the OBS
team would use if they were to provide metrics I went ahead and used those.

The code is built atop the osc-plugin-factory/osclib [1] and primarily uses /
search/request API call to retrieve requests for given projects of interest.
The code then walks the combined state changes of all the requests to re-
create the staging state at any point in time. A variety of metrics are
generated along the way and some quite interesting graphs can be produced in
Grafana. Additionally, I plan to create an event log [2] for use by the
release team both for SLE and openSUSE to provide a clear overview of what is
being done by the various staging masters.

Long story short, I plan to cleanup and push the code which can be used to
update the metrics on the regular basis. The original responses indicated that
if something is setup there may be interest in moving forward with the
metrics. Is there interest in hosting an InfluxDB and Grafana instance that
could also be expanded with metrics added directly by OBS? If there is no
interest I will look into setting up a proper host for the release teams.

The URL to my local instances has been shared a few times, but I would rather
not include it in a public mailing list since it is not setup for any sort of
load or real public access. If individuals are interested feel free to reply
or contact me in IRC.

As a teaser, here are some of the work-in-progress graphs which only include
accepted submit requests. The negative dip on the first Factory graph is
documented in the OBS bug regarding dates [3]. In fact the graph would be much
worse without some major workarounds to detect and avoid such corrupted data.

- https://imgur.com/a/lqMPo

Lastly, I had quite a time walking the data structure to extract the desired
values and documented a number of bugs and problems [4].

[1] https://github.com/openSUSE/osc-plugin-factory/tree/master/osclib
[2] https://github.com/openSUSE/osc-plugin-factory/issues/730
[3] https://github.com/openSUSE/open-build-service/issues/3858
[4] https://github.com/openSUSE/open-build-service/issues/3898

--
Jimmy
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-buildservice+owner@xxxxxxxxxxxx

< Previous Next >