Mailinglist Archive: opensuse-buildservice (176 mails)

< Previous Next >
Re: [opensuse-buildservice] Track statistics on the openSUSE staging process to gain feedback on changes
On Mittwoch, 1. März 2017, 01:48:19 CET wrote Jimmy Berry:
I am looking to provide a variety of statistics and metrics relating to the
staging workflow on OBS in onder to see the impact of automation and tune the
tools. I have spent some time setting up a local OBS, obs_factory engine,
looking through the database structure, and reviewing tools designed to
present metrics. My understanding from @hennevogel is that Influx Data is
planned to be used to provide this type of information and it makes sense to
get a feel for the existing plans and how this may fit into that.

The biggest hurdle seems to be trying to create timeseries information from
event data by walking the events backwards from the known state (ie current)
to find states of interest. At some point whatever solution is built will
need
access to the relevant data to do some aggregation and store intermediate
results that can then be drawn on for presentation. Alternatively, I had some
success using sub-selects, but that is likely not the most performant way
forward.

Given the design of the submit request/staging workflow, that no events are
recorded after a request is declined, it is not possible to determine when a
obsolete request was unstaged. The overall state of stagings cannot be
accurately determined when such a workflow has occurred. It will likely also
be difficult/impossible to determine when a build state complete, re-entered
building due to manual rebuilds or re-freeze, or failed/passed testing. The
dashboard [1] already provides this information, but re-creating a history in
an aggregate form is likely near impossible if not very difficult. As such it
likely makes sense to create a new polling job that stores facets of the
information collected by the dashboard in a timerseries database.

For the rest of the information tied specific to individual requests it seems
possible to figure everything out from a handful of tables in the OBS
database.

We did this for the maintenance statistics:

osc api /statistics/maintenance_statistics/openSUSE:Maintenance:6433

it also handles assignments from a group to a user to calculate how
long the group took to review.

Alternatively, I can write something that polls and scrapes data
using the APIs into local storage, but that seems like unnecessary extra
work.
Is it feasible for me to be granted read access to the few tables of interest
or the Influx/similar tool setup with access so that I can begin setting up
some metrics of interest?

Your code is very much isolated, so I can't really judge about it.
Just one hint, your code runs in an environemnt and host which is critical
for our security. And also for all people using repositories from it.

So another service means another potential weakness, it would be good if
that does not need to run on our main server at least.

A bit more on the topic of generating timeseries data. As an example,
consider
presenting a graph of the request backlog against Factory over time, the time
until first staging, or the number of empty stagings over time. The event
information is collected in the form of reviews. Anytime the request is
staged
or re-staged reviews for the particular staging project are added or
accepted.
Accepting or declining the request unfortunately stops future changes from
being recorded which means the staging tools cannot indicate when the request
is removed from a staging, but for simplicity it can be assumed complete
after
one of those states.

I think most of this is generic and not specific to staging projects.
So it would be good to extend the generic request system with statistics IMHO.

Assuming one has a known state from which to start it should be possible to
walk the event history and annotate states of interest. Given that the
current
state can be queried which indicates what requests are currently in a staging
that provides a starting point. Assuming the script can be stopped anytime it
encounters an annotation again (or other mechanism) the job can be run on
some
timely basis to annotate the desired information.

The polling technique could be used to avoid all this walking the event
history which is simpler, but cannot backfill the data. As such it is
preferable to walk the event tree where possible.

I look forward to your thoughts.


[1] https://build.opensuse.org/project/staging_projects/openSUSE:Factory




--

Adrian Schroeter
email: adrian@xxxxxxx

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284
(AG Nürnberg)

Maxfeldstraße 5
90409 Nürnberg
Germany


--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-buildservice+owner@xxxxxxxxxxxx

< Previous Next >
List Navigation
Follow Ups
References