On Wednesday, March 1, 2017 9:54:14 AM CST Adrian Schröter wrote:
On Mittwoch, 1. März 2017, 01:48:19 CET wrote Jimmy Berry:
I am looking to provide a variety of statistics and metrics relating to the staging workflow on OBS in onder to see the impact of automation and tune the tools. I have spent some time setting up a local OBS, obs_factory engine, looking through the database structure, and reviewing tools designed to present metrics. My understanding from @hennevogel is that Influx Data is planned to be used to provide this type of information and it makes sense to get a feel for the existing plans and how this may fit into that.
The biggest hurdle seems to be trying to create timeseries information from event data by walking the events backwards from the known state (ie current) to find states of interest. At some point whatever solution is built will need access to the relevant data to do some aggregation and store intermediate results that can then be drawn on for presentation. Alternatively, I had some success using sub-selects, but that is likely not the most performant way forward.
Given the design of the submit request/staging workflow, that no events are recorded after a request is declined, it is not possible to determine when a obsolete request was unstaged. The overall state of stagings cannot be accurately determined when such a workflow has occurred. It will likely also be difficult/impossible to determine when a build state complete, re-entered building due to manual rebuilds or re-freeze, or failed/passed testing. The dashboard [1] already provides this information, but re-creating a history in an aggregate form is likely near impossible if not very difficult. As such it likely makes sense to create a new polling job that stores facets of the information collected by the dashboard in a timerseries database.
For the rest of the information tied specific to individual requests it seems possible to figure everything out from a handful of tables in the OBS database.
We did this for the maintenance statistics:
osc api /statistics/maintenance_statistics/openSUSE:Maintenance:6433
it also handles assignments from a group to a user to calculate how long the group took to review.
This looks somewhat similar although the example provided does not have any reviews, but based on your comment I can assume what that might look like. What tool consumes this API? Presumably the tool then has to scrape all the information from this API in a similar manor to what I was trying to avoid.
Alternatively, I can write something that polls and scrapes data using the APIs into local storage, but that seems like unnecessary extra work. Is it feasible for me to be granted read access to the few tables of interest or the Influx/similar tool setup with access so that I can begin setting up some metrics of interest?
Your code is very much isolated, so I can't really judge about it. Just one hint, your code runs in an environemnt and host which is critical for our security. And also for all people using repositories from it.
So another service means another potential weakness, it would be good if that does not need to run on our main server at least.
Any code I mention at this point is running on my local machine. If read access to the source tables is not possible the code can be hosted entirely separate from OBS.
A bit more on the topic of generating timeseries data. As an example, consider presenting a graph of the request backlog against Factory over time, the time until first staging, or the number of empty stagings over time. The event information is collected in the form of reviews. Anytime the request is staged or re-staged reviews for the particular staging project are added or accepted. Accepting or declining the request unfortunately stops future changes from being recorded which means the staging tools cannot indicate when the request is removed from a staging, but for simplicity it can be assumed complete after one of those states.
I think most of this is generic and not specific to staging projects. So it would be good to extend the generic request system with statistics IMHO.
That was my understanding as well, which is why I am not proceeding any further until I get an understanding of any existing plans surrounding OBS metrics.
Assuming one has a known state from which to start it should be possible to walk the event history and annotate states of interest. Given that the current state can be queried which indicates what requests are currently in a staging that provides a starting point. Assuming the script can be stopped anytime it encounters an annotation again (or other mechanism) the job can be run on some timely basis to annotate the desired information.
The polling technique could be used to avoid all this walking the event history which is simpler, but cannot backfill the data. As such it is preferable to walk the event tree where possible.
I look forward to your thoughts.
[1] https://build.opensuse.org/project/staging_projects/openSUSE:Factory
-- Jimmy -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org