[opensuse-buildservice] openSUSE staging workflow metrics path forward
A while back you may remember a request for read access to a few tables or a snapshot of them from which one could work on extracting metrics of interest. That request never went anywhere so when revisiting this subject recently I went for a completely external approach using the API and some processing code. Since InfluxDB and Grafana were mentioned as the tools which the OBS team would use if they were to provide metrics I went ahead and used those. The code is built atop the osc-plugin-factory/osclib [1] and primarily uses / search/request API call to retrieve requests for given projects of interest. The code then walks the combined state changes of all the requests to re- create the staging state at any point in time. A variety of metrics are generated along the way and some quite interesting graphs can be produced in Grafana. Additionally, I plan to create an event log [2] for use by the release team both for SLE and openSUSE to provide a clear overview of what is being done by the various staging masters. Long story short, I plan to cleanup and push the code which can be used to update the metrics on the regular basis. The original responses indicated that if something is setup there may be interest in moving forward with the metrics. Is there interest in hosting an InfluxDB and Grafana instance that could also be expanded with metrics added directly by OBS? If there is no interest I will look into setting up a proper host for the release teams. The URL to my local instances has been shared a few times, but I would rather not include it in a public mailing list since it is not setup for any sort of load or real public access. If individuals are interested feel free to reply or contact me in IRC. As a teaser, here are some of the work-in-progress graphs which only include accepted submit requests. The negative dip on the first Factory graph is documented in the OBS bug regarding dates [3]. In fact the graph would be much worse without some major workarounds to detect and avoid such corrupted data. - https://imgur.com/a/lqMPo Lastly, I had quite a time walking the data structure to extract the desired values and documented a number of bugs and problems [4]. [1] https://github.com/openSUSE/osc-plugin-factory/tree/master/osclib [2] https://github.com/openSUSE/osc-plugin-factory/issues/730 [3] https://github.com/openSUSE/open-build-service/issues/3858 [4] https://github.com/openSUSE/open-build-service/issues/3898 -- Jimmy -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Is it safe to say the silence indicates no interest and I should move forward with hosting for the releaseteam? -- Jimmy -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hey, On 22.09.2017 22:25, Jimmy Berry wrote:
Is there interest in hosting an InfluxDB and Grafana instance that could also be expanded with metrics added directly by OBS?
There is interest, as you already know we are planing to use those tools for application health monitoring. But we have no ETA for this, sorry. If you need something now, you have to host it yourself. Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
For those interested my VM request was fulfilled and I setup the machine [1]. Still some issues to be worked out and improvements that can be made to graph titles and descriptions, but nice to have a solid basis publicly available. Besides RAM limitations during Factory ingest the primary item to resolve is a clean update path rather than re-requesting all data. See #4108: [api] allow ordering to be controlled on /search/requests [2]. [1] https://metrics.opensuse.org/ [2] https://github.com/openSUSE/open-build-service/issues/4108 -- Jimmy -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (2)
-
Henne Vogelsang
-
Jimmy Berry