Mailinglist Archive: opensuse-buildservice (176 mails)

< Previous Next >
Re: [opensuse-buildservice] Track statistics on the openSUSE staging process to gain feedback on changes
  • From: Jimmy Berry <jberry@xxxxxxxx>
  • Date: Wed, 01 Mar 2017 15:23:45 -0600
  • Message-id: <5577656.3uS8eOQHpl@boomba.local>
On Wednesday, March 1, 2017 5:44:58 PM CST Henne Vogelsang wrote:
Hey,

On 01.03.2017 15:43, Jimmy Berry wrote:
That was my understanding as well, which is why I am not proceeding any
further until I get an understanding of any existing plans surrounding OBS
metrics.

Apart from the tool we want to use to store time series data (influxdb),
the tool we want to use to send data there (influxer) and the tool we
want to use to show metrics (grafana) we don't have much of a plan. I
guess it's up to you to figure out how you can make sense out of this
for your use case :-)

If you need to record some extra time series data for your staging
workflow engine you can do that, as your engine always runs in the
context of the OBS instance it's mounted on top of. So it will also have
access to the influxdb instance etc.

Same is BTW true for access to the SQL database, your engine has the
same access as the Rails app it's mounted from.

As I would expect. I was looking for access to develop against since it is
difficult to recreate an accurate facsimile of the OBS instance and near
impossible to simulate the variety of workflows through which requests have
gone. It would also be good to see if pulling certain metrics directly from
the source tables is performant enough.

When I worked on the tooling used by the development site for other open
source projects it was possible to get a sanitized database dump or staging
environment that had access to both a clone of production and read access to
production. These resources were invaluable for validating data migrations and
tools before deployment. Without such access it was impossible to predict all
the ways in which data can be either inconsistent, corrupted, or odd edge-
cases.

Given that storing additional information will not cover all the desired
metrics it is likely more effective to just record timeseries data. I'll have
to look at the tool in question, but I would expect a background job to run
that periodically writes a record to the timeseries database. Such a
background job that will end up storing data outside of the scope of
obs_factory. On that note, are the various influx software pieces setup and
hosted or has nothing been done short of selecting the desired tool?

Short of database read access to where I can potentially run some of these
tools myself and figure out how to set things up I am not really sure how I
can proceed. Either I spend my time scraping the data via the APIs or writing
a scripts to generate data to develop against. Both of which seem like
unnecessary extra effort given the real deal already exists.

I am happy to put in effort to make this happen, but I'd rather not beat
around the bush recreating data that may or may not properly represent the
real data. Even if I can somehow put everything in the obs_factory engine that
does not help me develop it.


I hope that helps,

Henne

Thanks,

--
Jimmy
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-buildservice+owner@xxxxxxxxxxxx

< Previous Next >
List Navigation
Follow Ups