[yast-devel] RFC: Yast Logging Format
Hi Yasters, There's a new feature in the feature tracking tool that requests Yast to log in the same format as dmesg and journalctl. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Business Case: Easier to analyse logs in bug reports. Description: YaST should log the time in its log-file like 'dmesg' and 'journalctl --output short-monotonic' so that times in the various logs can be compared and events can be synchronized. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - I don't want to spoil your mind by the feature itself as there is already some discussion happening. What I actually want is you to think a bit deeper about this request from your POV. What would it change for us or others, what would it be good or bad for, what the requester actually wanted and how to make them happy and the team as well if possible. Thanks in advance Lukas -- Lukas Ocilka, Systems Management (Yast) Team Leader SLE Department, SUSE Linux -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, 21 Jan 2016 10:26:41 +0100 Lukas Ocilka <lukas.ocilka@suse.com> wrote:
Hi Yasters,
There's a new feature in the feature tracking tool that requests Yast to log in the same format as dmesg and journalctl.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Business Case: Easier to analyse logs in bug reports. Description: YaST should log the time in its log-file like 'dmesg' and 'journalctl --output short-monotonic' so that times in the various logs can be compared and events can be synchronized. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
I don't want to spoil your mind by the feature itself as there is already some discussion happening. What I actually want is you to think a bit deeper about this request from your POV. What would it change for us or others, what would it be good or bad for, what the requester actually wanted and how to make them happy and the team as well if possible.
Thanks in advance Lukas
Well, good think is that we use one formatter, so it is easy to change format. Disadvantage is that old old tools for coloring logs will need to be adapted. Otherwise I do not care much about format. Just that maybe we can reuse some other tools if we use more standard format. Josef -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 21.1.2016 10:29, Josef Reidinger wrote:
Well, good think is that we use one formatter, so it is easy to change format. Disadvantage is that old old tools for coloring logs will need to be adapted. Otherwise I do not care much about format. Just that maybe we can reuse some other tools if we use more standard format.
Could you be more specific on the standard format? Is there some description/RFC for that? Did you have some specific "other tools" in mind? What about the current usage of the y2log? Would it change? Thanks Lukas -- Lukas Ocilka, Systems Management (Yast) Team Leader SLE Department, SUSE Linux -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, 21 Jan 2016 16:23:52 +0100 Lukas Ocilka <lukas.ocilka@suse.com> wrote:
On 21.1.2016 10:29, Josef Reidinger wrote:
Well, good think is that we use one formatter, so it is easy to change format. Disadvantage is that old old tools for coloring logs will need to be adapted. Otherwise I do not care much about format. Just that maybe we can reuse some other tools if we use more standard format.
Could you be more specific on the standard format? Is there some description/RFC for that?
e.g. syslong log format? https://tools.ietf.org/html/rfc5424#section-6
Did you have some specific "other tools" in mind?
nothing particular in mind.
What about the current usage of the y2log? Would it change?
Well, my usage is reading logs manually, so nothing change from my POV. What would be nice is extra info like version of package reporting something. Maybe easier mass processing as there is already tools for syslog. Josef
Thanks Lukas
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 21.01.2016 17:49, Josef Reidinger wrote:
e.g. syslong log format? https://tools.ietf.org/html/rfc5424#section-6
No sub-second resolution in the timestamp? This is crucial for syncing some things like e.g. the installation macro. Only seconds are just not precise enough; a lot of things happen within one second. Kind regards -- Stefan Hundhammer <shundhammer@suse.de> YaST Developer SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, Jan 25, 2016 at 11:16:48AM +0100, Stefan Hundhammer wrote:
On 21.01.2016 17:49, Josef Reidinger wrote:
e.g. syslong log format? https://tools.ietf.org/html/rfc5424#section-6
No sub-second resolution in the timestamp?
This is crucial for syncing some things like e.g. the installation macro. Only seconds are just not precise enough; a lot of things happen within one second.
Indeed, higher precision has been missing for about a decade, see https://github.com/yast/yast-core/blob/master/liby2util-r/src/y2log.cc#L230. Unfortunately changing the time format breaks the old testsuite that greps with regexs in the logs (or something just as mad). ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 21.01.2016 10:26, Lukas Ocilka wrote:
There's a new feature in the feature tracking tool that requests Yast to log in the same format as dmesg and journalctl.
Bad idea.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Business Case: Easier to analyse logs in bug reports.
Easier for whom? Certainly not for us. And it's us who this log is for. If we need a customer log, by all means, yes, let's create one. But that would be a completely separate log that has nothing to do with our y2log. We could for example duplicate log.info, log.error, log.warning and write that to such a customer log. As a user, I'd also welcome a log with commands that were executed, files that were changed (or use a Git repo for them anyway?), but that's yet another level. But our y2log is for us for debugging. If we need to comply with weird formats, we won't be able to do many things we are doing today any more, so we will no longer be able to maintain the product. For example, look at the new format of the storage "target map" in current y2logs. Formatting this to be bug compatible with systemd would mean to get back the one-line format which is completely unusable (I spent many hours reformatting that stuff manually just to figure out what happened in bug situations). Our y2log is our most important tool. It's not a playground for people who won't be responsible for debugging stuff. So, bottom line: We might want to write an additional customer/user log, but our y2log is holy. Amen. ;-) CU -- Stefan Hundhammer <shundhammer@suse.de> YaST Developer SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 21.1.2016 11:42, Stefan Hundhammer wrote:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Business Case: Easier to analyse logs in bug reports.
Easier for whom? Certainly not for us. And it's us who this log is for. If we need a customer log, by all means, yes, let's create one. But that would be a completely separate log that has nothing to do with our y2log.
In fact, the feature lacks a real example (reason / use case) to justify its existence, but it has actually been filed by a Yast developer.
We could for example duplicate log.info, log.error, log.warning and write that to such a customer log.
Probably a different use case, I /think/ it's about looking into journalctl and dmesg output and trying to identify a certain point in time in Yast log (or vice versa) to find out what was happening there and why.
As a user, I'd also welcome a log with commands that were executed, files that were changed (or use a Git repo for them anyway?), but that's yet another level.
Yes, yet another feature, but doable via SCR or Cheetah or Augeas if we stick to using standardized API (we usually do anyway). Please file it if you think it would help you as a user and our customers.
But our y2log is for us for debugging. If we need to comply with weird formats, we won't be able to do many things we are doing today any more, so we will no longer be able to maintain the product.
For example, look at the new format of the storage "target map" in current y2logs. Formatting this to be bug compatible with systemd would mean to get back the one-line format which is completely unusable (I spent many hours reformatting that stuff manually just to figure out what happened in bug situations).
+1 (unless there is a workaround) On the other hand, we have a simple tool/script to reformat and colorize the log for you.
Our y2log is our most important tool. It's not a playground for people who won't be responsible for debugging stuff.
So, bottom line: We might want to write an additional customer/user log, but our y2log is holy. Amen. ;-)
I see your point, go in peace :) Lukas -- Lukas Ocilka, Systems Management (Yast) Team Leader SLE Department, SUSE Linux -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Hello, Am Donnerstag, 21. Januar 2016 schrieb Lukas Ocilka:
On 21.1.2016 11:42, Stefan Hundhammer wrote: [...] In fact, the feature lacks a real example (reason / use case) to justify its existence, but it has actually been filed by a Yast developer. [...] On the other hand, we have a simple tool/script to reformat and colorize the log for you.
Our y2log is our most important tool. It's not a playground for people who won't be responsible for debugging stuff.
So, bottom line: We might want to write an additional customer/user log, but our y2log is holy. Amen. ;-)
I see your point, go in peace :)
Hmm, now I'm slightly confused ;-) You need (and have) a tool to make the log readable, and still you refuse to improve the log format? BTW: how does the log look after the showy2log formatting? Some example lines or a screenshot on paste.opensuse.org would be nice ;-) Regards, Christian Boltz -- Es gibt in Afrika einen Stamm, die stehen auf einem Bein. Das ist deren Standart. Und weil sie immer so stehen, ist deren Standart bei denen Standard. [Steffen Schmidt in de.comm.infosystems.www.pages.misc] -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 23.1.2016 v 00:11 Christian Boltz napsal(a):
Hmm, now I'm slightly confused ;-)
You need (and have) a tool to make the log readable, and still you refuse to improve the log format?
It's not a reformatting tool it just colorizes the output (errors in red, warnings orange, etc...). Just to make the errors emphasized in the terminal to easily look for possible problems.
BTW: how does the log look after the showy2log formatting? Some example lines or a screenshot on paste.opensuse.org would be nice ;-)
See http://paste.opensuse.org/65261426 I run it using the "y2tool showy2log -- tail -f" command to colorize just the new lines. Optionally it can also reformat the maps and arrays (split to several lines) but I do not use that feature (the log then usually scrolls faster and takes much more lines). -- Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 23.01.2016 00:11, Christian Boltz wrote:
You need (and have) a tool to make the log readable, and still you refuse to improve the log format?
That format is not an improvement. It's a step backwards in many aspects. Kind regards -- Stefan Hundhammer <shundhammer@suse.de> YaST Developer SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, Jan 21, 2016 at 04:32:17PM +0100, Lukas Ocilka wrote:
On 21.1.2016 11:42, Stefan Hundhammer wrote:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Business Case: Easier to analyse logs in bug reports.
Easier for whom? Certainly not for us. And it's us who this log is for. If we need a customer log, by all means, yes, let's create one. But that would be a completely separate log that has nothing to do with our y2log.
In fact, the feature lacks a real example (reason / use case) to justify its existence, but it has actually been filed by a Yast developer.
The use-case is: You have a bug report where some error is logged with seconds since system start (journalctl). Even if you have all logs there is currently no way to know what YaST did at that time. So I want "various logs can be compared and events can be synchronized". And the feature request does not ask to drop the time in human readable format. So just logging the journalctl time at the start would be also fine. ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 25.1.2016 16:20, Arvin Schnell wrote:
And the feature request does not ask to drop the time in human readable format. So just logging the journalctl time at the start would be also fine.
I've added this use case to that feature, thanks. I'm still waiting for the discussion to come up with some solution or to reject that feature with a reason. So, please, Yast Hackers, what do you think of that? Do you want a `journalctl time` to be in Yast log at some place? Where? How to log it? Any great idea? Thanks in advance Lukas -- Lukas Ocilka, Systems Management (Yast) Team Leader SLE Department, SUSE Linux -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, 25 Jan 2016 16:25:51 +0100, Lukas Ocilka <lukas.ocilka@suse.com> wrote:
On 25.1.2016 16:20, Arvin Schnell wrote:
And the feature request does not ask to drop the time in human readable format. So just logging the journalctl time at the start would be also fine.
I've added this use case to that feature, thanks.
I'm still waiting for the discussion to come up with some solution or to reject that feature with a reason. So, please, Yast Hackers, what do you think of that? Do you want a `journalctl time` to be in Yast log at some place? Where? How to log it? Any great idea?
I think that adding another timestamp could be a simplest solution. However it makes log a bit less readable for those who don't need to know such value. Another approach could be extend logging capabilities a bit (e.g. allow some time formating "marks" which could be added into log message) M. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 21.1.2016 v 10:26 Lukas Ocilka napsal(a):
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Business Case: Easier to analyse logs in bug reports. Description: YaST should log the time in its log-file like 'dmesg' and 'journalctl --output short-monotonic' so that times in the various logs can be compared and events can be synchronized. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
I can see the point, but it has both advantages and disadvantages: + The monotonic time is measured in seconds from the kernel boot. That means e.g. a time zone change, DST change, the clock adjustments done by NTP, etc... do not affect the logged time and you can easily compare several logs. - Sometimes you really need to know the "real" time. If a customer by mistake attaches an old log you can easily spot it. (I have really seen this issue...) A solution could be to log the current "real" time at the beginning, then continue to use the monotonic time. The "showy2log" tool could be adjusted to optionally convert the time stamps. Is there any other reason for changing the format? Any other benefit? Or any other possible improvements? If we want to change it we should do it at once.
I don't want to spoil your mind by the feature itself as there is already some discussion happening. What I actually want is you to think a bit deeper about this request from your POV. What would it change for us or others, what would it be good or bad for, what the requester actually wanted and how to make them happy and the team as well if possible.
Personally I do not mind changing a bit the logging format provided we keep the current data (component, file location, PID, etc...). So far I only know the "showy2log" tool which deals with the y2log content. We would need to adapt it. But what about NTS, maybe they have some tools for processing y2log...?? -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (7)
-
Arvin Schnell -
Christian Boltz -
Josef Reidinger -
Ladislav Slezak -
Lukas Ocilka -
Michal Filka -
Stefan Hundhammer