[opensuse] Experimenting with rsyslog message formats
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'm trying to get rsyslog to output the messages in a format I would like. I'm, using the "RSYSLOG_SyslogProtocol23Format" template for one of the log files, which prints like this: <30>1 2010-11-22T16:29:04.707021+01:00 Telcontar smartd 3860 - - Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 29 to 28 For this I use this rule: *.* -/var/log/allmessages;My_SyslogProtocol23Format The problem starts with the templates: the definitions are not documented, I had to dig them out of the sources: static uchar template_SyslogProtocol23Format[] = "\"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n\""; I would like the timestamp to be like "2010-11-22 16:29:04". To this end I define my own template derived from that one: $template My_SyslogProtocol23Format,"<%PRI%> %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n" which works - ie, it is the same as the original one. So far, so good. But I can't find documentation for the "TIMESTAMP" formats supported. In the file "features.html" I see: +++······················ Current Features ... • good timestamp format control; at a minimum, ISO 8601/RFC 3339 second-resolution UTC zone ······················++- date-rfc3339 produces output like 2010-11-22T17:09:20.201284+01:00, which is different from what the CLI equivalent does: cer@Telcontar:~> date --iso=ns ; date --rfc-3339=ns 2010-11-22T17:13:58,866637161+0100 2010-11-22 17:13:58.868123100+01:00 (the one I want is --rfc-3339=seconds, wich in "date" does not print the "T") I think that rsyslog got the specification reversed. Anyway, if I try the "other one" in the template: $template My_SyslogProtocol23Format,"<%PRI%> %TIMESTAMP:::date-rfc8601% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n" I get this output: <46> Nov 22 17:16:48 Telcontar rsyslogd - - - ... which means it was not recognised at all. I have also tried "date-rfc8601=seconds" and "date-rfc33390seconds", none works. There is no mention in the sources of "rfc8601", only date-rfc3164 and date-rfc3339. So the question is, what are the syntax timestamp modifiers, what modifiers are really available? - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) iEYEARECAAYFAkzqm1QACgkQtTMYHG2NR9XB/QCdFplR7PrrUNB68IMeexmbqhsV tkgAnii2U2fSQdb/ayyznyN0HmtyJHHL =ZfGN -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2010-11-22 at 17:33 +0100, Carlos E. R. wrote:
Hi,
I'm trying to get rsyslog to output the messages in a format I would like. I'm, using the "RSYSLOG_SyslogProtocol23Format" template for one of the log files, which prints like this:
...
But I can't find documentation for the "TIMESTAMP" formats supported. In the file "features.html" I see:
...
So the question is, what are the syntax timestamp modifiers, what modifiers are really available?
None, it turns out. AFAIK. First, the rfc3339 date format supported is a subset, a stricter one. I found the definition in the "draft-ietf-syslog-protocol-23" definition - link: <http://zinfandel.levkowetz.com/html/draft-ietf-syslog-protocol-23#section-6.2.3> +++··························· 6.2.3. TIMESTAMP The TIMESTAMP field is a formalized timestamp derived from [RFC3339]. Whereas [RFC3339] makes allowances for multiple syntaxes, this document imposes further restrictions. The TIMESTAMP value MUST follow these restrictions: o The "T" and "Z" characters in this syntax MUST be upper case. o Usage of the "T" character is REQUIRED. o Leap seconds MUST NOT be used. The originator SHOULD include TIME-SECFRAC if its clock accuracy and performance permit. The "timeQuality" SD-ID described in Section 7.1 allows the originator to specify the accuracy and trustworthiness of the timestamp. A syslog application MUST use the NILVALUE as TIMESTAMP if the syslog application is incapable of obtaining system time. ···························++- Digging in the code I found that the date formats allowed by the rsyslog program are: rfc 3339, rfc 3164, pgsql and mysql. There is also "subseconds", but useless. Samples: 0 1 2 3 1 5 0 5 0 5 0 date-rfc3339 2010-12-05T02:21:41.889482+01:00 date-rfc3164 Dec 5 02:21:13 date-pgsql 2010-12-05 02:27:34 date-mysql 20101205022845 date-subseconds 529067 So the closest to my liking is "date-pgsql", and my definition ends like this: $template My_SyslogProtocol23Format,"<%PRI%> %TIMESTAMP:::date-pgsql% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n" I did not find references to modifications of those timestamps - I would simply like to reduce the precision (ie, eliminate or limit the subseconds). [...] Correction. I did find something else - by wild guessing! %TIMESTAMP::3:date-rfc3339% means that the maximum width of the field is 3 digits. It prints like: <46> 201 Hostnane.... and a ::22: yields: <46> 2010-12-05T03:15:13.85 Hostname A :22:: yields the same. But "TIMESTAMP:22:2:date-rfc3339" yields: 0 1 2 3 1 5 0 5 0 5 0 <46> 010-12-05T03:22:07.10 Hostname <-- :22:2: <46> 10-12-05T03:23:48.76 Hostname <-- :22:3: It seems that one trims from the left and the other from the right. Curious, is it not? And as far as I know, absolutely not documented. Correction again. It is briefly documented - now that I know what to look for: +++··························· Syslog message properties are used inside templates. They are accessed by putting them between percent signs. Properties can be modified by the property replacer. The full syntax is as follows: %propname:fromChar:toChar:options% ···························++- So, another possible template would be: $template My_SyslogProtocol23Format,"<%PRI%> %TIMESTAMP::22:date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n" which yields: <46> 2010-12-05T03:35:39.644 Telcontar rsyslogd - - - [origin software="rsyslogd" swVersion="4.4.1" x-pid="7344" x-info="http://www.rsyslog.com"] (re)start Not bad, but the 'T' in between is confusing to my sight. Y prefer the pgsql timestamp. I now have all logs in that format, except the allmessages file (full date-rfc3339), when needed. It seems that all this is documented in the file "property_replacer.html" of the sources. Even regex expressions in there! :-O - -- Cheers, Carlos E. R. (from 11.2 x86_64 "Emerald" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) iEYEARECAAYFAkz6/SwACgkQtTMYHG2NR9U5NgCfUaYtuK+WO6k+WRCnKzsGYTte 66EAn0d2HOFj8CMKDnv54nK2M2dXF2n+ =gJhd -----END PGP SIGNATURE-----
participants (1)
-
Carlos E. R.