[opensuse] Need help creating a syslog -ng config file to perform a Syslog Daemon's Work.
I want to build a Syslog Server. I have a Linux Log file viewer so most of the work is done. http://www.kiwisyslog.com/log-viewer-v2-beta-info.htm I needs syslog-ng to listen to UDP/514 and write a continuous file on the information it hears. Fortunately I do not need any log rotation as the file is only text base and although it has the potential to reach large sizes I can deal with a lot of space. Syslog-ng appears to have many config files and I am not sure which to modify. Can anyone assist me with this short line of syntax, given the above Linux Log file's ability to display the file as it changes and the various parameters it uses, some of which I understand but not all. The ability to NOT have to maintain a M$ PC just to be a Syslog +daemon would be a breakthrough for so many sysop's who require real time syslog data. Data from my multiple IDS's is sent to my current M$ Windows Syslog+Daemon, however I do have a large Linux IDS Management Module that does number crunching, provides warnings and reports but cannot display the data in realtime. Syslog data is sent to UDP/514 to Facility's numbering Local 0-7. The text stream looks something like [2007-04-21 17:31:55] <6>EFW: ALG: prio=1 algmod=http algsesid=70500 action=close reason=backlisted_url url="www.download.windowsupdate.com/msdownload/update/v3-19990518/ca" peer=client connipproto=TCP connrecvif=LAN connsrcip=192.168.100.40 connsrcport=3767 conndestif=core conndestip=202.158.212.136 conndestport=80 origsent=364 termsent=84 Where the number enclosed by < > is equal to 0 Emergency: system is unusable 1 Alert: action must be taken immediately 2 Critical: critical conditions 3 Error: error conditions 4 Warning: warning conditions 5 Notice: normal but significant condition 6 Informational: informational messages 7 Debug: debug-level message If anyone is really board and wants to learn about the convention there is a short war and piece version at http://www.faqs.org/rfcs/rfc3164.html Dont worry about understand the text, thats my job. I just offer it as an example for delineation purposes. I know this is a big ask, but no one but no one currently produces as Linux Syslog Daemon + Log Viewer. In my reading of my 2000 page into to C++, I have only got to page 95 and I know this is a 3 line entry into a config. Please tell me if I ask too much. Many thanks if anyone can assist. Scott :'(
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Saturday 2007-04-21 at 18:05 +1000, Registration Account wrote:
Syslog-ng appears to have many config files and I am not sure which to modify.
I see only one - where are you looking at? /etc/syslog-ng/syslog-ng.conf
Can anyone assist me with this short line of syntax, given the above Linux Log file's ability to display the file as it changes and the various parameters it uses, some of which I understand but not all. The ability to NOT have to maintain a M$ PC just to be a Syslog +daemon would be a breakthrough for so many sysop's who require real time syslog data.
To log external sources, I add: source ext { udp(ip("0.0.0.0") port(514)); }; below the existing "source src {... };" section. Later on, I add, for instance: filter f_router { host("router"); }; ... destination router { file("/var/log/router"); }; log { source(ext); filter(f_router); destination(router); };
I know this is a big ask, but no one but no one currently produces as Linux Syslog Daemon + Log Viewer.
Viewer? I just use plain "less /var/log/file" in an xterm. Or "tailf ..." for a continuous display with less resources spent. Viewing the log is a completely diferent task from logging it. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGKiCftTMYHG2NR9URAsQaAKCSg2Y/j+x31ETWWYO5eOdL7F0OowCfXZ0u rByKHrIV6E/H1C0iiA+6vpk= =DAes -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thank you Carlos, Those few lines of code are just what I need. Yes off course I can use KsystemLog - its all set up to chase the file as it grows by the millisecond and has a wide application use. An Xterm will not offer as much I feel. With respect to the substitution of the Priority codes below in place of the value contained the string below as is that also as easy to achieve. Please let me know where to send chocolate! With great thanks and appreciation Scott 2007-04-21 17:31:55] <6>EFW: ALG: prio=1 algmod=http algsesid=70500 action=close reason=backlisted_url url="www.download.windowsupdate.com/msdownload/update/v3-19990518/ca" peer=client connipproto=TCP connrecvif=LAN connsrcip=192.168.100.40 connsrcport=3767 conndestif=core conndestip=202.158.212.136 conndestport=80 origsent=364 termsent=84 Where the number enclosed by < > is equal to 0 Emergency: system is unusable 1 Alert: action must be taken immediately 2 Critical: critical conditions 3 Error: error conditions 4 Warning: warning conditions 5 Notice: normal but significant condition 6 Informational: informational messages 7 Debug: debug-level message Carlos E. R. wrote:
The Saturday 2007-04-21 at 18:05 +1000, Registration Account wrote:
Syslog-ng appears to have many config files and I am not sure which to modify.
I see only one - where are you looking at?
/etc/syslog-ng/syslog-ng.conf
Can anyone assist me with this short line of syntax, given the above Linux Log file's ability to display the file as it changes and the various parameters it uses, some of which I understand but not all. The ability to NOT have to maintain a M$ PC just to be a Syslog +daemon would be a breakthrough for so many sysop's who require real time syslog data.
To log external sources, I add:
source ext { udp(ip("0.0.0.0") port(514)); };
below the existing "source src {... };" section. Later on, I add, for instance:
filter f_router { host("router"); }; ... destination router { file("/var/log/router"); }; log { source(ext); filter(f_router); destination(router); };
I know this is a big ask, but no one but no one currently produces as Linux Syslog Daemon + Log Viewer.
Viewer? I just use plain "less /var/log/file" in an xterm. Or "tailf ..." for a continuous display with less resources spent.
Viewing the log is a completely diferent task from logging it.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Sunday 2007-04-22 at 07:47 +1000, Registration Account wrote:
Those few lines of code are just what I need. Yes off course I can use KsystemLog - its all set up to chase the file as it grows by the millisecond and has a wide application use. An Xterm will not offer as much I feel.
Oh, yes, xterm is much faster than any other gui app. Try, leave an xterm with "tailf logfile".
With respect to the substitution of the Priority codes below in place of the value contained the string below as is that also as easy to achieve.
I'm not sure what you want there... syslog has standard priority values, but the priority is not printed, its just used to filter them out to different destination files if wanted. For instance: filter f_mailinfo { level(info) and facility(mail); }; filter f_mailwarn { level(warn) and facility(mail); }; filter f_mailerr { level(err, crit) and facility(mail); }; filter f_mail { facility(mail); }; filter f_myemail { level(notice) and facility(mail) and not (program("amavis") and match("Passed CLEAN,")); }; # info o notice ... destination maildebug { file("/var/log/mail.debug" ); }; log { source(src); filter(f_mail); destination(maildebug); }; log { source(src); filter(f_myemail); destination(mail); }; log { source(src); filter(f_mail); destination(mail); }; The "/var/log/mail.debug" file will contain all the mail messages of any level, but the "/var/log/mail" will only contain those of lever "notice" and higher importance, except those comming from the program "amavis" with certain string. But I don't know how to insert an arbitrary string indicating the level; for that I think you will need to hack the syslog-ng code.
Please let me know where to send chocolate!
Ugh, I have half a kilo downstairs I shouldn't even look at... leave it as virtual ;-) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGKo2DtTMYHG2NR9URAjY2AJwLvbfhiK7obQOfdGeMCBoHL2WeAACeIZxG +hh7l6pBk5ykFo03xQobqv0= =52N0 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Dear Carlos, With your already wonderful script I can log the file as received however as I am aware of the RFC which defines syslog rules and conventions found at http://www.faqs.org/rfcs/rfc3164.html I need to substitute the value in for the following before the log is created. This is where we get the definitions of Where the number enclosed by < > is equal to 0 Emergency: system is unusable 1 Alert: action must be taken immediately 2 Critical: critical conditions 3 Error: error conditions 4 Warning: warning conditions 5 Notice: normal but significant condition 6 Informational: informational messages 7 Debug: debug-level message Within standard syslog information as I quoted [2007-04-21 17:31:55] *<6>*EFW: ALG: prio=1 algmod=http algsesid=70500 action=close reason=backlisted_url.......... In the above this represents an Informational event. It it were to be a 4 this would substitute the work 'warning' As ALL syslog messages conform to at least these 2 mandatory fields can I incorporate your code and see any value enclosed in *< >* and have it substitute for the correct Event Title. Normal expectations of messages are about. 38,000 mph - again dependant on staff numbers. I am in the process of building a dedicated Linux PC to perform just this function. under normal usage you would expect up to 38,000 messages per hour during heavy traffic. Hence I have a management model to do all the statistical work and trends. (Linux) After I succeed I will be happy to provide the result. There is an enormous requirement for a Linux Syslog. If you wish you may wish to publish on web. Kind Regards Scott :-) Carlos E. R. wrote:
The Sunday 2007-04-22 at 07:47 +1000, Registration Account wrote:
Those few lines of code are just what I need. Yes off course I can use KsystemLog - its all set up to chase the file as it grows by the millisecond and has a wide application use. An Xterm will not offer as much I feel.
Oh, yes, xterm is much faster than any other gui app. Try, leave an xterm with "tailf logfile".
With respect to the substitution of the Priority codes below in place of the value contained the string below as is that also as easy to achieve.
I'm not sure what you want there... syslog has standard priority values, but the priority is not printed, its just used to filter them out to different destination files if wanted.
For instance:
filter f_mailinfo { level(info) and facility(mail); }; filter f_mailwarn { level(warn) and facility(mail); }; filter f_mailerr { level(err, crit) and facility(mail); }; filter f_mail { facility(mail); }; filter f_myemail { level(notice) and facility(mail) and not (program("amavis") and match("Passed CLEAN,")); }; # info o notice
...
destination maildebug { file("/var/log/mail.debug" ); }; log { source(src); filter(f_mail); destination(maildebug); };
log { source(src); filter(f_myemail); destination(mail); }; log { source(src); filter(f_mail); destination(mail); };
The "/var/log/mail.debug" file will contain all the mail messages of any level, but the "/var/log/mail" will only contain those of lever "notice" and higher importance, except those comming from the program "amavis" with certain string.
But I don't know how to insert an arbitrary string indicating the level; for that I think you will need to hack the syslog-ng code.
Please let me know where to send chocolate!
Ugh, I have half a kilo downstairs I shouldn't even look at... leave it as virtual ;-)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2007-04-23 at 05:17 +1000, Registration Account wrote:
Dear Carlos, With your already wonderful script I can log the file as received however as I am aware of the RFC which defines syslog rules and conventions found at
http://www.faqs.org/rfcs/rfc3164.html
I need to substitute the value in for the following before the log is created.
Yes, I understand that, but the linux syslog daemong doesn't record it in the files.
After I succeed I will be happy to provide the result. There is an enormous requirement for a Linux Syslog. If you wish you may wish to publish on web.
There is a linux syslog daemon, you will have to rename it somehow or people will not understand you. I don't suppose it would be difficult to modify the existing service to record the severity number: but don't look at me, I have never done serious programming in linux. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGK7urtTMYHG2NR9URArgGAJ9v2AT7K8Sd+SgLHaSezzuZApb1LgCfdys+ O0srR05eLR49w4+aCx47TX0= =CIKe -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
I agree the PC-DOS-Windows industry has bastardised the concept of a syslog daemon. Syslog-ng IS the Linux system log daemon, however it performs function in much the same way as a Mainframe Daemon works and depending on my audience I have to be very specific with the reference 'syslog daemon' due its multiple meanings. If I had a big enough business I would always go for a Main Frame and rock solid O/S in lieu of a blade server and clustering despite the O/S. I still have issues with insecure comms that we use every day in the form of the collection TCP/IP. SNA forever Keep smiling Scott Carlos E. R. wrote:
The Monday 2007-04-23 at 05:17 +1000, Registration Account wrote:
Dear Carlos, With your already wonderful script I can log the file as received however as I am aware of the RFC which defines syslog rules and conventions found at
I need to substitute the value in for the following before the log is created.
Yes, I understand that, but the linux syslog daemong doesn't record it in the files.
After I succeed I will be happy to provide the result. There is an enormous requirement for a Linux Syslog. If you wish you may wish to publish on web.
There is a linux syslog daemon, you will have to rename it somehow or people will not understand you. I don't suppose it would be difficult to modify the existing service to record the severity number: but don't look at me, I have never done serious programming in linux.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Tuesday 2007-04-24 at 17:58 +1000, Registration Account wrote:
I agree the PC-DOS-Windows industry has bastardised the concept of a syslog daemon.
Syslog-ng IS the Linux system log daemon, however it performs function in much the same way as a Mainframe Daemon works and depending on my audience I have to be very specific with the reference 'syslog daemon' due its multiple meanings.
I simply meant that the syslog or syslog-ng daemon in linux doesn't do what you want, ie, record the priority level in the string sent to the file, nor do I know how to write it, short of modifying the source code, which I haven't even inspected to estimate the dificulty of such a simple modification. :-) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGLe1ltTMYHG2NR9URAg0eAKCRrs6x1TX6etRZRZ6KtlWdtD0nqgCeOFJx W96wveuNin7YkKXbQBQhe5c= =o1eH -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
S O L V E D ! Thanks to all who participated especially Darryl - The answer was staring me in the face all the time. At the very top of the file gives the clue All that is required is simply source src {internal(); udp(ip(0.0.0.0) port(514)); }; This line is present in PC IP that receives the data and without a destination will default to the system log "messages" Open KSystem Log - Default log is the system or messages file - The data is all present and changes before my eyes. We all did it and personally I would like to know if anyone has success in creating an additional source. I really don't believe its possible. Good Night All 00:27 Scott Registration Account wrote:
I want to build a Syslog Server. I have a Linux Log file viewer so most of the work is done. http://www.kiwisyslog.com/log-viewer-v2-beta-info.htm I needs syslog-ng to listen to UDP/514 and write a continuous file on the information it hears. Fortunately I do not need any log rotation as the file is only text base and although it has the potential to reach large sizes I can deal with a lot of space. Syslog-ng appears to have many config files and I am not sure which to modify. Can anyone assist me with this short line of syntax, given the above Linux Log file's ability to display the file as it changes and the various parameters it uses, some of which I understand but not all. The ability to NOT have to maintain a M$ PC just to be a Syslog +daemon would be a breakthrough for so many sysop's who require real time syslog data. Data from my multiple IDS's is sent to my current M$ Windows Syslog+Daemon, however I do have a large Linux IDS Management Module that does number crunching, provides warnings and reports but cannot display the data in realtime. Syslog data is sent to UDP/514 to Facility's numbering Local 0-7. The text stream looks something like
[2007-04-21 17:31:55] <6>EFW: ALG: prio=1 algmod=http algsesid=70500 action=close reason=backlisted_url url="www.download.windowsupdate.com/msdownload/update/v3-19990518/ca" peer=client connipproto=TCP connrecvif=LAN connsrcip=192.168.100.40 connsrcport=3767 conndestif=core conndestip=202.158.212.136 conndestport=80 origsent=364 termsent=84
Where the number enclosed by < > is equal to
0 Emergency: system is unusable 1 Alert: action must be taken immediately 2 Critical: critical conditions 3 Error: error conditions 4 Warning: warning conditions 5 Notice: normal but significant condition 6 Informational: informational messages 7 Debug: debug-level message
If anyone is really board and wants to learn about the convention there is a short war and piece version at http://www.faqs.org/rfcs/rfc3164.html
Dont worry about understand the text, thats my job. I just offer it as an example for delineation purposes.
I know this is a big ask, but no one but no one currently produces as Linux Syslog Daemon + Log Viewer. In my reading of my 2000 page into to C++, I have only got to page 95 and I know this is a 3 line entry into a config. Please tell me if I ask too much.
Many thanks if anyone can assist.
Scott :'(
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Thursday 2007-05-24 at 00:32 +1000, Registration Account wrote:
All that is required is simply source src {internal(); udp(ip(0.0.0.0) port(514)); };
This line is present in PC IP that receives the data and without a destination will default to the system log "messages"
Obviously.
We all did it and personally I would like to know if anyone has success in creating an additional source. I really don't believe its possible.
But of course we do... and told you how. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD0DBQFGVKFJtTMYHG2NR9URAkBNAI9dArfA/6YAzO+1bS7LsXcPdTgAoI7gsr0y 9kKGpmnoPmPLqaJXkMPn =MVkB -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
I have created a how to page at http://en.opensuse.org/SDB:Syslog_server In reading this please bear in mind the overriding principal is Simplicity. At some stage other will need to separate local0 from local1 and this is easily done by making adjustments to the lines placement and the facility entries that combine all local0-7. It would be easy to work out. Yes I totally agree - you gave me perfect syntax by the book. My query was not inane
We all did it and personally I would like to know if anyone *has success in creating an additional source*. I really don't believe its possible.
By this I mean I would like to know if anyone is successful in both defining another source AND getting the syslog-ng.conf not error on you. All of our syntax seems perfect - by the book not issue an error response on restart of syslog-ng. The offer hence is If someone else can define another source and I will push it to destination file AND get it to parse correctly without error I would like to know and thus learn more. Kind Regards to all End Thread for now Scott Carlos E. R. wrote:
The Thursday 2007-05-24 at 00:32 +1000, Registration Account wrote:
All that is required is simply source src {internal(); udp(ip(0.0.0.0) port(514)); };
This line is present in PC IP that receives the data and without a destination will default to the system log "messages"
Obviously.
We all did it and personally I would like to know if anyone has success in creating an additional source. I really don't believe its possible.
But of course we do... and told you how.
participants (2)
-
Carlos E. R. -
Registration Account