On 12/19/2016 11:40 AM, Greg Freemyer wrote:

I'm hoping the discussion can stick to help understand what benefits journald and its binary logs brings to the table.

Well, here are some illustrations of things that are remarkable easy with journalctl but would be difficult with syslog. https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-vi... https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-vi... Under https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-vi... there is the example of output in JSON format. This gets back to my point about processing by such tools as PERL. Read in the JSON format as hashes and slice and dice and correlate to your heart's content :-) -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-19 12:24, Peter Ragosch wrote:

For me more important is the question

"How can I access the binary systemd log files if the system itself doesn't run anymore?"

Do I think to complicated?

Hints on recommended reading how to handle this situation are very appreciated.

Maybe this (man journalctl): --file=GLOB Takes a file glob as an argument. If specified, journalctl will operate on the specified journal files matching GLOB instead of the default runtime and system journal paths. May be specified multiple times, in which case files will be suitably interleaved. --root=ROOT Takes a directory path as an argument. If specified, journalctl will operate on catalog file hierarchy underneath the specified directory instead of the root directory (e.g. --update-catalog will create ROOT/var/lib/systemd/catalog/database). But probably has to be of the same version. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYA+cACgkQja8UbcUWM1wXhgD9Fsn1ABUbSBLTJQ+2juoaDl+S PGhUHTVyv8pNuVgkp+gA/1g33wH3jTLbWLuNnRzNOs5iiPZ14O13uEajvJHwj0pL =sNYY -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-19 16:29, Anton Aylward wrote:

On 12/18/2016 02:21 PM, Greg Freemyer wrote:

...

4) Syslog is only one of many log systems on local machines. Separate logs are kept for utmp/wtmp, lastlog, audit, kernel logs, firmware logs, and a multitude of application-specific log formats. This is not only unnecessarily complex, but also hides the relation between the log entries in the various subsystems.

Hmm. So if I run journald can I get rid of all those other log files?

What other log files? You mean non-syslog log files? Those other log files are not handled by journald, either. They are handled by the specific applications that create them.

either delete them and hope the subsystem won't recreate them or link them to /dev/null ?

More and more, logs are are getting to be a means of detection intrusions, hacks. But that means correlating logs, which used to be a tedious and error-prone process. We have in journald the opportunity to see all the activity in one log file, making correlation of events much easier.

No correlation unless you have a database viewer and analysis tool. But then, logs may be as difficult to analyze as they currently are in Windows. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYAxQACgkQja8UbcUWM1xzMQD/bYOxs07UsLzibXgc/8GshY1w Ht80nDT8rYrvDmydfcwA/3VfOLD8rmhJcCoC5Wx//4mG9+HivIuI4+nVB2HNOWKA =4zVW -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-19 18:24, Anton Aylward wrote:

On 12/19/2016 10:56 AM, Carlos E. R. wrote:

...

On 2016-12-19 16:29, Anton Aylward wrote:

...

On 12/18/2016 02:21 PM, Greg Freemyer wrote:

...

4) Syslog is only one of many log systems on local machines. Separate logs are kept for utmp/wtmp, lastlog, audit, kernel logs, firmware logs, and a multitude of application-specific log formats. This is not only unnecessarily complex, but also hides the relation between the log entries in the various subsystems.

...

Hmm. So if I run journald can I get rid of all those other log files?

What other log files? You mean non-syslog log files? Those other log files are not handled by journald, either. They are handled by the specific applications that create them.

In that any program can, yes, but that's not my point.

My point is that that journald logs so many of these things, or can.

journalctl _COMM=sshd journalctl _COMM=login journalctl _COMM=su journalctl _COMM=systemd-logind

Of course most access involves PAM one way or another so just make sure you make use of pam_logind !!!

That's just login sessions of different types. I don't remember where these where kept before.

if you really want to be paranoid, you can forget about the indexoing that journalctl can do and simple grep

journalctl |grep login | more

Takes minutes with rotating rust.

Oh My!

Well that soaks up quite a few of those log files: utmp/wtmp, lastlog

...

...

More and more, logs are are getting to be a means of detection intrusions, hacks. But that means correlating logs, which used to be a tedious and error-prone process. We have in journald the opportunity to see all the activity in one log file, making correlation of events much easier.

No correlation unless you have a database viewer and analysis tool.

Ah, tools!

Well, for a start, journalctl does index.

What does that mean? Examples, please.

Then we have a;; the regular tools, from 'cut'/'join' of V7 UNIX vintage, on though 'awk' (see the appropriate White Book for how to convert text to database and do correlations and produce nicely formatted output) on though Perl, which was designed for doing that kind of thing.

Nonono. I want ready made user end tools. I do not want to learn programming awk now or have to write my own tools.

You've been around long enough, Carlos, to know and have done this sort of thing. What's this, some anti-systemd feeling?

No, not at all.

...

But then, logs may be as difficult to analyze as they currently are in Windows.

This isn't Windows. <obscentity> Windows. part of the nature of Windows is to make you buy some third party tool to do things that with UNIX/Linux can be done with a few lies of scripting.

The journal thing in Linux is getting closer to what Windows does with logs. That's my fear. I'm unable to find events in Windows system log, that I know they are there. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYGZsACgkQja8UbcUWM1y0RAD8CLPB3Ruw2jSkTVi5fsdrMf6y qUR0g754lfaI6GNgIXcBAJ9uFDDxqZDmrmR6d8TcGZqNV/AbkWLCiBCLTVnbmezj =alyw -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-19 19:09, Anton Aylward wrote:

On 12/19/2016 12:32 PM, Carlos E. R. wrote:

...

On 2016-12-19 18:24, Anton Aylward wrote:

...

...

journalctl |grep login | more

Takes minutes with rotating rust.

Minutes to do what? On my rotating rust I get output immediately. Yes, if you want to wait for the very end and you are one of the people who have a gazillion of gigabytes of log files, but that reasoning applies too if you want to get every login from every individual source (since this covers in my case dovecot access, smptd/postfix access, Xlog and more, it ends up being about 10 different files).

Rotating rust of syslog files isn't going to be any faster.

No, it is not. I measured the speed of the same test against the traditional log and journal logs, and finding entries with grep was an order of magnitude faster. I demonstrated that months ago on this list. I can not prove it today again because journalctl output is empty in this machine. I disabled it.

...

...

...

No correlation unless you have a database viewer and analysis tool.

...

Ah, tools!

...

Well, for a start, journalctl does index.

What does that mean? Examples, please.

I gave some examples, perhaps you didn't notice.

Something like journalctl _COMM=sshd vs journalctl | grep ssh

Ah, that's an index. New to me. Thanks. What are the possible indexes, how does it work?

...

Nonono. I want ready made user end tools. I do not want to learn programming awk now or have to write my own tools.

That's fine. Go off and use Windows then, accept someone elses' idea of how the UI should look, what data you're allowed to display and how to see it and what you can do with it and more to the point what it won't permit you to do with it.

You really shouldn't be using Linux if that's your attitude.

Well, if all people help that way, we are going very far... In Windows I accesses such things using Microsft Access, for instance, where I could apply any filter I could desire. now, if you tell me how I can import the journal into Libre Office...

...

...

You've been around long enough, Carlos, to know and have done this sort of thing. What's this, some anti-systemd feeling?

No, not at all.

Well, that's how its coming across.

It is all in the eyes of the beholder. You are waiting to see something, so you see it even if it is not there.

...

...

...

But then, logs may be as difficult to analyze as they currently are in Windows.

...

This isn't Windows. <obscentity> Windows. part of the nature of Windows is to make you buy some third party tool to do things that with UNIX/Linux can be done with a few lies of scripting.

The journal thing in Linux is getting closer to what Windows does with logs. That's my fear. I'm unable to find events in Windows system log, that I know they are there.

There are forms of anti-systemd manifestation. A reluctance to learn how to manage the new facilities is one of them.

Then teach!

This is all new to e as well, but I'm willing to accept it, not fight it, to experiment and learn. I figure that Lennart Poettering is smarter than me. What I understand makes sense and what I understand allows me to, slowly, learn more.

UNIX, Linux has always had for me a joy in learning and a WOW! factor and a 'hey, that's neat!" aspect. I've never had that with Windows (which was always A source of frustration) or even VMS (and yes even when I used the RATFOR Software Tools).

I'm not fighting it. I simply do not embrace new things blindly. Just exposing the problems is not being against it. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYWBkACgkQja8UbcUWM1zgzgD/VFak5qG1uLHT5vK2SsswSxAs Gc2B3JJFyMRk30vEjukA/jJKRB/IfqlRRaK6Jg8el+ZSr+usWtoP40v5t5hj8jCl =z+AS -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 00:03, Anton Aylward wrote:

On 12/19/2016 04:58 PM, Carlos E. R. wrote:

...

Ah, that's an index. New to me. Thanks. What are the possible indexes, how does it work?

Well, when I RTFM and experimented, I learnt about tab expansion

journalctl _COMM=<tab><tab>

Does not work here. I get: minas-tirith:~ # journalctl _COMM= Display all 186 possibilities? (y or n) .ICEauthority .xauthTWEnQl .Xauthority .xauthTfEeG2 .Xresources~ .xauthVDMJu3 .bash_history .xauthWJGMVo and many more. It is the current directory listing. I understand it is a new feature that my 13.1 does not have. Ok, I have a 42.2 machine available. Let's try: cer@Isengard:~> journalctl _COMM= accounts-daemon cleanup gnomesu-pam-bac logger pickup sddm start-ntpd systemd-journal umount alsactl config_postfix groupadd login pkexec sddm-greeter su systemd-localed unix2_chkpwd audispd console-kit-dae haveged lxdm polkitd sddm-helper swapoff systemd-logind useradd auditctl cron helloworld master postlog (sd-pam) systemctl systemd-shutdow usermod auditd dbus-daemon hp-upgrade mcelog pulseaudio smartd systemd systemd-sleep wicked avahi-daemon display-manager httpd-prefork ModemManager purge-kernels sm-notify (systemd) systemd-sysctl wickedd bluetoothd dnsmasq iscsiadm mount python smtp systemd-coredum systemd-sysuser wickedd-dhcp4 boot.apparmor dracut-cmdline killproc mtp-probe qmgr smtpd systemd-cryptse systemd-udevd wickedd-dhcp6 bounce echo lightdm nscd rpc.statd sshd systemd-fsck trivial-rewrite xfpm-power-back checkproc gnome-keyring-d local ntpd rtkit-daemon sshd-gen-keys-s systemd-hiberna udisksd cer@Isengard:~> journalctl _COMM= That's not correct. I see there "helloworld" which I recognize as a command I wrote, a script: /usr/local/bin/helloworld - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYbP0ACgkQja8UbcUWM1wRRgD/VFF/36wQEMj/S1a+MIS7WQWy +c4Cn1eaML1Biob1BlsA+wWMHbBB0c1Q3ddLMkD+FvUG7nYHFi8iyIdEigQeXRdD =esw7 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 00:33, Greg Freemyer wrote:

On Mon, Dec 19, 2016 at 6:27 PM, Carlos E. R. <> wrote:

...

That's not correct. I see there "helloworld" which I recognize as a command I wrote, a script:

Same here. Is there something we need to install first? (Leap 42.2)

We must be missing some tab completion package. cer@Isengard:~> ls /etc/bash_completion.d/ aria2c dbus-bash-completion.sh dracut grub lsinitrd scout.sh subversion yast2-completion.sh zypper.sh cer@Isengard:~> - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYcdEACgkQja8UbcUWM1z55QD/Q4AvvNStxykVWPtHVEPhytVK xw4JcynuJWs1zJ3PiVwA/jHViC+jY2RaMH3njifN18WBnkdH1jo1gRjssQRbMo0U =/qvj -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 01:15, Anton Aylward wrote:

On 12/19/2016 06:48 PM, Carlos E. R. wrote:

...

On 2016-12-20 00:33, Greg Freemyer wrote:

...

On Mon, Dec 19, 2016 at 6:27 PM, Carlos E. R. <> wrote:

...

...

That's not correct. I see there "helloworld" which I recognize as a command I wrote, a script:

...

Same here. Is there something we need to install first? (Leap 42.2)

We must be missing some tab completion package.

cer@Isengard:~> ls /etc/bash_completion.d/ aria2c dbus-bash-completion.sh dracut grub lsinitrd scout.sh subversion yast2-completion.sh zypper.sh

Hmm, I don't know about that. As I understood it, the BASH completion is to list the files.

No, it is more complex than that.

I was under the impression systemctl had its own completion mechanism.

systemctl has to tell bash what bash completion to use, by installing a file into that directory. It is bash which is reading the keyboard.

I'm running systemd-210.1477297097.83231f8-25.51.1.x86_64

What do you have in -- Reboot -- Dec 19 09:48:05 Mainbox echo[1753]: Starting mail service (Postfix)

Is there a file 'systemctl' there?

In where? :-? Where comes that "reboot" into?

I do and its from systemd-bash-completion-210.1477297097.83231f8-25.51.1.noarch

I have: Isengard:~ # rpm -qa | grep systemd-bash-completion systemd-bash-completion-228-13.1.noarch Isengard:~ # Isengard:~ # rpm -ql systemd-bash-completion /usr/share/bash-completion /usr/share/bash-completion/completions /usr/share/bash-completion/completions/bootctl /usr/share/bash-completion/completions/busctl /usr/share/bash-completion/completions/coredumpctl /usr/share/bash-completion/completions/hostnamectl /usr/share/bash-completion/completions/journalctl <====== /usr/share/bash-completion/completions/kernel-install /usr/share/bash-completion/completions/localectl /usr/share/bash-completion/completions/loginctl /usr/share/bash-completion/completions/machinectl /usr/share/bash-completion/completions/systemctl /usr/share/bash-completion/completions/systemd-analyze /usr/share/bash-completion/completions/systemd-cat /usr/share/bash-completion/completions/systemd-cgls /usr/share/bash-completion/completions/systemd-cgtop /usr/share/bash-completion/completions/systemd-delta /usr/share/bash-completion/completions/systemd-detect-virt /usr/share/bash-completion/completions/systemd-nspawn /usr/share/bash-completion/completions/systemd-run /usr/share/bash-completion/completions/timedatectl /usr/share/bash-completion/completions/udevadm Isengard:~ # Still, something is wrong, it does not work in our 42.2 machines. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYe+UACgkQja8UbcUWM1wg1wD+K2bKUi17uXsLkmcmG4rA98/b zZveHI+ws/zpGih0MVEA/0RmPZ7dkIQRIYp0AwU3oKVD4aWnMPwqfpg6/qGTg8/m =1cqa -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/19/2016 06:27 PM, Carlos E. R. wrote:

That's not correct. I see there "helloworld" which I recognize as a command I wrote, a script:

/usr/local/bin/helloworld

And so? What's the output if you do a journalctl _COMM=helloworld Just to make the point, when I <tab><tab> I get all the things I've set up and run like docecot, postfix, and even "echo". YES! even "echo". -- Reboot -- Dec 19 09:48:05 Mainbox echo[1753]: Starting mail service (Postfix) -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/19/2016 04:58 PM, Carlos E. R. wrote:

... now, if you tell me how I can import the journal into Libre Office...

OK, assuming you are applying the various filters to extract the items you want... I'm assuming you don't mean input as text cut and past into a "Word" document much as you might cut and paste text into a mail message, as we do quite regularly, that you mean paste it into the spreadhseet too. For that I suggest you use the JSON export option. Again, RTFM or google, it very well documented and described. https://www.freedesktop.org/wiki/Software/systemd/json/ Of course if you see no other way, you can convert JSON to CSV and import the CSV to OO http://www.convertcsv.com/json-to-csv.htm Or, if you prefer, there's a macro to convert JSON that's been pasted into one cell. https://extensions.libreoffice.org/extensions/libreoffice-getrest-plugin-1 It depends, entirely, on your specific needs. Some of us are familiar enough with things like Perl, for which JSON is a native format, that we'd have no problem manipulating it. IIR there's an option in OO/LO to use python as the scripting language, and a JSON importer in python is a well documented tool. Most certainly, JSON is a more robust, more portable, more application-friendly (particularly for transmission over the 'Net) format than CSV. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward wrote:

Rotating rust of syslog files isn't going to be any faster.

It was already pre-sorted w/syslog. If you used ng-syslog, you had all the filtering of grep available on each log message, so zero waiting for filtered syslog output.

You're clutching at straws here, Carlos.

Hardly.

That's fine. Go off and use Windows then, accept someone elses' idea of how the UI should look, what data you're allowed to display and how to see it and what you can do with it and more to the point what it won't permit you to do with it.

You really shouldn't be using Linux if that's your attitude.

---- If you want text logs by default, you shouldn't be using linux? What should you be using?

There are forms of anti-systemd manifestation. A reluctance to learn how to manage the new facilities is one of them.

--- When you only see in black and white, everything and everyone that is not with us is against us, right? Shows how religious/biblical this topic is to some.

This is all new to e as well, but I'm willing to accept it, not fight it, to experiment and learn. I figure that Lennart Poettering is smarter than me. What I understand makes sense and what I understand allows me to, slowly, learn more.

==== You want something new? How about logs in rot13? That's the next step to prevent casual browsing of system data entries (as MS uses in the registry).

UNIX, Linux has always had for me a joy in learning and a WOW! factor and a 'hey, that's neat!" aspect. I've never had that with Windows (which was always A source of frustration)

Did you ever read "Windows Internals" (by Russinovich)? Eventually divided into 2 volumes with other contributors... The earlier versions were better. Shows you how to dig in as deep as you want. You just had to work harder for your "oh wow".

or even VMS (and yes even when I used the RATFOR Software Tools).

---- Tools? Was only used to the compiler to generate a C like language (Rational Fortran)... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

20.12.2016 06:43, Carlos E. R. пишет: ...

...

You want something new? How about logs in rot13? That's the next step to prevent casual browsing of system data entries (as MS uses in the registry).

You got to be kidding. Really? MS uses rot13 in the registry? Wow.

Apparently one single application storing data in registry does use it. I have not seen any evidence there is general rule or format requirement. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Tue, Dec 20, 2016 at 12:29 PM, ianseeks wrote: ...

...

Why did I disable the journal on disk (temporary and long term)? Because it takes gigabytes...

I have an nntp local server (proxy), and it produces tons of log messages per day. With syslog I can dump them all after, say, a week. With journal, I can't. If I want to keep a reasonable long term journal of all important events in my machine, it also keeps the useless debug logs of everything else... I can not purge the database of selected content.

Are you sure about that?

Yes.

There are loads of configuration options in journald.conf regarding storage and storage sizes and retention times

If you take time to actually read about these configuration options you will see that they refer to total size or retention time of *all* journal entries. So you can trivially cause mild DoS by simply filling logs with garbage, thus causing important entries to be displaced out of journal. Syslog allows filtering of logs into different places for long term storage and you can configure individual retention rules for each (although that is strictly speaking unrelated to syslog). Systemd author is actively opposed to any idea to filter log entries in journald. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 06:58 AM, ianseeks wrote:

I admit my knowledge on the journal is not great and this is why i ask questions to hopefully get the answers

Despite everything, neither is mine. Its a continual learning process for me and threads like this encourage me to probe and learn more. I know more, much more that when this thread started, about journalctl, and realise some of what I said earlier in the thread ... well if not actually wrong is certainly in need of revision! An important part of learning the new is being able to let go of the old. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 04:43 AM, Anton Aylward wrote:

An important part of learning the new is being able to let go of the old.

Exactly. Being as systemd and journald is several years old now, and the systemd wars are all fought and decided, I have to wonder why this thread is raging on at at such length at this time. There are settings to pass through all the logging to old school logs. Use those or don't, its up to you. Either use it or configure your system so you don't need to. Lets move on, for pete sake people let it go. -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 04:58 PM, Carlos E. R. wrote:

we want settings that do not exist.

I wonder some times, googling around and seeing what's being done on Redhat and Arch for example, if the Suse policy of not adopting the latest upstream revisions or altering them in various way is to our benefit. We seem well behind the curve as far as systemd goes. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Tuesday, 20 December 2016 15:17:43 GMT Carlos E. R. wrote:

On 2016-12-20 12:58, ianseeks wrote:

...

On Tuesday, 20 December 2016 14:24:08 GMT Andrei Borzenkov wrote:

I don't understand how it can displace important entries from the journal. As far as i know, it continues into a new file once it reaches its limit.

You can journalctl --vacuum-time=xmonths/days any redundant journal files away once you have dealt with them.

...

Syslog allows filtering of logs into different places for long term storage and you can configure individual retention rules for each (although that is strictly speaking unrelated to syslog).

Systemd author is actively opposed to any idea to filter log entries in journald.

I admit my knowledge on the journal is not great and this is why i ask questions to hopefully get the answers

Yes, you don't understand.

Say that I have to keep logs of events for a year or two. This is not rare (you can be mandated for legal reasons). Yes, I can configure journal to do this, no problem so far. Now, journal keeps *all* entries of *everything*. That is the problem.

Suppose I have a service that talks a lot, debug entries. Say nntp, for instance. I'm only interested in a week of those, for instance. But journal does not allow me to keep a week and delete the rest, while keeping mail log for two years.

This nntp service may write to the log, for instance, ten megabytes of entries per day. On a year, that is 3650 megabytes (I have seen my journal have two gigabytes after an uptime of a month).

If I limit the size of the log to a reasonable size, say a hundred megabytes, the log will be full of nntp entries and little else. It will be difficult to investigate a crash or a reboot that happened two weeks ago because due to the amount of entries from nntp, it keeps less than ten days.

The solution is to *erase* all debug entries from nntp older than a week, say. But journal does not allow this, and the developer refuses to do it.

So no, journal is not useful to me, I have to keep syslog. Yes, I know how to do it and I do. On some machines I keep both, on some only syslog.

Ok, thanks for the explanation -- opensuse:tumbleweed:20161217 Qt: 5.7.0 KDE Frameworks: 5.29.0 KDE Plasma: 5.8.4 kwin5-5.8.4-171.1.x86_64 Kernel: 4.8.13-1-default Nouveau: 1.0.13_2.1 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward wrote:

On 12/20/2016 09:17 AM, Carlos E. R. wrote:

...

Say that I have to keep logs of events for a year or two. This is not rare (you can be mandated for legal reasons). Yes, I can configure journal to do this, no problem so far. Now, journal keeps *all* entries of *everything*. That is the problem.

Um, NOT.

Agrees, it's not up to the journal to filter stuff. It's easily filtered out afterwards. -- Per Jessen, Zürich (2.0°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd wrote:

Le 20/12/2016 à 15:53, Anton Aylward a écrit :

...

the when. That amounts to *everything* in so far as every access, every reboot, every reconfiguration.

if a log is too intensive, may be blame the original application, not the logger, may be ask the application to log elsewhere?

Yep.

ùmay be this is obsolete, but still a good stating point? https://www.eyrie.org/~eagle/software/inn/docs-2.4/install.html#S10

The openSUSE default for innd is to log to separate files under /var/log/news/ - Carlos is probably running leafnode or something like that. -- Per Jessen, Zürich (1.9°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 20/12/2016 à 17:37, Carlos E. R. a écrit :

On 2016-12-20 16:25, Per Jessen wrote:

...

The openSUSE default for innd is to log to separate files under /var/log/news/ - Carlos is probably running leafnode or something like that.

Correct, I use leafnode. It is a proxy server.

then http://leafnode.sourceforge.net/wuerzburg/faq.html this thread: https://forums.opensuse.org/showthread.php/506886-systemd-journald-Settings-... discuss your problem. what I see id that syslog uses a special file for leafnode. You still can use that and don't use the standard log system for debugging purpose jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 20/12/2016 à 18:24, Andrei Borzenkov a écrit :

20.12.2016 19:55, jdd пишет:

...

what I see id that syslog uses a special file for leafnode.

It CAN BE CONFIGURED BY ADMINISTRATOR to use special file for leafnode. That is exactly what is not possible with journal.

really? I don't use leafnode, but don't see why the official instructions couldn't work, even is that mean use an other system than journalctl logs

...

You still can use that and don't use the standard log system for debugging purpose

Which rather defeats the very idea of having standard log system in the first place.

why? the old way still ask to use a special file Not to mention that this presumes "debugging" is something

performed by developers only. In reality I have to deal with those logs every time I investigate problem on customer systems. They are extremely helpful in finished programs, the more logs - the better. But you also have no reasons to keep them for years.

and? they have to be used by admins. Anyway, It's unuseful to discuss this very problem only. debug mode is pretty common and everybody know it can generate a lot of logs. So the question on subject should be really: "how to manage debug mode" and not challenge what is one of the main journalctl feature (binary). I don't see neither why a debug mode should be kept for month if the logs are not analyzed in details. We should really focus on trying to solve real problems, not imaginary ones nor try to promote not viable solutions thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-20 16:20, jdd wrote:

Le 20/12/2016 à 15:53, Anton Aylward a écrit :

...

the when. That amounts to *everything* in so far as every access, every reboot, every reconfiguration.

if a log is too intensive, may be blame the original application, not the logger, may be ask the application to log elsewhere?

No, because with syslog and variants you can keep debug logs for a different period of time than main logs. The applications were designed with that in view, spewing log entries at different "priorities". Thus, for example, you can keep news.info and news.debug files, and rotate them differently.

ùmay be this is obsolete, but still a good stating point?

https://www.eyrie.org/~eagle/software/inn/docs-2.4/install.html#S10

I don't use inn :-) -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-20 17:59, jdd wrote:

Le 20/12/2016 à 17:36, Carlos E. R. a écrit :

...

I don't use inn :-)

well, it's the same problem. Don't use standard system log for debug an application. you could as well flood /var/log/messages with this

Traditional syslog has a method to cope with that flood of messages, journal doesn't. Which is why I do not use journal. It does not work. -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 22:38, Anton Aylward wrote:

On 12/20/2016 01:14 PM, Carlos E. R. wrote:

...

On 2016-12-20 17:59, jdd wrote:

...

Le 20/12/2016 à 17:36, Carlos E. R. a écrit :

...

I don't use inn :-)

well, it's the same problem. Don't use standard system log for debug an application. you could as well flood /var/log/messages with this

Traditional syslog has a method to cope with that flood of messages, journal doesn't. Which is why I do not use journal. It does not work.

You have that exactly backwards.

Syslog needs those rate-limiting tools to deal with the flood because it can't keep up. This has been a long standing traditional problems with syslog and one reason why its UDP based, so that packets from remote hosts that flood can be dropped with impunity. We've had to get faster and dedicated syslog servers to be able to keep up in large commercial settings.

Journald was, however, designed to be able to cope with the floods. That's why it has not such mechanism. Go read the design documents. Its there.

Go read my posts, it is not there. You have it wrong. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhZqbwACgkQja8UbcUWM1xAFAD/ZJswus6MzZVQGlD1fn/PQBqQ Yg5IPLaDyDnWpowMyg0A/RIaSA/YilziXYhirV7vajQKXVi+P/94XyEP95iwmy5B =krI+ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/21/2016 03:13 AM, Per Jessen wrote:

The problem Carlos describes is real, and just one of many reasons for continuing to use a syslog daemon.

we have, as I've implied before, a bifurcation here. The problem Carlos has is a real problem for Carlos. As I've said, the comprehensive, no filter etc no limiting approach of jounald is what Big Business is requiring this, along with Docker rather than VM, Cgroups and more, are things that are of value to Big business, banks, Brokerages, Telcos, "Wall Street". They are necessary if Linux is to be part of the IT DNA there. Traditionally UNIX and Linux were counter-culture. FOSS was reactionary. It started as a hobby project and has been a source of joy and sideline activity for programmers and and people who want to drill down and experiment for some decades now. There's a bifurcation. What Carlos want journald to be isn't going to happen. EVER. No Way, simply because that would be contrary to the needs of Big Business in the ways I've outlined in past posts. Its a "that was then, this is now" situation, just like "I lived in a small town when I was a kid and no-one locked their doors, now I live in a big city and we do ..." sort of 'that was then, this is now'. Yes, there are people who attend RenFairs and dress up in medieval costume, but they omit that health care was lacking back then. There are people who maintain steam engines and biplanes. And of course KDE3. But the mainstream moves on. So we have a bifurcation: Linux for the Big Business and Linux for the small guys. It seems that Suse has chose the "Big Business" path. That there has been the integration we see in Leap should be taken as an indicator of that. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 21/12/2016 à 16:58, Anton Aylward a écrit :

What Carlos want journald to be isn't going to happen. EVER. No Way, simply

who knows :-), after all we are on the opensource side :-) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/21/2016 08:19 PM, Carlos E. R. wrote:

On 2016-12-21 17:06, jdd wrote:

...

Le 21/12/2016 à 16:58, Anton Aylward a écrit :

...

What Carlos want journald to be isn't going to happen. EVER. No Way, simply Because the main systemd developer is very thick headed and he does not listen.

You are assigning to him an emotional childish churlishness that is not apparent in his reasoning for the design decisions he has made. I would say that he simply has a different agenda. A reasonable and reasoned one. Just different from yours. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/21/2016 08:37 PM, Carlos E. R. wrote:

On 2016-12-22 02:34, Anton Aylward wrote:

...

On 12/21/2016 08:19 PM, Carlos E. R. wrote:

...

On 2016-12-21 17:06, jdd wrote:

...

Le 21/12/2016 à 16:58, Anton Aylward a écrit :

...

What Carlos want journald to be isn't going to happen. EVER. No Way, simply Because the main systemd developer is very thick headed and he does not listen.

...

You are assigning to him an emotional childish churlishness that is not apparent in his reasoning for the design decisions he has made.

...

I would say that he simply has a different agenda. A reasonable and reasoned one. Just different from yours.

Not only mine. Obviously other people asked him to add the features and he refused. Something that many people have said about him.

So? Without seeing that those features are and wht his reasons for refusing the request were that's just hearsay. Phrasing it that way is argumentative and prejudicial.

I do not have an agenda,

Oh yes you do, as you go on to make quite clear!

I simply want to have a complete feature set, so that new programs have the same features as the old, and more, new, features. But not less. Not missing features.

That isn't what progress is about. With that attitude the car is would have ropes and purs rather than a steering wheel and that it has a steering wheel and an accelerator pedal. so as to compatible with a horse. progress is about the new, a different way of doing things. It requires that you let go of the old way of doing things, that the means of evaluating worthiness changes. That the tools and the methods of support and deployment change. That has always been the way of progress. https://www.youtube.com/watch?v=yUQRbqc2qtY We've had this debate before. Rob Pike said that 'each thing should do one thing and only one thing' and that lead to be debate about line numbering in 'cat' http://harmful.cat-v.org/cat-v/ Here, we have a debate about the purity of log collection vs the use of external tools put together in a more 'traditional' UNIX manner as scripts using pipes and filters in scripting languages. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/22/2016 07:25 AM, Carlos E. R. wrote:

This is not progress, it is retrocesion.

https://www.youtube.com/watch?v=xzm9urjQbWU -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/23/2016 08:24 PM, Carlos E. R. wrote:

On 2016-12-22 14:38, Anton Aylward wrote:

...

On 12/22/2016 07:25 AM, Carlos E. R. wrote:

...

This is not progress, it is retrocesion.

...

https://www.youtube.com/watch?v=xzm9urjQbWU

Whatever the point may be is totally lost on me. English nursery rymms means nothing to me.

The point is that just like the guy in the nursery rhyme you are making a (multi stage) circular argument. its only justification is its own premise. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-22 08:43, jdd wrote:

Le 22/12/2016 à 02:19, Carlos E. R. a écrit :

...

Yes, now someone will tell me to branch systemd.

- --

if you are the only one complaining, yes. If there are many people, there should be enough man power to do the job.

I know I'm not the only one, as the request for those changes has been made and rejected. Here, others do not talk because they do not want to be accused of things by the systemd zeleots. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhbxr4ACgkQja8UbcUWM1wVNgD/Yw05TaM/vqFdUnhB9rvhUwHF HYBYBRRyiBZjIybfOnsA/3tg7sKm++M0WiPCfB2tQg6FbWaHZULDs19ftVWWq1X8 =Ni17 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward wrote:

On 12/20/2016 11:59 AM, jdd wrote:

...

Le 20/12/2016 à 17:36, Carlos E. R. a écrit :

...

I don't use inn :-)

well, it's the same problem. Don't use standard system log for debug an application. you could as well flood /var/log/messages with this

I think what JDD means, and I agree with him here, is that using syslog for debug is a bit of a kludge.

Well, syslog() has had the LOG_DEBUG level for decades. It may not be exactly "debug" messages, but simply an extensive audit trail. -- Per Jessen, Zürich (1.0°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/21/2016 05:14 AM, Carlos E. R. wrote:

On 2016-12-21 09:15, Per Jessen wrote:

...

Anton Aylward wrote:

...

On 12/20/2016 11:59 AM, jdd wrote:

...

Le 20/12/2016 à 17:36, Carlos E. R. a écrit :

...

...

I think what JDD means, and I agree with him here, is that using syslog for debug is a bit of a kludge.

...

Well, syslog() has had the LOG_DEBUG level for decades. It may not be exactly "debug" messages, but simply an extensive audit trail.

Yes. It allows, for instance, tracking where an email went wrong, tracing how it went through the different daemons.

The information is different from the audit the authorities want for email: from, to, subject, date. At least, maybe all the headers. Dunno.

And debug messages do not flood /var/log/messages if you want. They would go to a different file, with earlier rotation. In case of email, it can be mail.debug, but the convention on openSUSE was /var/log/mail (there is mail.err and mail.info, maybe mail.warn)

You are making a very good case for there being a completely separate mechanism of DEBUG messaging from management messaging and audit trail messaging. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Hello, On Tue, 20 Dec 2016, jdd wrote:

Le 20/12/2016 à 17:36, Carlos E. R. a écrit :

...

I don't use inn :-)

well, it's the same problem. Don't use standard system log for debug an application. you could as well flood /var/log/messages with this

Bog standard openSUSE syslog-ng default config (/usr/share/doc/packages/syslog-ng/syslog-ng.conf.default) has this: ==== [..] filter f_console { level(warn) and facility(kern) and not filter(f_iptables) or level(err) and not facility(authpriv); }; filter f_newsnotice { level(notice) and facility(news); }; filter f_newscrit { level(crit) and facility(news); }; filter f_newserr { level(err) and facility(news); }; filter f_news { facility(news); }; [..] filter f_messages { not facility(news, mail) and not filter(f_iptables); }; filter f_warn { level(warn, err, crit) and not filter(f_iptables); }; filter f_alert { level(alert); }; [..] # # Enable this, if you want to keep all messages in one file: # (don't forget to provide logrotation config) # #destination allmessages { file("/var/log/allmessages"); }; #log { source(src); destination(allmessages); }; [..] # # News-messages in separate files: # destination newscrit { file("/var/log/news/news.crit" suppress(30) owner(news) group(news)); }; log { source(src); source(chroots); filter(f_newscrit); destination(newscrit); }; destination newserr { file("/var/log/news/news.err" suppress(30) owner(news) group(news)); }; log { source(src); source(chroots); filter(f_newserr); destination(newserr); }; destination newsnotice { file("/var/log/news/news.notice" suppress(30) owner(news) group(news)); }; log { source(src); source(chroots); filter(f_newsnotice); destination(newsnotice); }; # # and optionally also all in one file: # (don't forget to provide logrotation config) # #destination news { file("/var/log/news.all"); }; #log { source(src); source(chroots); filter(f_news); destination(news); }; [..] # # All messages except iptables and the facilities news and mail: # destination messages { file("/var/log/messages" suppress(30) owner(-1) group(-1) perm(-1)); }; log { source(src); source(chroots); filter(f_messages); destination(messages); }; [..] # # Warnings (except iptables) in one file: # destination warn { file("/var/log/warn" suppress(30) fsync(yes)); }; log { source(src); source(chroots); filter(f_warn); destination(warn); }; ==== That means you can pick and choose which files stuff goes where and choose to keep each file as long as you want, e.g. deleting newsnotice after a week, but keep mail logs for years. That's what Carlos was talking about. And BTW: syslog-ng has a sql() destination option, so you could log (some messages) directly into a database[1], e.g. just the news notice stuff: ==== destination d_newsnotice_sql { sql(type(pgsql) host("localhost") username("syslog-ng") password("password") database("logs_newsnotice") table("messages_${HOST}_${R_YEAR}${R_MONTH}${R_DAY}") columns("datetime", "host", "program", "pid", "message") values("{$R_DATE}", "${HOST}", "${PROGRAM}", "${PID}", "${MSGONLY}") indexes("datetime", "host", "program", "pid", "message")); }; log { source(src); source(chroots); filter(f_newsnotice); destination(d_newsnotice_sql); }; ==== Or whatever. Just grabbed the first example from [1]. And BTW: I have logs since Jan. 2011 in my /var/log (and there's quite a bit of debugging noise in them). # du -hs /var/log 249M /var/log And logging coredumps??? -dnh [1] Currently the Microsoft SQL (MSSQL), MySQL, Oracle, PostgreSQL, and SQLite databases are supported. https://www.balabit.com/documents/syslog-ng-ose-3.8-guides/en/syslog-ng-ose-... -- Get back there in front of the computer NOW. Christmas can wait. -- Linus "the Grinch" Torvalds, 24 Dec 2000 on linux-kernel -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 06:24 AM, Andrei Borzenkov wrote:

Systemd author is actively opposed to any idea to filter log entries in journald.

So here I am at a bank or a brokerage firm or some other regulated commodity (and yes, I've been though similar at a electricity provider and natural gas distributor) and its important that many things, event logs, business transaction logs, various types of communication, NOT be filtered and in some cases not deleted, perhaps never, perhaps only for seven years. The ones I've worked -- oh and many major NOCs I've visited and interviewed at as well - at have a dedicated team with dedicated high powered machines and huge amounts of disk devoted to logging and log analysis. Some are dealing with attacks and intrusions, insider violations, and of course maintenance events. As I've posted before, one such was simply a ITIL event management interface: if anything happened at one of nearly 50,000 branches of the bank across Canada and the Caribbean they knew about it, if it was more than just an out-of-paper or paper jam they'd notify the office or the support contractor for that area. This is how BIG FIRMS operate. (You could say that this is how IBM taught them to operate by supplying this elves of service for them back in the days ...) The reality is that more and more Linux, as well as Windows, is addressing the needs of Business, and Big Business/Wall Street at that, rather than the singleton home user, the hobbyist. This has become true of the desktop and the laptop as well, with RAID on the laptop, BtrFS for snapshotting, lots more. To be more comprehensive: the market has polarized. The casual user of old has not moved to the phone or tablet for browsing, email/gmail, news pages, Twitter, Facebook. The high demand singleton users are gamers who probably spend more on a graphics board than on the rest of the machine. Other such 'verticals'. That makes many of us 'hobbyists' on PC a bit of an anachronism. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 09:21 AM, Carlos E. R. wrote:

I a professional setup I would keep syslog, never rely on journal alone. At worst, I would have to keep both, because journal is not up to the task.

In what way is it not up to the task? -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-20 10:29, ianseeks wrote:

On Tuesday, 20 December 2016 04:43:03 GMT Carlos E. R. wrote:

...

I have an nntp local server (proxy), and it produces tons of log messages per day. With syslog I can dump them all after, say, a week. With journal, I can't. If I want to keep a reasonable long term journal of all important events in my machine, it also keeps the useless debug logs of everything else... I can not purge the database of selected content.

Are you sure about that? There are loads of configuration options in journald.conf regarding storage and storage sizes and retention times

Yes, I can control size. My current solution is zero size. It is not that. What I need is a method to delete from the journal thousands of messages automatically that match a certain criteria. Say, debug level messages older than a week for facility nntp. Ie, purging.

...

With syslog, I can keep logs dating back for a year or two and it is no issue. I simply dump the verbose debug logs of some facilities like nntp. Syslog/rsyslog can now read all log records directly from the journal (might depend on the version you use) and then you could continue as you normally do, just set storage to "volatile" or "none" and forward records to syslog socket (according to journald.conf)

Yes, that's is what I do, because I can't do it with journal, because it grows to an insane size. -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

* ianseeks [12-19-16 13:45]:

On Monday, 19 December 2016 18:32:11 GMT Carlos E. R. wrote:

...

On 2016-12-19 18:24, Anton Aylward wrote:

...

On 12/19/2016 10:56 AM, Carlos E. R. wrote:

...

On 2016-12-19 16:29, Anton Aylward wrote:

...

On 12/18/2016 02:21 PM, Greg Freemyer wrote:

...

4) Syslog is only one of many log systems on local machines. Separate logs are kept for utmp/wtmp, lastlog, audit, kernel logs, firmware logs, and a multitude of application-specific log formats. This is not only unnecessarily complex, but also hides the relation between the log entries in the various subsystems.

Hmm. So if I run journald can I get rid of all those other log files?

What other log files? You mean non-syslog log files? Those other log files are not handled by journald, either. They are handled by the specific applications that create them.

In that any program can, yes, but that's not my point.

My point is that that journald logs so many of these things, or can.

journalctl _COMM=sshd journalctl _COMM=login journalctl _COMM=su journalctl _COMM=systemd-logind

Of course most access involves PAM one way or another so just make sure you make use of pam_logind !!!

That's just login sessions of different types. I don't remember where these where kept before.

...

if you really want to be paranoid, you can forget about the indexoing that journalctl can do and simple grep

journalctl |grep login | more

Takes minutes with rotating rust.

...

Oh My!

Well that soaks up quite a few of those log files: utmp/wtmp, lastlog

...

...

More and more, logs are are getting to be a means of detection intrusions, hacks. But that means correlating logs, which used to be a tedious and error-prone process. We have in journald the opportunity to see all the activity in one log file, making correlation of events much easier.

No correlation unless you have a database viewer and analysis tool.

Ah, tools!

Well, for a start, journalctl does index.

What does that mean? Examples, please.

...

Then we have a;; the regular tools, from 'cut'/'join' of V7 UNIX vintage, on though 'awk' (see the appropriate White Book for how to convert text to database and do correlations and produce nicely formatted output) on though Perl, which was designed for doing that kind of thing.

Nonono. I want ready made user end tools. I do not want to learn programming awk now or have to write my own tools.

...

You've been around long enough, Carlos, to know and have done this sort of thing. What's this, some anti-systemd feeling?

No, not at all.

...

...

But then, logs may be as difficult to analyze as they currently are in Windows.

This isn't Windows. <obscentity> Windows. part of the nature of Windows is to make you buy some third party tool to do things that with UNIX/Linux can be done with a few lies of scripting.

The journal thing in Linux is getting closer to what Windows does with logs. That's my fear. I'm unable to find events in Windows system log, that I know they are there.

Windows is a special case, they do make some things as difficult as possible to hide things they don't want you to see.

...

-- Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 "Bottle" (Minas Tirith))

-- opensuse:tumbleweed:20161217 Qt: 5.7.0 KDE Frameworks: 5.29.0 KDE Plasma: 5.8.4 kwin5-5.8.4-171.1.x86_64 Kernel: 4.8.13-1-default Nouveau: 1.0.13_2.1

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

while you are unable to trim *anything* :( and add nothing of any value to the conversation :0: * ^From:.*\bingmybong@btinternet.com /dev/null I will no longer be bothered! -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 Photos: http://wahoo.no-ip.org/piwigo @ http://linuxcounter.net -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-19 21:52, Felix Miata wrote:

Patrick Shanahan composed on 2016-12-19 15:33 (UTC-0500):

...

I will no longer be bothered!

Announcing who and how you killfile I do not get. You think people who don't trim even understand what you've done? What's the point?

What's happened today? I see people very grumpy. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYWKgACgkQja8UbcUWM1wJIQD/VVrDSEMTzjMSscvD0/KxhvtH /tUMkCK+xPvvy63SQjQA/0AfPAHuml82m/z5PBv4tGUKSe4X+fPonGb/eE5AMqxs =RTua -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/19/2016 5:01 PM, Carlos E. R. wrote:

What's happened today? I see people very grumpy.

- -- Cheers / Saludos, give me coffee and no one gets hurt? :-) here it is either that or insufficient donuts.

Damon Register -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

* L A Walsh [12-19-16 22:07]:

Problems such as not trimming, seem to be more prevalent on lists where users demand bottom posting on the basis of it providing a top-down readable history of the conversation.

In order for bottom posters to be able to read previous conversation, they often include the full text of such conversations.

On lists that don't force bottom posting, the users quickly learn to just include the relevant parts they are responding to.

(just my experience and certainly not a 100% rule)...

-l

Patrick Shanahan wrote:

...

while you are unable to trim *anything* :( and add nothing of any value to the conversation

I would hazzard that you are mis-speaking. *Most* top poster cannot be bothered with *anything* below their comment. But you have continually exhibited actions contrary to the opensuse list netiquette statements including but not limited to posting to the list and duplicating to the poster(s). i'm done! -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 Photos: http://wahoo.no-ip.org/piwigo @ http://linuxcounter.net -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Patrick Shanahan wrote:

* L A Walsh [12-19-16 22:07]:

...

On lists that don't force bottom posting, the users quickly learn to just include the relevant parts they are responding to.

I would hazzard that you are mis-speaking. *Most* top poster cannot be bothered with *anything* below their comment.

That is my impression too. When you're bottom posting, you're forced to page through whatever it is you're responding to. Kind of inspires one to chop away at it. -- Per Jessen, Zürich (0.7°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 03:29 AM, jdd wrote:

some users answer through smartphones

trimming is very difficult with the ones I know, so I usually wait I have a real keyboard to write and only use smartphone to read/delete

+1 to all that. We really need to complain to the developers of the UIs for phones about this. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 20/12/2016 à 10:33, ianseeks a écrit :

Seems if you can be deemed a **** if you top post, bottom post, trim and no trim. I've seen moaning about all of those actions.

It's always better to trim. I mostly don't read posts written below my screen write bottom if you answer to a sentence of the post (like I do here). write top if your answer is more general, not related to the post content (for example to answer the subject if the thread derived to other content, like it could be here). keep the previous content is the reader may not have read it, for example restarting an old thread or answering to a commercial letter the sender may not have archived my two cents :-) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/19/2016 11:58 PM, Per Jessen wrote: <- possibly important context snipped ->

That is my impression too. When you're bottom posting, you're forced to page through whatever it is you're responding to. Kind of inspires one to chop away at it.

Indeed. This is why there's justification for both. There are certain environments where you want to carry thread history for future reference. These would tend to be professional team incident response and operational threads. I personally prefer top-posting for everything, but I'm not religiously fanatic about it. In the immortal words of Rodney King: "...can we all get along?" Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-19 19:44, ianseeks wrote:

...

The journal thing in Linux is getting closer to what Windows does with logs. That's my fear. I'm unable to find events in Windows system log, that I know they are there.

Windows is a special case, they do make some things as difficult as possible to hide things they don't want you to see.

The thing in Windows is that you have to know the correct "token" to do the filter, or you find nothing. Like getting the output on a particular index in journalctl. Either you know how the particular item you seek for is indexed, or you don't get anything. There is no text see it all output. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYWTQACgkQja8UbcUWM1ze5gD+LP7ohKB6WB+qtKvRWzuJXJ+D 9R+dYI8i+JoaRMBbeqAA/1DXvWP43ZhxxzGqy06P7m+WE0166bsMyQg/nZbamgF7 =RD7u -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 00:12, Anton Aylward wrote:

On 12/19/2016 05:03 PM, Carlos E. R. wrote:

...

On 2016-12-19 19:44, ianseeks wrote:

...

...

The journal thing in Linux is getting closer to what Windows does with logs. That's my fear. I'm unable to find events in Windows system log, that I know they are there.

...

Windows is a special case, they do make some things as difficult as possible to hide things they don't want you to see.

The thing in Windows is that you have to know the correct "token" to do the filter, or you find nothing. Like getting the output on a particular index in journalctl. Either you know how the particular item you seek for is indexed, or you don't get anything. There is no text see it all output.

Yes, but journalctl will tell you - see my post about <tab><tab>

Well, it doesn't work on the two machines I have available... :-? - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhYbzMACgkQja8UbcUWM1y1nAD/UNF+AN994vYOHqDB6ReImBtm T9M4kf1Mn3vL9zzzMCoA/2cvQZ5GZvcaa284BzKoabcKhOgRZ7dPaJh5386vbmwg =aeiX -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Mon, Dec 19, 2016 at 10:29 AM, Anton Aylward wrote:

On 12/18/2016 02:21 PM, Greg Freemyer wrote:

...

4) Syslog is only one of many log systems on local machines. Separate logs are kept for utmp/wtmp, lastlog, audit, kernel logs, firmware logs, and a multitude of application-specific log formats. This is not only unnecessarily complex, but also hides the relation between the log entries in the various subsystems.

Hmm. So if I run journald can I get rid of all those other log files? either delete them and hope the subsystem won't recreate them or link them to /dev/null ?

More and more, logs are are getting to be a means of detection intrusions, hacks. But that means correlating logs, which used to be a tedious and error-prone process. We have in journald the opportunity to see all the activity in one log file, making correlation of events much easier.

we just need tools!

RE: tools I don't know all the details, but: libprelude is an opensource log centralization toolkit. It allows a central logserver to collect logs from various other servers/devices. I believe there is a lot of IT security community support behind libprelude. libprelude went into factory 6 months ago and is in Leap 42.2 The Prelude SIEM builds on libprelude and does event correlation and notification. It too is now in factory and Leap 42.2. Suricata is also an opensource SIEM. Better known than the Prelude SIEM I think. It layers on top of libprelude as well. It is in OBS, but not factory or Leap. I don't know if there is any interrelationship between journald and libprelude? So there is lots of activity in the system logging space, but I've fallen behind. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-20 03:43, L A Walsh wrote:

Anton Aylward wrote:

...

More and more, logs are are getting to be a means of detection intrusions, hacks. But that means correlating logs, which used to be a tedious and error-prone process. We have in journald the opportunity to see all the activity in one log file, making correlation of events much easier.

--- That was always your choice. If you found multiple log files unsuitable for your purposes there was no one forcing you to keep them that way. They way they were configure was _ONE_ way out of hundreds or thousands of ways you could choose to have your log pre-parsed and pre-stored.

Yes, you can have all syslog entries stored in a single file. The syslog configuration usually contains a commented out entry for /var/log/allmessages. But he doesn't refer to that one. He refers to other services that use their independent log system, like the login services. Have all services log to the same log service, the journal. Apache, ssh, etc. I don't know. Without a database viewer, like an automatic import to LibreOffice where we can choose filters to see what we want or not, I see no advantage in a database. Somewhere where we can click on an event, follow it, discard garbage, etc. No ready made tools that I know of. -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 12:35 AM, Anton Aylward wrote: phpMyAdmin is network bnased. Journal can be accessed over the net as well See https://www.freedesktop.org/software/systemd/man/systemd-journal-gatewayd.se... Maybe your hypothetical GUI would like a real-time feed? -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 02:55 AM, Per Jessen wrote:

Anton Aylward wrote:

...

In the mean time, the real issue isn't that you can't import a snapshot of the journalctl into oocalc, its that ooclc ignores the JSON standard unless you have the add-in for it. Seems dumb to me that it should handle CSV and not JSON in this day and age.

For largish amounts of data (megabytes), JSON or XML is overkill, but CSV is very efficient.

if you're talking "byte efficient" as in low overhead, then yes. But the whole point of something like XML is not about byte efficiency, it is about semantic clarity and error handling. A few garbled bytes of a CSV writes off everything that follows. A few garbled bytes of a XML transmission resyncs on the following stanza because of the semantic structure. JSON is similar though the contrast between XML and JSON is rather like between ALGOL and C with respect to keywords vs curly brackets. Error resilience, semantic coherency, correct structuring, are going to be more important when dealing with large amounts of data in a stream. The present costs of storage and bandwidth mean the ideas of 'efficiency' we old fashioned engineers were brought up to now take second place. There is in my database of DotSigQuotes; ------------------ [Alice] has courage which can only be described as awesome. Against all odds, over a noisy telephone line, tapped by the tax authorities and the secret police, Alice will happily attempt, with someone [Bob] she doesn't trust, whom she cannot hear clearly, and who is probably someone else, to fiddle her tax returns and to organize a coup d'etat, while at the same time minimizing the cost of the phone call. A coding theorist is someone who doesn't think Alice is crazy. -------------------- The moral here is that most people are not coding theorists. Or engineers striving for 'byte efficiency'. The efficiencies of XML and JSON have more to do with Alice's noisy telephone line in that they are streaming protocols, the data being supplied in large amounts from a remote source over a high speed network. The cost, considering the large mount of data, of going back to to the start if a burst error occurs, is enormous. Not just in terms of "the phone call" but in terms of time, since this is real-time streaming data. Yes, I'm perfectly aware that you cold code up a packetized protocol for CSV with checksum for each packet and acknowledgement of good receipt vs request of retransmission, sort of like TCP writ large. But pretty soon you get into the overhead and you get into all the issues with queueing and flow control that has produced hundreds of studies and papers about TCP packet management over the decades. You really don't want to go there. Use XML or JSON instead. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 12/20/2016 08:27 AM, Per Jessen wrote:

Anton, you're not actually trying to argue that you would use XML to ensure proper data transmission? :-)

LOL! No, I use streaming XHTML, or for some data feeds stream JSON. We all do, in case you hadn't noticed. For example, I use Firefox and store my session snapshots as JSON. I store all my mindmaps as XML - the semantic content is important. I get my weather reports and other streaming data from services that supply it as streaming JSON. If you use any web application that does behind the scenes updating of data it probably uses the AJAX/JavaScript mechanisms to get asynchronous data from the server. you see that with google when you type a string and it presents suggestions as as pull-down options. What's going on behind the scenes uses data transmission in XML or more likely JSON. The whole point of JSON, which, lets face it, is not that much of an overhead wrt CSV, is the structuring of the data. It makes streaming assimilation much easier and as I say, had some error handling capabilities that do not exist with CSV. Why do I think streaming is so important? Well, try this. Instead of using a web browser, access all your pages by downloading them using wget or curl and then displaying them. you have to wait for the whole file to be dowloaded, all the CSS to be downloaded, all the images to be downloaded all before you see ANYTHING AT ALL. Most sites now make use of streaming mechanism. They may not display the whole page, but you don't get too frustrated as they display the structure, title, the take-away synopses and keep up with you as you read down the page and scroll. To do this there must be structural and possibly semantic data n the stream so that the parser & display know where to put things. Yes, sometimes they get it wrong and piss poor design or piss poor implementation means you page jitters around :-( There are so many situations where real time data flow from remote sources is important. Databases? Well I've seen brokerage terminals where the database of stock trades, performance is both historic and up to the minute because its drawing on both the archived data and the streaming current data. The volumes involved are very large and they are continuous. But their structure and semantic content is very important as the INTEGRITY must be guaranteed. You can't do that with CSV. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward wrote:

Why do I think streaming is so important?

Well, try this.

Instead of using a web browser, access all your pages by downloading them using wget or curl and then displaying them. you have to wait for the whole file to be dowloaded, all the CSS to be downloaded, all the images to be downloaded all before you see ANYTHING AT ALL.

Most sites now make use of streaming mechanism.

--- You are confusing incremental display & page update with streaming. They aren't the same. Example. If you store a jpg, you have the option to optimize for the web. Part of that is displaying, say, every 4th row from line 0 of the image every forth row on your screen (0 4 8...). In 1/4th the time of displaying the whole image, you have a low-ish resolution picture. Then it does the same offset by 2 (2 6 10...). Now your pic has 2x the resolution and looks clearer. Finally rows 1 and 3 are filled in in each group of 4. A way that looks slower and is more frustrating is strictly going from top to bottom (0,1,2,3,4...). However, in both cases, you are getting a stream of binary data from the server. Similarly, when a webpage displays, you first see text content, then it jumps into position as the css stylesheet is next downloaded and applied, then pictures will start filling in. Those are the result of how the browser asks for information. URL 1st, sees 'CSS' stylesheet ref at top and asks for it while displaying text from the URL. As the text is parsed, images and other embed's are requested by the client. Multiple requests are usually being downloaded at the same time and filled in **incrementally** (!=streaming). No web brower waits until everything is downloaded to display anything -- its *ALWAYS* been that way and has nothing to do with streaming. -l -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On 2016-12-20 06:35, Anton Aylward wrote:

...

On 12/19/2016 10:52 PM, Carlos E. R. wrote:

...

Yes, as I've mentioned,

journalctl -o json | json-to-csv

or similar lets you import a snapshot into ooclac.

Minas-Anor:~ # journalctl -o json | json-to-csv If 'json-to-csv' is not a typo you can use command-not-found to lookup the package that contains it, like this: cnf json-to-csv Minas-Anor:~ # cnf json-to-csv json-to-csv: command not found

Take it as an example, it would be straight forward to write such a converter. You can on doubt find one out there already. -- Per Jessen, Zürich (2.0°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-20 16:29, Anton Aylward wrote:

On 12/20/2016 10:11 AM, Per Jessen wrote:

...

Carlos E. R. wrote:

...

On 2016-12-20 06:35, Anton Aylward wrote:

...

On 12/19/2016 10:52 PM, Carlos E. R. wrote:

...

Yes, as I've mentioned,

journalctl -o json | json-to-csv

or similar lets you import a snapshot into ooclac.

Minas-Anor:~ # journalctl -o json | json-to-csv If 'json-to-csv' is not a typo you can use command-not-found to lookup the package that contains it, like this: cnf json-to-csv Minas-Anor:~ # cnf json-to-csv json-to-csv: command not found

Take it as an example, it would be straight forward to write such a converter. You can on doubt find one out there already.

Yes, IIR giving an example. There are libraries for this in PHP, Python, Perl and Ruby, as well as on-line conversion services.

Useless to me. I can not write that tool. Thus, "no tools" -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 18:19, Andrei Borzenkov wrote:

20.12.2016 19:41, Carlos E. R. пишет:

...

Useless to me. I can not write that tool. Thus, "no tools"

https://github.com/jehiah/json2csv

Apparently, I need to do: go get github.com/jehiah/json2csv First I have to install "go" (zypper install go). Then I try: cer@Isengard:~> md src cer@Isengard:~> cd src cer@Isengard:~/src> md json2csv cer@Isengard:~/src> cd json2csv cer@Isengard:~/src/json2csv> go get github.com/jehiah/json2csv package github.com/jehiah/json2csv: cannot download, $GOPATH not set. For more details see: go help gopath cer@Isengard:~/src/json2csv> Too complicated, I will not go that route. Removing go. If those fantastic tools are not in oss repo, they do not exist as far as I'm concerned. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhZrM8ACgkQja8UbcUWM1x4ewD9FQUw/HTi8/YXgsT4Se5LE0RP vzq6wRqTXByi2HtPjHwBAJ3jl6UUUH72zSWiMnkIjwkluNVD10NXXf4I1FzZMoE8 =v7Zm -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 22:04, Anton Aylward wrote:

On 12/20/2016 11:41 AM, Carlos E. R. wrote:

...

On 2016-12-20 16:29, Anton Aylward wrote:

...

...

Yes, IIR giving an example. There are libraries for this in PHP, Python, Perl and Ruby, as well as on-line conversion services.

Useless to me. I can not write that tool. Thus, "no tools"

so you don't use shell either? you can't download something already written using zypper?

Well, how about running a web browser? http://www.convertcsv.com/json-to-csv.htm

Online tool? No way, no privacy. You are grasping at straws. There are no tools to view directly journal logs in all the complexity. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhZrToACgkQja8UbcUWM1zSagD/aD91M2rE3HKlA9LHf9nBAOp1 K9d4cT9cp4Hvw++SbWMBAIJvtow+YaQwQ5UUxc4Y/dKd7QSI+D6HXsOyxbQNiGEh =Y+SU -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-20 08:53, Per Jessen wrote:

Carlos E. R. wrote:

...

I don't know. Without a database viewer, like an automatic import to LibreOffice

Way too inefficient, *Office is not suited for processing large amounts(*) of data. We rotate all logs at midnight, then rsync them to a central archive and run daily analysis during the night. All command line tools.

(*) I am assuming large amounts because small amounts don't require any tools, really.

I mention LO because that's the only powerful table viewer I know in Linux that can apply filters and such. Do you know some viewer for the journal, where I can, say, filter out of display all nntp messages? Without coding. -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-20 16:44, Anton Aylward wrote:

On 12/20/2016 09:30 AM, Carlos E. R. wrote:

...

...

I mention LO because that's the only powerful table viewer I know in Linux that can apply filters and such. Do you know some viewer for the journal, where I can, say, filter out of display all nntp messages?

Without coding.

That's being ridiculous.

But if you want you could take the JSON output, read it into a MySQL database and use the very power phpMyAdmin took, which has lots of filter and slicing and dicing and querying capabilities.

That is absurd. You are telling me to make my own tools, and I say I want open source tools that directly read the journal and dynamically apply to it point and click functions like filter or follow event. Unless the journal developers create those tools, the journal is irrelevant to me. -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 22:00, Anton Aylward wrote:

On 12/20/2016 11:49 AM, Carlos E. R. wrote:

...

...

That is absurd. You are telling me to make my own tools, and I say I want open source tools that directly read the journal and dynamically apply to it point and click functions like filter or follow event.

Unless the journal developers create those tools, the journal is irrelevant to me.

UNIX, and by implication Linux, has always been about making your own tools and assembling them using pipes and filters and regular expressions and the like.

No, I will not accept that argument. The truth is, those fantastic tools to analyze the journal do not exist, you have to create them yourself. Well, no, I prefer to keep using syslog with the current tools. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhZrcQACgkQja8UbcUWM1wUlQD/VhpOsWWlcyJHhN6r7LUXnnx/ 4vZJHfEHS68BrANrcnYA/ir2qZgnh2M9T4QDZtkeXmkHCX1W966hpa3UYD1W9liO =/xjO -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-21 08:04, jdd wrote:

Le 20/12/2016 à 23:16, Carlos E. R. a écrit :

...

yourself. Well, no, I prefer to keep using syslog with the current tools.

ypu said the journald dev didn't azccept, do you have a pointer to the discussion?

No. Andrei mentioned this fact on another post, so perhaps he has it.

did you ask leafnode dev?

Why? Leafnode is doing the correct thing. For decades. It is the newcomer which doesn't, IMO. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhaV5IACgkQja8UbcUWM1zjFwD/W7l+d7b04k9waSVkOjEfnEgg YX2MlkyjZzqrWbPl7E8A/1EGcXKtD8I+RZRXU3+9sFZwf4DPpiOGPLYU7kySqbfJ =bKUs -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-22 02:08, L A Walsh wrote:

Anton Aylward wrote:

...

UNIX, and by implication Linux, has always been about making your own tools and assembling them using pipes and filters and regular expressions and the like.

---- Except that the new log format isn't something easily parsed by all the text tools on Unix/Linux. It went binary. Text tools don't do binary very well if at all.

A log format that turns its back on the *nix toolset by going binary needs some tool set other than the *nix one.

That's right. It needs new tools that have not being designed. Yes, you can build your own, apparently. Not good. Or you can convert to text and use traditional tools on that text. Then there is no advantage... - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhbKwoACgkQja8UbcUWM1y9uwEAj3DEg2Xqf3snOTaWFjKYU2/D d5z6f1FSMxsiXglA03YBAIY59gLxCkdedQQdR4OjgWiZiJmDpFHPbj4pWFT44RQ6 =ygpo -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 21:08, Greg Freemyer wrote:

Leap 42.2 and factory have prewikka. I think prewikka can be used for advanced log viewing.

I know it works with syslog. I don't know about journald logs.

Anyone used prewikka? Anyone know if it can pull in logs for journald

Seems to be too complex. I installed it, could not start it. Requires being root. And: cer@Isengard:~> rpm -ql prewikka | grep bin/ /usr/sbin/prewikka-httpd cer@Isengard:~> it appears to be a daemon. Removing. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhZr/MACgkQja8UbcUWM1x+fQD/WVNrhgPP4iBFrpU8A6PObPoS dPw6VTTj/5GLU7w9/m0BAIaJgNvsau9+pfDTbxY2A2mhnfkX2CEYO68wmvLaSlKk =AbGR -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Tue, Dec 20, 2016 at 10:44 PM, Andrei Borzenkov wrote:

20.12.2016 23:08, Greg Freemyer пишет:

...

On Tue, Dec 20, 2016 at 10:44 AM, Anton Aylward wrote:

...

...

I mention LO because that's the only powerful table viewer I know in Linux that can apply filters and such. Do you know some viewer for the journal, where I can, say, filter out of display all nntp messages?

Without coding.

That's being ridiculous.

Leap 42.2 and factory have prewikka. I think prewikka can be used for advanced log viewing.

Is it not a frontend to prelude? Just to be sure we mean the same thing?

The prelude SIEM user interface, yes.

...

I know it works with syslog. I don't know about journald logs.

Anyone used prewikka? Anyone know if it can pull in logs for journald

If this is indeed frontend to prelude, we need to look for prelude driver (or is it called sensor) for journald.

Agreed. prelude-lml is the syslog sensor. https://www.prelude-siem.org/projects/prelude/wiki/PreludeLml I don't know if it can also handle journald. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-21 08:03, nicholas cunliffe wrote:

to be objective and constructive: 1- could someone articulate clearly what we want to achieve. (except journal selective editing) 2- there seem to be rants about viewing, filtering, template-like alerts?, forwarding logs to server? whats the use case of oocalc?

calc can import databases, and there is also a database component on LO/OO. With that you can read directly the syslog mysql database as generated by modern daemons. Once displayed in LO, you can apply filters, sorts, the lot of things you can do in a database with a powerful database viewer.

3- ALL the tools are already available, the code would be TRIVIAL. (from a quick view prelude etc look convoluted)

I'm reading about that. As I'm no longer being payed to examine logs, I will not develop code to inspect the journal, or invest much time on learning complex tools, but I'm interested in knowing if the tools exist. Tools ready made, not code to be done.

4- there is already a (toy) journal viewer in yast, if this is the kind of thing wanted, what is missing?

It seems a text viewer, does not separate fields in columns, no sort. I did not try the filter. It can not follow trails. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhaWjoACgkQja8UbcUWM1yDOQEAg45tG9Gn02dJG3K0DflJ2SzV ZEeGFWVQmzsYeIjNlscA/juh98hIKc4hfEBrSMy5sA1BhQag7YoHCPXIeYHX03wE =ANz8 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-20 21:11, Greg Freemyer wrote:

We have good databases in openSUSE.

We have at least one log aggregator in openSUSE (libprelude).

We have at least 2 log correlation engines in openSUSE (Prelude SIEM and Suricata (only in OBS)).

Again, I have no first hand experience, but it is time for some experimenting I do believe.

If I were still paid to do log analysis, yes, I would. As it is, I do not see those fantastic tools to see the journal, which would be the main justification for having binary logs. The tools have not been developed, you have to write your own. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhZsLAACgkQja8UbcUWM1wXZAD/T+gZ1OK5MQX+csJCzFCpqtEs v3S8oQQbY9NCxWBqacAA/3CY9PhNCX1th8xQjHh5YVWoKjvPqLbukH1bA5FwyP7p =drHG -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On 2016-12-20 08:53, Per Jessen wrote:

...

Carlos E. R. wrote:

...

...

I don't know. Without a database viewer, like an automatic import to LibreOffice

Way too inefficient, *Office is not suited for processing large amounts(*) of data. We rotate all logs at midnight, then rsync them to a central archive and run daily analysis during the night. All command line tools.

(*) I am assuming large amounts because small amounts don't require any tools, really.

I mention LO because that's the only powerful table viewer I know in Linux that can apply filters and such. Do you know some viewer for the journal, where I can, say, filter out of display all nntp messages?

grep -v ? -- Per Jessen, Zürich (1.6°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On 2016-12-20 17:00, Per Jessen wrote:

...

Carlos E. R. wrote:

...

...

I mention LO because that's the only powerful table viewer I know in Linux that can apply filters and such. Do you know some viewer for the journal, where I can, say, filter out of display all nntp messages?

grep -v ?

Of course, that's what I use, but on syslog, not on the journal.

It'll work on the journal too. -- Per Jessen, Zürich (0.9°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

On 2016-12-20 21:03, Per Jessen wrote:

...

Carlos E. R. wrote:

...

...

...

grep -v ?

Of course, that's what I use, but on syslog, not on the journal.

It'll work on the journal too.

Of course. But first you have to expand it, and this operation can take minutes on rotating rust.

Apparently yes, but the solution works nonetheless. As for a log viewer - I don't have a need myself, but I feel certain there are tools out there that will let you filter adhoc, colourise lines etc. Not a text-processor like *office, a log-processor. I don't think they belong as part of the journal though. I googled it - how about "colourtail", "multitail", "grc" or "view" (with vi syntax-highlighting) ?

And then there is the problem that due to the debug messages in the logs, the journal for a year can be really huge, so I don't keep it.

Yes, I do see that as a problem, especially when other messages are discarded when it grows too big. Same thing happens with the kernel ring buffer - firewall logging will often make it wrap. I'm implementing ulog, I just haven't quite got it working yet. If you have or want these copious amounts of logging and you want to use the journal, I see no other solution but making the space available. Or maybe direct those debug logs to disk?

In this machine I disabled completely the journal: This works with rsyslog, but not with syslog-ng, I believe.

I expect it would work with both, I'm sure they use the same interface. Hmm, I have just changed to storage=none on a test system, and syslog-ng does indeed not work. Interesting, I wonder why that is. Well, not a real problem, we use the default journal setup anyway. -- Per Jessen, Zürich (0.9°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-21 10:04, Andrei Borzenkov wrote:

On Wed, Dec 21, 2016 at 10:22 AM, Per Jessen wrote:

...

...

In this machine I disabled completely the journal: This works with rsyslog, but not with syslog-ng, I believe.

I expect it would work with both, I'm sure they use the same interface.

Hmm, I have just changed to storage=none on a test system, and syslog-ng does indeed not work. Interesting, I wonder why that is.

It reads and tracks the files.

Default configuration of syslog-ng is using system() source and system() source on Linux adds systemd-journal() source if it detect that it runs under systemd. systemd-journal() source actively pulls log entries from journal files. "storage=none" means no journal files - no log entries for syslog-ng source as well.

Actually I'm rather puzzled why rsyslog works for you at all :) On test TW installation I do not see any journal-specific part in rsyslog configuration and ForwardToSyslog apparently defaults to "no" so rsyslog should not receive data even via socket ... indeed, after installing and starting rsyslog I do not get any test message via it.

You need to have both initially, journal and rsyslog. Once rsyslog is working, disable storage on journal. This works on 13.1 at least, I have not tested in recent versions. It works because it doesn't pull the messages from the files. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhaW18ACgkQja8UbcUWM1zdOwEAg7opIfQloPVrQi7NLVmnSGyM nSxAWlfzcttOBGRhA84A/3y86m2gu9JpyAHcEvRN5tXQJYjhaZ+DmKZCWU+fzj00 =4pNw -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-22 14:56, Carlos E. R. wrote:

On 2016-12-21 10:04, Andrei Borzenkov wrote:

...

On Wed, Dec 21, 2016 at 10:22 AM, Per Jessen <> wrote:

...

...

In this machine I disabled completely the journal: This works with rsyslog, but not with syslog-ng, I believe.

I expect it would work with both, I'm sure they use the same interface.

Hmm, I have just changed to storage=none on a test system, and syslog-ng does indeed not work. Interesting, I wonder why that is.

Default configuration of syslog-ng is using system() source and system() source on Linux adds systemd-journal() source if it detect that it runs under systemd. systemd-journal() source actively pulls log entries from journal files. "storage=none" means no journal files - no log entries for syslog-ng source as well.

Actually I'm rather puzzled why rsyslog works for you at all :) On test TW installation I do not see any journal-specific part in rsyslog configuration and ForwardToSyslog apparently defaults to "no" so rsyslog should not receive data even via socket ... indeed, after installing and starting rsyslog I do not get any test message via it.

Well, quick test: I rebooted to 42.2 and applied the change. Firstly I installed rsyslog, enabled it (with some problems), checked that it was getting messages, then configured storage=none in journal. Rebooted, and syslog was still getting messages.

:-)

Journal says: cer@Minas-Anor:~> journalctl --follow -- Logs begin at Sun 2016-12-11 17:25:41 CET. -- Dec 22 14:06:34 Minas-Anor systemd[1]: Stopped Create Static Device Nodes in /dev. Dec 22 14:06:34 Minas-Anor systemd[1]: Stopped Create list of required static device nodes for the current kernel. Dec 22 14:06:34 Minas-Anor systemd[1]: Closed udev Kernel Socket. Dec 22 14:06:34 Minas-Anor systemd[1]: Closed udev Control Socket. Dec 22 14:06:34 Minas-Anor systemd[1]: Starting Cleanup udevd DB... Dec 22 14:06:34 Minas-Anor systemd[1]: Started Cleanup udevd DB. Dec 22 14:06:34 Minas-Anor systemd[1]: Reached target Switch Root. Dec 22 14:06:34 Minas-Anor systemd[1]: Starting Switch Root... Dec 22 14:06:34 Minas-Anor systemd[1]: Switching root. Dec 22 14:06:34 Minas-Anor systemd-journald[116]: Journal stopped and syslog says: 2016-12-22T14:01:16.180279+01:00 Minas-Anor rsyslogd: [origin software="rsyslogd" swVersion="8.4.0" x-pid="3986" x-info="http://www.rsyslog.com"] start 2016-12-22T14:01:16.181997+01:00 Minas-Anor systemd[1]: Listening on Syslog Socket. 2016-12-22T14:01:16.182028+01:00 Minas-Anor systemd[1]: Starting System Logging Service... 2016-12-22T14:01:16.182438+01:00 Minas-Anor systemd[1]: Started System Logging Service. 2016-12-22T14:01:16.184725+01:00 Minas-Anor kernel: [ 0.000000] microcode: CPU0 microcode updated early to revision 0xa0b, date = 2010-09-28 2016-12-22T14:01:16.184737+01:00 Minas-Anor kernel: [ 0.000000] Initializing cgroup subsys cpuset 2016-12-22T14:01:16.184739+01:00 Minas-Anor kernel: [ 0.000000] Initializing cgroup subsys cpu 2016-12-22T14:01:16.184741+01:00 Minas-Anor kernel: [ 0.000000] Initializing cgroup subsys cpuacct So it captured the current boot or close to it. -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-12-22 15:55, Andrei Borzenkov wrote:

On Thu, Dec 22, 2016 at 5:39 PM, Carlos E. R. <> wrote:

...

So it captured the current boot or close to it.

That's only kernel messages that it likely gets directly from kmsg. Try logger - does it appear in /var/log/messages?

Sure :-) cer@Minas-Anor:~> logger hello ^C cer@Minas-Anor:~> 2016-12-22T17:32:36.704131+01:00 Minas-Anor dbus[821]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' 2016-12-22T17:32:36.704728+01:00 Minas-Anor systemd[1]: Started Network Manager Script Dispatcher Service. 2016-12-22T17:33:14.696189+01:00 Minas-Anor cer: hello -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-22 18:19, Andrei Borzenkov wrote:

22.12.2016 19:34, Carlos E. R. пишет:

...

On 2016-12-22 15:55, Andrei Borzenkov wrote:

...

On Thu, Dec 22, 2016 at 5:39 PM, Carlos E. R. <> wrote:

...

...

So it captured the current boot or close to it.

That's only kernel messages that it likely gets directly from kmsg. Try logger - does it appear in /var/log/messages?

Sure :-)

cer@Minas-Anor:~> logger hello ^C cer@Minas-Anor:~>

2016-12-22T17:32:36.704131+01:00 Minas-Anor dbus[821]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' 2016-12-22T17:32:36.704728+01:00 Minas-Anor systemd[1]: Started Network Manager Script Dispatcher Service. 2016-12-22T17:33:14.696189+01:00 Minas-Anor cer: hello

Yes, on Leap 42.2 ForwardToSylog default to "yes". On TW it is disabled. TW actually follows upstream default, although I could not find any changelog describing it.

Arguably default rsyslog configuration on TW should then include journald module. Anyone cares enough to open bug report?

I don't use TW... It will have to be you ;-) I can report it, but I can not easily run tests. Only on an old virtual install. I'd have first to try reproduce the issue. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhdCikACgkQja8UbcUWM1zQQwEAiUPbR8RWkxzEvA5FHn49ZdKk vHpLaGSZt1l7W6ZOxLcA/0bg/btgLUxGcuegIfjoHBxb8zMtVr7g2m0fef56Rvng =LkxY -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 21.12.2016 19:50, John Andersen wrote:

But also, someone with some influence over the KDE developers needs to point out to those guys they they simply MUST STOP sending every useless debug message out with "warning" or higher severity levels.

They (KDE devs) are by far the most egregious offenders, and I have actually had some email exchanges directly with these developers trying to convince them of their folly, and had exactly zero success. The've never listened to end users on any subject whatsoever, but they might listen to Opensuse.

I could only find https://bugs.kde.org/show_bug.cgi?id=368357, which was actually caused by a version mismatch on your system. Could you please list the other bug numbers in bugs.kde.org? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-21 08:22, Per Jessen wrote:

Carlos E. R. wrote:

...

...

...

...

grep -v ?

Of course, that's what I use, but on syslog, not on the journal.

It'll work on the journal too.

Of course. But first you have to expand it, and this operation can take minutes on rotating rust.

Apparently yes, but the solution works nonetheless.

If that's what I must use, I prefer working on syslog than on journal.

As for a log viewer - I don't have a need myself, but I feel certain there are tools out there that will let you filter adhoc, colourise lines etc. Not a text-processor like *office,

I did not suggest to use a text processor, but a database or a table processor.

a log-processor. I don't think they belong as part of the journal though.

I googled it - how about "colourtail", "multitail", "grc" or "view" (with vi syntax-highlighting) ?

I do not know. I may try them, time permitting. :-)

...

And then there is the problem that due to the debug messages in the logs, the journal for a year can be really huge, so I don't keep it.

Yes, I do see that as a problem, especially when other messages are discarded when it grows too big. Same thing happens with the kernel ring buffer - firewall logging will often make it wrap. I'm implementing ulog, I just haven't quite got it working yet. If you have or want these copious amounts of logging and you want to use the journal, I see no other solution but making the space available. Or maybe direct those debug logs to disk?

My solution is to use rsyslog, not journal. Problem solved.

...

In this machine I disabled completely the journal: This works with rsyslog, but not with syslog-ng, I believe.

I expect it would work with both, I'm sure they use the same interface.

No, they don't. At least, not on 13.1

Hmm, I have just changed to storage=none on a test system, and syslog-ng does indeed not work. Interesting, I wonder why that is. Well, not a real problem, we use the default journal setup anyway.

And Andrei said that it does not work for him on TW with rsyslog. I want to know if this is a change somewhere that makes my trick impossible, or some config error that can be solved. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhbLW8ACgkQja8UbcUWM1wFEAD+IPaZ8CFnf4aFz1Qk/KZzpV9/ 2/ODnHhvYAtMTe09nbAA/R/2Ffls3y6+cFNB5GGxl7pUCm86afYDoa+7gyMD+QMR =AbWQ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-12-22 08:36, Per Jessen wrote:

Carlos E. R. wrote:

...

I did not suggest to use a text processor, but a database or a table processor.

I thought you mentioned *office, that's a text-processor in my book. It's basically not meant for handling tens of thousands of lines.

No no. Libre Office is a suite, has different programs. One is the text processor, but another is a calc program, similar to Excel, which is capable of importing databases into a table. Another is "lobase", a database eumm... processor.

...

...

Hmm, I have just changed to storage=none on a test system, and syslog-ng does indeed not work. Interesting, I wonder why that is. Well, not a real problem, we use the default journal setup anyway.

And Andrei said that it does not work for him on TW with rsyslog. want to know if this is a change somewhere that makes my trick impossible, or some config error that can be solved.

There's something for you to do over the Xmas holidays. :-)

Yes, but I need access to my main computer, to create virtual machines into which experiment. On the laptop, 42.2 means rebooting to the test partition. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlhbyI8ACgkQja8UbcUWM1ypTQEAh1vlgGLBR7YFxR+11qG/+oBQ UGxaYCtPx0yrPFoKUQ8A/3xeyFGQbbDIdN3/P70uPU0YHcx2gc29dkf4VRXK02I8 =9z09 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org