On 2017-03-06 04:35, L A Walsh wrote:
Stefan Seyfried wrote:
Am 03.03.2017 um 19:01 schrieb Carlos E. R.:
About the logs. You know that you can keep syslog standard logs, too. And even disable systemd writing logs.
No you can't. It will always put the journal in /run. And even with the journal on a ramdisk, it performs abysmally. (It takes more than 8 hours to read 4GB of journal (default size) from the ramdisk and output it to stdout, I'll see if I can find the bugzilla with all the dirty details, but since it's against SLES, it will probably be secret anyway).
--- 8 hours?! Even if it was compressed with "xz -9", it wouldn't take that long to decompress. Being on ram or disk shouldn't matter, since 4GB would fit in memory for many or most systems these days.
Example on machine with SSD, both journal and syslog: cer@Isengard:~> journalctl --disk-usage Archived and active journals take up 856.0M on disk. cer@Isengard:~> time journalctl | wc -l 733258 <====== real 0m31.853s user 0m27.219s sys 0m7.402s cer@Isengard:~> cer@Isengard:~> free -h total used free shared buffers cached Mem: 7.7G 7.5G 139M 66M 33M 6.8G -/+ buffers/cache: 694M 7.0G Swap: 9.0G 92M 8.9G cer@Isengard:~> cer@Isengard:~> du -chs /var/log/journal/af5da4da5956dc08183725bc583a0cd5/ 857M /var/log/journal/af5da4da5956dc08183725bc583a0cd5/ 857M total cer@Isengard:~> cer@Isengard:~> time cat /var/log/journal/af5da4da5956dc08183725bc583a0cd5/* > /dev/null real 0m0.905s user 0m0.006s sys 0m0.468s cer@Isengard:~> cer@Isengard:~> cat /proc/cpuinfo ... processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 76 model name : Intel(R) Pentium(R) CPU N3710 @ 1.60GHz Cache does not help: cer@Isengard:~> time journalctl | wc -l 733314 real 0m30.573s user 0m27.079s sys 0m6.980s cer@Isengard:~> time journalctl | wc -l 733314 real 0m31.139s user 0m27.175s sys 0m7.997s cer@Isengard:~> er@Isengard:~> uptime 04:48 up 27 days 23:52, 5 users, load average: 0.37, 0.53, 0.39 cer@Isengard:~> It is always the same time. Almost all ram is cache. Machine is not busy. Almost all the time used for reading the journal is user cpu time, not sys. So it is time decoding the files, not reading them. Syslog has about the same lines as the journal, compressed in much less size (16 megs), and reads much faster. I don't understand why the journal is so big in size; there must be redundancy of some sort. cer@Isengard:~> time ( xzcat /var/log/messages*z | wc -l ; cat /var/log/messages | wc -l ; cat /var/log/pruned | wc -l ; xzcat /var/log/pruned*z | wc -l ; cat /var/log/mail | wc -l ) 701597 2251 63888 340224 16634 real 0m1.182s <========== user 0m1.095s sys 0m0.269s cer@Isengard:~> My interpretation is that it has to read, decompress, and sort. Compression is probably done per record, not per file. And only the text. I hope.
Even if it was on a hard disk, the linux kernel, would, by default, buffer the file in memory if it was randomly accessed all over the place and used small I/O sizes. Even SSD's can be slow if you only do small I/O's. Sending a large email from Windows to my linux box -- when it stores it to the remote IMAP store, uses 4K packets.
For a 24MB file, it took 225 seconds meaning I was getting ~110KB/s -- over a 10Gb link (SMB2 file read/write speed is about 200MB/s).
Whereas on the IMAP server side, the disk BW is near 1GB/s and locally its coming off 4 SSD's in a RAID0 -- easily capable of over 200MB/s (near 600MB/s new). So disk I/O isn't the bottleneck it's the small I/O size that lowers BW by 2048!
Also, possible slowing it down would be if it was decompression using direct-I/O rather than buffered I/O, or possible telling the kernel not to cache the journal under decompression -- so as to never get any benefit from the kernel's buffering.
That just sounds too slow to be real -- i.e. you are talking about 146KB/s... and that's not even over a network.
Just do your own measurements... I'll leave this machine generating logs, see how big they get and what are the timings. I don't know how to find out the current size and time limits.
*suspicious* -- wondering about presence of delay loops to discourage people dumping logs...
Huh. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" (Minas Tirith))