Can systemd's journald be configured to write logs to RAM and then periodically save them to disk? I know it can write to RAM and I know it can write to disk, but I don't know whether it can combine the two. The use case is to reduce the number of write operations on a memory card. PS An actual configuration example would be a lot more use than a simple yes or no in answer to my question :) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Friday, 15 March 2019 12:33:00 GMT Dave Howorth wrote:
Can systemd's journald be configured to write logs to RAM and then periodically save them to disk?
I know it can write to RAM and I know it can write to disk, but I don't know whether it can combine the two. I'm not speaking from any experience of this.
I think you can limit the size of the journal file before it rolls over to another file - can that be monitored and as a new file is created the old one is moved to disk? There is also journalctl --rotate that might help to create a new file when requested before you move the old one to disk.
The use case is to reduce the number of write operations on a memory card.
PS An actual configuration example would be a lot more use than a simple yes or no in answer to my question :)
-- opensuse:tumbleweed:20190312 Qt: 5.12.0 KDE Frameworks: 5.55.0 - KDE Plasma: 5.15.2 - kwin 5.15.2 kmail2 5.10.3 - akonadiserver 5.10.3 - Kernel: 4.20.13-1-default - xf86-video-nouveau: 1.0.15 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 15/03/2019 13.33, Dave Howorth wrote:
Can systemd's journald be configured to write logs to RAM and then periodically save them to disk?
I know it can write to RAM and I know it can write to disk, but I don't know whether it can combine the two.
The use case is to reduce the number of write operations on a memory card.
PS An actual configuration example would be a lot more use than a simple yes or no in answer to my question :)
I don't know of a setting for this. I assume it writes to ram initially and soon to disk. I don't know how long it waits before commiting to disk. Ok, found one (man journald.conf): SyncIntervalSec= The timeout before synchronizing journal files to disk. After syncing, journal files are placed in the OFFLINE state. Note that syncing is unconditionally done immediately after a log message of priority CRIT, ALERT or EMERG has been logged. This setting hence applies only to messages of the levels ERR, WARNING, NOTICE, INFO, DEBUG. The default timeout is 5 minutes. Well, there you have. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On Sat, 16 Mar 2019 14:10:05 +0100 "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 15/03/2019 13.33, Dave Howorth wrote:
Can systemd's journald be configured to write logs to RAM and then periodically save them to disk?
I know it can write to RAM and I know it can write to disk, but I don't know whether it can combine the two.
The use case is to reduce the number of write operations on a memory card.
PS An actual configuration example would be a lot more use than a simple yes or no in answer to my question :)
I don't know of a setting for this. I assume it writes to ram initially and soon to disk. I don't know how long it waits before commiting to disk.
Ok, found one (man journald.conf):
SyncIntervalSec= The timeout before synchronizing journal files to disk. After syncing, journal files are placed in the OFFLINE state. Note that syncing is unconditionally done immediately after a log message of priority CRIT, ALERT or EMERG has been logged. This setting hence applies only to messages of the levels ERR, WARNING, NOTICE, INFO, DEBUG. The default timeout is 5 minutes.
Well, there you have.
Thanks, Carlos. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
"Carlos E. R." <robin.listas@telefonica.net> wrote:
On 15/03/2019 13.33, Dave Howorth wrote:
Can systemd's journald be configured to write logs to RAM and then periodically save them to disk? Ok, found one (man journald.conf):
SyncIntervalSec= The timeout before synchronizing journal files to disk. After syncing, journal files are placed in the OFFLINE state. Note that syncing is unconditionally done immediately after a log message of priority CRIT, ALERT or EMERG has been logged. This setting hence applies only to messages of the levels ERR, WARNING, NOTICE, INFO, DEBUG. The default timeout is 5 minutes.
Hmm, so a supplementary question for extra points :) I'm considering an environment where the 'hard disk' is an SD card (not an SSD or HD) and so minimising writes to it is still a relevant goal. It's possible to use journald to write all the logs to persistent storage, which causes it to write to disk once every five minutes, except in an emergency. That solves the problem but ... Most applications still use separate log files under /var/log so the strategy above requires every application to be customised to use journald instead. That means customising lots of conf files in various formats at best and ninja acrobatics at worst. Plus customising some of the systemd unit files as well. And maintaining it all through updates. So ... another approach is to mount a tmpfs in place of /var/log and let the applications continue to log as normal. That works, but loses all the logs on crash or power fail :( Not good. So there's at least one solution out there - a thing called log2ram - that does a couple of things. It regularly updates disk copies of RAM-based log files so they're reasonably up-to-date after a reboot. And it copies the directory structure under /var/log from RAM to disk whilst the system is running and then copies the directory structure back from disk to RAM when the system reboots. If we then treat journald as a special case that writes to disk always so we benefit from its crash-saving feature, then we have another solution. So my question is whether this log2ram idea looks like it will hold water? Or if there are other alternatives out there? (It seems like a fairly common problem) Or any other thoughts from the deep-thinking denizens here? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 17/03/2019 21.32, Dave Howorth wrote:
"Carlos E. R." <robin.listas@telefonica.net> wrote:
On 15/03/2019 13.33, Dave Howorth wrote:
Can systemd's journald be configured to write logs to RAM and then periodically save them to disk? Ok, found one (man journald.conf):
SyncIntervalSec= The timeout before synchronizing journal files to disk. After syncing, journal files are placed in the OFFLINE state. Note that syncing is unconditionally done immediately after a log message of priority CRIT, ALERT or EMERG has been logged. This setting hence applies only to messages of the levels ERR, WARNING, NOTICE, INFO, DEBUG. The default timeout is 5 minutes.
Hmm, so a supplementary question for extra points :)
I'm considering an environment where the 'hard disk' is an SD card (not an SSD or HD) and so minimising writes to it is still a relevant goal.
Hum.
It's possible to use journald to write all the logs to persistent storage, which causes it to write to disk once every five minutes, except in an emergency. That solves the problem but ...
Most applications still use separate log files under /var/log so the strategy above requires every application to be customised to use journald instead. That means customising lots of conf files in various formats at best and ninja acrobatics at worst. Plus customising some of the systemd unit files as well. And maintaining it all through updates.
Hum.
So ... another approach is to mount a tmpfs in place of /var/log and let the applications continue to log as normal. That works, but loses all the logs on crash or power fail :( Not good. So there's at least one solution out there - a thing called log2ram - that does a couple of things. It regularly updates disk copies of RAM-based log files so they're reasonably up-to-date after a reboot. And it copies the directory structure under /var/log from RAM to disk whilst the system is running and then copies the directory structure back from disk to RAM when the system reboots. If we then treat journald as a special case that writes to disk always so we benefit from its crash-saving feature, then we have another solution.
Hum.
So my question is whether this log2ram idea looks like it will hold water? Or if there are other alternatives out there? (It seems like a fairly common problem) Or any other thoughts from the deep-thinking denizens here?
There is something, but I have never tried. Not that extreme, at least. Adjust the kernel filesystem parameters so that it doesn't write the disk cache or buffers frequently. You need also lots of RAM. Now I don't know off hand how to do that, but I know a program that does just that: laptop-mode-tools or some similar name. <https://wiki.archlinux.org/index.php/Laptop_Mode_Tools> <https://github.com/rickysarraf/laptop-mode-tools/wiki/FAQ#i-have-a-solid-state-disk-ssd-in-my-machine-should-i-enable-any-of-the-disk-related-parts-of-laptop-mode-tools-or-are-they-irrelevant> You don't need to use the package, just study it to find out what parameters it changes. Or actually use the package with the adjustments to your choosing. HTH :-) -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
participants (3)
-
Carlos E. R. -
Dave Howorth -
Ianseeks