On Tuesday 2023-05-16 11:21, Andrei Borzenkov wrote:
Date: Tue, 16 May 2023 11:21:33 From: Andrei Borzenkov <arvidjaar@gmail.com> To: users@lists.opensuse.org Subject: Re: "background idle" syncing of filesystems
On Tue, May 16, 2023 at 11:33 AM Paul Neuwirth via openSUSE Users <users@lists.opensuse.org> wrote:
Hello,
I recently upgraded RAM on my Desktop computer (running openSUSE Leap 15.4) to 512GiB ECC DDR5. It's a great speed up, as I do a lot of disk I/O (home filesystem). But if it comes to syncing filesystes, it takes up to twenty minutes to sync them. Hence shutdown/reboot takes ages. (Hardware is a SAS Hardware RAID 1 (megaraid_sas with 2 2TB SAS HDDs)).
is it possible to do a regular sync of the filesystem in idle I/O class, e.g. a crontab "ionice -c 3 sync", or would it cause performance issues or some sort of locks?
By default the kernel should flush old dirty pages (change more than 30 seconds ago), the kernel task to do it is called every 5 seconds. This is controlled by /proc/sys/vm/dirty_expire_centisecs and /proc/sys/vm/dirty_writeback_centisecs. If the system accumulates a lot of dirty pages, they are probably constantly changing.
I do not want to loose the speedup by async mounts and buffer/caching. But also worried about possible data loss, e.g. on AC fail.
Default is to start background flushing of dirty pages to disk when they grow over 10% of available memory. With 512GiB this means about 50G. You may consider lowering this limit so background flushing happens more often. See /proc/sys/vm/dirty_background_bytes and /proc/sys/vm/dirty_background_ratio. There are also global dirty_bytes and dirty_ratio which will force synchronous writes.
If some program generates a lot of dirty data, it may be possible to run it in its own cgroup and restrict the total amount of available memory. The above vm sysctls should be applied proportionally to cgroup memory. I think it needs cgroup v2.
Thank you for that information, I thus monitored nr_dirty and nr_dirty_threshold in /proc/vmstat. the highest value for nr_dirty I got so far is as following: nr_dirty 7686 nr_dirty_threshold 23977156 it seems "dirty" data is being written quite regularly to disk. But why does "sync" (or a sysrq emergency sync) take ages then on shutdown? maybe I also need to monitor that over a longer period of time. currently a "sync" is finished in parts of a second. regards, paul