On 11/30/23 02:37, Per Jessen wrote:
I have a new storage server running LIO as an iscsi target. Disk space is a RAID6 array, 12 x 2Tb drives under mdraid, managed under LVM.
Since we moved to this config, a couple of months back, I have had issues roughly every Monday, continuing on to middle of the week. iscsi volumes are unmounted, or go read-only.
It turns out to be the Sunday run of mdcheck "MD array scrubbing". It is supposedly limited to 6 hours, which is fine, but my client systems have died long before that. Every day an mdcheck_continue runs at 0105, to resume the scrubbing.
Has anyone else experienced this and if so, what did you do? is there maybe a way of running the scrubbing "in the background", letting iscsi services take priority? As it is, it simply doesn't work.
Not with iscsi directly, I have 4T of SATA/SAS raid1 arrays and scrubbing takes very close to 2/hr per T. So if you have 12 2T drives (not sure how many arrays you have), but presuming you are using all 2T per array, that's going to take 4 hr/per array. Like you I start at 1:00 am and it takes until 9:00 am to run straight through. I just try and keep a light demand until it completes. There is a good article on the Arch wiki on it https://wiki.archlinux.org/title/RAID#RAID_Maintenance -- David C. Rankin, J.D.,P.E.