[opensuse] Disk cloning with ddrescue (gnu)
Hi, I have a suspicious hard disk (suspicion of failure), and I have cloned it to a new one. Exact same model and version. I did it this way: Andor:~/Mantenimientos/migracion_disco_C_2T_2020 # time ddrescue --force --ask --retry-passes=3 /dev/sdc /dev/sdd ddrescue.mapfile --log-events=ddrescue.logfile --log-rates=ddrescue.logrates GNU ddrescue 1.23 About to copy 4000 GBytes from '/dev/sdc' [UNKNOWN] (4000787030016) to '/dev/sdd' [UNKNOWN] (4000787030016) Proceed (y/N)? y Press Ctrl-C to interrupt ipos: 4000 GB, non-trimmed: 0 B, current rate: 47996 kB/s opos: 4000 GB, non-scraped: 0 B, average rate: 75418 kB/s non-tried: 0 B, bad-sector: 0 B, error rate: 0 B/s rescued: 4000 GB, bad areas: 0, run time: 14h 44m 7s pct rescued: 100.00%, read errors: 0, remaining time: n/a time since last successful read: n/a Finished real 884m53.976s user 0m28.480s sys 41m34.528s Andor:~/Mantenimientos/migracion_disco_C_2T_2020 # That is, 14,73 hours. A lot, for a disk with no errors found. ddrescue said initially (yesterday night) that it would take 5:45 hours. This morning it was still running: it said 7 minutes remaining, it took about 14. The thing is, I could see in gkrellm that ddrescue was alternating reading from one disk and writing on the other, instead of doing it simultaneously. Why? :-o I would understand that when it finds a read error it drops to alternating the operations, but when there is no problem, why? If a disk is prone to failure, the longer it takes, the danger increases, right? -- Cheers, Carlos E. R. (from openSUSE Leap 15.1 x86_64 (Minas Tirith)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2020-09-09 11:58, Carlos E. R. wrote:
That is, 14,73 hours. A lot, for a disk with no errors found. ddrescue said initially (yesterday night) that it would take 5:45 hours. This morning it was still running: it said 7 minutes remaining, it took about 14.
A buddy of mine came over with a disc a couple of years ago and we rescued some photo gallery's from a NTFS-partition. During that process I also had problem with slow read of the disc with ddrescue. After we had done the rescuing I kept the disc to experiment on. After some tests I found the -O and -a switch. You can read about it in the manual but my conclusion was that with the switches I could regain the initial read speed. But I suspect that it's a bit of a risk to use that switch due to maybe faulty hw getting even more faulty. Unfortunatley I don't remember the exact command I used at that time. It might have been -a 0. -a bytes --min-read-rate=bytes Minimum read rate of good non-tried areas, in bytes per second. If the read rate falls below this value during the first two passes of the copying phase, ddrescue will skip ahead a variable amount depending on rate and error histories. The skipped blocks are tried in additional passes (before trimming). If bytes is 0 (auto), the minimum read rate is recalculated every second as (average_rate / 10). -O --reopen-on-error Close infile and then reopen it after every read error encountered during the copying phase. If '--min-read-rate' is set, also close and reopen infile after every slow read encountered during the first two passes of the copying phase. Use this option if you notice a permanent drop in transfer rate after finding read errors or slow areas. But be warned that most probably the slowing-down is intentionally caused by the kernel in an attempt to increase the probability of reading data from the device. -- /bengan -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
El 2020-09-09 a las 16:59 +0200, Bengt Gördén escribió:
On 2020-09-09 11:58, Carlos E. R. wrote:
That is, 14,73 hours. A lot, for a disk with no errors found. ddrescue said initially (yesterday night) that it would take 5:45 hours. This morning it was still running: it said 7 minutes remaining, it took about 14.
A buddy of mine came over with a disc a couple of years ago and we rescued some photo gallery's from a NTFS-partition. During that process I also had problem with slow read of the disc with ddrescue. After we had done the rescuing I kept the disc to experiment on. After some tests I found the -O and -a switch. You can read about it in the manual but my conclusion was that with the switches I could regain the initial read speed. But I suspect that it's a bit of a risk to use that switch due to maybe faulty hw getting even more faulty. Unfortunatley I don't remember the exact command I used at that time. It might have been -a 0.
-a bytes --min-read-rate=bytes Minimum read rate of good non-tried areas, in bytes per second. If the read rate falls below this value during the first two passes of the copying phase, ddrescue will skip ahead a variable amount depending on rate and error histories. The skipped blocks are tried in additional passes (before trimming). If bytes is 0 (auto), the minimum read rate is recalculated every second as (average_rate / 10).
-O --reopen-on-error Close infile and then reopen it after every read error encountered during the copying phase. If '--min-read-rate' is set, also close and reopen infile after every slow read encountered during the first two passes of the copying phase. Use this option if you notice a permanent drop in transfer rate after finding read errors or slow areas. But be warned that most probably the slowing-down is intentionally caused by the kernel in an attempt to increase the probability of reading data from the device.
I don't think it was slow read, but that ddrescue can not do read and write simultaneously, as dd does. i/o graph, more or less: sdc (read): ******** ******* * * * * * * * * ***** ********* ******* sdd (write): ***** ******** ******* * * * * * * * * * * ******** ******** ******* -- Cheers Carlos E. R. (from openSUSE Leap 15.1 x86_64 (Minas Tirith))
On 9/9/20 2:21 PM, Carlos E. R. wrote:
El 2020-09-09 a las 16:59 +0200, Bengt Gördén escribió:
-a bytes --min-read-rate=bytes Minimum read rate of good non-tried areas, in bytes per second. If the read rate falls below this value during the first two passes of the copying phase, ddrescue will skip ahead a variable amount depending on rate and error histories. The skipped blocks are tried in additional passes (before trimming). If bytes is 0 (auto), the minimum read rate is recalculated every second as (average_rate / 10).
-O --reopen-on-error Close infile and then reopen it after every read error encountered during the copying phase. If '--min-read-rate' is set, also close and reopen infile after every slow read encountered during the first two passes of the copying phase. Use this option if you notice a permanent drop in transfer rate after finding read errors or slow areas. But be warned that most probably the slowing-down is intentionally caused by the kernel in an attempt to increase the probability of reading data from the device.
I don't think it was slow read, but that ddrescue can not do read and write simultaneously, as dd does.
i/o graph, more or less:
sdc (read):
******** ******* * * * * * * * * ***** ********* *******
sdd (write):
***** ******** ******* * * * * * * * * * * ******** ******** *******
The algorithm is designed to be somewhat slow, much slower than dd according to https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Algorithm and unless the -O or --reopen-on-error isn't modified, then the kernel may intentionally further slow the reads even more to attempt a more full recovery. I can see this becoming more of an issue on multi-terrbyte drives. We are victims of our own success (excess). -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/09/2020 07.58, David C. Rankin wrote:
On 9/9/20 2:21 PM, Carlos E. R. wrote:
El 2020-09-09 a las 16:59 +0200, Bengt Gördén escribió:
The algorithm is designed to be somewhat slow, much slower than dd according to https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Algorithm and unless the -O or --reopen-on-error isn't modified, then the kernel may intentionally further slow the reads even more to attempt a more full recovery.
I can see this becoming more of an issue on multi-terrbyte drives. We are victims of our own success (excess).
But I do not see the point of going slow till it finds errors. I will have to try dd_rhelp the next time. If not, plain dd and hope for the best. Fortunately I was asleep most of the time, but... -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On 2020-09-09 21:21, Carlos E. R. wrote:
I don't think it was slow read, but that ddrescue can not do read and write simultaneously, as dd does.
GNU dd does not read/write simultaneously either. That would require threading or multi-processing. E.g. rsync does. I can't tell for ddrescue, but for dd(1) it makes a huge difference if you use the default block size (4k) or a custom one like e.g. bs=32M. Larger block sizes make the (good) reads/writes faster. The problem comes when there's a read error. AFAIR this is ddrescue's domain, because it re-tries with smaller block to find the physical bad block(s) ... which it will set to NULs when writing the output. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/09/2020 18.57, Bernhard Voelker wrote:
On 2020-09-09 21:21, Carlos E. R. wrote:
I don't think it was slow read, but that ddrescue can not do read and write simultaneously, as dd does.
GNU dd does not read/write simultaneously either.
But it does manage to maximize the i/o, the graph doesn't show alternating phases. It shows more or less continuous, with speed around 150 Mbytes or more.
That would require threading or multi-processing. E.g. rsync does.
I can't tell for ddrescue, but for dd(1) it makes a huge difference if you use the default block size (4k) or a custom one like e.g. bs=32M. Larger block sizes make the (good) reads/writes faster.
I do use larger blocks with dd, but I did not see an equivalent setting for ddrescue. I hoped it would do it automatically.
The problem comes when there's a read error. AFAIR this is ddrescue's domain, because it re-tries with smaller block to find the physical bad block(s) ... which it will set to NULs when writing the output.
Certainly. The question is why doesn't ddrescue go fast when there are no errors, and thread carefully when errors are detected. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
participants (4)
-
Bengt Gördén -
Bernhard Voelker -
Carlos E. R. -
David C. Rankin