On 02/13/2014 06:05 PM, Greg Freemyer wrote:
Since you seem to know the details, can walk me through a simple example.
As I already said, I'm not yet sure about the correlation between the two cases where a) the unreadable part of a bad read (on failing disks) is padded out with NULs with "conv=noerror,sync", versus b) the regular "short read" case which mainly happens on pipes/fifos where the provider does not have enough data ready to be read by dd(1).
Assume we have a block device with only 8 sectors and the 4th sector is bad:
GGGBGGGG - G=Good - B=Bad
"dd bs=4K conv=noerror,sync" will produce:
GGG00000 - 0= a sector full of nulls.
It does that because it only invokes read once, and when if gets a short read, it null fills the rest of the block. The end result is 4 sectors of valid data is not copied, even though it is readable.
As said above, I can't tell for bad reads (yet), but for "short" reads, e.g. in a fifo case, the result is completely different ... probably as a big surprise to everyone who does not know the implication of a short read playing together with conv=sync and *bs= sizes. I reduced your example to bs=4 for illustration purposes. $ mkfifo fifo # Start dd to read from the fifo with bs=ibs=obs=4. $ dd bs=4 conv=noerror,sync < fifo > out 2> err & # Now write 3 bytes, then sleep and write the other 5 bytes. $ { printf GGG ; sleep 10 ; printf GGGGG; } > fifo # Look at the result: oops 12 bytes! $ ls -ldog out -rw-r--r-- 1 12 Feb 14 01:21 out Oops, 12 bytes? How that? # Look at the stderr diagnostic: $ cat err 1+2 records in 3+0 records out 12 bytes (12 B) copied, 10.0007 s, 0.0 kB/s Ah, there were 3 read() invocations where 2 were "short" reads (1+2 records in), and 3 records out. # Now looking at the output file with od(1): $ od -tx1 out 0000000 47 47 47 00 47 47 47 47 47 00 00 00 0000014 The first read() has returned 3 'G's. and since the obs size is the same as ibs, it has written 4 bytes (G G G NUL). The second read() was successful, and the 4x 'G's are written to the output file as expected. The third read() is a short read and returns only the 8th 'G'; as obs=ibs=4, four bytes are written padded out with 3 NULs. I'd guess 90% of the users would not have expected this, right? The reason for this is the conv=sync padding. Without it, the result looks as expected: $ rm fifo out err $ mkfifo fifo $ dd bs=4 conv=noerror < fifo > out 2> err & $ { printf GGG ; sleep 10 ; printf GGGGG; } > fifo $ cat err $ 1+2 records in $ 1+2 records out 8 bytes (8 B) copied, 10.0018 s, 0.0 kB/s $ od -tx1 out 0000000 47 47 47 47 47 47 47 47 0000010 Although there were 2 short reads, the output has been copied as expected. Looking at ith with "strace -e read,write dd ...", you can see it: ,,, read(0, "GGG", 4) = 3 write(1, "GGG", 3) = 3 read(0, "GGGG", 4) = 4 write(1, "GGGG", 4) = 4 read(0, "G", 4) = 1 write(1, "G", 1) = 1 read(0, "", 4) = 0 ... Okay, this is all the pipe/fifo case. For a failing disk, the user would want to have the same size of the output file as the input block device file. Therefore, he would add the conv=sync option again plus the iflag=fullblock option. I think it's most obvious with the strace output how fullblock works: $ rm fifo out err $ mkfifo fifo $ strace -e read,write -o log dd bs=4 conv=noerror,sync iflag=fullblock < fifo > out 2> err & $ { printf GGG ; sleep 10 ; printf GGGGG; } > fifo $ cat err 2+0 records in 2+0 records out 8 bytes (8 B) copied, 9.99723 s, 0.0 kB/s $ od -tx1 out 0000000 47 47 47 47 47 47 47 47 0000010 $ cat log ... read(0, "GGG", 4) = 3 read(0, "G", 1) = 1 write(1, "GGGG", 4) = 4 read(0, "GGGG", 4) = 4 write(1, "GGGG", 4) = 4 read(0, "", 4) = 0 ... With iflag=fullblock, dd reads until the bs=4 buffer is completely filled before writing it to stdout. As said several times, this is the behavior for "short" reads. I have to dive into the sources again to see how "conv=noerror,sync" interferes with "iflag=fullblock" for *bad* reads. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org