[opensuse] dd to copy a partition?
Hi Folks, One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in: dd if=/dev/sda1 of=/dev/sda2 She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination. I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong? I think she was using 42.3, probably with ext4. Definitely not btrfs, if that matters. Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2018-04-11 16:51, Lew Wolfgang wrote:
Hi Folks,
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in:
dd if=/dev/sda1 of=/dev/sda2
She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination.
I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong? I think she was using 42.3, probably with ext4. Definitely not btrfs, if that matters.
It certainly works if it is the exact same size, or larger destination. If the partition is a logical one, somewhere in the partition head there must be a pointer to the next partition and perhaps to the size. I don't know for sure if this is inside the partition "data area" or out. So I have a doubt if the partition is sda5 or greater. -- Cheers / Saludos, Carlos E. R. (from 42.3 x86_64 "Malachite" at Telcontar)
On 04/11/2018 09:51 AM, Lew Wolfgang wrote:
Hi Folks,
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in:
dd if=/dev/sda1 of=/dev/sda2
She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination.
I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong? I think she was using 42.3, probably with ext4. Definitely not btrfs, if that matters.
Regards, Lew
Lew, there is probably a flaky sector on the partition she is copying from (which is probably the reason for the copy to begin with). dd will quite readily choke and quit copying when it encounters a sector it cannot read, unless the conv=noerror,sync options were given. Also, the smaller the 'bs' setting (e.g. 4096 = 1 sector) will minimize data loss if there is a bad sector present. (I've seen some give `bs=32M` when attempting copies) Which is why we have ddrescue -- for copying partitions with flaky sectors. Find out exactly what options she used, and if possible re-attempt with ddrescue or parted. (I would suggest gparted, but for completely insane reasons, it pulls in 1/2 the btrfs dependencies, yast-plugins, snappers, etc.. -- even when there is no btrfs 50 miles from the install -- really, that's a bit whacko) Find out exactly what options she used, especially if she failed to use 'conv=noerror,sync', and go from there. -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Lew Wolfgang composed on 2018-04-11 07:51 (UTC-0700):
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in:
dd if=/dev/sda1 of=/dev/sda2
She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination.
I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong?
dd if=/dev/sda1 of=/dev/sda2 stands a significant chance of failure on an MBR disk. Before it became standard practice to leave a gap of 1.0MiB or 2.0MiB in front of the first partition, sda1 would be a different size in sectors from an ostensibly identically sized sda2, sda3 or sda4. The latter three would typically be larger than sda1 by 62 sectors. The latter three would also be larger than a "same" size logical. IIUC, with a standard gap from recent partitioning tools the difference between the first and up to three "identical" primaries would be gone, but not the difference between primaries and logicals. So, sda2 or sda3 to sda5 or sda13 should fail in any event. Were the destination partition a nominal size larger than the source, then dd could do OK, and there would be some unused sectors at the end of the partition. Another failure mode would be making an exact clone and not immediately accounting for having multiple filesystems with identical UUIDs and possibly volume labels, whether MBR disk or GPT. One or the other would need to be differentiated (e.g. with tune2fs -U and maybe -L) before either were put into use via mounting. Another failure mode would be attempting to clone an in use (mounted) partition, also same whether MBR or GPT, with files open and who knows what else that would result in filesystem inconsistency when attempting to put it into use. -- "Wisdom is supreme; therefore get wisdom. Whatever else you get, get wisdom." Proverbs 4:7 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/11/2018 10:30 AM, Felix Miata wrote:
Lew Wolfgang composed on 2018-04-11 07:51 (UTC-0700):
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in:
dd if=/dev/sda1 of=/dev/sda2
She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination.
I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong?
dd if=/dev/sda1 of=/dev/sda2 stands a significant chance of failure on an MBR disk. Before it became standard practice to leave a gap of 1.0MiB or 2.0MiB in front of the first partition, sda1 would be a different size in sectors from an ostensibly identically sized sda2, sda3 or sda4. The latter three would typically be larger than sda1 by 62 sectors. The latter three would also be larger than a "same" size logical. IIUC, with a standard gap from recent partitioning tools the difference between the first and up to three "identical" primaries would be gone, but not the difference between primaries and logicals. So, sda2 or sda3 to sda5 or sda13 should fail in any event.
Were the destination partition a nominal size larger than the source, then dd could do OK, and there would be some unused sectors at the end of the partition.
Another failure mode would be making an exact clone and not immediately accounting for having multiple filesystems with identical UUIDs and possibly volume labels, whether MBR disk or GPT. One or the other would need to be differentiated (e.g. with tune2fs -U and maybe -L) before either were put into use via mounting.
Another failure mode would be attempting to clone an in use (mounted) partition, also same whether MBR or GPT, with files open and who knows what else that would result in filesystem inconsistency when attempting to put it into use.
I recently discovered I couldn't dd a partition via sudo. I had to actually su - (or use ctrl-alt-Fx) where it worked when launching it as a properly logged in root. I was following the instructions for getting SLES Raspberry-Pi installed. https://www.suse.com/documentation/suse-best-practices/sles-for-arm-raspberr... It wasn't till the second cup of coffee that I figure it out. (Sles on the Pi works nicely, but it is significantly slower than Raspian. Some minor issues to work through.) -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Apr 11, 2018 at 1:30 PM, Felix Miata <mrmazda@earthlink.net> wrote:
Lew Wolfgang composed on 2018-04-11 07:51 (UTC-0700):
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in:
dd if=/dev/sda1 of=/dev/sda2
She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination.
I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong?
dd if=/dev/sda1 of=/dev/sda2 stands a significant chance of failure on an MBR disk. Before it became standard practice to leave a gap of 1.0MiB or 2.0MiB in front of the first partition, sda1 would be a different size in sectors from an ostensibly identically sized sda2, sda3 or sda4. The latter three would typically be larger than sda1 by 62 sectors. The latter three would also be larger than a "same" size logical. IIUC, with a standard gap from recent partitioning tools the difference between the first and up to three "identical" primaries would be gone, but not the difference between primaries and logicals. So, sda2 or sda3 to sda5 or sda13 should fail in any event.
Were the destination partition a nominal size larger than the source, then dd could do OK, and there would be some unused sectors at the end of the partition.
Another failure mode would be making an exact clone and not immediately accounting for having multiple filesystems with identical UUIDs and possibly volume labels, whether MBR disk or GPT. One or the other would need to be differentiated (e.g. with tune2fs -U and maybe -L) before either were put into use via mounting.
Another failure mode would be attempting to clone an in use (mounted) partition, also same whether MBR or GPT, with files open and who knows what else that would result in filesystem inconsistency when attempting to put it into use.
In addition to Felix's comments, dd aborts by default on a read or write error. I don't use dd often for that job, but when I do it's "dd if=/dev/sda1 of=devsda2 bs=1M conv=noerror,sync" noerror says not to abort on an error and sync says to null fill any failed reads. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Lew Wolfgang composed on 2018-04-11 07:51 (UTC-0700):
Another failure mode would be attempting to clone an in use (mounted) partition, also same whether MBR or GPT, with files open and who knows what else that would result in filesystem inconsistency when attempting to put it into use.
Another reason to use LVM. You can 'clone' a 'partition' with an active file system on it, not a problem. In LVM terminology this is called a 'snapshot'. You create a logical volume - aka partition - that is a snapshot of an existing one. where possible, LVM snapshots try to use the efficient Copy-on-write mechanism and pointers, so not, the LV isn't copied in the same sense that DD might. In that case you might want to look to use 'rsync' or simply 'cp'. It really depends on what the purpose of the copy is. If this is a copy for a disk-to-disk-to-tape backup then the LVM mechanism is just fime. If it is a copy to another spindle that is going to be removed, ten a COW model won't work. As I say, Context is Everything and without the context of the reason for the copy what is good, better/best according to whatever parameters that might be evaluated, cannot be asserted. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Greg Freemyer wrote:
On Wed, Apr 11, 2018 at 1:30 PM, Felix Miata <mrmazda@earthlink.net> wrote:
Lew Wolfgang composed on 2018-04-11 07:51 (UTC-0700):
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in: dd if=/dev/sda1 of=/dev/sda2 She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination. I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong? dd if=/dev/sda1 of=/dev/sda2 stands a significant chance of failure on an MBR disk. Before it became standard practice to leave a gap of 1.0MiB or 2.0MiB in front of the first partition, sda1 would be a different size in sectors from an ostensibly identically sized sda2, sda3 or sda4. The latter three would typically be larger than sda1 by 62 sectors. The latter three would also be larger than a "same" size logical. IIUC, with a standard gap from recent partitioning tools the difference between the first and up to three "identical" primaries would be gone, but not the difference between primaries and logicals. So, sda2 or sda3 to sda5 or sda13 should fail in any event.
Disclaimer -- when I copy partition, it's either to another disk or one end is a file. I've never had such a copy NOT work and I've used a few extreme cases, like copy old image that was a RAID0 from 4 disks ~240GB each to a 2TB hard disk where it used the 1st 1TB (approx) of disk space. From that I copied it to a new 960GB SSD (the HD was too slow even though it was temporary). And eventually copied it to another RAID, but RAID10 with 4 512MB HD's making the RAID. So, you are telling me it's not suppose to work? or does the fact that they were all disks (of a sort) mean that it does? I have copied partitions when whole disk didn't work -- 1st copy copied the partition table, then next 2 copies copied the 2 partitions (for some reason copying the HD resulted in a short copy due to hitting and end of file on the source disk...weird). But copying the 2 partitions to the ones on the other disk (where the parition table had already been copied successfully) worked fine. I don't remember trying to go partition to another partition on the same disk -- would be way too slow.
Another failure mode would be attempting to clone an in use (mounted) partition, also same whether MBR or GPT, with files open and who knows what else that would result in filesystem inconsistency when attempting to put it into use.
In addition to Felix's comments, dd aborts by default on a read or write error.
I used a quiescent and sync'ed system. If it was more important than normal, I might remount it read-only. Also when copying, I made sure it did 'fullblocks' only. I usually dispose of disks that start developing errors... it's not a good sign, IMO. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2018-06-16 02:04, L A Walsh wrote:
Greg Freemyer wrote:
On Wed, Apr 11, 2018 at 1:30 PM, Felix Miata <mrmazda@earthlink.net> wrote:
Lew Wolfgang composed on 2018-04-11 07:51 (UTC-0700):
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in: dd if=/dev/sda1 of=/dev/sda2 She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination. I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong? dd if=/dev/sda1 of=/dev/sda2 stands a significant chance of failure on an MBR disk. Before it became standard practice to leave a gap of 1.0MiB or 2.0MiB in front of the first partition, sda1 would be a different size in sectors from an ostensibly identically sized sda2, sda3 or sda4. The latter three would typically be larger than sda1 by 62 sectors. The latter three would also be larger than a "same" size logical. IIUC, with a standard gap from recent partitioning tools the difference between the first and up to three "identical" primaries would be gone, but not the difference between primaries and logicals. So, sda2 or sda3 to sda5 or sda13 should fail in any event.
Disclaimer -- when I copy partition, it's either to another disk or one end is a file.
I've never had such a copy NOT work and I've used a few extreme cases, like copy old image that was a RAID0 from 4 disks ~240GB each to a 2TB hard disk where it used the 1st 1TB (approx) of disk space.
Notice that the above paragraph you referenced is about particular clone practice. There are many others in which it works. Imaging a primary partition to a logical partition can fail because they don't have the same gaps. And imaging an MBR disk p1 to p2 (same or different disk) can fail because the gaps at the start have changed. Also sector size has changed. -- Cheers / Saludos, Carlos E. R. (from 42.3 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
Notice that the above paragraph you referenced is about particular clone practice. There are many others in which it works.
Oh...
Imaging a primary partition to a logical partition can fail because they don't have the same gaps.
---- I can see there might be problems...but wouldn't automatically assume. However, the original poster's looked like a primary->primary copy. I often see where gaps are left in front of or at end of a partition -- but they aren't inside the partition, unless you override formatting defaults and disable DOS compatibility, for example.
And imaging an MBR disk p1 to p2 (same or different disk) can fail because the gaps at the start have changed.
The partition doesn't include the gaps. I've looked at many partitions, and they always start with the FS-data.
Also sector size has changed.
---- ???! not if they are on the same disk -- and hopefully the person would know if they were copying between disks with different sector sizes ... even still most 4k sector disks emulate 512B sectors (though at a significant write penalty). Nevertheless, they should work... What would be most likely to fail would be on some Win OS which, upon seeing duplicate disk labels would likely disable one of the partitions. If they are on the same disk, it's not likely to be useful to make such a copy, if on separate disks, unplugging one brings up the other copy. Pah.... It's not standard practice, I'll admit...and if you do it, be prepared to have to do it again if it doesn't work the first time... ;-) More than a few things I try involve more than one attempt... Just have to be willing to try again if something doesn't work. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2018-06-16 07:07, L A Walsh wrote:
Carlos E. R. wrote:
Notice that the above paragraph you referenced is about particular clone practice. There are many others in which it works.
Oh...
Imaging a primary partition to a logical partition can fail because they don't have the same gaps.
---- I can see there might be problems...but wouldn't automatically assume. However, the original poster's looked like a primary->primary copy.
The original post was back in April, I don't know why the thread has resurrected now, there is no new data. The copy was done by someone else and Lew did not have factual data.
I often see where gaps are left in front of or at end of a partition -- but they aren't inside the partition, unless you override formatting defaults and disable DOS compatibility, for example.
And imaging an MBR disk p1 to p2 (same or different disk) can fail because the gaps at the start have changed.
The partition doesn't include the gaps. I've looked at many partitions, and they always start with the FS-data.
AFAIK they affect to the size of the partition, how many sectors are available.
Also sector size has changed.
???! not if they are on the same disk -- and hopefully the person would know if they were copying between disks with different sector sizes ... even still most 4k sector disks emulate 512B sectors (though at a significant write penalty). Nevertheless, they should work...
No, Lew did not know what had actually been done :-) Greg was commenting on possibilities. -- Cheers / Saludos, Carlos E. R. (from 42.3 x86_64 "Malachite" at Telcontar)
On 04/11/2018 04:51 PM, Lew Wolfgang wrote:
Hi Folks,
One of my acolytes reported that she used dd to copy one disk partition to another partition on the same disk, as in:
dd if=/dev/sda1 of=/dev/sda2
She said it failed, but was a little fuzzy about exactly what happened (I wasn't there), just that the destination partition appeared to be "empty" after the operation. I don't know how she made that determination.
I've been doing UNIX stuff for a long time but I never would have thought to use dd in this fashion. Maybe instinct? But assuming that the two partitions are exactly the same size, what could possibly go wrong? I think she was using 42.3, probably with ext4. Definitely not btrfs, if that matters.
"short reads", use 'iflag=fullblock': <https://www.gnu.org/software/coreutils/dd> 'conv=noerror,sync' is also useful if the disk starts failing. Other than than, dd(1) is fine for such work. I'd use larger 'bs' than the default for faster copying, e.g. 'bs=32M'. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (10)
-
Anton Aylward -
Bernhard Voelker -
Carlos E. R. -
David C. Rankin -
ellanios82 -
Felix Miata -
Greg Freemyer -
John Andersen -
L A Walsh -
Lew Wolfgang