-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tuesday, 2013-04-23 at 16:07 -0400, Greg Freemyer wrote:

That is with just noerror an error in block 3 would get you:

1 2 4 5

For a filesystem with millions of blocks, have all the ones after the readerror offset by one makes it useless. With sync you get:

1 2 0 4 5

Obviously the block with an error is garbage but now the rest of the drive is properly aligned.

I like that explanation. In those cases, though, I prefer to use "dd_rescue", or rather, the script "dd_rhelp".

fyi: If you want to save the output to a file instead of as a clone, then I suggest you use ewfacquire. It is specially designed to do that one job. It puts internal CRCs within the "image" and it can calculate various hashes to let you use ewfverify to check the overall consistency of the image. You use ewfexport to pull the data back out.

Interesting. I'll have a look at that. For 12.1 and 11.4 it is in the security repo. - -- Cheers, Carlos E. R. (from 12.1 x86_64 "Asparagus" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAlF3J34ACgkQtTMYHG2NR9VvQQCdE0rYbPwI6NE5pFy3kj18D8DU +mMAnRCJocA3aEzavyW3qSiMpu28NriG =oz4H -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Greg Freemyer wrote:

On Tue, Apr 23, 2013 at 1:39 PM, ellanios82 wrote:

...

probably best do from terminal :

~ maybe use "dd' :

something like :

dd if=/dev/[source-drive] of=/dev/[target-drive]

Add "bs=4K conv=noerror, sync"

If it has 64M buffer, best to use the buffer size to achive optimal throughput.

4K can be as big as you want, but the default 512 bytes is smaller than the linux kernel pagesize and at least in some scenarios it makes the copy run dog slow. Also the default behavior of dd to a media error is to abort. You don't want that, so noerror says to continue even if you hit a media error. Sync is a modifier to noerror that says if a read error is encountered, pad the output with nulls to keep everything in sync.

--- Ug... I would rather have it die and try again -- sometimes a reread will work... if you have to copy a known bad disk, then what you suggest is reasonable, but on disks with no known problems...??? Also use.... dd besides the if=dev of=dev & bs=64M, you would want iflag=direct, oflag=direct. No GUI will beat it's speed... The direct flags will net you an extra 20-30% speed. If the disk buffers are 32M, use that as the copy size...(etc...)... I duped a windows disk using cygwin's 'dd' from an old SSD to a 1TB drive, then to the new SSD -- worked great! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Hearns, John wrote:

--- Ug... I would rather have it die and try again -- sometimes a reread will work... if you have to copy a known bad disk, then what you suggest is reasonable, but on disks with no known problems...???

On the subject of using 'dd' to read from a disk which has known bad sectors, Have a look at the 'ddrescue' utility which will retry reads: http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html

Good idea. If you have a bad disk, you should use a special tool for it. Somehow this topic seems to have migrated from how to do disk to disk copy w/o using command line (and so many of us try to tell him the command line is by far the best for something this simple), to dealing with bad sector in a source disk... which fortunately for me, is a rare situation. But seriously dd is your most reliable be. A gui won't have the decades of use behind it. dd if=/dev/sda of=/dev/sdb bs=64M iflag=direct oflag=direct & pid=$! ^ ^ output file (destination) ^^ says to not waste time w/extra mem copies -input file (source disk) the & on the end puts it in background, and the pid=$! stores the process number of it then "kill -USR1 $pid" every once in a while and it will print out how much it has done: 18335302+0 records in 18335302+0 records out 9387674624 bytes (9.4 GB) copied, 34.6279 seconds, 271 MB/s Note, for the bs, above, adjust it to the size of the cache memory on your destination disk and be sure write-caching is enabled. That way when you write to it, it goes into the write cache FAST, and dd, can start reading the next block size while the destination disk is flushing it's buffer to disk... I.e. you'll get much better performance in doing the copy. But it's not a GUI...so it doesn't really answer your question...but I'd strongly recommend against using an arbitrary GUI product for something this simple.... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Greg Freemyer wrote:

Linda Walsh wrote:

...

Somehow this topic seems to have migrated from how to do disk to disk copy w/o using command line (and so many of us try to tell him the command line is by far the best for something this simple), to dealing with bad sector in a source disk... which fortunately for me, is a rare situation.

Linda,

I do this procedure as part of my day job.

--- My condolences! ;-)

(That's why I packaged ewfacquire, I use it routinely.)

I am sure it is better for treating bad disks than 'dd'

The "subject drives" which I read from are a random collection of customer drives. They can be a almost new drive in a new machine, all the way to a 10 year old drive in a computer sitting out in a shed that was almost forgotten about. Most are from desktop/laptop PCs a few years old and routinely in use.

I don't keep stats, but I would guess between 5 and 10% of them have at least one bad sector. Having a significant number of bad sectors I agree is rare, but having one or two I would say is almost routine.

---- Um, are you saying a typical user should expect to see 1-2 disk errors on a disk->disk copy? Isn't it fair to say that many consumer level drives not only have 1-2 disk errors, but already have such sectors remapped to per-track spares when new? For that matter, if it isn't a drive nearing it's "end of useful life", would you expect users to actually see or notice such an error -- or wouldn't it be handled by the drives internal firmware -- w/recovery via internal ECC and remapping all handled on the fly? Isn't, by 'SMART' standards, a drive at the end of its useful lifespan when it can no longer automatically relocate such data automatically? Isn't it *normally* the case that a user will only see disk errors on a drive that can no longer remap sectors?

In fact, I think the spec for new drives is no more than one bad sector in 10e10. 500gb drives have a billion sectors, so even with brand new drives having 1 in 10 have a bad sector would be in spec.

---- But you are talking raw sectors -- not formatted capacity, no? Wouldn't the MTBF say, a new, 5-year warranty Hitachi 4TB drive rated at 1-2 million hours (for DeskStar V. Ultrastar models) sort of imply that most users will never see a disk error during the useful life of that disk?

Greg

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh wrote:

Greg Freemyer wrote:

...

Linda Walsh wrote:

...

Somehow this topic seems to have migrated from how to do disk to

disk

...

...

copy w/o using command line (and so many of us try to tell him the command line is by far the best for something this simple), to dealing with bad sector in a source disk... which fortunately for me, is a rare situation.

Linda,

I do this procedure as part of my day job.

---

My condolences! ;-)

...

(That's why I packaged ewfacquire, I use it routinely.)

I am sure it is better for treating bad disks than 'dd'

...

The "subject drives" which I read from are a random collection of customer drives. They can be a almost new drive in a new machine, all the way to a 10 year old drive in a computer sitting out in a shed that was almost forgotten about. Most are from desktop/laptop PCs a few years old and routinely in use.

I don't keep stats, but I would guess between 5 and 10% of them have at least one bad sector. Having a significant number of bad sectors I agree is rare, but having one or two I would say is almost routine.

---- Um, are you saying a typical user should expect to see 1-2 disk errors on a disk->disk copy?

I'm saying 90% of the time they should see 0 bad sectors, but 1 or 2 the other 10% of the time is expected and normal.

Isn't it fair to say that many consumer level drives not only have 1-2 disk errors, but already have such sectors remapped to per-track spares when new? For that matter, if it isn't a drive nearing it's "end of useful life", would you expect users to actually see or notice such an error -- or wouldn't it be handled by the drives internal firmware -- w/recovery via internal ECC and remapping all handled on the fly? Isn't, by 'SMART' standards, a drive at the end of its useful lifespan when it can no longer automatically relocate such data automatically?

You don't seem to understand the bad sector life cycle. Here you go: - Sector magnetism becomes degraded. - Time passes (hours, days, years) - A sector read of that specific sector occurs. - A sector error is returned to the userspace app and smart marks the sector bad - all subsequent reads continue to attempt to read from the same bad sector. That is they typically fail as well. - time passes (seconds, days, years) - a write to the bad sector occurs - drive controller notes this sector is bad, time to remap it - sector remapped and new data written - reads now succeed

Isn't it *normally* the case that a user will only see disk errors on a drive that can no longer remap sectors?

No, see the life cycle and the 2 different places an error can exist for years. Most raid1/5/6 solutions run scrubbers once a month or so to detect and correct these failures without degrading the raid array. Ie. They force the reading of every sector of the array. When an error is found they use the raid correction logic to calculate what it should have been and write the valid data back out. It is the write that triggers the remap. Thus a freshly scrubbed raid array should have 0 bad sectors, but a failure may occur before the next scrub. That is why you want to run a scrub every month or so. Note that for the raid scenario you don't want the drive to automatically retry failed reads. Instead you want it to try exactly once and fail immediately if there is a problem. That lets the raid solution handle it. Most will immediately write out the calculated data and thus fix the bad sector as soon as it is detected. Therefore a raid edition sata drive is actually less reliable than a consumer edition drive. The only difference is the firmware. It is the raid solution that makes the overall system more robust. That means if you are buying drives for a raid, then definately buy a raid edition drive, but if it is for standalone use avoid raid edition drives, they don't have auto retry logic built-in. Back to consumer systems. If a sector holds changing data, then the error will be found and corrected quickly, but assume it is part of unallocated space near the end of the drive. With a modern 500 gb disks, many of the sectors at the end of the drive will NEVER be read or written in the life of the drive under normal conditions. Bad sectors there just sit there in a failed state forever.

...

In fact, I think the spec for new drives is no more than one bad sector in 10e10. 500gb drives have a billion sectors, so even with brand new drives having 1 in 10 have a bad sector would be in spec.

---- But you are talking raw sectors -- not formatted capacity, no? Wouldn't the MTBF say, a new, 5-year warranty Hitachi 4TB drive rated at 1-2 million hours (for DeskStar V. Ultrastar models) sort of imply that most users will never see a disk error during the useful life of that disk?

No. Most new drives today do have zero bad sectors, but run smart on a several year old drive and you will rarely find one with zero remapped sectors. For it to be remapped it had to report bad at some point. That means a media error was reported at least once for every remapped sector reported by smart. (Remember the drives / kernel have retry logic so the error may not propagate to user space so even a tool like dd won't see all the bad sectors.) Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 23/04/2013 19:39, ellanios82 a écrit :

On 04/23/2013 08:07 PM, Jesse Palser wrote:

...

Copy One Hard Drive To Another Hard Drive(Backup)?

Hi,

I was running Win 8 until the release of 12.3. I am happy with 12.3 and will continue to use it.

My question now is how would I copy a 1TB HDD to another 1TB HDD for purposes of backup of data?

I was hoping there was a program somewhere for 12.3 that would do the above using a convenient GUI, but I can't find one. (I would prefer a program with a GUI - not Terminal commands or BootCD)

---

probably best do from terminal :

~ maybe use "dd' :

something like :

dd if=/dev/[source-drive] of=/dev/[target-drive]

..............

or . . . rsync is really good . . . something like :

* dd is not safe. * For such a size, avoid graphical tools. * So prepare your partition, and use rsync instead : rsync -r -P -t --progress source destination Dsant, from France -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 24/04/2013 19:35, Mark Hounschell a écrit :

On 04/24/2013 12:21 PM, Dsant wrote:

...

* dd is not safe. * For such a size, avoid graphical tools. * So prepare your partition, and use rsync instead : rsync -r -P -t --progress source destination

Could you explain why "dd is not safe"?

mark

Tell me if I am wrong, but dd does not know about partition format consistency , files and folders, or checksums. So if you have the choice between dd and rsync, rsync will complain if CRC is wrong. The best would be that the tool would re-read what have been written, as for CD-RW. Dsant, from France -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Dave Howorth wrote:

Don't forget unison

I'm surprised Emacs can't be used for this. ;-) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On Wednesday, 2013-04-24 at 10:03 +0100, Dave Howorth wrote:

...

Don't forget unison

No, unison is not a backup tool. It copies in any direction so that you have the newest copy on both sides. It is wonderful if, for example, you work on the same set of files sometimes on the desktop, sometimes in the laptop; it copies from one to the other (or the other to the one) so that both sides have the same copy (the newest one).

It is not appropriate to keep a backup copy of one machine on another or on another disk. It can do it, but it is not the best.

unison is a lot more flexible than you suggest. But even using it in the simple fashion you suggest will suffice to make a backup copy. Since by definition the newest files are on the active machine, they will be copied to the backup. It has a lot more options than that. You might also want to look at the section in the manual entitled "Keeping Backups", which describes keeping multi-generational backups. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org