[opensuse] dmraid and files
Is it possible to use dmraid with files rather than physical devices? I've made copies with dd of two RAID0 disks as files. Is there any way to access the contents of the RAID or do I need to copy them onto other physical disks first? Thanks, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hi! Am Donnerstag, 17. April 2008 16:06 schrieb Dave Howorth:
Is it possible to use dmraid with files rather than physical devices? I've made copies with dd of two RAID0 disks as files. Is there any way to access the contents of the RAID or do I need to copy them onto other physical disks first?
You could try "mdadm --assemble /dev/md/0 file1 file2 ...". Don't know whether it works so. Afterwards you should be able to mount /dev/md0 as usual. Regards, Matthias -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Matthias Bach wrote:
Hi!
Am Donnerstag, 17. April 2008 16:06 schrieb Dave Howorth:
Is it possible to use dmraid with files rather than physical devices? I've made copies with dd of two RAID0 disks as files. Is there any way to access the contents of the RAID or do I need to copy them onto other physical disks first?
You could try "mdadm --assemble /dev/md/0 file1 file2 ...". Don't know whether it works so. Afterwards you should be able to mount /dev/md0 as usual.
I tried mdadm --assemble /dev/md0 /tmp/mnt/sda /tmp/mnt/sdb but it says: mdadm: /tmp/mnt/sda is not a block device mdadm: /tmp/mnt/sda has no superblock - assembly aborted Oh well. Thanks for the idea, though. Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
----- Original Message -----
From: "Dave Howorth"
Matthias Bach wrote:
Hi!
Am Donnerstag, 17. April 2008 16:06 schrieb Dave Howorth:
Is it possible to use dmraid with files rather than physical devices? I've made copies with dd of two RAID0 disks as files. Is there any way to access the contents of the RAID or do I need to copy them onto other physical disks first?
You could try "mdadm --assemble /dev/md/0 file1 file2 ...". Don't know whether it works so. Afterwards you should be able to mount /dev/md0 as usual.
I tried
mdadm --assemble /dev/md0 /tmp/mnt/sda /tmp/mnt/sdb
but it says:
mdadm: /tmp/mnt/sda is not a block device mdadm: /tmp/mnt/sda has no superblock - assembly aborted
Oh well. Thanks for the idea, though.
What you want is called a loop device in linux. (other os's call it a file-backed ram disk) losetup is your friend here. Assume your disk images are stored in file0, file1, file2 I'm working with three blank 512 meg files for the sake of the examples here. Assign loop devices to your 3 files: mybox:~ # losetup /dev/loop0 file0 mybox:~ # losetup /dev/loop1 file1 mybox:~ # losetup /dev/loop2 file2 Assemble the array: mybox:~ # mdadm -A /dev/md3 /dev/loop[0-2] mdadm: /dev/md3 has been started with 3 drives. View the array status: mybox:~ # cat /proc/mdstat Personalities : [raid5] [raid4] md3 : active raid5 loop0[0] loop2[2] loop1[1] 1048448 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> Mount the filesystem: mybox:~ # mkdir /md3 mybox:~ # mount /dev/md3 /md3 View the filesystem: mybox:~ # mount [...] /dev/md3 on /md3 type ext2 (rw) mybox:~ # df -h Filesystem Size Used Avail Use% Mounted on [...] /dev/md3 1008M 20K 957M 1% /md3 mybox:~ # 3 x 512 megs in raid5 = 1gig Now, that assumed the disk images were actually partition images, which I'm guessing you didn't think to collect but instead collected images of the full raw disks, such that each file contains the boot block, partition table and multiple partitions all in one file. In that case it's a bit trickier.Then you have to collect the offsets of the different partitions and use them with the -o offset option to losetup, or extract the partitions into new fils with dd and the same offset numbers. Assuming all three disk images are exactly the same sizes and have exactly the same size partitions, collect the numbers from any one disk, if the disks and/or partitions are different from each other, then do the following for each & every disk : mybox:~ # fdisk -ul /dev/loop0 Disk /dev/loop0: 536 MB, 536870912 bytes 255 heads, 63 sectors/track, 65 cylinders, total 1048576 sectors Units = sectors of 1 * 512 = 512 bytes Device Boot Start End Blocks Id System /dev/loop0p1 63 112454 56196 83 Linux /dev/loop0p2 112455 1044224 465885 83 Linux Assuming the 2nd partition is the important one: p2 starts at 512 bytes (per block) x 112455 (blocks) = byte numbr 57576960 There is actually no such file for the partitions like "/dev/loop0p2". We do need to use offsets into the full file with losetup -o offset or dd seek/skip=offset At this point we could either create more loop devices 3,4,5 mapped to chunks of loop0,1,2, such that /dev/loop0 is the full disk and /dev/loop3 would be partition 2 within loop0 but we don't actually need the full disk image loop devices any more, lets un-map them and map them to the partitions we want instead so that loop0 becomes partition 2 of file0, loop1 becomes partition 2 of file1, etc... To me this is less confusing, but do whatever is less confusing for you. Assuming my way, deconstruct the existing loopbacks: mybox:~ # losetup -d /dev/loop0 mybox:~ # losetup -d /dev/loop1 mybox:~ # losetup -d /dev/loop2 And reconnect to the partitions within the same files using -o offset: mybox:~ # losetup -o 57576960 /dev/loop0 file0 mybox:~ # losetup -o 57576960 /dev/loop1 file1 mybox:~ # losetup -o 57576960 /dev/loop2 file2
From here on it's exactly the same as the first example:
Assemble the array: mybox:~ # mdadm -A /dev/md3 /dev/loop[0-2] mdadm: /dev/md3 has been started with 3 drives. Check it out: mybox:~ # cat /proc/mdstat Personalities : [raid5] [raid4] md3 : active raid5 loop0[0] loop2[2] loop1[1] 935936 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> Mount the filesystem: mybox:~ # mount /dev/md3 /md3 Check it out: mybox:~ # mount [...] /dev/md3 on /md3 type ext2 (rw) mybox:~ # df -h Filesystem Size Used Avail Use% Mounted on [...] /dev/md3 900M 20K 854M 1% /md3 Notice the size of the array is smaller than my first example now that I'm using a 50 meg offset into each file instead of the full file. Now just pull out your data at will: mkdir /restore rsync -av --sparse /md3/* /restore or star -copy -v -C /md3 . /restore Don't use tar for a full filesystem like that, use cpio or afio or star or rsync. Just in case it's confusing how a given set of files could make a valid array and a valid filesystem both from the full file and from a 50 meg offset within it, It can't, I did new fdisk, mdadm -C, and mkfs commands on the files between the first & second examples to simulate the 2 different scenarios. -- Brian K. White brian@aljex.com http://www.myspace.com/KEYofR +++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++. filePro BBx Linux SCO FreeBSD #callahans Satriani Filk! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Brian K. White wrote:
Am Donnerstag, 17. April 2008 16:06 schrieb Dave Howorth:
Is it possible to use dmraid with files rather than physical devices? I've made copies with dd of two RAID0 disks as files. Is there any way to access the contents of the RAID or do I need to copy them onto other physical disks first?
What you want is called a loop device in linux. (other os's call it a file-backed ram disk) losetup is your friend here.
Brian, thanks for this - and thanks also to Philipp who made the same suggestion. But you went above and beyond with your detailed reply! :)
<snip> Now, that assumed the disk images were actually partition images, which I'm guessing you didn't think to collect but instead collected images of the full raw disks,
You had me going there, but you're still thinking ahead of me. That is indeed what I did.
such that each file contains the boot block, partition table and multiple partitions all in one file. In that case it's a bit trickier.Then you have to collect the offsets of the different partitions and use them with the -o offset option to losetup, or extract the partitions into new fils with dd and the same offset numbers. Assuming all three disk images are exactly the same sizes and have exactly the same size partitions, collect the numbers from any one disk, if the disks and/or partitions are different from each other, then do the following for each & every disk :
mybox:~ # fdisk -ul /dev/loop0
This is where it gets sad :( I have two identical disks and the images are wrapped up as loop1 and loop2: # fdisk -ul /dev/loop1 # fdisk -ul /dev/loop2 Disk /dev/loop2: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes Disk /dev/loop2 doesn't contain a valid partition table So the first one produced no output. That may not be surprising because that's the one that is believed to be faulty, so we don't know how much data in the image is valid, if any. The source machine is dual boot. Vista (spit!) boots fine and works to a first approximation, though it reports a hard disk fault. Suse boots into single user mode, saying that it can't assemble the raid. I know next to nothing about this type of 'raid', except not to use it! But I guess that means the start of the first disk is lost. Thanks again for the very clear, comprehensive guide! Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2008-04-21 at 10:01 +0100, Dave Howorth wrote:
from any one disk, if the disks and/or partitions are different from each other, then do the following for each & every disk :
mybox:~ # fdisk -ul /dev/loop0
This is where it gets sad :( I have two identical disks and the images are wrapped up as loop1 and loop2:
# fdisk -ul /dev/loop1 # fdisk -ul /dev/loop2
Disk /dev/loop2: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes
Disk /dev/loop2 doesn't contain a valid partition table
So the first one produced no output. That may not be surprising because that's the one that is believed to be faulty, so we don't know how much data in the image is valid, if any.
Is this a linux software raid 1? If it is, you can simply mount one side of the raid. Just try mount loop1 or loop2 without telling it it is a raid, like a plain partition. I did that once and was able to recover the data. Otherwise, it should be possible to be mounted as raid with one side not available, in degraded mode. If "dmraid" is something special... then I don't know. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFIDGOVtTMYHG2NR9URAsYAAKCWiZhPgzD+IBTWWxKTDwYFwM7RdACghMVA 0oXFW95wVoVBy3eXT8CQrAE= =59Q5 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos E. R. wrote:
The Monday 2008-04-21 at 10:01 +0100, Dave Howorth wrote:
from any one disk, if the disks and/or partitions are different from each other, then do the following for each & every disk :
mybox:~ # fdisk -ul /dev/loop0
This is where it gets sad :( I have two identical disks and the images are wrapped up as loop1 and loop2:
# fdisk -ul /dev/loop1 # fdisk -ul /dev/loop2
Disk /dev/loop2: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes
Disk /dev/loop2 doesn't contain a valid partition table
So the first one produced no output. That may not be surprising because that's the one that is believed to be faulty, so we don't know how much data in the image is valid, if any.
Is this a linux software raid 1?
No, it's a fake raid 0 :( Specifically, isw : (+) Intel Software RAID Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2008-04-21 at 11:07 +0100, Dave Howorth wrote:
Is this a linux software raid 1?
No, it's a fake raid 0 :( Specifically, isw : (+) Intel Software RAID
Argh! :-( I dislike those kind of solutions, precisely as proved here. If you run "file /image", where "/image" is the image file you dd'ed from the raids, what does it say? - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFIDIIptTMYHG2NR9URAg9GAJ45WeUBfJesdtour9+iXTTguqrHvgCgiYx3 VRTSRlbF3me8Xks82qETYck= =wmfR -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
----- Original Message -----
From: "Dave Howorth"
Carlos E. R. wrote:
The Monday 2008-04-21 at 10:01 +0100, Dave Howorth wrote:
from any one disk, if the disks and/or partitions are different from each other, then do the following for each & every disk :
mybox:~ # fdisk -ul /dev/loop0
This is where it gets sad :( I have two identical disks and the images are wrapped up as loop1 and loop2:
# fdisk -ul /dev/loop1 # fdisk -ul /dev/loop2
Disk /dev/loop2: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes
Disk /dev/loop2 doesn't contain a valid partition table
So the first one produced no output. That may not be surprising because that's the one that is believed to be faulty, so we don't know how much data in the image is valid, if any.
Is this a linux software raid 1?
No, it's a fake raid 0 :( Specifically, isw : (+) Intel Software RAID
Ahhh, yeah, you have the worst of both worlds there. Sorry to hear that. As you say, "now you know" the hard way.
Software raid is fine, requires more foo to use, but you can do anything and you never care what kind of controller you happen to have today or used to have yesterday. As per this thread, you don't even need any controller as it can all be manipulated in ram or as files.
Hardware raid is fine, far less flexable in that the only way to access an array is with the same brand, family, and type of card that created it, but generally effortless to use and generally far more robust as well.
But with fake raid you get the worst of both worlds and the benefits of neither.
Well, one slight benefit from each. The low cost of software raid and the ability to boot from raid0.
It may actually still be possible to assembly your array, but may require more exotic surgery than the simple commands I showed.
Ultimately, most raids, hardware and software and fake, are all pretty much the same. If you happen to know the exact layout of the array, how large the stripes were, it should be possible to do this:
mdadm -B -l 0 -n 2 -c 64 --assume-clean /dev/md3 /dev/loop0 /dev/loop1
Where 64 is the stripe size in kbytes, and the loop devices are for the full disk images. (since you had fake raid, then the full disk image is the correct and only kind of image to collect after all. The partitions will only be visible from within the assembled array. That command assumes the array was a raid0 with 2 drives
If you don't remember what the stripe size was, maybe you can google up the default for that motherboard, bios, version, and maybe you never changed it?
If you are careful not to write to the array, then you can safely just try several common stripe sizes. 32K 64K 128K etc... try 64, then try to read the partition table within the array:
fdisk -ul /dev/md3
or
sfdisk -l /dev/md3
If it produces junk or nothing, then just stop the current array and rerun the mdadm -B with -c 128 and try sfdisk -l again.
mdadm -S /dev/md3
mdadm -B -l 0 -n 2 -c 128 --assume-clean /dev/md3 /dev/loop0 /dev/loop1
If you ever get a meaningful looking partition table listing, then you can try read-only mounting a filesystem:
Create a loop device to one of the partitions similar to what I described before,
losetup -o
Brian K. White wrote:
Dave Howorth wrote:
No, it's a fake raid 0 :( Specifically, isw : (+) Intel Software RAID
Ahhh, yeah, you have the worst of both worlds there. Sorry to hear that. As you say, "now you know" the hard way.
Luckily for me, it's not my machine and even better, I don't support it :) But it does belong to a colleague who doesn't have a full backup so I tried to help. It's now up to the support guys to try to recover it and if they can't we'll probably send it to a specialist to try to recover the data. I use 3ware RAID5 :) and full nightly backups. I guess after this, my colleague might not use fake raid and might be more consistent with backups. Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Thu, 17 Apr 2008 15:06:24 +0100, Dave Howorth wrote:
Is it possible to use dmraid with files rather than physical devices? I've made copies with dd of two RAID0 disks as files. Is there any way to access the contents of the RAID or do I need to copy them onto other physical disks first?
I don't know if it works, but I'd try loop mounting each file and then pass the loop devices to mdadm. Maybe that works. Philipp -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (5)
-
Brian K. White -
Carlos E. R. -
Dave Howorth -
Matthias Bach -
Philipp Thomas