If you are going to use rsynce consider that you may want to use some additional options to make the copy more faithful to the original...
rsync -a -H -A -X -S /from/ /to/
See the man page.
Before going to 12.1 I used rsync to duplicate / to a spare partition (I always leave make two OS partitions on every drive - can't have too many now that disk is cheap). I then manually edited /etc/fstab /boot/grub/menu.1st to make sure it would boot, that way I could boot the upgraded 12.1 or the old 11.4. Note this was in addition to offline backups (also using rsync). If you brought a machine down to runlevel 1 it would be reasonably safe to just rsync the live machine.
On Sun, 26 Feb 2012 14:01:07 Carl Hartung wrote:
On Sat, 25 Feb 2012 16:07:58 -0800 Steven Hess flamebait@gmail.com wrote:
I am getting ready to do a zypper dup to 12.1 from 11.4 on my main desktop system. I would like to clone my current / drive to a back up disk (attached via USB). I have used pmagic for this in the past. Is it possible to do this from within the openSUSE 11.4 KDE environment?
Thanks Steven
Hi Steven,
This is exactly what I did before upgrading my system to 12.1. Good thing in my case because I just couldn't adapt to GNOME 3. After about a week I reverted back to my 11.4 installation using this method. I also used this method to store the 12.1 installation on another drive to try again later, after more updates have been released.
Notes: This procedure assumes you've got '/' and '/home' on separate partitions and that the source and target partitions are properly created and formatted. Of course, you'll want to verify and/or modify device paths and numbering to match your environment.
The basic procedure I follow is this:
Boot into rescue mode using the 11.4 installation DVD (login as 'root'; does not require a password.)
Plug in your external usb drive and wait a few seconds for it to power up and 'settle.'
cd /mnt mkdir sda1 sda2 sdb1 sdb2 mount /dev/sda1 sda1 mount /dev/sda2 sda2 mount /dev/sdb1 sdb1 mount /dev/sdb2 sdb2 mount (review and ensure everything is mounted as expected) rsync -av --delete sda1/ sdb1 rsync -av --delete sda2/ sdb2 umount reboot
Unplug the external drive when the system is 'down,' before it boots again. Replace the 11.4 installation DVD with your 12.1 media and install. You should install without touching the existing bootloader in case, like me, you decide to revert back.
The procedure I've outlined above creates proper 'mirrors' of the two partitions, keeping timestamps and permissions intact. A side benefit is each file is written in one pass, which eliminates a lot of fragmentation. You can drop the '--delete' flag if you know for a fact that your target partition(s) are empty. Finally, with rsync, the trailing '/' in the source path is important. It tells the program 'copy everything *under* the specified directory without recreating the directory, itself, on the target.'
hth & regards,
Carl
On Mon, 27 Feb 2012 17:59:09 +1300 michael@actrix.gen.nz wrote:
If you are going to use rsynce consider that you may want to use some additional options to make the copy more faithful to the original...
rsync -a -H -A -X -S /from/ /to/
See the man page.
Before going to 12.1 I used rsync to duplicate / to a spare partition (I always leave make two OS partitions on every drive - can't have too many now that disk is cheap). I then manually edited /etc/fstab /boot/grub/menu.1st to make sure it would boot, that way I could boot the upgraded 12.1 or the old 11.4. Note this was in addition to offline backups (also using rsync). If you brought a machine down to runlevel 1 it would be reasonably safe to just rsync the live machine.
Hi Michael,
Please don't 'top post' as reading (and posting) 'top to bottom' is the convention on this list.
It's been my experience that the '-a' flag creates totally "faithful" archives (a.k.a. backups,) in the use case presented by the OP.
What scenarios do you have in mind where the additional flags come into play? Is it 'normal' to find hard links in one's '/' or '/home' partitions? Ditto ACLs or extended attributes?
Clearly the additional flags you've recommended have a purpose. But I have yet to encounter a failed restoration from the archives I've been creating. Have I just been lucky? ;-) Are there any potential drawbacks?
TIA & regards,
On Mon, 27 Feb 2012 22:55:56 Carl Hartung wrote:
On Mon, 27 Feb 2012 17:59:09 +1300 michael@actrix.gen.nz wrote:
If you are going to use rsynce consider that you may want to use some additional options to make the copy more faithful to the original...
rsync -a -H -A -X -S /from/ /to/
See the man page.
Before going to 12.1 I used rsync to duplicate / to a spare partition (I always leave make two OS partitions on every drive - can't have too many now that disk is cheap). I then manually edited /etc/fstab /boot/grub/menu.1st to make sure it would boot, that way I could boot the upgraded 12.1 or the old 11.4. Note this was in addition to offline backups (also using rsync). If you brought a machine down to runlevel 1 it would be reasonably safe to just rsync the live machine.
Hi Michael,
Please don't 'top post' as reading (and posting) 'top to bottom' is the convention on this list.
It's been my experience that the '-a' flag creates totally "faithful" archives (a.k.a. backups,) in the use case presented by the OP.
What scenarios do you have in mind where the additional flags come into play? Is it 'normal' to find hard links in one's '/' or '/home' partitions? Ditto ACLs or extended attributes?
Clearly the additional flags you've recommended have a purpose. But I have yet to encounter a failed restoration from the archives I've been creating. Have I just been lucky? ;-) Are there any potential drawbacks?
TIA & regards,
Omitting any of the extra flags isn't likely to cause a system to fail in any major way.
But if ls -l /bin /usr/bin you will see that some files are hard linked. Losing the association will burn a little space. As long as the package manager restores links when updating a package all will be well. I'd just rather avoid any confusion in the mean time because I sometimes have to manually hack my system (rare these days admittedly).
As for ACL's or extended attributes, my main concern is that I'm not really aware whether any packages are using them or not. I preserve them just in case. The mount options for / normally include user_xattr,acl, so better safe than sorry.
If anything has created a 4GB sparse file, I don't want to lose that space when I copy it (/ is only 20GB) - so I preserve sparse files.
I want a faithful copy to be sure nothing will mysteriously or silently fail on me. But you are right, I bet almost all commonly used things will work just fine - but for a small cost I avoid making the bet.
Sorry about breaking the conventions for the list - thanks for bringing to my attention.
On Tue, 28 Feb 2012 00:03:07 +1300 michael@actrix.gen.nz wrote:
On Mon, 27 Feb 2012 22:55:56 Carl Hartung wrote:
On Mon, 27 Feb 2012 17:59:09 +1300 michael@actrix.gen.nz wrote:
If you are going to use rsynce consider that you may want to use some additional options to make the copy more faithful to the original...
rsync -a -H -A -X -S /from/ /to/
See the man page.
Before going to 12.1 I used rsync to duplicate / to a spare partition (I always leave make two OS partitions on every drive - can't have too many now that disk is cheap). I then manually edited /etc/fstab /boot/grub/menu.1st to make sure it would boot, that way I could boot the upgraded 12.1 or the old 11.4. Note this was in addition to offline backups (also using rsync). If you brought a machine down to runlevel 1 it would be reasonably safe to just rsync the live machine.
Hi Michael,
Please don't 'top post' as reading (and posting) 'top to bottom' is the convention on this list.
It's been my experience that the '-a' flag creates totally "faithful" archives (a.k.a. backups,) in the use case presented by the OP.
What scenarios do you have in mind where the additional flags come into play? Is it 'normal' to find hard links in one's '/' or '/home' partitions? Ditto ACLs or extended attributes?
Clearly the additional flags you've recommended have a purpose. But I have yet to encounter a failed restoration from the archives I've been creating. Have I just been lucky? ;-) Are there any potential drawbacks?
TIA & regards,
Omitting any of the extra flags isn't likely to cause a system to fail in any major way.
But if ls -l /bin /usr/bin you will see that some files are hard linked. Losing the association will burn a little space. As long as the package manager restores links when updating a package all will be well. I'd just rather avoid any confusion in the mean time because I sometimes have to manually hack my system (rare these days admittedly).
As for ACL's or extended attributes, my main concern is that I'm not really aware whether any packages are using them or not. I preserve them just in case. The mount options for / normally include user_xattr,acl, so better safe than sorry.
If anything has created a 4GB sparse file, I don't want to lose that space when I copy it (/ is only 20GB) - so I preserve sparse files.
I want a faithful copy to be sure nothing will mysteriously or silently fail on me. But you are right, I bet almost all commonly used things will work just fine - but for a small cost I avoid making the bet.
Sorry about breaking the conventions for the list - thanks for bringing to my attention.
Thank you very much for bringing this to my attention! Sure enough, /etc/fstab shows acl & user_xattr on both / and /home :-) It is certainly worth investing a couple of extra keystrokes, as you've said. No need to apologize for something that you weren't aware of.
Much appreciated!
Carl
On Mon, 2012-02-27 at 17:59 +1300, michael@actrix.gen.nz wrote:
If you are going to use rsynce consider that you may want to use some additional options to make the copy more faithful to the original...
rsync -a -H -A -X -S /from/ /to/
Why would the oo want to use rsync? It is a great tool for handling files,
For devices (and every horrible detail beneath it (soft raids, lvm, encryption, ...) the more appropiate tool remains "dd" It might me a waste of time, as it also copies empty parts of the file system though.
(see its manpage)
hw
Le 27/02/2012 15:11, Hans Witvliet a écrit :
It might me a waste of time, as it also copies empty parts of the file system though.
yes, and waste of space
jdd
On Mon, Feb 27, 2012 at 1:23 PM, jdd jdd@dodin.org wrote:
Le 27/02/2012 15:11, Hans Witvliet a écrit :
It might me a waste of time, as it also copies empty parts of the file system though.
yes, and waste of space
jdd
If you install ewftools from the security repo (or factory if you're brave), there is a command "ewfacquire".
http://linux.die.net/man/1/ewfacquire
It is basically like dd on steroids, but uses a very different syntax. It has compression built in by design as well internal CRC checksums and an overall hash (md5). It is also smart enough to look for sectors full of nulls and compress them extra well.
All that is kept within the image file set. (By default it breaks the image_file down in the 1.4GB chunks. ie. image_chunks.*)
Then you can use ewfexport to restore the disk to how it was originally. (ie. ewfexport -t - image_chunks.* > /dev/sda)
fyi: this type of image is called an EnCase image or Expert Witness Format (ewf) image. It is widely used by computer forensics professionals. Many consider it more robust than just plain dd, and the compression feature is nice too since it knows its working with disks / sectors.
fyi2: ewf does not ignore unused space. It still considers that important, so a tool that ignores freespace could be even more efficient.
fyi3: I'm trying to package guymager now. It's a QT front-end instead of CLI and supports more image formats than just ewf. But it may not be in security for a week or two. It has a cool look and feel:
http://guymager.sourceforge.net/
Greg
On Tue, 28 Feb 2012 03:11:19 Hans Witvliet wrote:
On Mon, 2012-02-27 at 17:59 +1300, michael@actrix.gen.nz wrote:
If you are going to use rsynce consider that you may want to use some additional options to make the copy more faithful to the original...
rsync -a -H -A -X -S /from/ /to/
Why would the oo want to use rsync? It is a great tool for handling files,
For devices (and every horrible detail beneath it (soft raids, lvm, encryption, ...) the more appropiate tool remains "dd" It might me a waste of time, as it also copies empty parts of the file system though.
(see its manpage)
hw
If I was cloning a partition to a duplicate partition, a larger partition, or replacing a disk, I often do use dd or in some cases tools like gparted. So I agree, if you are truely "cloning a drive" as the topic heading implies then look at these kinds of tools.
If you intend to just keep the dd image then, as jdd mentioned, it's size can be a problem. I guess you can pipe the image into gzip/bzip2/xz.
If the destination partition is smaller, then rsync is an alternate option.
If I want to minimise down time, I can use rsync to prepare the way by copying the live system to the destination. I'd then drop down to run level 1 and do a quick rsync to eliminating any problems due to copying it live. (After this I'd most likely also update the copy, and the original's grub, so that the copy it is a dual boot option.)
If I was copying a really full partition into a relatively empty one I'd consider using rsync or similar so that the OS gets a chance to better allocate blocks in the less crowded partition.
Michael
Le 27/02/2012 20:54, michael@actrix.gen.nz
If I want to minimise down time, I can use rsync to prepare the way by copying the live system to the destination. I'd then drop down
the software I copied i better suited to backup system partitions.
mu own system partition are usually 20Bb, with less than 10Gb oused (on desktops, less than 3Gb on servers), so the downtime to back them up is around 30 minutes.
The rest of the backuop can often be done on the running machine. I don't run any very intensively working server though (no big database).
Backing a large database needs other tools is the base always have writes
jdd
On Tue, 2012-02-28 at 08:54 +1300, michael@actrix.gen.nz wrote:
On Tue, 28 Feb 2012 03:11:19 Hans Witvliet wrote:
On Mon, 2012-02-27 at 17:59 +1300, michael@actrix.gen.nz wrote:
If you are going to use rsynce consider that you may want to use some additional options to make the copy more faithful to the original...
rsync -a -H -A -X -S /from/ /to/
Why would the oo want to use rsync? It is a great tool for handling files,
For devices (and every horrible detail beneath it (soft raids, lvm, encryption, ...) the more appropiate tool remains "dd" It might me a waste of time, as it also copies empty parts of the file system though.
(see its manpage)
hw
If I was cloning a partition to a duplicate partition, a larger partition, or replacing a disk, I often do use dd or in some cases tools like gparted. So I agree, if you are truely "cloning a drive" as the topic heading implies then look at these kinds of tools.
If you intend to just keep the dd image then, as jdd mentioned, it's size can be a problem. I guess you can pipe the image into gzip/bzip2/xz.
If the destination partition is smaller, then rsync is an alternate option.
If I want to minimise down time, I can use rsync to prepare the way by copying the live system to the destination. I'd then drop down to run level 1 and do a quick rsync to eliminating any problems due to copying it live. (After this I'd most likely also update the copy, and the original's grub, so that the copy it is a dual boot option.)
If I was copying a really full partition into a relatively empty one I'd consider using rsync or similar so that the OS gets a chance to better allocate blocks in the less crowded partition.
Michael
Indeed, if you choose wisely you have have the best of both worlds. For instance, _if_ you have separate logical volumes, you can use either tool on them. dd: if you want to clone just a part. or a part you cannot decypher. rsync if you can access the file system.
And dd gives you access before, in between and after the partitions...
hw
On 02/27/2012 06:06 PM, Hans Witvliet wrote:
And dd gives you access before, in between and after the partitions...
and if it is just a copy of 'files' on a partition or drive-to-drive (on the local machine) without the MBR or partition boundary information, good 'ole:
cp -a
works just fine. I like Carl's rsync solution, but personally if I were going to copy my / to another drive and I wanted to make it a bootable replacement for the current drive, I would use 'dd'. That way you could copy the MBR, etc. and simply swap drives and boot back to what you have. gparted works well for copying entire partitions from drive-to-drive as well.
David C. Rankin said the following on 02/27/2012 07:53 PM:
and if it is just a copy of 'files' on a partition or drive-to-drive (on the local machine) without the MBR or partition boundary information, good 'ole:
cp -a
works just fine.
Well maybe. You might want to add a recursion and that still wont copy hidden files.
You could run multiple commands with
cp -ra ${SRC}/* ${DST} cp -ra ${SRC}/.[^.]* ${DST}
or you could turn on the globbing of hidden files:
• Set shopt dotglob option to enable. (shopt -s dotglob) • Copy hidden files and folder. • Set shopt dotglob option to disable.
See the man page for BASH for details (that's RTFM or 'go google')
-
Anton Aylward -
Carl Hartung -
David C. Rankin -
Greg Freemyer -
Hans Witvliet -
jdd -
michael＠actrix.gen.nz