[opensuse] How to move completely to new HDD?
I have previously used clonezilla moving from one HDD to 'identicyl' new HDD because of hdd failures on the first one. First I used clonezilla the wrong or hard way and I learned it by making huge mistakes apparently, so it messed up the grub2 install of the leap 42.1 HDD. Then the second attempt I sector by sector copied the whole hdd same amount of blocks to second hdd of the same type which has the same number of blocks. everything worked that way, I had deselected all those additional clonezilla settings that it would not attempt to install grub on its own and all that stuff. eventually i removed hdd-1 and used hdd-2 and then i fired up the leap 42.1 usb key to boot into recovery mode and there I only made sure that /boot/grub2/.... wouldnt point anywhere to the wrong old hdd1 via serial number of the hdd1 and so on. the leap 42.1 booted up fine from the hdd2 after that. Now I have again a similar situation only this time I cannot use the same hdd again so different manufacturer this time and maybe or most probably different number of sectors here, so I cannot sector by sectory copy I guess. what is the official way to re-create the boot mechanism of an opensuse / leap 42.1 properly from say usb key and rescue mode and then going into chroot? how can I make absolutely sure to be able to boot without hassle from my new hdd2 from the different manufacturer. unfortunately there are coming more and more obstacles in our way as linux kernel and components evolve, back in old days it was easy with stuff such as /dev/sda and similar, today we have all sorts of guid and partition uid and lables and then hassle with grub2 and quite a number of places and config files where old device names can hide. think I found leftovers in /etc/default.d/ or similar directories and places which grub2 installer or scripts would take into account Any simple and easy steps or yast2 modules to repair and rebuild a pretty normal single hddl partition setup? with / and /home as partitions next to swap as the only places that exist thanks for helpers. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
cagsm wrote:
I have previously used clonezilla moving from one HDD to 'identicyl' new HDD because of hdd failures on the first one.
First I used clonezilla the wrong or hard way and I learned it by making huge mistakes apparently, so it messed up the grub2 install of the leap 42.1 HDD.
Then the second attempt I sector by sector copied the whole hdd same amount of blocks to second hdd of the same type which has the same number of blocks.
everything worked that way, I had deselected all those additional clonezilla settings that it would not attempt to install grub on its own and all that stuff.
eventually i removed hdd-1 and used hdd-2 and then i fired up the leap 42.1 usb key to boot into recovery mode and there I only made sure that /boot/grub2/.... wouldnt point anywhere to the wrong old hdd1 via serial number of the hdd1 and so on.
the leap 42.1 booted up fine from the hdd2 after that.
Now I have again a similar situation only this time I cannot use the same hdd again so different manufacturer this time and maybe or most probably different number of sectors here, so I cannot sector by sectory copy I guess.
what is the official way to re-create the boot mechanism of an opensuse / leap 42.1 properly from say usb key and rescue mode and then going into chroot?
how can I make absolutely sure to be able to boot without hassle from my new hdd2 from the different manufacturer.
unfortunately there are coming more and more obstacles in our way as linux kernel and components evolve, back in old days it was easy with stuff such as /dev/sda and similar, today we have all sorts of guid and partition uid and lables and then hassle with grub2 and quite a number of places and config files where old device names can hide.
think I found leftovers in /etc/default.d/ or similar directories and places which grub2 installer or scripts would take into account
Any simple and easy steps or yast2 modules to repair and rebuild a pretty normal single hddl partition setup?
with / and /home as partitions next to swap as the only places that exist
thanks for helpers. I do not know the official way and I am interested in recommendations too.
I would go the following steps to replace a hard disk. This way is not the fastest, but you have some flexibility with partitions, encryption, RAID, filesystems etc.: * install the new hard disk into PC * install openSUSE Leap 42.1 (or the matching version on the old hard disk) to this hard disk: o with desired partition format (BIOS or GPT) o with desired partitions o with desired filesystems o ... * boot the new hard disk and do some checks * boot a rescue system from USB * delete files from the new hard disk with some exceptions: /boot, other grub and kernel related files like /etc/default/grub, /etc/fstab, /etc/crypttab, /lib/modules, /etc/sysconfig/kernel * copy files from old to new hard disk (do not overwrite files on new hard disk) * boot the new hard disk * upgrade the distribution on new hard disk Greetings, Björn -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
cagsm wrote:
Now I have again a similar situation only this time I cannot use the same hdd again so different manufacturer this time and maybe or most probably different number of sectors here, so I cannot sector by sectory copy I guess.
what is the official way to re-create the boot mechanism of an opensuse / leap 42.1 properly from say usb key and rescue mode and then going into chroot?
I'm not sure quite what you're asking, but I copy systems around fairly frequently, either for cloning an existing system, moving physical to virtual or just moving hardware. a) prepare the target filesystem - mkfs <fs-of-your-choice>. Depending on the situation, to do this you may need to boot a rescue system or Knoppix or some such. b) mount target filesystem, rsync <sourcefs> to <targetfs> (excluding /dev, /proc and /sys). mkdir /dev /proc /sys. c) re/create the initrd on target system: (mount -bind /proc proc ..... chroot, dracut ...). d) reinstall your favourite boot-loader. (I use lilo, never got used to grub). I've probably left something out. -- Per Jessen, Zürich (4.8°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, Mar 14, 2016 at 3:43 PM, Per Jessen <per@computer.org> wrote:
cagsm wrote:
Now I have again a similar situation only this time I cannot use the same hdd again so different manufacturer this time and maybe or most probably different number of sectors here, so I cannot sector by sectory copy I guess.
what is the official way to re-create the boot mechanism of an opensuse / leap 42.1 properly from say usb key and rescue mode and then going into chroot?
I'm not sure quite what you're asking, but I copy systems around fairly frequently, either for cloning an existing system, moving physical to virtual or just moving hardware.
a) prepare the target filesystem - mkfs <fs-of-your-choice>. Depending on the situation, to do this you may need to boot a rescue system or Knoppix or some such.
Using default Leap layout this could be quite challenging - you need to create the same subvolumes and mount in the same place. So it is more than just mkfs.
b) mount target filesystem, rsync <sourcefs> to <targetfs> (excluding /dev, /proc and /sys). mkdir /dev /proc /sys.
Before continuing: Edit /etc/fstab, /etc/default/grub_installdevice, may be /boot/grub2/device.map, may be /etc/crypttab, may be /etc/mddadm.conf (different device paths, different UUIDs - replace with new ones).
c) re/create the initrd on target system: (mount -bind /proc proc ..... chroot, dracut ...).
d) reinstall your favourite boot-loader. (I use lilo, never got used to grub).
I've probably left something out.
In general, any place where disk device is referenced directly may need update. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-03-14 13:43, Per Jessen wrote:
a) prepare the target filesystem - mkfs <fs-of-your-choice>. Depending on the situation, to do this you may need to boot a rescue system or Knoppix or some such.
I like to create fresh a new but small Linux install on the target disk, and I use this as rescue system to do the migration, and as a guide of settings. As new disks are now huge, dedicating 15 GB or (no separate home) to this is not an issue. Copying everything with rsync has the advantage that I can rethink the partition distribution (sizes and mount points and format types) Otherwise I would use the rescue usb stick of the same openSUSE release, but there is none for Leap. I was trying to create one, but I got stuck at some point. I need to dedicate time to it again. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 03/14/2016 04:46 AM, cagsm wrote:
I have previously used clonezilla moving from one HDD to 'identicyl' new HDD because of hdd failures on the first one.
To be completely honest up front, I haven't tried this with a Linux drive yet. I've cloned several Windows drives with no issues. We bought an Aukey drive cloning box. It wasn't very expensive. http://www.newegg.com/Product/Product.aspx?Item=9SIA5NV2AJ7420 It will clone any SATA drive to one of equal or larger capacity. Put the old drive in one slot. Format and put the new drive in the other slot and push a button. It will also work as an external, drop in, drive by USB cable. -- Fast is fine, but accuracy is final. You must learn to be slow in a hurry. -Wyatt Earp- _ _... ..._ _ _._ ._ ..... ._.. ... .._ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-03-14 14:14, Billie Walsh wrote:
On 03/14/2016 04:46 AM, cagsm wrote:
I have previously used clonezilla moving from one HDD to 'identicyl' new HDD because of hdd failures on the first one.
To be completely honest up front, I haven't tried this with a Linux drive yet. I've cloned several Windows drives with no issues.
Typically those tools replicate the boot system by copying all sector, whereas on Linux they try to recreate boot. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
I've done this a number of times. Of course it helps to have backups :-) Regular readers will recall that U use LVM for everything except the bootloader and /boot. This makes it *VERY* easy to do a migration, provided a few things are kept in mind. I'll get to those. LVM supports multi-spindle groups. It also has control over how those spindles are managed - mirroring, striping. it also has the ability to move completely OFF a particular spindle. So to move to a new drive, I can use 'pvmove'. Simple as that! And yes I've done this and I've done this in a corporate setting. Its also useful for dealing with LVM managed RAID (which is where the corporate 'move' occurred). Carlos makes the point that the new disk should be prepared using fdisk. Conventional tools are also used for dealing with the MBR and with moving /boot. Now one reason I've not had problems with this is that my grub uses 'disk-by-name' not UUID. So it really doesn't matter here the /dev/vgmain/vROOT resides. I can use any allocation strategy I want with the spindles I have, mirror, strip or migrate volume by volume rather than the volume group in one go. This can be useful with disk which are dying, or which have a specific volume that has become unusable on one spindle. Think of it as a 'soft landing'. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
cagsm composed on 2016-03-14 10:46 (UTC+0100): ...
what is the official way to re-create the boot mechanism of an opensuse / leap 42.1 properly from say usb key and rescue mode and then going into chroot?
how can I make absolutely sure to be able to boot without hassle from my new hdd2 from the different manufacturer. .. Any simple and easy steps or yast2 modules to repair and rebuild a pretty normal single hddl partition setup?
You seem to be overthinking the process into avoidable complications. 1-Shut down 2-Connect new disk >= used size of old[1] 3-Boot something other than anything that lives on the source or target disk 4-Ensure nothing on source or target gets mounted 5-Clone with any true cloner, the entirety of the source, sector by sector, be it with dd, dd_rescue, clonezilla, whatever. I've never used Clonezilla. Mostly I use DFSee[2], but with a failing source I have used dd_rescue. 6-Shut down 7-Remove old disk 8-Move new disk to connector formerly used by old That's it, as long as you never try to boot either source or target without appropriate configuration modifications while both are connected to the same PC. If you need to do that, then you first need to either wipe one of them (possibly only selectively), or change UUIDs and volume labels on one of them (possibly only selectively). All that said, I don't often do ("move") as you propose. I almost always want the new to be different in some way, obviating the possibility of cloning an entire disk. In part this is why I use DFSee's large multi-platform tool chest for the bulk of disk configuration processes. The biggest thing it can't do is manage FOSS bootloaders. Typically with a wiped or new disk that can't be initialized via temporary installation in a machine booted to an installed OS, I boot Knoppix to create any initial filesystems that aren't to be provided via cloning, and to configure the first bootloader. [1] if old is old enough to use legacy 512b sectors, and new has 4k sectors, you may wish to reconsider cloning at all. Better to partition, format, and install at a minimum bootloader, probably plus OS, fresh, then use cp or rsync to copy files; this to avoid potential performance degradation. [2] DFSee is not FOSS. http://www.dfsee.com/dfsee/ -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/14/2016 02:46 AM, cagsm wrote:
I have previously used clonezilla moving from one HDD to 'identicyl' new HDD because of hdd failures on the first one.
First I used clonezilla the wrong or hard way and I learned it by making huge mistakes apparently, so it messed up the grub2 install of the leap 42.1 HDD.
Then the second attempt I sector by sector copied the whole hdd same amount of blocks to second hdd of the same type which has the same number of blocks.
everything worked that way, I had deselected all those additional clonezilla settings that it would not attempt to install grub on its own and all that stuff.
eventually i removed hdd-1 and used hdd-2 and then i fired up the leap 42.1 usb key to boot into recovery mode and there I only made sure that /boot/grub2/.... wouldnt point anywhere to the wrong old hdd1 via serial number of the hdd1 and so on.
the leap 42.1 booted up fine from the hdd2 after that.
Now I have again a similar situation only this time I cannot use the same hdd again so different manufacturer this time and maybe or most probably different number of sectors here, so I cannot sector by sectory copy I guess.
what is the official way to re-create the boot mechanism of an opensuse / leap 42.1 properly from say usb key and rescue mode and then going into chroot?
how can I make absolutely sure to be able to boot without hassle from my new hdd2 from the different manufacturer.
unfortunately there are coming more and more obstacles in our way as linux kernel and components evolve, back in old days it was easy with stuff such as /dev/sda and similar, today we have all sorts of guid and partition uid and lables and then hassle with grub2 and quite a number of places and config files where old device names can hide.
think I found leftovers in /etc/default.d/ or similar directories and places which grub2 installer or scripts would take into account
Any simple and easy steps or yast2 modules to repair and rebuild a pretty normal single hddl partition setup?
with / and /home as partitions next to swap as the only places that exist
thanks for helpers.
I have dine this a few times to create bootable USB hard drives, and then use them to bring copies of my system to other machines. In summary, what I have done is 1)use the Leap DVD distribution to build the partitions I wish on the USB drive (usually without snapshots enabled), 2) install the openSuse on the USB drive, 3) boot back into the original system and use rsync to copy the existing media to my USB drive. There is not much problem with GRUB2 if the USB drive and the original system are running the same version of the kernel. All I have to do is edit USB:/boot/grub2/grub.cfg to change all of the UUID numbers from those of the original system to that given by blkid for the root partition on the USB drive (25 occurrences last count). I exclude certain files from the rsync copy using the --exclude-from option to rsync, shown below. I currently issue two rsync commands, one for the root and the other for /home: sudo mount /dev/sdd2 /mnt/usr11 sudo mount /dev/sdd3 /mnt/usr12 sudo rsync -rlpgovtSAXHDc --progress -v --delete --exclude-from=/home/don/.rsync-btrfs-backup_exclude / /mnt/usr11 sudo rsync -rlpgouvtSAXHD --modify-window=1 -v --delete /home/ /mnt/usr12/ My backup_exclude file contains: /etc/fstab /etc/hostname /etc/HOSTNAME /etc/ssh/* /etc/sysconfig/network/ifcfg-enp9s0 /etc/sysconfig/network/ifcfg-wlp11s0 /.snapshots/* /proc/* /sys/* /run/* /dev/* /tmp/* /var/tmp/* /mnt/cdrom/* /mnt/mcc/* /mnt/nas/* /mnt/usr11/* /mnt/usr12/* /mnt/usr13/* /mnt/usr14/* /usr1/* /home/* You may need to modify the above depending on your network cards etc. Also, I added the /usr1 to my fstab, as suggested on this list, so that I could have a simple way to access directories under /home. /home /usr1 none rbind 0 3 I hope this can be of some help. I should offer that not many of the list members appear to support this approach. But it works for me:-) Don -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
If the installation is on Btrfs, then it's a problem to do an exact backup and restore. The simplest is to only backup and restore /home, which by default is on XFS, and plan on doing a clean installation and updates of the OS itself. The problem with clonezilla/rsync with Btrfs is it will neither copy everything, nor restore it, because it's not Btrfs aware. Most of the Btrfs layout is actually not in the mounted path from root '/' (on Tumbleweed at least), which actually is a good thing because if it were, a bunch of the same thing would get sync'd (i.e. the duplication of snapshots would be seen as unique directories so they'd all get duplicated; the deduplication is not preserved). Further the subvolumes aren't preserved, they become just directories on the backup. And then on restore it's the same, you end up with a totally flat restore, and it wouldn't boot because the restored fstab and bootloader expects a completely different layout. The problem with block level cloning (dd) is in the Btrfs wikie. The gist is that the exact cloning means there can be ambiguity with two identical UUIDs available and it's not difficult to corrupt one or both volumes in short order. https://btrfs.wiki.kernel.org/index.php/Gotchas It is possible to boot off live media, make the Btrfs root a seed device, mount the seed (ro), add another device (partition or LV), remount rw, then remove the seed. This will create a new volume UUID on the added partition/LV, and copy the entire file system over. It will preserve the entire structure, subvolumes, snapshots, and deduplicated/shared extents. However, there isn't a straightforward way to keep this separate volume up to date (there's no way to update the seed, then incrementally update the 2nd device). Anyway, I find it easier to just backup and restore /home, and bet on having to clean install the OS, should a catastrophic failure happen. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 17/03/2016 08:25, Chris Murphy a écrit :
The problem with clonezilla/rsync with Btrfs
as far as I understand, clonezilla and rsync are two completely different things. rsync only copies the "visible" data, so in restore all the snapshot info will be lost, but the system should works, depending of the new hardware config clonezilla makes a sector by sector backup and can only restore to a new hardware (replacing the old). I don't know how btrfs is managed, but the clonezilla site list it as supported: http://clonezilla.org/ the problem is restoration. full restoration *test* is very hard and long. I doubt if many people do it. I use rsync mostly to restore individual files, not system. I use system backups to have a copy of every bit of setup, but never had to restore one, as, like you do, I like better to install a new system. Faster than dealing with small changes a restore needs and a reason to have a clean system jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, Mar 17, 2016 at 1:43 AM, jdd <jdd@dodin.org> wrote:
Le 17/03/2016 08:25, Chris Murphy a écrit :
The problem with clonezilla/rsync with Btrfs
as far as I understand, clonezilla and rsync are two completely different things.
rsync only copies the "visible" data, so in restore all the snapshot info will be lost, but the system should works, depending of the new hardware config
clonezilla makes a sector by sector backup and can only restore to a new hardware (replacing the old). I don't know how btrfs is managed, but the clonezilla site list it as supported:
the problem is restoration.
full restoration *test* is very hard and long. I doubt if many people do it.
It's true. I have two subvolumes, root and home. I used to do separate boot, I don't anymore. Simple version is this: cd /boot tar -acf bootefi.tar.gz efi/ mount -o subvolid=5 /dev/sdX /mnt cd /mnt btrfs sub snap -r root root.6 btrfs sub snap -r home home.6 btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups' So those are incremental sends, and are quite fast. The resulting root.6 and home.6 subvolumes on the remote backup contain all files in the current state they're in on my laptop so those two subvolumes are all I have to restore. The restore is booting from a live OS, enabling ssh, partitioning and formatting, and then from the remote (yes I use ssh to get into the remote and then push the subvolume from remote to laptop): btrfs send root.6 | ssh chris@ip 'btrfs receive /mnt' btrfs send home.6 | ssh chris@ip 'btrfs receive /mnt' Back to local shell: cd /mnt btrfs sub snap root.6 root btrfs sub snap home.6 home cd / umount /mnt mount -o subvol=root /dev/sda1 /mnt mount /dev/sda2 /mnt/boot/efi mount -B /dev /mnt/dev mount -B /sys /mnt/sys mount -B /proc /mnt/proc chroot /mnt vi /etc/fstab ### fix up the UUIDs for the new ESP and Btrfs volume cd /boot tar -xf bootefi.tar.gz grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg exit Then umount everything and reboot. Pretty rudimentary, might have forgotten a step, but I've done this maybe a dozen times. It's fast, and pretty easy to remember. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Mar 18, 2016 at 2:39 AM, Chris Murphy <lists@colorremedies.com> wrote: ...
I have two subvolumes, root and home. I used to do separate boot, I don't anymore. Simple version is this:
cd /boot tar -acf bootefi.tar.gz efi/ mount -o subvolid=5 /dev/sdX /mnt cd /mnt btrfs sub snap -r root root.6 btrfs sub snap -r home home.6 btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups'
Do you need to create subvolumes on destination in advance or are they created automatically by btrfs receive? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Mar 18, 2016 at 3:51 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
On Fri, Mar 18, 2016 at 2:39 AM, Chris Murphy <lists@colorremedies.com> wrote: ...
I have two subvolumes, root and home. I used to do separate boot, I don't anymore. Simple version is this:
cd /boot tar -acf bootefi.tar.gz efi/ mount -o subvolid=5 /dev/sdX /mnt cd /mnt btrfs sub snap -r root root.6 btrfs sub snap -r home home.6 btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups'
Do you need to create subvolumes on destination in advance or are they created automatically by btrfs receive?
They're created by receive. The very first send is different, it omits -p because it's not an incremental send. And with -p option on the send side, the receive side must already have that subvolume. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
19.03.2016 04:00, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 3:51 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
On Fri, Mar 18, 2016 at 2:39 AM, Chris Murphy <lists@colorremedies.com> wrote: ...
I have two subvolumes, root and home. I used to do separate boot, I don't anymore. Simple version is this:
cd /boot tar -acf bootefi.tar.gz efi/ mount -o subvolid=5 /dev/sdX /mnt cd /mnt btrfs sub snap -r root root.6 btrfs sub snap -r home home.6 btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups'
Do you need to create subvolumes on destination in advance or are they created automatically by btrfs receive?
They're created by receive. The very first send is different, it omits -p because it's not an incremental send. And with -p option on the send side, the receive side must already have that subvolume.
How does it verify that baseline snapshot is really unchanged? Using name only? I.e. if I rename completely unrelated subvolume as home.5 on destination, will it apply diff between source home.5 and home.6? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Mar 18, 2016 at 11:32 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
19.03.2016 04:00, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 3:51 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
On Fri, Mar 18, 2016 at 2:39 AM, Chris Murphy <lists@colorremedies.com> wrote: ...
I have two subvolumes, root and home. I used to do separate boot, I don't anymore. Simple version is this:
cd /boot tar -acf bootefi.tar.gz efi/ mount -o subvolid=5 /dev/sdX /mnt cd /mnt btrfs sub snap -r root root.6 btrfs sub snap -r home home.6 btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups'
Do you need to create subvolumes on destination in advance or are they created automatically by btrfs receive?
They're created by receive. The very first send is different, it omits -p because it's not an incremental send. And with -p option on the send side, the receive side must already have that subvolume.
How does it verify that baseline snapshot is really unchanged? Using name only? I.e. if I rename completely unrelated subvolume as home.5 on destination, will it apply diff between source home.5 and home.6?
Only read-only snapshots can be sent received. Hence the -r when the snapshot is taken. I haven't looked at the code, but I expect names are only used by user space, where the subvol uuids are used behind the scene: each subvolume has its own uuid, a parent uuid, and a received uuid to keep track of relationships on and across volumes. Like I mentioned in the example, home.5 must already be on the destination to incrementally send home.6. How do you rename an unrelated subvolume to match an existing subvolume name? That would fail. And even if it succeeded, the "fake" home.5 would not have the same parent uuid or received uuid as what's expected when using -p on the send side. So the receive would fail. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
19.03.2016 21:35, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 11:32 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
19.03.2016 04:00, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 3:51 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
On Fri, Mar 18, 2016 at 2:39 AM, Chris Murphy <lists@colorremedies.com> wrote: ...
I have two subvolumes, root and home. I used to do separate boot, I don't anymore. Simple version is this:
cd /boot tar -acf bootefi.tar.gz efi/ mount -o subvolid=5 /dev/sdX /mnt cd /mnt btrfs sub snap -r root root.6 btrfs sub snap -r home home.6 btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups'
Do you need to create subvolumes on destination in advance or are they created automatically by btrfs receive?
They're created by receive. The very first send is different, it omits -p because it's not an incremental send. And with -p option on the send side, the receive side must already have that subvolume.
How does it verify that baseline snapshot is really unchanged? Using name only? I.e. if I rename completely unrelated subvolume as home.5 on destination, will it apply diff between source home.5 and home.6?
Only read-only snapshots can be sent received. Hence the -r when the snapshot is taken. I haven't looked at the code, but I expect names are only used by user space, where the subvol uuids are used behind the scene: each subvolume has its own uuid, a parent uuid, and a received uuid to keep track of relationships on and across volumes.
Like I mentioned in the example, home.5 must already be on the destination to incrementally send home.6. How do you rename an unrelated subvolume to match an existing subvolume name?
mv /home.5 /some-other-name mv /unrelated-subvolume /home.5
That would fail.
Why?
And even if it succeeded, the "fake" home.5 would not have the same parent uuid or received uuid as what's expected when using -p on the send side.
How send side knows what parent uuid to expect on receive side in the first place? This information should be kept somewhere. How can I see what parent uuid is expected for next operation? Not that I really understand what "parent" in case of btrfs means ... So the receive would fail.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, Mar 21, 2016 at 1:29 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
19.03.2016 21:35, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 11:32 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
19.03.2016 04:00, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 3:51 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
On Fri, Mar 18, 2016 at 2:39 AM, Chris Murphy <lists@colorremedies.com> wrote: ...
I have two subvolumes, root and home. I used to do separate boot, I don't anymore. Simple version is this:
cd /boot tar -acf bootefi.tar.gz efi/ mount -o subvolid=5 /dev/sdX /mnt cd /mnt btrfs sub snap -r root root.6 btrfs sub snap -r home home.6 btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups'
Do you need to create subvolumes on destination in advance or are they created automatically by btrfs receive?
They're created by receive. The very first send is different, it omits -p because it's not an incremental send. And with -p option on the send side, the receive side must already have that subvolume.
How does it verify that baseline snapshot is really unchanged? Using name only? I.e. if I rename completely unrelated subvolume as home.5 on destination, will it apply diff between source home.5 and home.6?
Only read-only snapshots can be sent received. Hence the -r when the snapshot is taken. I haven't looked at the code, but I expect names are only used by user space, where the subvol uuids are used behind the scene: each subvolume has its own uuid, a parent uuid, and a received uuid to keep track of relationships on and across volumes.
Like I mentioned in the example, home.5 must already be on the destination to incrementally send home.6. How do you rename an unrelated subvolume to match an existing subvolume name?
mv /home.5 /some-other-name mv /unrelated-subvolume /home.5
That would fail.
Why?
home.5 is a read only subvolume, it can't be renamed, which also means it can't be moved either. Only read-only subvolumes (snapshots) can be sent/received. And they can't be modified on either side or you mess up the integrity of the whole thing. If you want to make a derivative for modification you need to make a snapshot of that without -r, so that it's a read write snapshot then you can do whatever you want with it.
And even if it succeeded, the "fake" home.5 would not have the same parent uuid or received uuid as what's expected when using -p on the send side.
How send side knows what parent uuid to expect on receive side in the first place? This information should be kept somewhere. How can I see what parent uuid is expected for next operation?
It's the receive side that expects this information to match because it's the receive side's job to effectively snapshot the parent already on the receive side, then modified that snapshot with the incremental send stream.
Not that I really understand what "parent" in case of btrfs means ...
So the receive would fail.
Yes and since it's an early check, it somehow backfires and fails send also, so it's not like you end up with a hung send process that keeps sending stuff that can't be received. Here's a "simplified version" where I've tacked on a single letter after each unique UUID. This is the "send" side: Name: original UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Parent UUID: - Received UUID: - Name: original.snapshot UUID: 80134f6c-8dda-df48-9e00-af0213bc1234 B Parent UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Received UUID: - Name: originalmodified.snapshot UUID: c2aa5bac-bb3a-8641-89c8-5d6da2b810d2 C Parent UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Received UUID: - ================ This is the "receive" side Name: original.snapshot UUID: bceda471-de5c-6d42-afa0-72ec47080625 D Parent UUID: - Received UUID: 80134f6c-8dda-df48-9e00-af0213bc1234 B Name: originalmodified.snapshot UUID: 29f83d8c-79f0-9f4f-b5bc-ff4fb4effb87 E Parent UUID: bceda471-de5c-6d42-afa0-72ec47080625 D Received UUID: c2aa5bac-bb3a-8641-89c8-5d6da2b810d2 C The receive was populated from the send side using commands: [root@f23s ~]# btrfs send original.snapshot | ssh root@10.0.0.4 'btrfs receive /root/' [root@f23s ~]# btrfs send -p original.snapshot originalmodified.snapshot/ | ssh root@10.0.0.4 'btrfs receive /root/' -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, Mar 22, 2016 at 5:28 PM, Chris Murphy <lists@colorremedies.com> wrote:
Only read-only subvolumes (snapshots) can be sent/received.
I wrote this in a confusing way. I didn't mean to indicate snapshots are read-only snapshots (or vice versa). I meant to indicate that "subvolume" and "snapshot" are effectively interchangeable in this context. A snapshot is a pre-populated subvolume. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
23.03.2016 02:28, Chris Murphy пишет:
On Mon, Mar 21, 2016 at 1:29 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
19.03.2016 21:35, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 11:32 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
19.03.2016 04:00, Chris Murphy пишет:
On Fri, Mar 18, 2016 at 3:51 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
On Fri, Mar 18, 2016 at 2:39 AM, Chris Murphy <lists@colorremedies.com> wrote: ... > > I have two subvolumes, root and home. I used to do separate boot, I > don't anymore. Simple version is this: > > cd /boot > tar -acf bootefi.tar.gz efi/ > mount -o subvolid=5 /dev/sdX /mnt > cd /mnt > btrfs sub snap -r root root.6 > btrfs sub snap -r home home.6 > btrfs send -p root.5 root.6 | ssh chris@ip 'btrfs receive /backups' > btrfs send -p home.5 home.6 | ssh chris@ip 'btrfs receive /backups' >
Do you need to create subvolumes on destination in advance or are they created automatically by btrfs receive?
They're created by receive. The very first send is different, it omits -p because it's not an incremental send. And with -p option on the send side, the receive side must already have that subvolume.
How does it verify that baseline snapshot is really unchanged? Using name only? I.e. if I rename completely unrelated subvolume as home.5 on destination, will it apply diff between source home.5 and home.6?
Only read-only snapshots can be sent received. Hence the -r when the snapshot is taken. I haven't looked at the code, but I expect names are only used by user space, where the subvol uuids are used behind the scene: each subvolume has its own uuid, a parent uuid, and a received uuid to keep track of relationships on and across volumes.
Like I mentioned in the example, home.5 must already be on the destination to incrementally send home.6. How do you rename an unrelated subvolume to match an existing subvolume name?
mv /home.5 /some-other-name mv /unrelated-subvolume /home.5
That would fail.
Why?
home.5 is a read only subvolume, it can't be renamed, which also means it can't be moved either.
Of course it can. Did you actually try it? I did before posting :) "Read-only" here refers to subvolume content, not to its "attachment point".
Only read-only subvolumes (snapshots) can be sent/received. And they can't be modified on either side or you mess up the integrity of the whole thing. If you want to make a derivative for modification you need to make a snapshot of that without -r, so that it's a read write snapshot then you can do whatever you want with it.
And even if it succeeded, the "fake" home.5 would not have the same parent uuid or received uuid as what's expected when using -p on the send side.
How send side knows what parent uuid to expect on receive side in the first place? This information should be kept somewhere. How can I see what parent uuid is expected for next operation?
It's the receive side that expects this information to match because it's the receive side's job to effectively snapshot the parent already on the receive side, then modified that snapshot with the incremental send stream.
Well, how does *receive* side know what snapshot on send side to expect then? Again - all that sounds like the only thing receive side knows is subvolume name and subvolume name can be changed arbitrarily.
Not that I really understand what "parent" in case of btrfs means ...
So the receive would fail.
Yes and since it's an early check, it somehow backfires and fails send also, so it's not like you end up with a hung send process that keeps sending stuff that can't be received.
Here's a "simplified version" where I've tacked on a single letter after each unique UUID.
This is the "send" side:
Name: original UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Parent UUID: - Received UUID: -
Name: original.snapshot UUID: 80134f6c-8dda-df48-9e00-af0213bc1234 B Parent UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Received UUID: -
Name: originalmodified.snapshot UUID: c2aa5bac-bb3a-8641-89c8-5d6da2b810d2 C Parent UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Received UUID: -
================ This is the "receive" side
Name: original.snapshot UUID: bceda471-de5c-6d42-afa0-72ec47080625 D Parent UUID: - Received UUID: 80134f6c-8dda-df48-9e00-af0213bc1234 B
Name: originalmodified.snapshot UUID: 29f83d8c-79f0-9f4f-b5bc-ff4fb4effb87 E Parent UUID: bceda471-de5c-6d42-afa0-72ec47080625 D Received UUID: c2aa5bac-bb3a-8641-89c8-5d6da2b810d2 C
Is received UUID actually present on receive side somewhere after "btrfs send | btrfs receive" has completed?
The receive was populated from the send side using commands: [root@f23s ~]# btrfs send original.snapshot | ssh root@10.0.0.4 'btrfs receive /root/'
[root@f23s ~]# btrfs send -p original.snapshot originalmodified.snapshot/ | ssh root@10.0.0.4 'btrfs receive /root/'
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, Mar 22, 2016 at 9:23 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
home.5 is a read only subvolume, it can't be renamed, which also means it can't be moved either.
Of course it can. Did you actually try it? I did before posting :)
Many, many times. And it seems that became a habit because I haven't tried in a while. I just tried again and it works. So the behavior has changed. I don't know which kernel/progs version first allowed this, but I'll guess it happens at the same time that "received uuid" came about which was very recent.
"Read-only" here refers to subvolume content, not to its "attachment point".
No that's not how it worked for YEARS. And even now you still can't move a read only subvolume. [root@f23m ~]# mv stuff.snap1 / mv: cannot move ‘stuff.snap1’ to ‘/stuff.snap1’: Read-only file system /dev/sda5 on / type btrfs (rw,noatime,seclabel,ssd,space_cache,subvolid=288,subvol=/f23w-root) Root is definitely not read only, but stuff.snap1 is read only. # btrfs property get stuff.snap1 ro ro=true
Only read-only subvolumes (snapshots) can be sent/received. And they can't be modified on either side or you mess up the integrity of the whole thing. If you want to make a derivative for modification you need to make a snapshot of that without -r, so that it's a read write snapshot then you can do whatever you want with it.
And even if it succeeded, the "fake" home.5 would not have the same parent uuid or received uuid as what's expected when using -p on the send side.
How send side knows what parent uuid to expect on receive side in the first place? This information should be kept somewhere. How can I see what parent uuid is expected for next operation?
It's the receive side that expects this information to match because it's the receive side's job to effectively snapshot the parent already on the receive side, then modified that snapshot with the incremental send stream.
Well, how does *receive* side know what snapshot on send side to expect then?
You should try doing a send only with -f and --no-data options and look inside the resulting file, which is in the send stream format. It contains the uuids and is clearly depending only on those. If I do a send | receive without options, the receive side doesn't need to know anything. It creates a new subvolume with a new UUID, and writes the contents of the send stream into that subvolume. The send stream includes the subvolume UUID of the subvolume being sent, and that becomes the "received UUID" on the new subvolume on the receive side. If I do an incremental send with -p option, then there are multiple subvolume UUIDs in the send stream, and the receive side makes sure it already has the -p subvolume by doing "received uuid" matching.
Again - all that sounds like the only thing receive side knows is subvolume name and subvolume name can be changed arbitrarily.
That was probably true at one time before "received uuid" existed and why read only subvolumes could not be renamed. But I just tried renaming a bunch of read only snapshots and doing incremental sends and it still works. The stream contains all the relevant uuids. [root@f23s ~]# hexdump -C output.btr 00000000 62 74 72 66 73 2d 73 74 72 65 61 6d 00 01 00 00 |btrfs-stream....| 00000010 00 52 00 00 00 02 00 ec bc 31 b0 0f 00 0e 00 6f |.R.......1.....o| 00000020 72 69 67 69 6e 61 6c 2e 73 6e 61 70 34 01 00 10 |riginal.snap4...| 00000030 00 66 70 e8 66 83 d4 13 49 bf 41 ec 62 d0 fe d4 |.fp.f...I.A.b...| 00000040 af 02 00 08 00 96 bc 02 00 00 00 00 00 14 00 10 |................| 00000050 00 10 33 94 f1 36 03 ec 4e 8b 0d e5 b6 3d 05 05 |..3..6..N....=..| 00000060 20 15 00 08 00 95 bc 02 00 00 00 00 00 34 00 00 | ............4..| 00000070 00 14 00 de df a0 47 0f 00 00 00 0b 00 0c 00 04 |......G.........| 00000080 bf f2 56 00 00 00 00 eb 35 f8 32 0a 00 0c 00 e7 |..V.....5.2.....| 00000090 be f2 56 00 00 00 00 e3 ea eb 12 09 00 0c 00 e7 |..V.............| 000000a0 be f2 56 00 00 00 00 e3 ea eb 12 32 00 00 00 0f |..V........2....| 000000b0 00 80 cf bd 52 0f 00 0c 00 73 6f 6d 65 74 65 78 |....R....sometex| 000000c0 74 2e 74 78 74 12 00 08 00 00 00 00 00 00 00 00 |t.txt...........| 000000d0 00 13 00 12 00 68 65 6c 6c 6f 0a 68 65 6c 6c 6f |.....hello.hello| 000000e0 20 61 67 61 69 6e 0a 1c 00 00 00 11 00 cc c9 f3 | again..........| 000000f0 ba 0f 00 0c 00 73 6f 6d 65 74 65 78 74 2e 74 78 |.....sometext.tx| 00000100 74 04 00 08 00 12 00 00 00 00 00 00 00 40 00 00 |t............@..| 00000110 00 14 00 fe e0 57 e1 0f 00 0c 00 73 6f 6d 65 74 |.....W.....somet| 00000120 65 78 74 2e 74 78 74 0b 00 0c 00 e7 be f2 56 00 |ext.txt.......V.| 00000130 00 00 00 e3 ea eb 12 0a 00 0c 00 05 bf f2 56 00 |..............V.| 00000140 00 00 00 d9 7d 3d 28 09 00 0c 00 05 bf f2 56 00 |....}=(.......V.| 00000150 00 00 00 d9 7d 3d 28 00 00 00 00 15 00 50 6c c9 |....}=(......Pl.| 00000160 9d |.| 00000161 [root@f23s ~]# btrfs sub show original.snap3 /root/original.snap3 Name: original.snap3 UUID: 103394f1-3603-ec4e-8b0d-e5b63d050520 Parent UUID: 10ac658c-2204-9448-aa53-6fa268a0e308 Received UUID: - Creation time: 2016-03-23 10:06:14 -0600 Subvolume ID: 496 Generation: 179349 Gen at creation: 179349 Parent ID: 371 Top level ID: 371 Flags: readonly Snapshot(s): [root@f23s ~]# btrfs sub show original.snap4 /root/original.snap4 Name: original.snap4 UUID: 6670e866-83d4-1349-bf41-ec62d0fed4af Parent UUID: 10ac658c-2204-9448-aa53-6fa268a0e308 Received UUID: - Creation time: 2016-03-23 10:06:37 -0600 Subvolume ID: 497 Generation: 179350 Gen at creation: 179350 Parent ID: 371 Top level ID: 371 Flags: readonly Snapshot(s): You can see in the stream that it contains the name and subvolume uuid for the subvolume being sent. Name: original.snap4 UUID: 6670e866-83d4-1349-bf41-ec62d0fed4af 66 70 e8 66 83 d4 13 49 bf 41 ec 62 d0 fe d4 |.fp.f...I.A.b...| 00000040 af That's how receive knows what to set for "received uuid" and also how to name the received subvolume. You can also see in the stream that there is no name for the -p parent subvolume, only its uuid, so the name change doesn't matter. Name: original.snap3 UUID: 103394f1-3603-ec4e-8b0d-e5b63d050520 10 33 94 f1 36 03 ec 4e 8b 0d e5 b6 3d 05 05 |..3..6..N....=..| 00000060 20
Not that I really understand what "parent" in case of btrfs means ...
So the receive would fail.
Yes and since it's an early check, it somehow backfires and fails send also, so it's not like you end up with a hung send process that keeps sending stuff that can't be received.
Here's a "simplified version" where I've tacked on a single letter after each unique UUID.
This is the "send" side:
Name: original UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Parent UUID: - Received UUID: -
Name: original.snapshot UUID: 80134f6c-8dda-df48-9e00-af0213bc1234 B Parent UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Received UUID: -
Name: originalmodified.snapshot UUID: c2aa5bac-bb3a-8641-89c8-5d6da2b810d2 C Parent UUID: 9e7736a6-ad72-4f43-b7c9-fc4bf73bf74b A Received UUID: -
================ This is the "receive" side
Name: original.snapshot UUID: bceda471-de5c-6d42-afa0-72ec47080625 D Parent UUID: - Received UUID: 80134f6c-8dda-df48-9e00-af0213bc1234 B
Name: originalmodified.snapshot UUID: 29f83d8c-79f0-9f4f-b5bc-ff4fb4effb87 E Parent UUID: bceda471-de5c-6d42-afa0-72ec47080625 D Received UUID: c2aa5bac-bb3a-8641-89c8-5d6da2b810d2 C
Is received UUID actually present on receive side somewhere after "btrfs send | btrfs receive" has completed?
? I'm not following. Look at the UUID labeled "B" on both send and receive side. They are the same subvolume. Because they're on different file systems so the receive side version of that same subvolume data gets its own uuid. But subvolume "B" and "D" are the same thing, just on different file systems. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Mar 23, 2016 at 10:50 AM, Chris Murphy <lists@colorremedies.com> wrote:
On Tue, Mar 22, 2016 at 9:23 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
home.5 is a read only subvolume, it can't be renamed, which also means it can't be moved either.
Of course it can. Did you actually try it? I did before posting :)
Many, many times. And it seems that became a habit because I haven't tried in a while. I just tried again and it works. So the behavior has changed. I don't know which kernel/progs version first allowed this, but I'll guess it happens at the same time that "received uuid" came about which was very recent.
Wow. So, kernel 3.11 allows me to rename read only subvolumes. The received uuid appeard with kernel and progs 4.1. So I guess I've just conflated inability to relocate with mv, with an inability to rename with mv. But I can rename, I just can't relocate. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
23.03.2016 19:50, Chris Murphy пишет:
that becomes the "received UUID" on the new subvolume on the receive side.
I appreciate efforts, but my question was more easily answered with "btrfs subvolume list -R" :) Thank you! It was quite useful. btrfs slowly gains at least some parity with zfs. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Mar 23, 2016 at 11:45 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
23.03.2016 19:50, Chris Murphy пишет:
that becomes the "received UUID" on the new subvolume on the receive side.
I appreciate efforts, but my question was more easily answered with "btrfs subvolume list -R" :)
Thank you! It was quite useful. btrfs slowly gains at least some parity with zfs.
Haha! The main thing ZFS has is ~ 6 years older and is considered enterprise stable. But already Btrfs has features and capabilities that ZFS doesn't. I expect that to continue. There are many developers working on it, and there are often thousands of code changes happening per kernel cycle. It's definitely progressing. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/23/2016 09:49 PM, Chris Murphy wrote:
On Wed, Mar 23, 2016 at 11:45 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
23.03.2016 19:50, Chris Murphy пишет:
that becomes the "received UUID" on the new subvolume on the receive side.
I appreciate efforts, but my question was more easily answered with "btrfs subvolume list -R" :)
Thank you! It was quite useful. btrfs slowly gains at least some parity with zfs.
Haha! The main thing ZFS has is ~ 6 years older and is considered enterprise stable. But already Btrfs has features and capabilities that ZFS doesn't. I expect that to continue. There are many developers working on it, and there are often thousands of code changes happening per kernel cycle. It's definitely progressing.
You've hit on The perfect definition of "Unstable" if you ask me. A simple thing like storage allocation has become a quagmire that nobody can understand even at a high level, and the types of questions we get on this list (and others) about BTRFS makes that perfectly clear. The tools need time to catch up. -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 24/03/2016 18:28, John Andersen a écrit :
A simple thing like storage allocation has become a quagmire that nobody can understand even at a high level, and the types of questions we get on this list (and others) about BTRFS makes that perfectly clear.
The tools need time to catch up.
true, but only mass deployment can reach that... That's open source it's the way to pay for what we have jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/24/2016 10:48 AM, jdd wrote:
Le 24/03/2016 18:28, John Andersen a écrit :
A simple thing like storage allocation has become a quagmire that nobody can understand even at a high level, and the types of questions we get on this list (and others) about BTRFS makes that perfectly clear.
The tools need time to catch up.
true, but only mass deployment can reach that...
That's open source
it's the way to pay for what we have
After the SECOND data-loss on BTRFS in 6 mohts, which required a total reinstall, I decided that price was too high. I don't know what the target market for BTRFS is. But I'm no longer buying at the price I had to pay. As things mature, the price comes down. I'll wait. -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-03-17 08:25, Chris Murphy wrote:
The problem with clonezilla/rsync with Btrfs is it will neither copy everything, nor restore it, because it's not Btrfs aware.
When those tools such as clonezilla don't understand a filesystem, they do a sector by sector blind copy. dd or equivalent. Don't place in the same bag clonezilla with rsync :-) -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 03/17/2016 12:25 AM, Chris Murphy wrote:
If the installation is on Btrfs, then it's a problem to do an exact backup and restore. The simplest is to only backup and restore /home, which by default is on XFS, and plan on doing a clean installation and updates of the OS itself.
The problem with clonezilla/rsync with Btrfs is it will neither copy everything, nor restore it, because it's not Btrfs aware. Most of the Btrfs layout is actually not in the mounted path from root '/' (on Tumbleweed at least), which actually is a good thing because if it were, a bunch of the same thing would get sync'd (i.e. the duplication of snapshots would be seen as unique directories so they'd all get duplicated; the deduplication is not preserved). Further the subvolumes aren't preserved, they become just directories on the backup. And then on restore it's the same, you end up with a totally flat restore, and it wouldn't boot because the restored fstab and bootloader expects a completely different layout.
The problem with block level cloning (dd) is in the Btrfs wikie. The gist is that the exact cloning means there can be ambiguity with two identical UUIDs available and it's not difficult to corrupt one or both volumes in short order. https://btrfs.wiki.kernel.org/index.php/Gotchas
It is possible to boot off live media, make the Btrfs root a seed device, mount the seed (ro), add another device (partition or LV), remount rw, then remove the seed. This will create a new volume UUID on the added partition/LV, and copy the entire file system over. It will preserve the entire structure, subvolumes, snapshots, and deduplicated/shared extents. However, there isn't a straightforward way to keep this separate volume up to date (there's no way to update the seed, then incrementally update the 2nd device).
Anyway, I find it easier to just backup and restore /home, and bet on having to clean install the OS, should a catastrophic failure happen.
I would like to repeat and emphasize that I turn off snapshots when I generate my file systems using the installation DVD. I have rather simple computer systems, just a bunch of Alienware laptops with one and two TB SSDs, and never understood or saw the need for btrfs. I once ran a multiuser facility (remember the VAX in the 80s?), so I do understand some of the problems that can arise. But I wonder what percentage of openSuse systems fall into that category? I keep the btrfs format so that 1) I can remain compatible with standard openSuse and 2) I can enable snapshots if the need arises, and 3) I once saw documentation stating that btrfs was very efficient at handling the block structure on SSDs. But that documentation was accessed a long time ago and do not have any references. I also have always found the "clean install" suggestion a difficult proposition. The are so many items that become customized during the life of my systems that restoration from scratch is a daunting proposition. I can only repeat that my backup/restore system works for the four laptops that I have used it on. YMMV:-) I am not trying to provoke any sort of flame war, just presenting another, perhaps simpler option. Don -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (12)
-
Andrei Borzenkov -
Anton Aylward -
Billie Walsh -
Bjoern Voigt -
cagsm -
Carlos E. R. -
Chris Murphy -
don fisher -
Felix Miata -
jdd -
John Andersen -
Per Jessen