On 08/13/2013 11:21 PM, Brian K. White wrote:
On 8/13/2013 8:21 PM, Doug wrote:
On 08/13/2013 06:39 PM, Brian K. White wrote:
On 8/13/2013 6:16 PM, Doug wrote:
On 08/13/2013 04:27 PM, Brian K. White wrote:
On 8/8/2013 5:42 AM, Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thursday, 2013-08-08 at 07:59 +0200, Per Jessen wrote: /snip/
explanation of filesystems and copying--well done!
You can proceed a few different ways.
The best way is probably create new empty filesystems manually on the new partitions, then use rsync to copy all your files over to them. The advantage here is that rsync is resumable (if you abort in the middle, it doesn't have to start over from the beginning), and it works for any kind of fs as long as you can mount that fs in linux.
Another way is dd the old fs to the new partition, then use whichever utilities are appropriate for the fs type to grow the fs to fill the new space. Not all fs's are growable but the common current linux ones are. At least ext3, ext4, reiser. They don't all use the same command, ie there isn't a growfs command. ext3/4 has resize2fs, reiser has resize_reiserfs, xfs probably has something, btrfs probably has something, etc. This all also assumes you aren't using lvm. This way might work if you had a non-linux fs from some other OS where you can't mount or resize the fs from linux, but you can dd any block of disk bytes, and then maybe the other OS can resize it's own fs itself later.
Generally, copying a fs to a larger partition or whole device (no partition table) does not affect the fs at all, it still works fine exactly as before. But it also does not use the extra space. You have to resize the fs to use the extra space. Not all fs types can be resized. Some can be resized but maybe there is no linux utility to do it, or not a safe one anyways. Some can only be grown and never shrunk, some can be grown or shrunk.
I was intending to format the partitions on the new drive with ext4 (for Linux) and NTFS (for Windows 8) so that the DDd files would arrive on formatted space, and the rest of the partition would be usable, being formatted and empty. I get the impression that I'm missing something here--please explain. Is there a difference between what you call a file system, and what I call a formatted partition? If so, what?
Thanx--doug
A filesystem is the formatting, but it need not be in a partition. It can be on a whole device (like a floppy, only you can do it to anything like a hard drive or a thumb drive or a ramdisk) or inside of a file.
dd does not copy files it copies raw data. That data may or may not come from a file or go into a file, the point is dd doesn't know or care what the data is. If you use dd to copy a filesystem, you are not "dding the files" you are dding one big solid hunk of raw meaningless data called the entire filesystem. dd does not know about all the individual files inside in the way that tar or cp or rsync or winzip does.
The filesystem is the "formatted partition" in one sense, in the sense that it is the formatting that usually occupies an entire partition. When you format a partition in windows, what is happening is windows creates an ntfs or fat32 filesystem
But a filesystem is also not a "formatted partition" in the sense that it is not the partition itself, just the data that was written to the partition. The partition is just an arbitrary block of disk space which is defined by the partition table. It's not really accurate to think of "formatting the partition" because for instance you can "format" just a part of a partition if you wanted to. You could write a filesystem that only uses 800 gigs of a 1tb partition. Is that partition then "formatted"?
Normally you would not do that but the distinction still matters because of your own current project.
You have an old drive that currently shows up as /dev/sda and a new larger drive that shows up as /dev/sdb.
The old drive has a partition /dev/sda1 which is 500 gigs, 100% filled with a 500 gig ext3 filesystem.
The new drive has no partitions and no filesystems. You create a partition table and make a 1.5tb partition on it. Now you have a /dev/sdb1 that is 1.5tb but it has no filesystem yet.
Instead of running "mkfs /dev/sdb1" to create one you use dd to copy the old one: dd if=/dev/sda1 of=/dev/sdb1
What happened was, the first 500G of /dev/sdb1 now has a carbon copy of /dev/sda1, followed by 1T of blank space. Not only is it blank, it's "invisible" to most utilities. If you mount the new copy:
mount /dev/sdb1 /mnt
Then look at /mnt, you will see that there is only 500G of space total in /mnt.
df -h /mnt
will show something like this: # df -h /mnt Filesystem Size Used Avail Use% Mounted on /dev/sdb1 512M 358M 154M 70% /mnt
That empty space is not merely unused like the 154M available above. It's not even available, because it lies outside the filesystem.
In the old old days, you couldn't resize filesystems, you could only make new ones and copy files from one to another. Today the popular filesystems can mostly be resized.
So now you can use resize2fs to resize /dev/sdb1 to fill up the new larger partition it is in. After that, the df command will show you that the Size and Avail are larger.
rsync would be a different story.
rsync DOES work on individual files. One reason to use rsync (or cp or tar or cpio or any of the backup programs) instead of dd, would be that if you copy the individual files instead of the whole flsystem, then the new filesystem can be a different type than the old one, as well as a different size. For instance, say the old filesystem is ext2. If you wanted the new fs to be ext3 or ext4 then you can upgrade an existing ext* to any higher version in-place, but that's a special case. In any other case, say you want the new filesystem to be reiserfs or xfs or btrfs or zfs, there is no way to convert one fs to another. So you couldn't copy the old fs with dd and then convert it from ext2 to reiserfs. Instead you have to make a new (empty) fs and copy the files out of the old one into the new one.
So, rsync (or cp or tar etc) does not care what kind of filesystem(s) the source or destination files reside in, except for the fact that different fs's have different capabilities, and so it's impossible to replicate some metadata from one fs to another. For instance, if you try to rsync from an ext3 fs to a dos fat16 fs, fat16 has no such things as symlinks, device nodes, named pipes, filenames longer than 8.3, case sensitivity, ownerships, permissions, suid/guid/sticky bits, accurate timestamps, or any extended attributes or acls, files larger than 2G, etc... rsync normally reproduces all of that, but it can't if the one or the other filesystem simply doesn't have it. But for most files going between most linux/unix fs's, most of those features are supported by all of them and so mostly you can copy files between them without ever knowing the difference.
I want to thank you very much for the clear description. I'm afraid I was pretty much in the dark about this whole situation, and now you have shed some light! --doug -- Blessed are the peacemakers..for they shall be shot at from both sides. --A.M.Greeley -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org