Mailinglist Archive: opensuse (880 mails)

< Previous Next >
Re: [opensuse] Creating the distro USB stick with a simple 'cp'
On 08/13/2013 11:21 PM, Brian K. White wrote:
On 8/13/2013 8:21 PM, Doug wrote:
On 08/13/2013 06:39 PM, Brian K. White wrote:
On 8/13/2013 6:16 PM, Doug wrote:
On 08/13/2013 04:27 PM, Brian K. White wrote:
On 8/8/2013 5:42 AM, Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On Thursday, 2013-08-08 at 07:59 +0200, Per Jessen wrote:
/snip/

explanation of filesystems and copying--well done!


You can proceed a few different ways.

The best way is probably create new empty filesystems manually on the
new partitions, then use rsync to copy all your files over to them. The
advantage here is that rsync is resumable (if you abort in the middle,
it doesn't have to start over from the beginning), and it works for any
kind of fs as long as you can mount that fs in linux.

Another way is dd the old fs to the new partition, then use whichever
utilities are appropriate for the fs type to grow the fs to fill the new
space. Not all fs's are growable but the common current linux ones are.
At least ext3, ext4, reiser. They don't all use the same command, ie
there isn't a growfs command. ext3/4 has resize2fs, reiser has
resize_reiserfs, xfs probably has something, btrfs probably has
something, etc. This all also assumes you aren't using lvm.
This way might work if you had a non-linux fs from some other OS where
you can't mount or resize the fs from linux, but you can dd any block of
disk bytes, and then maybe the other OS can resize it's own fs itself later.

Generally, copying a fs to a larger partition or whole device (no
partition table) does not affect the fs at all, it still works fine
exactly as before. But it also does not use the extra space. You have to
resize the fs to use the extra space. Not all fs types can be resized.
Some can be resized but maybe there is no linux utility to do it, or not
a safe one anyways. Some can only be grown and never shrunk, some can be
grown or shrunk.

I was intending to format the partitions on the new drive with ext4 (for
Linux) and NTFS (for Windows 8) so that the DDd files would arrive on
formatted space, and the rest of the partition would be usable, being
formatted and empty. I get the impression that I'm missing something
here--please explain. Is there a difference between what you call a
file system, and what I call a formatted partition? If so, what?

Thanx--doug

A filesystem is the formatting, but it need not be in a partition. It
can be on a whole device (like a floppy, only you can do it to anything
like a hard drive or a thumb drive or a ramdisk) or inside of a file.

dd does not copy files it copies raw data. That data may or may not come
from a file or go into a file, the point is dd doesn't know or care what
the data is. If you use dd to copy a filesystem, you are not "dding the
files" you are dding one big solid hunk of raw meaningless data called
the entire filesystem. dd does not know about all the individual files
inside in the way that tar or cp or rsync or winzip does.

The filesystem is the "formatted partition" in one sense, in the sense
that it is the formatting that usually occupies an entire partition.
When you format a partition in windows, what is happening is windows
creates an ntfs or fat32 filesystem

But a filesystem is also not a "formatted partition" in the sense that
it is not the partition itself, just the data that was written to the
partition. The partition is just an arbitrary block of disk space which
is defined by the partition table. It's not really accurate to think of
"formatting the partition" because for instance you can "format" just a
part of a partition if you wanted to. You could write a filesystem that
only uses 800 gigs of a 1tb partition. Is that partition then "formatted"?

Normally you would not do that but the distinction still matters because
of your own current project.

You have an old drive that currently shows up as /dev/sda
and a new larger drive that shows up as /dev/sdb.

The old drive has a partition /dev/sda1 which is 500 gigs, 100% filled
with a 500 gig ext3 filesystem.

The new drive has no partitions and no filesystems.
You create a partition table and make a 1.5tb partition on it.
Now you have a /dev/sdb1 that is 1.5tb but it has no filesystem yet.

Instead of running "mkfs /dev/sdb1" to create one you use dd to copy the
old one:
dd if=/dev/sda1 of=/dev/sdb1

What happened was, the first 500G of /dev/sdb1 now has a carbon copy of
/dev/sda1, followed by 1T of blank space. Not only is it blank, it's
"invisible" to most utilities. If you mount the new copy:

mount /dev/sdb1 /mnt

Then look at /mnt, you will see that there is only 500G of space total
in /mnt.

df -h /mnt

will show something like this:
# df -h /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 512M 358M 154M 70% /mnt

That empty space is not merely unused like the 154M available above.
It's not even available, because it lies outside the filesystem.

In the old old days, you couldn't resize filesystems, you could only
make new ones and copy files from one to another. Today the popular
filesystems can mostly be resized.

So now you can use resize2fs to resize /dev/sdb1 to fill up the new
larger partition it is in. After that, the df command will show you that
the Size and Avail are larger.

rsync would be a different story.

rsync DOES work on individual files. One reason to use rsync (or cp or
tar or cpio or any of the backup programs) instead of dd, would be that
if you copy the individual files instead of the whole flsystem, then the
new filesystem can be a different type than the old one, as well as a
different size. For instance, say the old filesystem is ext2. If you
wanted the new fs to be ext3 or ext4 then you can upgrade an existing
ext* to any higher version in-place, but that's a special case. In any
other case, say you want the new filesystem to be reiserfs or xfs or
btrfs or zfs, there is no way to convert one fs to another. So you
couldn't copy the old fs with dd and then convert it from ext2 to
reiserfs. Instead you have to make a new (empty) fs and copy the files
out of the old one into the new one.

So, rsync (or cp or tar etc) does not care what kind of filesystem(s)
the source or destination files reside in, except for the fact that
different fs's have different capabilities, and so it's impossible to
replicate some metadata from one fs to another. For instance, if you try
to rsync from an ext3 fs to a dos fat16 fs, fat16 has no such things as
symlinks, device nodes, named pipes, filenames longer than 8.3, case
sensitivity, ownerships, permissions, suid/guid/sticky bits, accurate
timestamps, or any extended attributes or acls, files larger than 2G,
etc... rsync normally reproduces all of that, but it can't if the one or
the other filesystem simply doesn't have it. But for most files going
between most linux/unix fs's, most of those features are supported by
all of them and so mostly you can copy files between them without ever
knowing the difference.

I want to thank you very much for the clear description. I'm afraid I
was pretty much in the dark about this whole situation, and now you have
shed some light!

--doug

--
Blessed are the peacemakers..for they shall be shot at from both sides.
--A.M.Greeley
--
To unsubscribe, e-mail: opensuse+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups