Mailinglist Archive: opensuse-factory (422 mails)

< Previous Next >
[opensuse-factory] Re: [opensuse-kernel] Squashfs
  • From: Rob OpenSuSE <rob.opensuse.linux@xxxxxxxxxxxxxx>
  • Date: Thu, 15 Jan 2009 14:53:33 +0000
  • Message-id: <ce9d8ed60901150653v6671862fn69c9c08eda5977f2@xxxxxxxxxxxxxx>
The problem that seems to be overlooked is that, SquashFS is only just
making it into the mainline kernel. initrd's are from kernel hacker
point of you, non-broken and functioning perfectly satisfactorily
using the cpio/gzip format.

So to utilise a SquashFS format initrd, requires this kernel change.
A distro version which then uses this format, is not going to be able
to support booting with older kernels, hence unsuitable for kernel
hackers and support, who may very well need to do such things, unless
the support is backported into their "stable" kernels.

It seems unlikely to me, that a initrd unification project could make
changes that might prove contraversial, more they're going to need to
build consensus, minimising changes that ppl may object to.

So perhaps a FUSE based initrd admin tool, or unpack and repack
scripts, is a more feasible solution to the difficulties disovered
with 11.1 mkinitrd for the sysadmin.


2009/1/15 Matt Sealey <matt@xxxxxxxxxxxxxx>:
Rob OpenSuSE wrote:

2009/1/14 Matt Sealey <matt@xxxxxxxxxxxxxx>:

Nobody is suggesting that this be a solution, although it IS the solution
used for mkinstallinitrd in installation-images - it makes a cpio.gz initrd
and stuffs in squashfs files full of modules and other stuff, and loop
mounts them via Linuxrc (after loading the squashfs module and loop module
which are the only ones in the cpio.gz).

That's already overcomplicated.

Agreed.

It would seem wise to err on side of heavier compression and
processing. bzImage has been used for a very long time, to decompress
the kernel on older CPUs with less memory than is standard now, so it
is likely pointless to go to a looser compression for decompression
speed and lower memory consumption.

bzImage etc. only exist for bootloaders which could not load files bigger
than a certain size, which was kind of a problem when the kernel and initrds
were getting rather large.

It was also nice, because you could boot a kernel off a floppy disk :)

Sine you have to decompress the kernel to run it, bzImage doesn't save any
memory at all (it just takes the bootloader longer to decompress it and bzip
requires far more memory to decompress than gzip). Once it's decompressed,
the original bzImage data is discarded and only the uncompressed kernel and
(and compressed initrd data) are left in RAM when the kernel entry point is
called.

True, but gzip is the kernel decrompession and uses less memory than
LZMA, and CPU. It's been used for a long time, so should not upset
folk with embedded class low power hardware, whereas LZMA might be a
problem on older CPUs. The "less memory" referred to systems where
gzip method was successfully used, not the effect of using bzImage or
gzip for initrd. We agree again there.

The boot loader doesn't decompress the kernel, as far as I understand,
it's the kernel that does that, effectively unpacking itself into it's
final position in memory. That feature allowed the kernel to work
with very simple minimal boot loaders on things like floppies.

Can that support live installion of new modules? Logically the initrd
has been a minimal size on typical running systems, and should ideally
be the minimal set of modules to get / (and /lib/modules) mounted.
Then "all" of the modules are in 1 place, and the initrd only needs
changing when a kernel update, or hardware change occurs.

As an example, you could build a standard initrd and then just append the
modules to it with mksquashfs, or add config options to append a new module
when a new kmp is installed which may be needed earlier in boot (for
instance making sure a new raid controller is brought up). This is as easy
with mksquashfs as it is with cpio because of the cpio format (iirc just
concatenating cpio archives works), but mksquashfs is more flexible about

The current mkinitrd wraps up stuff in a non obvious way. I suspect
FUSE, might provide a suitable alternative, to permit changes to the
generated initrd, where udev's copying of currently booted system.

But I much prefer the idea of being able to mount the initrd, like an
iso file and make changes, to current unpack, and repack scheme.

The real point is, the current mkinitrd in 11.1, has a flaw through
using udev, which fixed other issues (not ones that are clear to me)
that moving to a mountable format might ameliorate.

It's a Pro reason for Novell/SuSE to support the SquashFS initrd idea,
when Dave Jones is seeking input. Though I guess I could email him,
pointing out the bug report involved, I am sure there are other
Pro/Con that I am not fully party to.

Installers may benefit from the extra layer of mounting a special
purpose filesystem type, but if the kernel expects a cpio archive in
RAM, and not the SqaushFS format, then you gain nothing.

Squashfs patch makes the kernel expect Squashfs as well as a bunch of other
formats (cramfs, ext2 etc.) besides cpio that the kernel already supports.
mkinitrd (or dracut) gets run in the kernel-flavor.arch.rpm post.sh script
on INSTALLATION, which means basically you can tell when and if the right
filesystem is available in the kernel (this decision is left up to the
kernel maintainers who know the kernel version and RPM release the support
started, and can check for it - hell, there's not even a need for that,
CONFIG_SQUASHFS=y in /boot/config-VERSION-FLAVOR is enough notice that you
can support that format.

Great to make that point clear. But it means SqaushFS must be
compiled in, and not a loadable module, like most filesystems.

One reason perhaps for use of cpio, was that the format was absolutely
stably fixed. If SquashFS got used in the suggested way, then it's
format needs to be similarly stable, with a mature tool chain.

Phillip Lougher is doing this right now which is why squashfs is going into
mainline in 10.. 9.. 8...

The reason I bring it up is because it will be in mainline, it will be
available in every kernel-source after that without a patch, the format has
to be stable at that point (4.0) and basically it is a better solution in a
bunch of ways, some huge, some not so huge, some may even be negative, but
it is in common use, it is a flexible format and a real filesystem (I
remember when all our initrds were cramfs..) and has many other benefits
outside of initrds (Linuxrc already relies on it for the root filesystem,
common/rescue/etc. in the installer) and would provide a common filesystem
compression format to be used across the whole distribution.

If they ever manage to support LZMA in squashfs (which requires support in
the kernel first, which people seem to be dilly-dallying about right now),
then there is no reason why the DVD images (gnome/kde/x11/base) can't be
compressed in it either.

Now that the SquashFS has been accepted, it'll be easier to add kernel
support for LZMA, and then LZMA patches to SquashFS, if the change has
technical merit and meets user requirements.

Modern systems would likely benefit, without seeing the downsides,
that low powered hardware might experience.
--
To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-factory+help@xxxxxxxxxxxx

< Previous Next >
Follow Ups