Greg KH wrote:
On Wed, Jan 14, 2009 at 03:12:32PM -0600, Matt Sealey wrote:
I just noticed that while squashfs 3.4 kernel module is used on the -default kernel, the squashfs tools on both PowerPC and x86 repos are stuck
Someone needs to kick the owner of the userspace squashfs tools to get them to update them. If you could be so kind as to file a bug against the package, that would help.
I am doing it now, but also going through another bunch of bugs.. also I'm a little wary of the actual category to put it in but, oh well.. Other can do for now.
Also, more for the kernel guys.. is there any reason why the mkinitrd etc. stuff relies on cpio archives when the SUSE kernel is built with squashfs (granted as a module), all the rescue/install images are squashfs?
No specific reason, other than "that's just the way we've always done it."
Do you really want to touch that mkinitrd code to change this? :)
I was sure at some point I was reading mkinitrd code and saw some option for initrd_fs which specified a type, which said "ext2" was the default. This is obviously dead wrong (since it's always a cpio). Looking at it again, it turns out mk_image (argh what's with these names, I just had a fight with mkimage from U-Boot and jigdo) supports squashfs through the Conv2Image perl module. Aren't these two codebases doing EXACTLY the same thing but in different ways, here? Some room for consolidation, I'd think.
It does not provide any remarkably better compression - in fact I tested 100k larger filesystem with mksquashfs 3.2 but then mksquashfs 3.2 refuses to use large block sizes so the compression is lower than it could be, also mksquashfs 3.4 supposes a 10% improvement and unsquashfs supposes a 40% improvement in speed in creation and extraction (we will ignore the kernel improvements as, they are in the kernel already)..
.. But a few tests in performance usually always show that accessing a squashfs initrd mounted to RAM is faster than a cpio initrd in RAM.
Are you sure? What kind of performance difference is there?
I'm trying to benchmark that now, I'm wondering what I can actually do.. maybe I will grab the 11.1 common and make a cpio out of it and loop mount them both and do something with it, but it is sort of beyond my means at the moment. I've run out of hardware that's not in permanent use.. benchmarking on a system running an rpm build is not fair :) I will endeavour to poke at it though. In the meantime I bugged the author, since he would know :D
It also has a few extra little advantages here and there but I do not know if they are worth it, but I do know that managing squashfs filesystems is a billion times easier than pushing cpio archives around.
But these are generated "on-the-fly" and don't need to be managed by anything, right?
Indeed..
What advantages do you know of?
Appending files is a breeze compared to a cpio archive. Replacing the same file in a cpio usually means dumping and remaking it. mksquashfs can get rid of the original file and relink the new one at the end.. it would mean you can do clever tricks like building half the initrd or filesystem image, doing something else based on that result, and then appending further processed data. I guess it's just as easy with cpio with clever scripting. There's also the potential for slightly better compression if there was some known sort order of the files so they could be grouped together. Perhaps in the future LZO (not as great compression as gz but much faster to decompress and FAR less memory intensive of course) or LZMA (better, but a bit heavier of course. Always a trade-off) sometime, perhaps, maybe. I'm not counting that but it would be a benefit worth risking a little work on.
Since squashfs is going to be in the kernel mainline at some point (it's in Linus' tree)
Yes, it will be in 2.6.29, and is already there in 2.6.29-rc1.
and everyone and his dog makes squashfs initrds,
Who else?
Gentoo, GoboLinux.. umm.. Knoppix et al. :) For no other good reason than it ends up that someone wants to loop mount a root filesystem at some point during the life of the install, be it just during install (as is on the SUSE DVD common/rescue images etc.), or using it to pre-seed the root filesystem and then union mounting a real root on top (I can't remember exactly where I saw this trick now, but I think it was GoboLinux. All the kernel modules are in the initrd, not on the main filesystem.. it hasn't the finesse of the SUSE "dynamically make an initrd based on your machine", but it's no more clunky than the SUSE "the installer initrd cpio actually contains a big squashfs of all the modules and loop mounts it and sometimes in the betas it's not there (bug 437796)" :) Granted it's nothing you can't do with cpio, but.. some places people decided, if you're going to make a filesystem archive which gets loaded or mounted, they may as well all be the same type. And SUSE is already using it in the installer.
are we going to see a move to it at some point and have the thing built in with more use of the format?
I think the goal with mkinitrd is to work with the community to come up with a common tool that all the distros use. Dave Jones is currently working on a first cut at this, and then we will all get together to go from there. If it happens to use squashfs instead of cpio, well, that will be up to the community, not just us.
I don't think you would get too many objections.. there's also no reason (like mk_image in installation-images) that it can't use a bunch of different image types. Who knows, squashfs could be superceded by something else in the future. You know how usually goes.. it's better to be flexible :) -- Matt Sealey Genesi, Manager, Developer Relations -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org