Mailinglist Archive: opensuse (1126 mails)

< Previous Next >
Re: [opensuse] compression tool with high compression ratio, multiple files and fast random access?
  • From: "Carlos E. R." <robin.listas@xxxxxxxxx>
  • Date: Wed, 01 Sep 2010 11:31:07 +0200
  • Message-id: <4C7E1D5B.2080005@xxxxxxxxx>
On 2010-09-01 11:04, Per Jessen wrote:
Dave Howorth wrote:

A compressed filesystem?
squashfs?

Sorry, don't know much about the field.

Thanks Dave - it's a valid suggestion. I've already looked at squashfs
and I may have to take another look, but I think the problem was in the
mount time as well as in single-file extraction/retrieval time.

A compressed filesystem is indeed interesting for email.

The only only I tested is zisofs, used to create compressed CD/DVDs, directly
mountable in linux -
ie, access to a single file is direct. However, it does not seem very fast.

And the creation is cumbersome.


squashfs is also RO. The wikipedia says it uses gzip compression (lzma
projected).

http://en.wikipedia.org/wiki/Squashfs

]> SquashFS is used by the Live CD versions of Arch Linux, Debian, Fedora,
Gentoo Linux, Ubuntu and
on embedded distributions such as the OpenWRT and DD-WRT router firmware. It is
often combined with
a union mount filesystem, such as UnionFS or aufs, to provide a read-write
environment for live
Linux distributions. This takes advantage of both the SquashFS's high speed
compression abilities
with the ability to alter the distribution while running it from a live CD.
Distributions such as
Slax, Debian Live, Mandriva One and Puppy Linux use this combination.
]>
]> The on-disk format of SquashFS has stabilized enough that it has been merged
into the 2.6.29
version of the Linux kernel.[3] In that process, the backward-compatibility
code for older formats
was removed.[4]

And:

]> See also
]>
]> * Cloop is a compressed loopback device module for the Linux kernel
]> * Cramfs is another read-only compressed file system
]> * e2compr provides compression for ext2

http://en.wikipedia.org/wiki/Cloop

]> The compressed loopback device or cloop is a module for the Linux kernel. It
adds support for
transparently decompressed, read-only block devices. It is not a compressed
file system in itself.

http://en.wikipedia.org/wiki/Cramfs

The compressed ROM file system (or cramfs) is a free (GPL'ed) read-only Linux
file system designed for simplicity and space-efficiency. It is mainly used
in embedded systems and small-footprint systems.

Unlike a compressed image of a conventional file system, a cramfs image can
be used as it is i.e. without the need to decompress the image first. For
this reason, some Linux distributions also use cramfs for initrd images
(Debian 3.1 in particular) and installation images (SUSE Linux in
particular), where there are constraints on memory and image size.


http://en.wikipedia.org/wiki/E2compr#Compression_extension

]> e2compr is a modification to the ext2 file system driver in the Linux kernel
to support online
compression and decompression of files on file system level without any support
by user applications.
]>
]> e2compr is a small patch against the ext2 file system that allows on-the-fly
compression and
decompression. It compresses only regular files; the administrative data
(superblock, inodes,
directory files etc.) are not compressed (mainly for safety reasons). Access to
compressed blocks is
provided for read and write operations. The compression algorithm and cluster
size is specified on a
per-file basis. Directories can also be marked for compression, in which case
every newly created
file in the directory will be automatically compressed with the same cluster
size and the same
algorithm that was specified for the directory.
]>
]> e2compr is not a new file system. It is only a patch to the ext2 file system
made to support the
EXT2_COMPR_FL flag. It does not require you to make a new partition, and will
continue to read or
write existing ext2 file systems. One can consider it as simply a way for the
read and write
routines to access files that could have been created by a simple utility
similar to gzip or
compress. Compressed and uncompressed files coexist nicely on ext2 partitions.
]>
]> The latest e2compr-branch is available for current releases of 2.6 and 2.4
Linux kernels, but
development is stalled. There are also older branches for older 2.0 and 2.2
kernels, which are more
stable.



Pity that e2compr development has stalled, I love the R/W idea (to access a
mail folder many MUAs
want to write to it - meaning a DVD copy does not work, has to be copied to HD
first.



--
Cheers / Saludos,

Carlos E. R.
(from 11.2 x86_64 "Emerald" GM (Elessar))

< Previous Next >