On Sun, 9 Jun 2019 15:59:20 +0200
Michal Suchánek
On Sun, 09 Jun 2019 08:48:43 +0200 Achim Gratz
wrote: Michal Suchánek writes:
How does zstd compare to xz in robustness? https://www.nongnu.org/lzip/xz_inadequate.html
I haven't gone over the details, but the format is described here:
https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md
This seems to address most of the criticism the article you cited, although I'd prefer to have a more neutral checklist that doesn't grind an axe against one particular implementation.
The article mainly criticizes
- variable size of everything which prevents you making sense of stream with a bitflip in one of the headers that determine what size everything is. This makes recovering good data from slightly corrupted file very difficult. - nonsensical checksums that don't really add to data integrity. The situation with zstd is not much better. The optional checksum is attached at the very end of a series of variable-everything blocks. At least it checksums the decompressed data which gives some integrity check for RLE and raw blocks that don't benefit from internal checks of zstd algorithm itself. Unfortunately, you are required to write blocks as raw when you cannot compress them smaller.
From the point of view of data integrity and recovery the zstd format is not particularly awesome AFAICS.
That said, this is not particularly critical for use with rpm. The packages are protected by strong cryptographic hash and signature anyway so you should not even get a corrupted rpm package in hand. For the compression format in general it should either provide decent corruption protection or none at all. Both of the formats are flawed in this respect. This might be somewhat more relevant for something like initrd which tends to be transferred over networks and stored on media of questionable quality which are accessed by experimental hardware drivers. In this case you might want to do corruption protection independent of the compression. Thanks Michal -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org