Le 03/10/2016 à 21:10, John Andersen a écrit :

In those years, I've had a grand total of ZERO tar.gz archives fail.

I had some, mostly due to file transfer or hardware problems, but no file in it could be read jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd wrote:

Le 03/10/2016 à 21:10, John Andersen a écrit :

...

In those years, I've had a grand total of ZERO tar.gz archives fail.

I had some, mostly due to file transfer or hardware problems, but no file in it could be read

Second that. Had the same problem after an apparently failed SFTP transfer. But there was no error message. The copy seemed valid. If I hadn't packed the files, only single files probably were affected. - Chris -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Chris -- ...and then Chris said... % % Second that. Had the same problem after an apparently failed SFTP % transfer. But there was no error message. The copy seemed valid. If I % hadn't packed the files, only single files probably were affected. Just curious... Did you run a checksum on both ends after the copy? In my experience, I can write things perfectly happily but then trip over some write or read error that shows up in a checksum test. Even for local copies, if it's important I always diff the src and dst to ensure that I can read what I wrote before wiping the original I read. HAND :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos, et al -- ...and then Carlos E. R. said... % % On 2016-10-17 11:53, David T-G wrote: % > ... % > Just curious... Did you run a checksum on both ends after the copy? ... % % If the source is static and doesn't change, yes, that's a possibility. % % But I often do backups while the system is live. Well, yeah, that could make things more challenging. Of course, then you don't really know if you got a good copy at all, right? So snapshotting a mirror or breaking and then re-syncing would probably be a good approach there. All of this presumes, of course, that one is worried about integrity, which should be pretty simple in this day and age, but I have to admit that I spend more time than I should arguing with my spinning media :-( HAND :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 17/10/2016 à 12:44, Carlos E. R. a écrit :

But I often do backups while the system is live.

but then changing files are probably not usable anyway jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 17/10/2016 à 16:42, Carlos E. R. a écrit :

On 2016-10-17 15:27, jdd wrote:

...

but then changing files are probably not usable anyway

Most of them are temporary files used by the desktop and not that important. All data files are copied as the saved files that exist on disk. If you worry about a file that is changing at the same time as the backup is done, then do it twice.

most important is database. one should backup the database (as text) before any rsync, but who do? my own database is pretty rarely written (and only by me, most never at the time I do the cron jobbed backup :-), so it's not a problem, but else the database can't be recovered jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-10-17 17:50, Greg Freemyer wrote:

On Mon, Oct 17, 2016 at 10:42 AM, Carlos E. R. wrote:

...

In MsDOS you can try to acquire a lock on the file before backing it up. If the file is in use, it will fail, so the backup either waits or continues with another file, and try the locked file later. At the end, it can produce a list of failed-locked files.

I don't know if that is possible in Linux.

To my knowledge all file level locking is voluntary. ie. user space apps have to intentionally respect other programs locking a file.

No, there are also kernel locks. I read something about mandatory locking, but I don't know how good it is. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2016-10-18 05:33, Andrei Borzenkov wrote:

18.10.2016 00:21, Carlos E. R. пишет:

...

On 2016-10-17 17:50, Greg Freemyer wrote:

...

On Mon, Oct 17, 2016 at 10:42 AM, Carlos E. R. <> wrote:

...

...

To my knowledge all file level locking is voluntary. ie. user space apps have to intentionally respect other programs locking a file.

No, there are also kernel locks.

Advisory locks are also kernel locks.

...

I read something about mandatory locking, but I don't know how good it is.

File needs to be explicitly marked for allowing mandatory locking so it won't work in general case. See https://www.kernel.org/doc/Documentation/filesystems/mandatory-locking.txt.

Oh. Pity. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2016-10-17 23:25, Greg Freemyer wrote:

On Mon, Oct 17, 2016 at 9:27 AM, jdd wrote:

...

Le 17/10/2016 à 12:44, Carlos E. R. a écrit :

...

But I often do backups while the system is live.

but then changing files are probably not usable anyway

I didn't pay much attention to this thread, but LVM snapshots are a great way to handle changing files.

You make a snapshot, then mount the snapshot read-only and make your backup from there.

XFS volumes also have a way to do it. I have not used the method myself, but I read about it on some man page. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2016-10-04 08:14, Per Jessen wrote:

John Andersen wrote:

...

On 10/03/2016 11:59 AM, Carlos E. R. wrote:

...

Reason: a few (or one) byte error makes the tar unrecoverable.

In reality, Carlos, exactly how many of these have failed for you?

Yes, I have to wonder that too.

It is rare, but it is documented. Media failures are possible, and if one happens in the middle of a tar.gz you lose it completely.

...

In those years, I've had a grand total of ZERO tar.gz archives fail.

Ditto.

How many in floppies? -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2016-10-04 11:47, Per Jessen wrote:

Carlos E. R. wrote:

...

On 2016-10-04 08:14, Per Jessen wrote:

...

John Andersen wrote:

...

On 10/03/2016 11:59 AM, Carlos E. R. wrote:

...

Reason: a few (or one) byte error makes the tar unrecoverable.

In reality, Carlos, exactly how many of these have failed for you?

Yes, I have to wonder that too.

It is rare, but it is documented. Media failures are possible, and if one happens in the middle of a tar.gz you lose it completely.

Sure, but every archiving tool is vulnerable to media and network failures, tar no more so than any other.

Not tar. The problem is gz.

...

...

...

In those years, I've had a grand total of ZERO tar.gz archives fail.

Ditto.

How many in floppies?

Floppies??

LOL. Yes, the worst media I can think of where I stored archives some time ago, and where media error in later years were frequent. Hey, this desktop machine has a floppy drive. I haven't managed to make it work since I built it up, dunno if hardware problem or kernel problem. A single error, and the whole archive is lost. On the other hand, I still have backups made on a hundred floppies in working order. Made with pctools backup in the 80's. One has one error, but the restore program successfully regenerates the data with forward error recovery methods. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2016-10-04 14:03, Per Jessen wrote:

Carlos E. R. wrote:

...

Not tar. The problem is gz.

Oh? Well, then use another compressor. xz is good.

I made a review of several compressors, none is good enough. Almost none have error recovery.

Yes, that is true. Well, floppies were never meant for long-term storage, although I'm sure I still have some 5" up in the attic.

The original floppies sold in the 80's do last. Those made by the end of the 90's and later are terrible. Impossible to get one that lasts a month without an error. A box of ten has some that error out on format. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2016-10-04 16:30, jdd wrote:

Le 04/10/2016 à 14:39, Carlos E. R. a écrit :

...

I made a review of several compressors, none is good enough. Almost none have error recovery.

they should have:

http://www.7-zip.org/recover.html

at least the files out of the damaged part should be recoverable

Yes, I have to test it. If I recall correctly it was not proven reliable. One that is proven is rar, but it does not support all Linux attributes. Oh, looking at your link, it is terrible. With rar you simply tell the command to repair the archive and that's it.

...

...

Yes, that is true. Well, floppies were never meant for long-term storage, although I'm sure I still have some 5" up in the attic.

The original floppies sold in the 80's do last. Those made by the end of the 90's and later are terrible. Impossible to get one that lasts a month without an error. A box of ten has some that error out on format.

anyway, most people do not have floppy reader. or interface to make it work.

And those that have it, like me, would not think of doing a hard disk backup that way. LOL.

so, yes, when trying to make long term archives (one can't anymore speak of backup) have to think of the medium

for example, flash disks are known to have a short life (including ssd), I speak of time life, not number of cycles. Hard drives have good life expectancy but can dye unexpectedly.

once it was said than archive medium have to be updated at a bare minimum every 5 years. It's pretty easy as in this time price drop and size grow.

one can find dvd or BD said to be worth 1000 years. take appointment in 3000 to know :-))

Hah! :-)

https://www.nierle.com/en/article/32558/Verbatim_M-DISC_Blu-ray_BD-R_25_GB_-...

Ah. But we still do not have in Linux a backup program that gets close to what PCtools Backup did in the 80's. In this case, perform the backup to supported media with compression and forward recovery bits. See: https://en.wikipedia.org/wiki/Forward_error_correction The idea is that even if the DVD/BD gets scratches and broken sectors, up to a percent, *all* the files can be correctly recovered without errors. See also "par2". (https://en.wikipedia.org/wiki/Parchive). If you want it, build it yourself. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 10/04/2016 07:58 AM, Carlos E. R. wrote:

Yes, I have to test it. If I recall correctly it was not proven reliable.

To be perfectly pedantic about it, you can not prove any archive method reliable. It can only be proven un-reliable, and only then if the underlying storage media is reliable. Which does not happen in this world. But lets get real here. The topic is Backup a hard drive before clean install. So you use tar.gz and then compare or run some tests for readability. Then you carry on with your install and restore. Your media or your tar.gz is not likely to go bad in the 90 minutes it takes to re-install. Entire distros use tar.gz as package managers. The whole linux industry runs away from unreliability toward safety, yet you just don't hear anyone howling about tar.zs files. Where are the articles demanding everybody avoid them? How come they aren't deprecated loudly in the press and in distributions? Instead we run headlong into things like BTRFS (100% failure rate for me) and systemd and kde4 (years before it was useable) and other unproven crapware with glee while taunting the stragglers as if they were holocaust deniers. Then you turn around and suggest tar.gz will bit-rot inside of 90 minutes? Come on Carlos. -- After all is said and done, more is said than done.

On 10/04/2016 01:34 PM, Carlos E. R. wrote:

I said that /IF/ there is an error that corrupts one byte of a compressed tar, with most compressors used you lose the entire archive.

And it is for that precise reason that I do not use that method for backing up critical data. Not since I got bit on the ass by it once, which is all it takes for me to never use it again. Terabyte drives are under US$50 now so cloning a drive is cheap. Daily backups are cheap and fast now days. I certainly remember when they weren't. Etc. But whatever you do, if it's critical data, don't rely on a compressed tar file. Just IMHO, of course. You do whatever you wish. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-10-05 10:24, jdd wrote:

Le 05/10/2016 à 07:58, Stevens a écrit :

...

Terabyte drives are under US$50 now so cloning a drive is cheap. Daily backups are cheap and fast now days. I certainly remember when they weren't. Etc. But whatever you do, if it's critical data, don't rely on a compressed tar file. Just IMHO, of course. You do whatever you wish.

the large files do not compress well anyway (jpg and mpg/mp4 are already compressed). Only text files (.txt) do compress and they a rarely large nowadays

I want to consider compressed filesystems. btrfs has the feature, but I think it is still considered experimental. Combined with snapshots, it would do a nice backup system. But somehow it doesn't ring well, with btrfs... one seeks ultimate reliability. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

Stevens wrote:

On 10/04/2016 01:34 PM, Carlos E. R. wrote:

...

I said that /IF/ there is an error that corrupts one byte of a compressed tar, with most compressors used you lose the entire archive.

And it is for that precise reason that I do not use that method for backing up critical data. Not since I got bit on the ass by it once, which is all it takes for me to never use it again.

Terabyte drives are under US$50 now so cloning a drive is cheap. Daily backups are cheap and fast now days. I certainly remember when they weren't. Etc. But whatever you do, if it's critical data, don't rely on a compressed tar file. Just IMHO, of course. You do whatever you wish.

That's it. Why should one deal with a probable source of errors that's cheap to avoid? - Chris -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2016-10-05 11:56, Anton Aylward wrote:

On 10/04/2016 02:34 PM, Carlos E. R. wrote:

...

On 2016-10-04 19:03, John Andersen wrote:

...

On 10/04/2016 07:58 AM, Carlos E. R. wrote:

Entire distros use tar.gz as package managers.

With checksums. If there is a problem you download it again, so its not an issue.

Ahn, no, Carlos, you've confused the issue. You're doing to much of the Marshall McCluhan "Medium *IS* the message" thing.

Checksum on download of a ISO or RPM or whatever that contains a TAR or CPIO or whatever package that has been compressed served another purposes well. Not just 'is it corrupted in transmission" but 'is the copy you got the one that packagers intended or one that was created by hackers'.

It serves both purposes. An accidental alteration during transmission, and intentional alteration by hackers or something. And yes, accidental alterations do appear now and then in the mail lists. zypper complains, and does not recover automatically, you have to delete the file.

I thin this is the the case since back in 1982 I wrote a a low level disk driver for the RL02 on a PDP-11 based V6 UNIX for carrier grade application for a telco that did this. A repeated error caused the corrected data to be re-written elsewhere and the low level disk mapping in the driver taking care of this redirection.

This is now normal practice with modern disk drives and is taken care of by the on-baord electronics so that the computer operating system sees an unblemished linear array of sectors no matter how they might be organized at the physical level.

XFS is adding checksums for at least metadata sectors, and they are considering data checksums, so that file integrity is guaranteed. Btrfs I think is going a similar route, after all, they share several devs.

Perhaps TAR'ing up the whole system or FS and expecting there to be only one or two errors in something that large is what worries you? Well perhaps you shouldn't take that big a bite of the cake.

The problem you apparently don't understand is that a single error and the whole targz archive is lost. I'm not analyzing anything. I'm not saying that it happens often. I'm simply saying that I do not trust tar.gz for archival and backup, unless you take additional measures. And this is a fact that is well documented. A single byte error and the tgz is lost. You may say that it is very rare to have such an error. Accepted. But if things go wrong and you do get that single byte error, then the whole tgz is lost, big or small. It is a documented fact.

You might read this: https://www.g-loaded.eu/2007/12/01/choosing-a-format-for-data-backups-tar-vs...

I have not read that document, but yes, I'm aware of the advantages of cpio in this respect. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 10/05/2016 07:41 AM, Carlos E. R. wrote:

The problem you apparently don't understand is that a single error and the whole targz archive is lost. I'm not analyzing anything. I'm not saying that it happens often. I'm simply saying that I do not trust tar.gz for archival and backup, unless you take additional measures.

And this is a fact that is well documented. A single byte error and the tgz is lost. You may say that it is very rare to have such an error. Accepted. But if things go wrong and you do get that single byte error, then the whole tgz is lost, big or small. It is a documented fact.

Perhaps its documented by people who don't understand (or perhaps don't use) CRC, but the whole point of CRC is not just an integrity checksum but a error correction code. Yes there are checksums that don't give detailed error information. Yes it may be that the the file is so bit that the correction information is not a lot of use. As JDD said, large files do not compress well anyway and he also said that gigabytes tars are a pain. I put these in the class of "don't go there". If you're TARing up a whole filesystem, rather than using one of the alternative backup or archiving methods we've discussed, and so get into these considerations, then I think you a fool. My base SYSTEM is about 20G. My HOME is made up out of many small file system, apart from movies and music, none larger than 5G so I can backup each onto a DVD. I did use rsync to an off site, low cost repository, but my cable provider charges the earth for the extra bandwidth. Yes I'm changing provider to one that allows 'unlimited after 2am'. Carlos, I think you're getting ridiculous in your defence of that position, ignoring that modern compress uses CRC which is correctable and is not a mere checksum, and throwing up irrelevancies that have nothing to do with this issue. All of this is quite irrelevant to you original stated objective of backing up a hard drive before a clean install. As I said, the alligator in the swamp analogy holds. When you're up to your ass in alligators while draining the swamp its difficult to remember that your purpose was to build a luxury seafront code complex with a 9-hole golf course, tennis courts and spa. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 10/04/2016 01:03 PM, John Andersen wrote:

On 10/04/2016 07:58 AM, Carlos E. R. wrote:

...

Yes, I have to test it. If I recall correctly it was not proven reliable.

To be perfectly pedantic about it, you can not prove any archive method reliable. It can only be proven un-reliable, and only then if the underlying storage media is reliable. Which does not happen in this world.

But lets get real here. The topic is Backup a hard drive before clean install.

So you use tar.gz and then compare or run some tests for readability. Then you carry on with your install and restore.

Your media or your tar.gz is not likely to go bad in the 90 minutes it takes to re-install.

Entire distros use tar.gz as package managers. The whole linux industry runs away from unreliability toward safety, yet you just don't hear anyone howling about tar.zs files. Where are the articles demanding everybody avoid them? How come they aren't deprecated loudly in the press and in distributions?

Instead we run headlong into things like BTRFS (100% failure rate for me) and systemd and kde4 (years before it was useable) and other unproven crapware with glee while taunting the stragglers as if they were holocaust deniers.

Then you turn around and suggest tar.gz will bit-rot inside of 90 minutes?

Come on Carlos.

Very! Well! Said!, John, although as you say, you're being pedantic, which is usually something people expect of me :-) But let me say it more briefly 1 Its about risk management. That is the most important thing and should be of overarching concern. It should be the driver that led you to consider the upgrade in the first place. 2 Backups are different from archives. if you don't understand that then you will get yourself mightily confused. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 10/05/2016 05:49 AM, Carlos E. R. wrote:

On 2016-10-05 11:16, Anton Aylward wrote:

...

On 10/04/2016 01:03 PM, John Andersen wrote:

...

...

Then you turn around and suggest tar.gz will bit-rot inside of 90 minutes?

Come on Carlos.

Very! Well! Said!, John, although as you say, you're being pedantic, which is usually something people expect of me :-)

But let me say it more briefly

Look again at my answer to that, consider the emphasis, and then at Stevens answer.

You said

I said that /IF/ there is an error that corrupts one byte of a compressed tar, with most compressors used you lose the entire archive. </quote>

You seem to miss out on that "one bit" being correctable by CRC Yes, un-tarz doesn't do that but there are tools that do.. And you never got to my conclusion that perhaps TAR'ing/ziping is not the way to go given alternatives. As discussed, one alternative might be a new disk. Whether for backup or migration (which I've done since I use LVM, but you could use rsync) would be up to you. Disks have lifetimes so getting a new disk periodically and migrating is a goo strategy anyway :-) I think you're being a bit of an inflexible stick-in-the-mud by refusing to acknowledge (a) TARZ has CRC and is error correctable using suitable tools and (b) there are alternatives to achieve your objectives of a newly install OS & system while preserving your data.. You're obsessing about a particular method while failing to considerer the overall objectives. The dictum about alligators in the swamp comes to mind. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 10/05/2016 07:42 AM, Carlos E. R. wrote:

On 2016-10-05 12:13, Anton Aylward wrote:

...

You seem to miss out on that "one bit" being correctable by CRC

No if the only thing you have got is the tgz archive. Again, this problem is documented.

"behaves awfully:, "Gives a lot of hastle in doing a recovery" and "its too difficult for me" and even "I don't know how so I don't believe it can be done" are all common cases. The thing is that in many cases it can be done. The bigger the file -- and you're talking about doing whole filesystems, the less likely or the more hassle it is going to be. And anyway, since many of the files are likely to be compressed already, as JDD points out, an error in them renders them unusable and that has nothing to do with TAR or your compression algorithm. There would be little point in my compressing /home/anton as whole file system; the BIG parts of that tree are the music and the videos, which amount to over 80G, compared to the rest of the tree that amounts to about less that 25G. That being said, I use LVM and have a number of LVs and file systems to make that 25G manageable for backing up onto DVD. The DVDs are then mountable as ISO file systems and can be treated as archives to extract specific file or to do a complete snapshot restore. LVM lets me take snapshots to make the DVD from :-) In terms of acheive the stated objective of find a "Reliable way to backup hard drive before clean install" this sounds a good strategy. Saying "it can't EVER" be done only needs one case of it being done to prove you wrong. http://www.gzip.org/recover.txt Let us not forget, gzip work on single files, not groups of files. Plain old zip (and pkzip) operate on groups of files and have the concept of the archive built-in. When you're dealing with individual files the issue of CRC for individual files rather than the whole archive changes the landscape. Let us not forget that many are switchng to the use of XZ compressions for a variety of reasons. It is a block compression algorithm and if there is an unrecoverable error then only that one file wherein it occurred is affected, all the rest are unaffected. And finally.. Lets not forget that a CRC is not a checksum. A CRC is about error correction. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 10/04/2016 10:03 AM, John Andersen wrote:

To be perfectly pedantic about it, you can not prove any archive method reliable.

Of course it can be proven reliable. That statement along with the rest of your email complaining about systemd, kde4 and btrfs is just trollish at best. It can't even be taken remotely seriousy. On 10/04/2016 10:03 AM, John Andersen wrote:

It can only be proven un-reliable, and only then if the underlying storage media is reliable. Which does not happen in this world.

Oh, so in science something can only be proven unreliable. How can you now talk about the storage media being reliable? According to your logic that's not possible. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On 2016-10-04 14:03, Per Jessen wrote:

...

Carlos E. R. wrote:

...

...

Not tar. The problem is gz.

Oh? Well, then use another compressor. xz is good.

I made a review of several compressors, none is good enough. Almost none have error recovery.

Okay. I use tar to disk and to tape, it has never been an issue for me. -- Per Jessen, Zürich (15.6°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Quoting Per Jessen :

Carlos E. R. wrote:

...

On 2016-10-04 11:47, Per Jessen wrote:

...

Carlos E. R. wrote:

...

On 2016-10-04 08:14, Per Jessen wrote:

...

John Andersen wrote:

...

On 10/03/2016 11:59 AM, Carlos E. R. wrote: > Reason: a few (or > one) byte error makes the tar unrecoverable.

In reality, Carlos, exactly how many of these have failed for you?

Yes, I have to wonder that too.

It is rare, but it is documented. Media failures are possible, and if one happens in the middle of a tar.gz you lose it completely.

Sure, but every archiving tool is vulnerable to media and network failures, tar no more so than any other.

Not tar. The problem is gz.

Oh? Well, then use another compressor. xz is good.

All compression types are vulnerable to a single corrupted bit making the rest of the file unusable. Compression removes all redundancy. Unless there is error correction (additional bits), flipping a single bit is unrecoverable. Codes are variable length, so there is no way to know where the next valid codes starts. JPEG uses periodic reset codes that lets decoding/decompression to restart but everything between a corrupt bit and the next reset is lost. Archives usually have a table of contents, so the most a single bit error can corrupt is a file. Jeffrey -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 10/04/2016 10:13 PM, Jeffrey L. Taylor wrote:

Archives usually have a table of contents, so the most a single bit error can corrupt is a file.

Indeed, but calling TAR an archive is a misnomer. Its a streaming protocol intended for use with tapes, not 'direct access, random seek' devices like disk files. Recall the issue: a backup is not an archive. Yes, if you have a TAR file on a disk (compressed or not) you can use a tool - I use Konkeror - to open it up and at a application level read through the whole stream and construct that index, but that index, that image of the file as a file system hierarchy, is in the application, not in the file itself. TAR is NOT an archive format. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org