On Tue, Oct 31, 2017 at 11:45 AM, Wols Lists <antlists@youngman.org.uk> wrote:
On 31/10/17 15:31, Lew Wolfgang wrote:
On 10/31/2017 04:20 AM, Greg Freemyer wrote: <snip>
LTFS seems to work best with large files. rdiff-backup makes no effort to consolidate small files, but if your files being backed up, it should perform well.
Yes, lots of large files.
Spanning tapes with LTFS would be the big question. A quick google shows that it may be possible. I'll have to dig deeper.
If LTFS allows a raid-like solution over multiple tapes :-) it'll be similar to technology that was available long ago with tape libraries.
Okay, it needed fancy robotic self-loading tapes, but I'm sure I worked with it on a Perkin-Elmer (or was it UniVac) mainframe back in the 80s. Files that weren't accessed for a while were shuffled off to tape, and when you tried to access them they were retrieved.
It seems to me you're going to find LTFS a bit tricky to work into your backup solution unless you're planning just to dump incrementals to tape, in which case you're not really using LTFS as designed. But it certainly looks interesting technology.
Wol, I'm just coming up to speed myself on LTFS. But I know rdiff-backup. rdiff-backup on day one makes a copy of every file in the folder structure being backed up. You can use any file explorer to browse the created duplicate tree and find a copy of the files as of day one. Then every time you invoke it (daily?) it compares the primary instance of the file to the copy in the rdiff-backup set and saves the set of differences. Most files won't change daily, so those files don't get anything written to the secondary media. If there is a new file or a file changes, then either a copy of the file is made, or a changeset file is written. Once you've had it running for say 30 days, you can tell rdiff-backup to do a restore to any of the various point-in-time copies you made of the dataset. By design, rdiff-backup uses a filesystem to hold the secondary copy, possibly on a low cost NAS. Let's guess Lew would need 400TB of capacity to hold his backup including a month's worth of daily incremental changesets. That's 40 10TB drives without redundancy. Or about $16K worth of disks. Easily $20K when you include a couple chasis's to hold it all. It seems at this point that by using LTFS, that secondary storage can be replaced with a LTO tape library. A tape library with 1 tape drive and enough tape capacity to hold it all should be in the same price range (or so I assume). The real question is if Lew would rather have a tape backup or a NAS backup. I personally like the ability to take a tape set offsite occasionally. Or send it to Iron Mountain. It is that offsite ability that would lead me to choosing a tape solution if they are both near the same price. Unfortunately, a single LTO-8 tape isn't big enough to hold Lew's 300TB of data. He will need to be able to concatenate multiple tapes into one logical tape. I don't know if that last requirement can be met by LTFS or not. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org