On 30/10/17 19:35, Lew Wolfgang wrote:
Hi Folks,
BLUF: How to backup large data stores?
Say I've got 300-TB of on-line data, most if it is fairly static, while some of it changes frequently on a daily basis. I do daily incremental backups on the dynamic areas to disk filesystems on separate computers, with a monthly full disk image being stored off-line at an off-site location. But I'm worried about the bulk of the data that isn't being backed up.
Most of the data consists of lots of binary files stored on multiple hardware RAID-6 arrays. The arrays have hot-spare disks, and I've got spares on the shelf. But as the graybeards know, reliable RAID isn't backup!
One technology being considered is LTO tape. LTO-8 is due out any day now that claims to store 12-TB native. A drive or two with a tape library would possibly fill the requirement.
Does anyone have any thoughts/advice? What do clouds like Google and AWS do for backups?
Just done a quick look on Amazon ... A backup array you're looking at £3000 for the disks alone ... OUCH. I looked at 10TB Seagate Ironwolf raid drives, but Barracudas are even more expensive! and probably naffer drives. That said, it looks like you're looking at £1000 for a single set of LTO-8 tapes. What's your worlflow? Two 16-drive 10TB/drive arrays will give you near enough your 300TB. What filesystem are you using - btrfs? Is the all the actively changing stuff, stuff that's being worked on workstations? This might well not suit your situation, but I'd be inclined to store the stuff being worked on, on the workstations. Maybe raid-5, maybe no raid. Every night, they do a btrfs push to the big raid arrays (or, if they're not running btrfs, an in-place rsync). Every morning, the raid backups do a snapshot. That gives you a daily backup on-site. That might free up drives for an off-site raid setup, to which you can replicate over the internet. And then you can dump your raid arrays to tape as and when. Depends what level of backup and security you want, but for the price of three tape backups, you can have an off-site raid-array backup ... If you get a tape array as someone has suggested, I'm not sure how that will integrate with btrfs, but you could have a small offsite array that takes the btrfs pushes, and then stores it all on tape. Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org