On Fri, Nov 18, 2016 at 6:19 PM, Lew Wolfgang <wolfgang@sweet-haven.com> wrote:
On 11/18/2016 03:05 PM, Greg Freemyer wrote:
This email might be offtopic, not sure.
I have a minimum of 10 TB of data I want to consolidate off of multiple USB drives to free them up. The data is almost exclusively static and rarely accessed, but I need to maintain it.
I have 2 copies of this data in most cases currently, but it is spread around multiple USB3 drives. (I have dozens of them). The plan for now is just to consolidate one copy of the data.
I bought a 10 TB SATA drive to hold a first big chunk. I expect it will get filled quickly as I start to consolidate my backup copies, so I want to be able to grow the volume holding the data by adding disks to the pool and extending the volume.
I suspect I will also want to have more resiliency at some point. (ie. raid 0 => raid 5 => raid 6)
If I truly had confidence in this storage pool I might eliminate both copies of the data that is on USB drives currently. But even with raid 6, I think I would worry about a total LVM or Volume crash Or even a user error!
Even though I know LVM and MDraid somewhat, I don't know if is "dynamic" and "reliable" enough for what I want.
And advice out there?
Goals:
- Create a fileserver that I can add drives to from time to time and grow it's capacity. Probably 10TB drives so I don't have too many spindles in the mix. When bigger drives become available, I'd prefer to use them, so being stuck will all the same size drives is a negative.
- Performance is non-critical. I've used LTO-4 tapes to do this in the past, but I hope online is a better choice now. With LTO-4, once I had a new data set (typically 100GB - 2TB) I would make a backup with tape and put it away for the time I needed to ensure I still had it. (Often years).
- Share the exported volumes with Windows PCs. (Not critical, but preferred)
- have the ability to start the drive pool with a single drive and add to it over time
- Allow the added drive to be either as SATA USB3.1
- Allow the Raid "protection level" to be adjusted for a given volume from time to time.
=== I was actually planning to do this with Windows and its "Storage Spaces" solution. I just this afternoon put a new 10TB drive in a windows PC and added it to the "Storage Space" (like LVM).
But my reading says the "resilience level" of a volume has to be set at the time the volume is created. I can grow it later, but I can't change it from a raid 0, to a raid 1, to a raid 5, etc.
Hi Greg,
Why don't you try a Drobo?
I really like what I've read about the drobo, so it is an option. Fairly expensive for the chassis as I recall, but I don't see pricing on their site right now. It is definitely an option I'm going to compare to. My plan as of 3 hours ago was blown out of the water when I started digging into "Storage Spaces" and realized how inflexible it was. fyi: This fileserver is for my company, not a customer, so I'm fairly cost conscious. On the other hand, having a bunch of partially full USB drives isn't very cost effective at this point.
But I don't think it will change RAID configuration on-the-fly.
Actually, I think it can. They say one of the cons of traditional raid is that lack of flexibility: http://www.drobo.com/drobo/beyondraid/
BTW, I'd recommend against using RAID-5 for anything if reliably is very important. The most stressful time for a RAID is when it's rebuilding after loosing a drive, and if you loose a second disk during a rebuild you're toast. RAID-6 gives you significant cushion during rebuilds.
Yeah, RAID 5 would mandate keeping another copy of the data elsewhere. But I'm not sure RAID 6 would make me comfortable enough either to eliminate that second physical copy.
BTW, aren't all drives SATA? If a drive is USB it probably has a converter to SATA which just adds a layer of complexity to break.
All the USB-3 rotating drives I've disassembled did indeed have a SATA internally. But I already own numerous USB-3 enclosed SATA drives (1TB - 5TB). No desire to break them out of their cases. I'll probably buy more 10TB+ size drives to add internally to the chassis as funds are available. FYI: But don't forget NVMe / SCSI / etc.
Regards, Lew
BTW: DId I miss the part where you told me how to do this with Linux? Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org