I recommend using mdraid and LVM2. You can add disks to mdraids and you can change raid levels. See https://serverhorror.wordpress.com/2011/01/27/migrating-raid-levels-in-linux... Using md devices as physical devices for a lvm volume adds even more flexibility. If you swap old disks for new ones, you can add the new devices (as new md device) to expand the volume group, use pvmove to move data online to the new devices and then drop old devices from the volume group. Or better: you add new md devices and extend your logical volume. If you use large disks, use mirrored devices whenever possible. Raid5 is not an option (as Lew already noted) and raid 6 is really slow! Growing a raid6 with another 10TB drive may take days! So best is to add always 2 devices, create a mirror and add it to the logical volume. See manpages of mdadm, pvcreate, pvmove, vgextendd, vgreduce, lvextend On 19.11.2016 00:05, Greg Freemyer wrote:
This email might be offtopic, not sure.
I have a minimum of 10 TB of data I want to consolidate off of multiple USB drives to free them up. The data is almost exclusively static and rarely accessed, but I need to maintain it.
I have 2 copies of this data in most cases currently, but it is spread around multiple USB3 drives. (I have dozens of them). The plan for now is just to consolidate one copy of the data.
I bought a 10 TB SATA drive to hold a first big chunk. I expect it will get filled quickly as I start to consolidate my backup copies, so I want to be able to grow the volume holding the data by adding disks to the pool and extending the volume.
I suspect I will also want to have more resiliency at some point. (ie. raid 0 => raid 5 => raid 6)
If I truly had confidence in this storage pool I might eliminate both copies of the data that is on USB drives currently. But even with raid 6, I think I would worry about a total LVM or Volume crash Or even a user error!
Even though I know LVM and MDraid somewhat, I don't know if is "dynamic" and "reliable" enough for what I want.
And advice out there?
Goals:
- Create a fileserver that I can add drives to from time to time and grow it's capacity. Probably 10TB drives so I don't have too many spindles in the mix. When bigger drives become available, I'd prefer to use them, so being stuck will all the same size drives is a negative.
- Performance is non-critical. I've used LTO-4 tapes to do this in the past, but I hope online is a better choice now. With LTO-4, once I had a new data set (typically 100GB - 2TB) I would make a backup with tape and put it away for the time I needed to ensure I still had it. (Often years).
- Share the exported volumes with Windows PCs. (Not critical, but preferred)
- have the ability to start the drive pool with a single drive and add to it over time
- Allow the added drive to be either as SATA USB3.1
- Allow the Raid "protection level" to be adjusted for a given volume from time to time.
=== I was actually planning to do this with Windows and its "Storage Spaces" solution. I just this afternoon put a new 10TB drive in a windows PC and added it to the "Storage Space" (like LVM).
But my reading says the "resilience level" of a volume has to be set at the time the volume is created. I can grow it later, but I can't change it from a raid 0, to a raid 1, to a raid 5, etc.
Thanks Greg -- Greg Freemyer