* Lew Wolfgang <wolfgang@sweet-haven.com> [10-10-22 15:30]:
On 10/10/22 12:07, David C. Rankin wrote:
On 10/7/22 18:01, Lew Wolfgang wrote:
I'm thinking a fancy shell script is called for here. I bet that rsync should be leveraged to allow restarting from an aborted copy process.
Any ideas? It seems to be a common requirement, maybe someone's done it already?
Well...
That really depends on what is in the files. If they are binary, then you are going to need to split on a record boundary if you hope to mount some arbitrary JBOD copy somewhere and read something other than gibberish.
We don't want to split a file between disks. The files are all 7-GB binary, and the disks are 8-TB. That should give us about 1,100 files on a disk, we're not going to worry about the wasted space.
If they are text, that's a bit easier, but the storage would be less than optimal from a size standpoint.
There is a coreutils utility called 'split' that allows you to slice large files into smaller files by lines or bytes and it also has a --filter=command option to filter through a shell script which would give you the ability to umount/mount new JBOD devices between parts of the original being written.
Yes, split is great and would work, but we're not going to need it this time.
and once more, dar will accomplish all that for you very quickly, especially using the gui -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet oftc