Who would have thought that something as simple as copying a number of files would have created so much discussion? Thanks for that - I now understand a bit more of why it failed. One thing that will definitely be changed is to put it in various subdirectories to make it easier to copy in the future. The good news is that we managed to get the files copied using mc on a 64 bit machine - took 36 hours though. It was definitely faster on a 64 bit machine (Dual Xeon 3.4GHz, 2GB RAM) than on a 32 bit machine (P4 2.4GHz, 512MB RAM). I should have the disk back with me within the next week or so. Currently it is in the UK where they needed the images (graphic files) for a presentation. Unfortunately they have already prepared the drive for shipping before I could stop them to test some of your suggestions. I will try and see if I can get them to put it back into the machine. We can access a file if we know what the file name is. It is therefore possible to manipulate (rm/cp/ls) any single file and possibly a number of files. Each image is 125x125 pixels named according to a numeric value indicating a number on a grid which in turn represents the world. These images contain some demographic information compiled from various sources within our company. During development we ran into a problem where the graphics was created incorrectly. We then tried to delete the files but couldn’t. At the end we've found that mc can delete them - given enough time. I could possibly use the following to copy the files in to a more manageable directory structure: On Wednesday 31 August 2005 at about 17:46, Jerry Feldman wrote:
Assume that the source directory is mounted as /usr/localfoo and the target directory is mounted on /mnt: cd /usr/local/foo find . -type f -exec cp -p {} /mnt \; This is assuming that /usr/local/foo does not have subdirectories. You can set appropriate flags in find to prevent recursion. Another possibility is to partition the output into a number of subdirectories: Assume that you may have directories: /mnt/a, /mnt/b, ... find . -name \[aA\]* -exec cp -p {} /mnt/a \;
I will also try ls -U (to get an unsorted directory listing as suggested by Jim Cunning). Thanks again for all the suggestions. The response was fantastic. Albert -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.344 / Virus Database: 267.10.18/86 - Release Date: 31/08/2005
Albert wrote: Midnight Commander? Umm, ok.. but then you have to sit there while all the work is being done :-) There may also be solutions which are even easier than the ones suggested so far.
We can access a file if we know what the file name is. It is therefore possible to manipulate (rm/cp/ls) any single file and possibly a number of files. Each image is 125x125 pixels named according to a numeric value indicating a number on a grid which in turn represents the world. These images contain some demographic information compiled from various sources within our company.
Are they strictly numeric filenames? Try something like for a in {0..9}{0..9}{0..9}; do mkdir <target>/$a mv $a* <target>/$a/ done This will create 1000 separate directories, eg. <target>/005/ etc, and move every file beginning with a specific combination of 3 numbers into a directory of that name, ie. 005* are all moved into <target>/005/. Each mv command will involve about 1000 files. If you cannot handle moving about 1000 files at a time, then add another set of numbers: "for a in {0..9}{0..9}{0..9}{0..9}; ...." which would produce 10,000 directories of 100 files each. If you do not want so many subdirectories in <target>, add another "for" layer. For example, for a in {0..9}{0..9}; do mkdir <target>/$a for b in {0..9}{0..9}; do mv $a$b* <target>/$a done done Now you get 100 directories with about 10,000 files in each, and move 100 files at a time. If that is too many files per directory, you can do all sorts of variants on this theme. This one will move 100 files at a time, and give you 100 directories containing 100 subdirectories, each containing approximately 100 files: for a in {0..9}{0..9}; do mkdir <target>/$a for b in {0..9}{0..9}; do mkdir <target>/$a/$b for c in {0..9}{0..9}; do mv $a$b$c* <target>/$a/$b done done done You may not be able to produce the exact file sorting you want with a command structure involving only "for" groups, but at least you will certainly be able to move them into a manageable tree structure, and then finish the work manually.
On Thursday 01 September 2005 3:24 am, Albert wrote:
During development we ran into a problem where the graphics was created incorrectly. We then tried to delete the files but couldn’t. At the end we've found that mc can delete them - given enough time. The problem there was that you probably did something like 'rm -f *' And here you run into the shell limitations. I'm using shell generically because it applies to most of the common shells.
-- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
participants (3)
-
Albert -
Darryl Gregorash -
Jerry Feldman