[opensuse] Embedded Spaces in Pathnames
I have been trying to figure this one out all week both at work (Red Hat) and at home (OpenSuSE). I have a script that walks all of the ext3 filesystems on a system and detects those files and directories with either no user or no group (or both) creating an output file using the -ls option on the find command. On files/directories with no user, I want to change the user to nobody and likewise for the group. I could run the initial script twice, once to detect unowned files and the other to detect unowned groups and handle each separately. However, on some systems we have terabyte filesystems with millions of files and a single find command can run for days. Thus I would like to take this initial file and use that to correct both problems. The problem I keep running into is with pathnames with embedded spaces. I have tried everything I can think of using shell scripts, awk and perl and they all have problems with pathnames with embedded spaces. Can anyone point me to some good articles that may be of some help? Thank you, Lucky Leavell -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
El 29/10/10 21:22, Lucky Leavell escribió:
I have been trying to figure this one out all week both at work (Red Hat) and at home (OpenSuSE).
I have a script that walks all of the ext3 filesystems on a system and detects those files and directories with either no user or no group (or both) creating an output file using the -ls option on the find command.
Post the script. we dont know what you are doing. However, on some systems we have terabyte
filesystems with millions of files and a single find command can run for days.
Huh, you are doing it wrong then ;) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sat, 30 Oct 2010 10:52:56 Lucky Leavell wrote:
[...]
The problem I keep running into is with pathnames with embedded spaces. I have tried everything I can think of using shell scripts, awk and perl and they all have problems with pathnames with embedded spaces.
Can anyone point me to some good articles that may be of some help?
xargs is your friend. Use the -print0 option of find and pipe the output to xargs -0 (man find and man xargs). That takes care of spaces in paths (which are otherwise considered as delimiters between arguments). -- =================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au =================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hello, On Fri, 29 Oct 2010, Lucky Leavell wrote:
The problem I keep running into is with pathnames with embedded spaces. I have tried everything I can think of using shell scripts, awk and perl and they all have problems with pathnames with embedded spaces.
I don't believe you. Bash (+sed/awk/whatever): quote your variables properly. If you do it right, all characters but NUL and '/' are possible. If you pipe names between processes, you need to use NUL as delimiter to be safe. Perl: Simple. Use File::Find. Perl handles all the icky stuff. If you have problems, you're doing something wrong (easy with shell, in perl only if you're misusing 'system()' (and it's other forms like qx() and ``)). -dnh -- No trees were destroyed in the sending of this message, however, a significant number of electrons were terribly inconvenienced. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2010-10-29 at 20:22 -0400, Lucky Leavell wrote:
On files/directories with no user, I want to change the user to nobody and likewise for the group. I could run the initial script twice, once to detect unowned files and the other to detect unowned groups and handle each separately. However, on some systems we have terabyte filesystems with millions of files and a single find command can run for days. Thus I would like to take this initial file and use that to correct both problems.
The problem I keep running into is with pathnames with embedded spaces. I have tried everything I can think of using shell scripts, awk and perl and they all have problems with pathnames with embedded spaces.
I simply write the names to a file, one file per line. Then a while loop reads them, one by one: while read FILES ; do WorkOn "$FILES" done < $LISTOFFILES Calling commands with enclosed "$FILES" variable works with spaces. However, you would still traverse the list twice at least, because the attributes are not stored that way, only the names. You could use find with xargs, but you still have to access the disk several times. - -- Cheers, Carlos E. R. (from 11.2 x86_64 "Emerald" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) iEYEARECAAYFAkzMDIYACgkQtTMYHG2NR9WBhwCfTISuBAAlNKnYACWtdvZsOY97 iqwAoIaU6C84WCExp1DekC4f6LDabjut =5Tt6 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Saturday October 30 2010, Carlos E. R. wrote:
...
I simply write the names to a file, one file per line. Then a while loop reads them, one by one:
while read FILES ; do WorkOn "$FILES" done < $LISTOFFILES
I may be remembering the defaults incorrectly, but I'd reset IFS just to be sure spaces don't break the parse: IFS= while read targetFile; do # Do stuff with the file whose name is held in "$targetFile" done <"fileListfile"
Calling commands with enclosed "$FILES" variable works with spaces. However, you would still traverse the list twice at least, because the attributes are not stored that way, only the names.
You could use find with xargs, but you still have to access the disk several times.
Yes and no. In reality, the data structures holding the file system information will be cached by the kernel and making multiple passes over them has fairly low overhead.
-- Cheers, Carlos E. R.
Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Saturday, 2010-10-30 at 10:07 -0700, Randall R Schulz wrote:
On Saturday October 30 2010, Carlos E. R. wrote:
...
I simply write the names to a file, one file per line. Then a while loop reads them, one by one:
while read FILES ; do WorkOn "$FILES" done < $LISTOFFILES
I may be remembering the defaults incorrectly, but I'd reset IFS just to be sure spaces don't break the parse:
IFS= while read targetFile; do # Do stuff with the file whose name is held in "$targetFile" done <"fileListfile"
However, it is working for me :-) I wrote a script the other day to convert video files to .avi, and I'm using a loop like the one I said, sucesfully, without touching IFS. The trick is to surround each use of the variable with quotes. I suppose it will not work with new lines, but I have none of that. There is, however, a strange thing: while read FILES ; do ffmpeg whatever "$FILES" done < $LISTOFFILES only works on the first file and it exits, if there is a working call to ffmpeg. Aparently, ffmpeg somehow disrupts piping. So, instead I have to first read the file list to an array (bash), then loop over the array, not the input file.
You could use find with xargs, but you still have to access the disk several times.
Yes and no. In reality, the data structures holding the file system information will be cached by the kernel and making multiple passes over them has fairly low overhead.
Could be. If the OP says it takes days to run on those huge disks, then I would take the time to create a binary program to do the task. C. Me, pascal. The time to create the program is less than the running time of the script... - -- Cheers, Carlos E. R. (from 11.2 x86_64 "Emerald" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) iEYEARECAAYFAkzQJIQACgkQtTMYHG2NR9WAgACfXuDXHlz4jtL2BajVhlg/Vcne +d8AnRjTFqp+JeTE7sZsEgrd5TgmXdky =9zow -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (6)
-
Carlos E. R. -
Cristian Rodríguez -
David Haller -
Lucky Leavell -
Randall R Schulz -
Rodney Baker