[opensuse] Another Bash scripting question
Hi all. I have a few directories containing wav files that have been ripped from CDs and I'm trying to write a script to invoke sox to convert them to mp3 format. The filenames have whitespaces and hyphens in them e.g. 'artist - 01 - title with many words.wav'. I tried the following script but I'm obviously doing something wrong... --------------------------------------------------------------------------------------------------------- #!/bin/bash for file in `find * -print0 | xargs --null` do { INFILE=$file OUTFILE=${file/%wav/mp3} echo "$OUTFILE" # if [ ! -e "$file" ] # then # echo "$file does not exist."; echo # else # sox "$file" "$OUTFILE" # fi # if [ $? -eq 0 ] # then # rm -f "$file" # fi } done exit 0 ------------------------------------------------------------------------------------------------- In its current form each word of each filename is treated separately i.e. xargs --null appears to have no effect at all - it is interpreting whitespace as a delimiter. If I modify the first line as follows: for file in "`find * -print0 | xargs --null`" or for file in "$(find * -print0 | xargs --null)" then I get the entire directory listing (i.e. all the filenames) concatenated into a single parameter with only the last file extension replaced. What have I missed here? I'm assuming it is something to do with the pipe or redirection via xargs that is not working as expected. BTW, I also tried this: for file in $(xargs --null -- < `find * -print0`) but then I get an "ambiguous redirect" error. Using this: for file in `xargs --null -- < `find * -print0`` gives the following errors: command substitution: line 26: syntax error near unexpected token `newline' command substitution: line 26: `xargs --null -- < ' but then produces the output I'm expecting (i.e. a list of the output file names (keeping in mind the actual processing part of the script is currently commented out) with the exception that "find" is the first entry in the list and "-print0" is the last entry in the list; somehow the variable $OUTFILE appears to be getting the values "find" and "-print0" at the beginning and end of the loop. I seem to be just going around in circles here. Any suggestions will be greatly appreciated. Thanks in advance, -- =================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au =================================================== I have great faith in fools -- self confidence my friends call it. -- Edgar Allan Poe
On Saturday 17 May 2008 06:33, Rodney Baker wrote:
Hi all.
I have a few directories containing wav files that have been ripped from CDs and I'm trying to write a script to invoke sox to convert them to mp3 format.
The filenames have whitespaces and hyphens in them e.g. 'artist - 01 - title with many words.wav'. I tried the following script but I'm obviously doing something wrong...
#!/bin/bash
for file in `find * -print0 | xargs --null`
The way you're using xargs is essentially a no-op here, since the output is no different than if you skipped the xargs part. However, if you put the action code into a separately executable script file that takes multiple arguments and gave that command to xargs, you'd have what you want. So if your script was something like this: -==--==--==--==--==--==--==--==--==--==--==--==--==--==--==- #!/bin/bash --norc for arg; do # Do stuff with "$arg" (always quote it!) done -==--==--==--==--==--==--==--==--==--==--==--==--==--==--==- Then just invoke it like this: % for file in $(find * -print0 | xargs --null scriptName)
do { ... } done
By the way, these braces are redundant.
...
I seem to be just going around in circles here. Any suggestions will be greatly appreciated.
Thanks in advance,
Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sat, 17 May 2008 23:30:51 Randall R Schulz wrote:
[...]
#!/bin/bash
for file in `find * -print0 | xargs --null`
The way you're using xargs is essentially a no-op here, since the output is no different than if you skipped the xargs part.
However, if you put the action code into a separately executable script file that takes multiple arguments and gave that command to xargs, you'd have what you want.
So if your script was something like this:
-==--==--==--==--==--==--==--==--==--==--==--==--==--==--==- #!/bin/bash --norc
for arg; do
# Do stuff with "$arg" (always quote it!)
done -==--==--==--==--==--==--==--==--==--==--==--==--==--==--==-
Then just invoke it like this:
% for file in $(find * -print0 | xargs --null scriptName)
do { ... } done
By the way, these braces are redundant.
Thanks, Randall. That does make sense. Incidentally, using the following line: for file in `xargs --null < `find * -print0`` despite errors mentioned in my first post, the script actually does run and do exactly what I want it to do. Of course when it tries to operate on "find" and "-print0" it generates an error and continues on to the other files in the directory. That must be something to do with the nested command substitution... Anyway, thanks for your help. I'm always amazed the stuff one can learn on this list. Regards, Rodney. -- =================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ===================================================
On Sat, 17 May 2008 23:30:51 Randall R Schulz wrote:
[...]
So if your script was something like this:
-==--==--==--==--==--==--==--==--==--==--==--==--==--==--==- #!/bin/bash --norc
for arg; do
# Do stuff with "$arg" (always quote it!)
done -==--==--==--==--==--==--==--==--==--==--==--==--==--==--==-
Then just invoke it like this:
% for file in $(find * -print0 | xargs --null scriptName)
Randall, Further to my previous reply, I've changed it as per your suggestion and setup an alias for the above command line. It works a treat. I figure that it will be fairly easy now to add a coupe of extra parameters to the script so that I can specify the input and output formats rather than having them hardcoded; however that will be a project for another day. Time for bed now. Thanks again, Rodney. -- =================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au =================================================== Indifference will be the downfall of mankind, but who cares?
Rodney Baker wrote:
Hi all.
I have a few directories containing wav files that have been ripped from CDs and I'm trying to write a script to invoke sox to convert them to mp3 format.
The filenames have whitespaces and hyphens in them e.g. 'artist - 01 - title with many words.wav'. I tried the following script but I'm obviously doing something wrong...
--------------------------------------------------------------------------------------------------------- #!/bin/bash
for file in `find * -print0 | xargs --null`
do { INFILE=$file OUTFILE=${file/%wav/mp3}
^^^^^^^^^^^^^^^^^ WHAT????? Clue for the clueless... If you had bothered to COMMENT YOUR CODE, you would have discovered this error right off the bat. And if you get in the habit of writing comments BEFORE YOU WRITE THE CODE, you'll discover that your rate of bug production drops drastically (both in bugs/hour and also in bugs/lines of code).
echo "$OUTFILE"
# if [ ! -e "$file" ] # then # echo "$file does not exist."; echo
# else
# sox "$file" "$OUTFILE"
# fi
# if [ $? -eq 0 ] # then # rm -f "$file" # fi } done
exit 0 -------------------------------------------------------------------------------------------------
In its current form each word of each filename is treated separately i.e. xargs --null appears to have no effect at all - it is interpreting whitespace as a delimiter.
If I modify the first line as follows:
for file in "`find * -print0 | xargs --null`"
or
for file in "$(find * -print0 | xargs --null)"
then I get the entire directory listing (i.e. all the filenames) concatenated into a single parameter with only the last file extension replaced.
What have I missed here? I'm assuming it is something to do with the pipe or redirection via xargs that is not working as expected.
BTW, I also tried this:
for file in $(xargs --null -- < `find * -print0`)
but then I get an "ambiguous redirect" error.
Using this:
for file in `xargs --null -- < `find * -print0``
gives the following errors: command substitution: line 26: syntax error near unexpected token `newline' command substitution: line 26: `xargs --null -- < '
but then produces the output I'm expecting (i.e. a list of the output file names (keeping in mind the actual processing part of the script is currently commented out) with the exception that "find" is the first entry in the list and "-print0" is the last entry in the list; somehow the variable $OUTFILE appears to be getting the values "find" and "-print0" at the beginning and end of the loop.
I seem to be just going around in circles here. Any suggestions will be greatly appreciated.
Thanks in advance,
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sun, 18 May 2008, Sam Clemens wrote:-
Rodney Baker wrote:
OUTFILE=${file/%wav/mp3}
^^^^^^^^^^^^^^^^^ WHAT?????
Clue for the clueless...
Oh dear. That was a silly thing to do. I really do hope that you washed your feet recently because, if not, that's going to leave a bad taste behind if you haven't.
If you had bothered to COMMENT YOUR CODE, you would have discovered this error right off the bat.
What exactly is the error with that line? Do you have any idea what it does? Have you tried it to find out? I'm guessing you haven't because, if you had, you wouldn't have mentioned it. Here's an example of what it does: davjam@donnas:~> i="this is a test to see if the expression \${i/%wav/mp3} correctly removes a trailing.wav" davjam@donnas:~> echo "${i}" this is a test to see if the expression ${i/%wav/mp3} correctly removes a trailing.wav davjam@donnas:~> echo "${i/wav/mp3}" this is a test to see if the expression ${i/%mp3/mp3} correctly removes a trailing.wav davjam@donnas:~> echo "${i/%wav/mp3}" this is a test to see if the expression ${i/%wav/mp3} correctly removes a trailing.mp3 As you can see, it uses the string replacement function of bash, where ${var/something/else} looks at the contents of ${var} and replaces the first instance of "something" with "else". If ${var//something/else} was used instead, all instances of "something" would be replaced by "else". The addition of the '%' slightly modifies the behaviour so ${var/%something/else} looks for the last instance of "something" and replaces it with "else".
And if you get in the habit of writing comments BEFORE YOU WRITE THE CODE, you'll discover that your rate of bug production drops drastically (both in bugs/hour and also in bugs/lines of code).
Well, one bug I spotted was that there's at least one major problem that goes unchecked. What happens if ${INFILE} == ${OUTFILE}? I'd hazard a guess that sox would open ${file}[0] to read, then open ${OUTFILE} for writing, then see ${file} has just become a 0-length file and terminate. That sort of problem is easily fixed with a simple addition of: # skip if input and output files are the same # [ "${file}" == "${OUTFILE}" ] && continue to the loop. Another question is, is the script supposed to only process the files in the current directory, or all sub-directories as well. If it's just the current directory, it would be much simpler to use: # look for files only within the current directory # find -maxdepth 1 -type f -name "*.wav" | \ while read file do If it's the latter, using: # look for files within the current directory, and any sub-directories # find -type f -name "*.wav" | \ while read file do would ensure that only .wav files are processed, both of which removes the requirement for the "${file}" == "${OUTFILE}" check, and also makes sure that nothing happens when there are no files to process. The only time this is going to break is when a filename contains an ASCII character 0x0a, or line feed. However, anyone stupid enough to include one of those in a filename should expect breakage and can figure out how to fix the script to handle those files themselves. Regards, David Bolt -- Team Acorn: http://www.distributed.net/ OGR-P2 @ ~100Mnodes RC5-72 @ ~15Mkeys SUSE 10.1 32bit | openSUSE 10.2 32bit | openSUSE 10.3 32bit | openSUSE 11.0b1 SUSE 10.1 64bit | openSUSE 10.2 64bit | openSUSE 10.3 64bit RISC OS 3.6 | TOS 4.02 | openSUSE 10.3 PPC | RISC OS 3.11 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Monday 19 May 2008 02:47, David Bolt wrote:
...
Well, one bug I spotted was that there's at least one major problem that goes unchecked. What happens if ${INFILE} == ${OUTFILE}? I'd hazard a guess that sox would open ${file}[0] to read, then open ${OUTFILE} for writing, then see ${file} has just become a 0-length file and terminate. That sort of problem is easily fixed with a simple addition of:
# skip if input and output files are the same # [ "${file}" == "${OUTFILE}" ] && continue
to the loop.
If you want to make it really bulletproof and consider the kind of aliasing that could come about through hard or symbolic links, you could test for sameness by using the "stat" command and testing for equality of both the device and inode numbers: [ "$(stat -c '%D-%i' "$file")" == "$(stat -c '%D-%i' "$OUTFILE")" ] \ && continue
...
The only time this is going to break is when a filename contains an ASCII character 0x0a, or line feed. However, anyone stupid enough to include one of those in a filename should expect breakage and can figure out how to fix the script to handle those files themselves.
Well, that's your prerogative, but it's not too hard to write scripts with the necessary precautions.
Regards, David Bolt
Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Mon, 2008-05-19 at 10:47 +0100, David Bolt wrote:
# skip if input and output files are the same # [ "${file}" == "${OUTFILE}" ] && continue
Thanks for that bit it was just what I also needed to fix my script which strips spaces from names. ~/MyBash/test> cat RemoveSpaceInName.sh # +------------------------------------+ # | Loop over all files in a directory | # +------------------------------------+ # +--------------------------+ # | $i = current filename | # | remove blanks | # | emit the result | # +--------------------------+ for i in * ; do # echo $i NewName=`echo $i| tr -d ' '` # echo $NewName # skip if input and output files are the same # [ "${i}" == "${NewName}" ] && continue mv "$i" "$NewName" done -- _______ _______ _______ __ / ____\ \ / / ____|_ _\ \ / / | | \ \ /\ / / (___ | | \ \ / / | | \ \/ \/ / \___ \ | | \ \/ / | |____ \ /\ / ____) |_| |_ \ / \_____| \/ \/ |_____/|_____| \/ | \ /|\ || |\ / |~~\ /~~\ /~~| //~~\ | \ / | \ || | X |__/| || |( `--. |__ | | \| \_/ / \ | \ \__/ \__| \\__/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Fri, 11 Jul 2008, Carl Spitzer wrote:- <snip> You can get the same results as this:
NewName=`echo $i| tr -d ' '`
Without having to use pipes by using: NewName="${i// /}" The construct: ${VARIABLE/<STRING>/<REPLACEMENT>} replaces only the first instance of <STRING> with <REPLACEMENT>. The construct: ${VARIABLE//<STRING>/<REPLACEMENT>} does almost the same, but will replace all instances of <STRING> with <REPLACEMENT>. Regards, David Bolt -- Team Acorn: http://www.distributed.net/ OGR-P2 @ ~100Mnodes RC5-72 @ ~15Mkeys SUSE 10.1 32 | | openSUSE 10.3 32bit | openSUSE 11.0 32bit | openSUSE 10.2 64bit | openSUSE 10.3 64bit | openSUSE 11.0 64bit RISC OS 3.6 | TOS 4.02 | openSUSE 10.3 PPC | RISC OS 3.11 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (5)
-
Carl Spitzer
-
David Bolt
-
Randall R Schulz
-
Rodney Baker
-
Sam Clemens