Hello all! I have a small script to convert and resize images from my digital camera. #!/bin/bash ############################################################### # Converting Pentax .pef and .jpg files to 600x400 size jpegs # and store them in ./preview folder # Script must be run in the directory of the picturefolder you want to # convert from. # Utils used: 'dcraw', 'exiftools', 'ImageMagick' ############################################################### # # ToDO: Clean up the code. # Call with options or paths (from other scripts) # ############################################################### # # Tell us how many files we will convert. # echo `ls |wc -l` "files to convert" echo "creating ./preview folder" mkdir ./preview # Loop, convert and resize .pef files for infile in `ls *.pef |cut -d. -f1` # Find file name w/o extension do echo "converting RAW "$infile # Use dcraw to convert the RAW file and pipe to STD_OUT. # Grab the pipe directly into convert. dcraw -w -h -c $infile.pef |convert -resize 600x400 - ./preview/$infile.jpg # Use exiftool to write the metatags exiftool -q -overwrite_original -tagsfromfile $infile.pef ./preview/$infile.jpg done # Loop and convert (resize) the jpg's for infile in `ls *.jpg |cut -d. -f1` do echo "converting jpeg "$infile # No need to do anything but resize. convert -resize 600x400 $infile.jpg ./preview/$infile.jpg done ########################################## Is there anyway to thread this into multiple pipes?, so i can tell the script to convert two or more images at the time? Prefarably with a switch to set the number of threads. (ie. foo -n 3 to convert 3 images in parallell) -- /Rikard ----------------------------------------------------------------------------- email : rikard.j@rikjoh.com web : http://www.rikjoh.com mob : +46 (0)736 19 76 25 ------------------------ Public PGP fingerprint ---------------------------- < 15 28 DF 78 67 98 B2 16 1F D3 FD C5 59 D4 B6 78 46 1C EE 56 >
Rikard Johnels wrote:
Hello all!
Is there anyway to thread this into multiple pipes?, so i can tell the script to convert two or more images at the time?
Add '&' to start a separate process. Here's an example that might inspire you: http://www.mail-archive.com/linux-raid@vger.kernel.org/msg00745.html This script starts multiple threads to stress-test a file-system or disk-array. /Per Jessen, Zürich -- http://www.spamchek.com/ - managed anti-spam and anti-virus solution.
Rikard, On Thursday 22 December 2005 00:11, Rikard Johnels wrote:
Hello all!
I have a small script to convert and resize images from my digital camera.
#!/bin/bash ... ##########################################
Is there anyway to thread this into multiple pipes?, so i can tell the script to convert two or more images at the time? Prefarably with a switch to set the number of threads. (ie. foo -n 3 to convert 3 images in parallell)
Since the computation (image resizing / resampling) is probably CPU-bound, multiple threads are not likely to speed the overall completion much, if any, unless you have multiple processors.
-- /Rikard
Randall Schulz
On Thursday 22 December 2005 15:14, Randall R Schulz wrote:
Rikard,
On Thursday 22 December 2005 00:11, Rikard Johnels wrote:
Hello all!
I have a small script to convert and resize images from my digital camera.
#!/bin/bash ... ##########################################
Is there anyway to thread this into multiple pipes?, so i can tell the script to convert two or more images at the time? Prefarably with a switch to set the number of threads. (ie. foo -n 3 to convert 3 images in parallell)
Since the computation (image resizing / resampling) is probably CPU-bound, multiple threads are not likely to speed the overall completion much, if any, unless you have multiple processors.
-- /Rikard
Randall Schulz
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
Hehe.. Thats the point :) I DO have a dual CPU machine. (SGI Zx10 Dual PIII 1 Ghz) Thats why i was curious. Also to TRY to learn how to thread scripts in the first place.. -- /Rikard ----------------------------------------------------------------------------- email : rikard.j@rikjoh.com web : http://www.rikjoh.com mob : +46 (0)736 19 76 25 ------------------------ Public PGP fingerprint ---------------------------- < 15 28 DF 78 67 98 B2 16 1F D3 FD C5 59 D4 B6 78 46 1C EE 56 >
On 12/22/05, Rikard Johnels <rikard.j@rikjoh.com> wrote:
On Thursday 22 December 2005 15:14, Randall R Schulz wrote:
Rikard,
On Thursday 22 December 2005 00:11, Rikard Johnels wrote:
Hello all!
I have a small script to convert and resize images from my digital camera.
#!/bin/bash ... ##########################################
Is there anyway to thread this into multiple pipes?, so i can tell the script to convert two or more images at the time? Prefarably with a switch to set the number of threads. (ie. foo -n 3 to convert 3 images in parallell)
Since the computation (image resizing / resampling) is probably CPU-bound, multiple threads are not likely to speed the overall completion much, if any, unless you have multiple processors.
-- /Rikard
Randall Schulz
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
Hehe.. Thats the point :) I DO have a dual CPU machine. (SGI Zx10 Dual PIII 1 Ghz) Thats why i was curious. Also to TRY to learn how to thread scripts in the first place..
-- /Rikard
just call the processing command (I can imagine you use ImageMagic commands to process your images) with &, so you place them in background: convert -x -x -x & Or make the script to call itself with & and no threading option. Ie. lets say you run the script with myscript -t 3 (3 threads) pseudo code follows: if (OPTION_T is present) FOR (1 to NRTREADS) myscript & ELSE convert pictures .... ########## That way you will have the requested nr. ob background processes doing the job. I'm not a script expert, so it may be inpossible to recurevly call the same script, but then you can always create a wrapper. Cheers -- -- Svetoslav Milenov (Sunny)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Thursday 2005-12-22 at 09:59 -0600, Sunny wrote:
pseudo code follows:
if (OPTION_T is present) FOR (1 to NRTREADS) myscript & ELSE convert pictures ....
(myscript could be a bash function, which is simpler than calling a script) Yes, but you have to devise a method by which each thread picks a file to convert from the list, and make sure that no other thread has picked that one previously. That's not so simple. - -- Cheers, Carlos Robinson -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFDqwmktTMYHG2NR9URAmPdAJoDVg2CY7QRKjXOi6EvQwEjin5ANACfR/kN Sxt6HhI9OmvYum9Wl62E9rM= =LOxZ -----END PGP SIGNATURE-----
On 12/22/05, Carlos E. R. <robin1.listas@tiscali.es> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
The Thursday 2005-12-22 at 09:59 -0600, Sunny wrote:
pseudo code follows:
if (OPTION_T is present) FOR (1 to NRTREADS) myscript & ELSE convert pictures ....
(myscript could be a bash function, which is simpler than calling a script)
Yes, but you have to devise a method by which each thread picks a file to convert from the list, and make sure that no other thread has picked that one previously. That's not so simple.
- -- Cheers, Carlos Robinson
Thread synchronization is not simple anywhere (even in java and c#). As I said, I'm not script expert, but I do recall a while ago I have read some article or post about possible solutions. Not so efficient, but workable solution will be to use "wait". Also, there are workarounds. You can always split the list of the images by nr. of the threads. Or, you may use use another scripting language, like python. It has threads build-in as far as I know. -- -- Svetoslav Milenov (Sunny)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Thursday 2005-12-22 at 14:38 -0600, Sunny wrote:
Yes, but you have to devise a method by which each thread picks a file to convert from the list, and make sure that no other thread has picked that one previously. That's not so simple.
Thread synchronization is not simple anywhere (even in java and c#). As I said, I'm not script expert, but I do recall a while ago I have read some article or post about possible solutions. Not so efficient, but workable solution will be to use "wait".
Yes, I use that one, but in this case would be inefficient. It waits till "backgrounded" tasks returns; when they do, you can launch another batch.
Also, there are workarounds. You can always split the list of the images by nr. of the threads.
Yes, that's what I thought. But if one finishes earlier than another, its waisting resources. My idea would be to have another function that would return the name of the next file to process; but it would be needed to ensure that it can not be called twice simultaneously, or in other words, be "atomic". Semaphores in bash?
Or, you may use use another scripting language, like python. It has threads build-in as far as I know.
Yeap. But I don't know python ;-) Ah! Found it: NAME lockfile - conditional semaphore-file creator SYNOPSIS lockfile -sleeptime | -r retries | -l locktimeout | -s suspend | -! | -ml | -mu | filename ... function next_file_name { lockfile important.lock get next file from list rm -f important.lock } [....] No, it doesn't work. See my test code: #!/bin/bash NOMBRE=1 function next_file_name { lockfile important.lock NOMBRE=`expr $NOMBRE + 1` echo $NOMBRE rm -f important.lock } function proceso { next_file_name } proceso & proceso & proceso & proceso & wait The result is: cer@nimrodel:~> probar_thread 2 2 <---- delay here or next line, some times. 2 2 cer@nimrodel:~> The expected result (no threading) would be: cer@nimrodel:~> probar_thread 2 3 4 5 cer@nimrodel:~> Obviously, the lock file does not impede the function from entering again. It doesn't "lock". Or there is something else in play: perhaps the variables are read before entering the function. - -- Cheers, Carlos Robinson -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFDqzo9tTMYHG2NR9URAkSaAJ4tPlbk/lo53Ll30Vmtb71d6f3mAACeMArw QLJhiAnEW8RlqoKyfGxTgHM= =M9zx -----END PGP SIGNATURE-----
Rikard Johnels <rikard.j@rikjoh.com> writes:
I have a small script to convert and resize images from my digital camera. ... Is there anyway to thread this into multiple pipes?, so i can tell the script to convert two or more images at the time?
I use "make -j2" for two jobs and "make -jN" in general to do the job. It's both simple and robust. The only problem is that you must know how to write a short Makefile. I don't have time to write a working example now. Just a hint: Makefile: RAW = # see the make documentation how to get a list off all RAW files # in the current directory JPG = $(RAW:.raw=.jpg) .PHONY : all all : $(JPG) %.jpg : %.raw conversion_script $< $@ BTW, a compilation (file.c -> file.o) is a similar problem. -- A.M.
participants (6)
-
Alexandr Malusek -
Carlos E. R. -
Per Jessen -
Randall R Schulz -
Rikard Johnels -
Sunny