Mailinglist Archive: opensuse (3336 mails)
| < Previous | Next > |
Re: [SLE] Threading scripts?
- From: "Carlos E. R." <robin1.listas@xxxxxxxxxx>
- Date: Fri, 23 Dec 2005 00:43:47 +0100 (CET)
- Message-id: <Pine.LNX.4.61.0512222343120.4608@xxxxxxxxxxxxxxxx>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
The Thursday 2005-12-22 at 14:38 -0600, Sunny wrote:
> > Yes, but you have to devise a method by which each thread picks a file to
> > convert from the list, and make sure that no other thread has picked that
> > one previously. That's not so simple.
>
> Thread synchronization is not simple anywhere (even in java and c#).
> As I said, I'm not script expert, but I do recall a while ago I have
> read some article or post about possible solutions. Not so efficient,
> but workable solution will be to use "wait".
Yes, I use that one, but in this case would be inefficient. It waits till
"backgrounded" tasks returns; when they do, you can launch another batch.
> Also, there are workarounds. You can always split the list of the
> images by nr. of the threads.
Yes, that's what I thought. But if one finishes earlier than another, its
waisting resources.
My idea would be to have another function that would return the name of
the next file to process; but it would be needed to ensure that it can not
be called twice simultaneously, or in other words, be "atomic". Semaphores
in bash?
> Or, you may use use another scripting
> language, like python. It has threads build-in as far as I know.
Yeap. But I don't know python ;-)
Ah! Found it:
NAME
lockfile - conditional semaphore-file creator
SYNOPSIS
lockfile -sleeptime | -r retries |
-l locktimeout | -s suspend | -! | -ml | -mu | filename ...
function next_file_name
{
lockfile important.lock
get next file from list
rm -f important.lock
}
[....]
No, it doesn't work. See my test code:
#!/bin/bash
NOMBRE=1
function next_file_name
{
lockfile important.lock
NOMBRE=`expr $NOMBRE + 1`
echo $NOMBRE
rm -f important.lock
}
function proceso
{
next_file_name
}
proceso &
proceso &
proceso &
proceso &
wait
The result is:
cer@nimrodel:~> probar_thread
2
2 <---- delay here or next line, some times.
2
2
cer@nimrodel:~>
The expected result (no threading) would be:
cer@nimrodel:~> probar_thread
2
3
4
5
cer@nimrodel:~>
Obviously, the lock file does not impede the function from entering again.
It doesn't "lock". Or there is something else in play: perhaps the
variables are read before entering the function.
- --
Cheers,
Carlos Robinson
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Made with pgp4pine 1.76
iD8DBQFDqzo9tTMYHG2NR9URAkSaAJ4tPlbk/lo53Ll30Vmtb71d6f3mAACeMArw
QLJhiAnEW8RlqoKyfGxTgHM=
=M9zx
-----END PGP SIGNATURE-----
Hash: SHA1
The Thursday 2005-12-22 at 14:38 -0600, Sunny wrote:
> > Yes, but you have to devise a method by which each thread picks a file to
> > convert from the list, and make sure that no other thread has picked that
> > one previously. That's not so simple.
>
> Thread synchronization is not simple anywhere (even in java and c#).
> As I said, I'm not script expert, but I do recall a while ago I have
> read some article or post about possible solutions. Not so efficient,
> but workable solution will be to use "wait".
Yes, I use that one, but in this case would be inefficient. It waits till
"backgrounded" tasks returns; when they do, you can launch another batch.
> Also, there are workarounds. You can always split the list of the
> images by nr. of the threads.
Yes, that's what I thought. But if one finishes earlier than another, its
waisting resources.
My idea would be to have another function that would return the name of
the next file to process; but it would be needed to ensure that it can not
be called twice simultaneously, or in other words, be "atomic". Semaphores
in bash?
> Or, you may use use another scripting
> language, like python. It has threads build-in as far as I know.
Yeap. But I don't know python ;-)
Ah! Found it:
NAME
lockfile - conditional semaphore-file creator
SYNOPSIS
lockfile -sleeptime | -r retries |
-l locktimeout | -s suspend | -! | -ml | -mu | filename ...
function next_file_name
{
lockfile important.lock
get next file from list
rm -f important.lock
}
[....]
No, it doesn't work. See my test code:
#!/bin/bash
NOMBRE=1
function next_file_name
{
lockfile important.lock
NOMBRE=`expr $NOMBRE + 1`
echo $NOMBRE
rm -f important.lock
}
function proceso
{
next_file_name
}
proceso &
proceso &
proceso &
proceso &
wait
The result is:
cer@nimrodel:~> probar_thread
2
2 <---- delay here or next line, some times.
2
2
cer@nimrodel:~>
The expected result (no threading) would be:
cer@nimrodel:~> probar_thread
2
3
4
5
cer@nimrodel:~>
Obviously, the lock file does not impede the function from entering again.
It doesn't "lock". Or there is something else in play: perhaps the
variables are read before entering the function.
- --
Cheers,
Carlos Robinson
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Made with pgp4pine 1.76
iD8DBQFDqzo9tTMYHG2NR9URAkSaAJ4tPlbk/lo53Ll30Vmtb71d6f3mAACeMArw
QLJhiAnEW8RlqoKyfGxTgHM=
=M9zx
-----END PGP SIGNATURE-----
| < Previous | Next > |