hy people!, the thing is i have an AIX box that downloads things from a ftp and another program that moves everything that's in the directory to another FS the thing is that in aix if you try to cp a file that is in use it says resource is busy but now im migrating the program to suse linux and i find this problem: even if the download hasnt finished the cp or mv starts and my file gets copied or moved in a corrupt state, thats why i was looking for a ftp server that blocked the files until the download has finished . and the prob is i cant change the program to put in a scrip it has to stay as it is!. Help!!
On Tuesday 21 June 2005 17:21, daniel parkes wrote:
hy people!,
the thing is i have an AIX box that downloads things from a ftp and another program that moves everything that's in the directory to another FS the thing is that in aix if you try to cp a file that is in use it says resource is busy but now im migrating the program to suse linux and i find this problem:
even if the download hasnt finished the cp or mv starts and my file gets copied or moved in a corrupt state, thats why i was looking for a ftp server
You mean client, surely
that blocked the files until the download has finished . and the prob is i cant change the program to put in a scrip it has to stay as it is!.
Why? Since you are free to choose another ftp client, why not simply make it a simple script like cd $TMPDIR wget $FILENAME mv $FILENAME $FINAL_DESTINATION_DIR There is no way to "block" files, the only way I can see would be to play with permissions, but if you're free to pick an ftp client, why not do it the easy way?
On 6/21/05, Anders Johansson <andjoh@rydsbo.net> wrote:
On Tuesday 21 June 2005 17:21, daniel parkes wrote:
hy people!,
the thing is i have an AIX box that downloads things from a ftp and another program that moves everything that's in the directory to another FS the thing is that in aix if you try to cp a file that is in use it says resource is busy but now im migrating the program to suse linux and i find this problem:
even if the download hasnt finished the cp or mv starts and my file gets copied or moved in a corrupt state, thats why i was looking for a ftp server
You mean client, surely
that blocked the files until the download has finished . and the prob is i cant change the program to put in a scrip it has to stay as it is!.
Why?
Since you are free to choose another ftp client, why not simply make it a simple script like
cd $TMPDIR wget $FILENAME mv $FILENAME $FINAL_DESTINATION_DIR
There is no way to "block" files, the only way I can see would be to play with permissions, but if you're free to pick an ftp client, why not do it the easy way?
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
If I did get you right, I'd see some more ideas to this problem, as it is not really a new one.. mh.. well, I got no ftp-server in mind which takes care about this, but 1. you could play with "flock" if you are on the local filesystem if you cannot open this file with an exclusive lock, you know the file is still opened for writing (this also causes an exclusive lock). very easy that one, but with the limitation that it's only working on local fs ( ok, and shares .. ), which means you would have to write your own ftp-server, or you could find a workaround somehow.. 2. you could simply check the current filesize with your ftp-client script ( dunno, you mentioned it cannot or should not be altered, that's bad.. ) and keep this information. After a period, let's say 10seconds or so you check it again and compare the new filesize with the filesize from the last check. If they are identically, that means the file has not been changed for a single bit during the last X seconds and could be fetched.. 2. you could try the old fashioned way, and expect a md5-checksum-file for each incoming file.. this is the most secure way for your data, but I fear not the most favorized one. some ideas only.. hope that helps a little, markus
On 6/21/05, Markus Natter <markus.natter@gmail.com> wrote:
On 6/21/05, Anders Johansson <andjoh@rydsbo.net> wrote:
On Tuesday 21 June 2005 17:21, daniel parkes wrote:
hy people!,
the thing is i have an AIX box that downloads things from a ftp and another program that moves everything that's in the directory to another FS the thing is that in aix if you try to cp a file that is in use it says resource is busy but now im migrating the program to suse linux and i find this problem:
even if the download hasnt finished the cp or mv starts and my file gets copied or moved in a corrupt state, thats why i was looking for a ftp server
You mean client, surely
that blocked the files until the download has finished . and the prob is i cant change the program to put in a scrip it has to stay as it is!.
Why?
Since you are free to choose another ftp client, why not simply make it a simple script like
cd $TMPDIR wget $FILENAME mv $FILENAME $FINAL_DESTINATION_DIR
There is no way to "block" files, the only way I can see would be to play with permissions, but if you're free to pick an ftp client, why not do it the easy way?
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
If I did get you right, I'd see some more ideas to this problem, as it is not really a new one.. mh.. well, I got no ftp-server in mind which takes care about this, but
1. you could play with "flock" if you are on the local filesystem if you cannot open this file with an exclusive lock, you know the file is still opened for writing (this also causes an exclusive lock). very easy that one, but with the limitation that it's only working on local fs ( ok, and shares .. ), which means you would have to write your own ftp-server, or you could find a workaround somehow..
2. you could simply check the current filesize with your ftp-client script ( dunno, you mentioned it cannot or should not be altered, that's bad.. ) and keep this information. After a period, let's say 10seconds or so you check it again and compare the new filesize with the filesize from the last check. If they are identically, that means the file has not been changed for a single bit during the last X seconds and could be fetched..
2. you could try the old fashioned way, and expect a md5-checksum-file for each incoming file.. this is the most secure way for your data, but I fear not the most favorized one.
some ideas only..
hope that helps a little,
markus
oops, I forgot one possible solution to meantion.. maybe an important one for you.. if you'd use pure-ftpd as your ftp-server you could try using 2 different users and directories to solve it. pure-ftpd has been compiled with the option "CallUploadScript". If you set this option to yes, it could trigger a script that simply moves the completed upload file from your "upload-directory" to your "download-directory". Put these two directories on the same partition. see /etc/pure-ftpd/pure-ftpd.conf: # If your pure-ftpd has been compiled with pure-uploadscript support, # this will make pure-ftpd write info about new uploads to # /var/run/pure-ftpd.upload.pipe so pure-uploadscript can read it and # spawn a script to handle the upload. #CallUploadScript yes markus
oops, I forgot one possible solution to meantion..
maybe an important one for you..
if you'd use pure-ftpd as your ftp-server you could try using 2 different users and directories to solve it.
2 users and 2 directories?? i just dont seem to get your idea here? it sounds interesting could u please, explain it again?? THNxx!!!! pure-ftpd has been compiled with the option "CallUploadScript".
If you set this option to yes, it could trigger a script that simply moves the completed upload file from your "upload-directory" to your "download-directory". Put these two directories on the same partition.
see /etc/pure-ftpd/pure-ftpd.conf:
# If your pure-ftpd has been compiled with pure-uploadscript support, # this will make pure-ftpd write info about new uploads to # /var/run/pure-ftpd.upload.pipe so pure-uploadscript can read it and # spawn a script to handle the upload.
#CallUploadScript yes
markus
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
On 6/22/05, daniel parkes <liquidsmail@gmail.com> wrote:
oops, I forgot one possible solution to meantion..
maybe an important one for you..
if you'd use pure-ftpd as your ftp-server you could try using 2 different
users
and directories to solve it.
2 users and 2 directories?? i just dont seem to get your idea here?
it sounds interesting could u please, explain it again??
THNxx!!!!
if you'd configure the pure-ftpd to chroot everyone to the users home dir, you could use for example one specific user and its home directory for uploading files (e.g. upload and /home/upload and another user and home directory for downloading files (e.g. download /home/download). The idea I meant behind this is that, when an file upload is in progess, no incomplete file would be downloaded as it does not exist in the download directory, yet. Once the upload to the upload directory has finished, pure-ftpd would trigger a simple move script, which moves the uploaded file to the download user's home directory /home/download/. If both homedirs are on the same harddisk partition the mv command only renames the file (the inode stays the same) and does not copy the file's content itself. So, once the filename appears in the the download user's homedir , you could be shure it's complete. does this help? markus
pure-ftpd has been compiled with the option "CallUploadScript". If you set this option to yes, it could trigger a script that simply moves the completed upload file from your "upload-directory" to your "download-directory". Put these two directories on the same partition.
see /etc/pure-ftpd/pure-ftpd.conf:
# If your pure-ftpd has been compiled with pure-uploadscript support, # this will make pure-ftpd write info about new uploads to # /var/run/pure-ftpd.upload.pipe so pure-uploadscript can read it and # spawn a script to handle the upload.
#CallUploadScript yes
markus
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
HY thnx for your answers
server
You mean client, surely
Is there a free client with this option? .
Since you are free to choose another ftp client, why not simply make it a simple script like
cd $TMPDIR wget $FILENAME mv $FILENAME $FINAL_DESTINATION_DIR
But with this script where should i need to have the authentication to the ftpserver? in the wget sentence? Thnx alot!!! great help!
On Wednesday 22 June 2005 10:19, daniel parkes wrote:
cd $TMPDIR wget $FILENAME mv $FILENAME $FINAL_DESTINATION_DIR
But with this script where should i need to have the authentication to the ftpserver? in the wget sentence?
Yes, exactly. For example wget ftp://username:password@server/filename
Yes i hav something like this, anything you can think of to make it look better? #!/bin/bash TEMPDIR=/tmp FINALDIR=/root FILE=$1 cd $TEMPDIR wget -o /tmp/wget.ftp.log ftp://user:server@box/$FILE mv $FILE $FINALDIR thnX again!. On 22/06/05, Anders Johansson <andjoh@rydsbo.net> wrote:
cd $TMPDIR wget $FILENAME mv $FILENAME $FINAL_DESTINATION_DIR
But with this script where should i need to have the authentication to
On Wednesday 22 June 2005 10:19, daniel parkes wrote: the
ftpserver? in the wget sentence?
Yes, exactly. For example
wget ftp://username:password@server/filename
On Wed, 22 Jun 2005 15:32:02 +0200 daniel parkes <.> wrote:
Yes i hav something like this, anything you can think of to make it look better?
#!/bin/bash TEMPDIR=/tmp FINALDIR=/root FILE=$1 cd $TEMPDIR wget -o /tmp/wget.ftp.log ftp://user:server@box/$FILE mv $FILE $FINALDIR
#!/bin/sh FILE=$1 FINALDIR=/root TEMPDIR=/tmp wget -p $TEMPDIR -o $TEMPDIR/wget.ftp.log ftp://user:server@box/$FILE mv $TEMPDIR/$FILE $FINALDIR
participants (4)
-
Anders Johansson -
daniel parkes -
Markus Natter -
pelibali