[opensuse] ssh transfer reliability for big files when using WiFi
I have seen recently a similar topic about the best ways to transfer files between two Linux boxes. My experience with two PCs with openSUSE 11.1 connected through a wireless router (802.11g) showed that the ssh protocol is not reliable enough to transfer big files (~250 MB). The ssh protocol seems to die after some time. Resuming the transfer is not possible, the entire file being downloaded from the beginning. Do you have experienced similar problems? Is there a way to solve this issue ? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Bogdan Cristea pecked at the keyboard and wrote:
I have seen recently a similar topic about the best ways to transfer files between two Linux boxes. My experience with two PCs with openSUSE 11.1 connected through a wireless router (802.11g) showed that the ssh protocol is not reliable enough to transfer big files (~250 MB). The ssh protocol seems to die after some time. Resuming the transfer is not possible, the entire file being downloaded from the beginning. Do you have experienced similar problems? Is there a way to solve this issue ?
Use rsync with compression -z. -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Ken Schneider - openSUSE wrote:
Bogdan Cristea pecked at the keyboard and wrote:
I have seen recently a similar topic about the best ways to transfer files between two Linux boxes. My experience with two PCs with openSUSE 11.1 connected through a wireless router (802.11g) showed that the ssh protocol is not reliable enough to transfer big files (~250 MB). The ssh protocol seems to die after some time. Resuming the transfer is not possible, the entire file being downloaded from the beginning. Do you have experienced similar problems? Is there a way to solve this issue ?
Use rsync with compression -z.
Unless the files are already compressed. /Per -- Per Jessen, Zürich (23.4°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Ken Schneider - openSUSE wrote:
Bogdan Cristea pecked at the keyboard and wrote:
I have seen recently a similar topic about the best ways to transfer files between two Linux boxes. My experience with two PCs with openSUSE 11.1 connected through a wireless router (802.11g) showed that the ssh protocol is not reliable enough to transfer big files (~250 MB). The ssh protocol seems to die after some time. Resuming the transfer is not possible, the entire file being downloaded from the beginning.
Use rsync with compression -z.
From my experience, ssh (rsh) is reliable for very large files. I routinely
Doesn't rsync usually use ssh for transport? Ssh also compresses by default, I don't think you can compress a compress data stream and gain anything. Does rsync turn off ssh compression when told to compress on its own? transfer multiple gigabyte-length files on a local network. I recall having problems with WiFi dropouts until I replaced the router. I think Bogdan may just be seeing a flaky router. I've had good luck with my current D-Link DIR-655. Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Lew Wolfgang wrote:
Ken Schneider - openSUSE wrote:
Use rsync with compression -z.
Doesn't rsync usually use ssh for transport? Ssh also compresses by default,
It does?? That's news to me. I don't think mine does.
From my experience, ssh (rsh) is reliable for very large files. I routinely transfer multiple gigabyte-length files on a local network.
Every night, I transfer about two gigs over the internet from Germany. No probs. /Per -- Per Jessen, Zürich (22.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, 11 Aug 2009 17:56:38 +0200, Per Jessen wrote:
Lew Wolfgang wrote:
Ken Schneider - openSUSE wrote:
Use rsync with compression -z.
Doesn't rsync usually use ssh for transport? Ssh also compresses by default,
It does?? That's news to me. I don't think mine does.
It doesn't here, either, unless I add "Compression yes" to the .ssh/ config file for the connection in question. There is an option in /etc/ ssh_config to enable it, but the default is "no".
From my experience, ssh (rsh) is reliable for very large files. I routinely transfer multiple gigabyte-length files on a local network.
Every night, I transfer about two gigs over the internet from Germany. No probs.
Same here, but I have found that ssh is really touchy when it comes to lost packets. I used to tunnel VNC over SSH for a remote connection with a flakey NIC, and the ssh session was always dropping. I'd suggest that Bogdan might look to see if there's any packet loss taking place, because that would very likely cause a problem. Jim -- Jim Henderson Please keep on-topic replies on the list so everyone benefits -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Jim Henderson a écrit :
Same here, but I have found that ssh is really touchy when it comes to lost packets.
yes. I had problems with a hosted server, one byte lost every 2Gb approx! ssh (scp) stopped immediately. Seemed to be a link problem, the hosting company changed the cable and the switch and now it works (1 week investigation!) jdd -- http://www.dodin.net http://valerie.dodin.org http://news.opensuse.org/2009/04/13/people-of-opensuse-jean-daniel-dodin/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tuesday 11 August 2009 19:23:12 Jim Henderson wrote:
Same here, but I have found that ssh is really touchy when it comes to lost packets. I used to tunnel VNC over SSH for a remote connection with a flakey NIC, and the ssh session was always dropping. I'd suggest that Bogdan might look to see if there's any packet loss taking place, because that would very likely cause a problem.
The ssh connection seems to be unreliable only when using WiFi, wired connection works much better and faster. Using the same WiFi connection I am able to transfer several gigs using the samba client from a Win XP server. That's why my feeling was that is something wrong with the ssh client/server from openSUSE. I'll try to follow Dave suggestions and to investigate more carefully this issue. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, 11 Aug 2009 19:42:44 +0300, Bogdan Cristea wrote:
Using the same WiFi connection I am able to transfer several gigs using the samba client from a Win XP server. That's why my feeling was that is something wrong with the ssh client/server from openSUSE.
I don't think it's that there's something wrong with openSSH (per se), I think it's that Samba has better recovery mechanisms. It may be that you're seeing decreased performance from Samba as well due to retries. Jim -- Jim Henderson Please keep on-topic replies on the list so everyone benefits -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, Aug 11, 2009 at 12:42 PM, Bogdan Cristea
The ssh connection seems to be unreliable only when using WiFi, wired connection works much better and faster.
I can get 10-11MB/s with wired and only 1-2.5MB/s with wireless(and that's with the laptop right beside the router). I've never gotten anywhere near the 5.4MB/s max of a wifi connection.
Using the same WiFi connection I am able to transfer several gigs using the samba client from a Win XP server. That's why my feeling was that is something wrong with the ssh client/server from openSUSE.
What type of router are you using? Wireless card? If you are using the ath5k driver for atheros based cards, I've found that it is very unreliable. I always build the madwifi driver because it's stable and gets much higher throughput(as well as actually making the lights on my pcmcia card work properly :-) ). My main router is a Netgear WGR416 v6. It's flakey on occasion, but better than my WGT624 v2(which drops at least once a day). I have a WRT54G router(flashed with DD-WRT), but it's being used as a wireless bridge until I get the time to actually run some wires. The stock firmware on the WRT54G was unreliable. I was gonna pitch it until I realized I could flash it. Been 100% rock solid since. Good luck. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Bogdan Cristea wrote:
My experience with two PCs with openSUSE 11.1 connected through a wireless router (802.11g) showed that the ssh protocol is not reliable enough to transfer big files (~250 MB). The ssh protocol seems to die after some time.
Unless I'm missing something doesn't this mean that either: (1) the software installation is broken on one or both machines, or (2) the network is broken to the point that it can't run TCP reliably? Sorting out the underlying problem seems a better idea than applying a band-aid to the file transfer. What error message is produced when the session fails? (see -o LogLevel) Can you run any other TCP-based transfer of > 250 MB over this link? (e.g. FTP, HTTP or even telnet + cat) Cheers, Dave PS I agree with Lew about a likely cause. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (8)
-
Bogdan Cristea
-
Dave Howorth
-
jdd (kim2)
-
Jim Henderson
-
Ken Schneider - openSUSE
-
Larry Stotler
-
Lew Wolfgang
-
Per Jessen