[opensuse] SFTP performance degradation under Dolphin and Konqueror
Hello everyone, Something very curious has just been discovered here. We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why? Boris. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 05/18/2010 08:59 AM, Boris Epstein wrote:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Boris.
Boris, I haven't tried that with dolphin in 4.4.3, but I have notice several other regression in 4.4.3. Better to just file the bug so they can take a look at the ssl_kio (or whatever handles the transfer) to see if an old piece of code slipped back in. I know the file-save dialogs have gone to hell again. Just file -- I guess with suse and that should get the ball rolling. -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, May 18, 2010 at 5:17 PM, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
On 05/18/2010 08:59 AM, Boris Epstein wrote:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Boris.
Boris,
I haven't tried that with dolphin in 4.4.3, but I have notice several other regression in 4.4.3. Better to just file the bug so they can take a look at the ssl_kio (or whatever handles the transfer) to see if an old piece of code slipped back in. I know the file-save dialogs have gone to hell again. Just file -- I guess with suse and that should get the ball rolling.
-- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks David, I would be thinking along the same lines as you if it wasn't for the fact that under Ubuntu things seem to be exactly the same as under OpenSUSE. Boris. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 05/18/2010 04:34 PM, Boris Epstein wrote:
Thanks David,
I would be thinking along the same lines as you if it wasn't for the fact that under Ubuntu things seem to be exactly the same as under OpenSUSE.
Boris.
No doubt it is a kde issue, but at least the last time it was discussed, the openSuSE folks wanted the KDE bugs affecting the present release filed here and then they coordinate it with bugs.kde.org. If that's changed, then I guess just file at bugs.kde.org. I filed the file-saveas regression at bugs.kde.org: https://bugs.kde.org/show_bug.cgi?id=238101 because I'm still on 11.0 and basically it doesn't exist anymore from suse's perspective. Good luck! -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, May 18, 2010 at 5:54 PM, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
On 05/18/2010 04:34 PM, Boris Epstein wrote:
Thanks David,
I would be thinking along the same lines as you if it wasn't for the fact that under Ubuntu things seem to be exactly the same as under OpenSUSE.
Boris.
No doubt it is a kde issue, but at least the last time it was discussed, the openSuSE folks wanted the KDE bugs affecting the present release filed here and then they coordinate it with bugs.kde.org. If that's changed, then I guess just file at bugs.kde.org.
I filed the file-saveas regression at bugs.kde.org:
https://bugs.kde.org/show_bug.cgi?id=238101
because I'm still on 11.0 and basically it doesn't exist anymore from suse's perspective.
Good luck!
-- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Well, I actually doubt it is a KDE issue as I am using Gnome on all the machines in question. Now, I even doubt it is a Gnome issue - I think it is a graphical vs non-graphical issue - perhaps the graphical transfer does too many stops-and-goes to assess the speed on the transfer and Windows (on the server side) is not keeping up switching contexts as it often does. Just a guess, but who knows... Boris. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Am Dienstag 18 Mai 2010, 15:59:21 schrieb Boris Epstein:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Dolpin and konqueror use kio_sftp to use the sftp protocol. kio_sftp uses the libssh library. It's likely that it is a problem of this library, which isn't part of KDE. Herbert -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, May 18, 2010 at 6:05 PM, Herbert Graeber <hgraeber@opensuse.org> wrote:
Am Dienstag 18 Mai 2010, 15:59:21 schrieb Boris Epstein:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Dolpin and konqueror use kio_sftp to use the sftp protocol. kio_sftp uses the libssh library. It's likely that it is a problem of this library, which isn't part of KDE.
Herbert -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks Herbert! You might be on to something, however, wouldn't the command line client (sftp) also be using that same library, libssh? Things go fast using that client. Boris. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Wednesday 19 May 2010 00:24:42 Boris Epstein wrote:
On Tue, May 18, 2010 at 6:05 PM, Herbert Graeber <hgraeber@opensuse.org> wrote:
Am Dienstag 18 Mai 2010, 15:59:21 schrieb Boris Epstein:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Dolpin and konqueror use kio_sftp to use the sftp protocol. kio_sftp uses the libssh library. It's likely that it is a problem of this library, which isn't part of KDE.
Herbert -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks Herbert!
You might be on to something, however, wouldn't the command line client (sftp) also be using that same library, libssh?
Things go fast using that client.
Herbert is right. In KDE < 4.3 times, kio_sftp called the sftp command line binary. Since KDE SC 4.4, it uses libssh directly, which is more robust and (theoretically) faster. KDE already converted kio_smb from using the smbclient commandline to using libsmbclient. However, libssh currently only supports synchronous I/O. The sftp command uses async I/O for better performance. libssh.org are working on implementing async I/O too. This is the reason that KDE sftp ioslave performance is currently worse than it has been in the past. sftp does not use the libssh codebase. It uses the original openssh codebase, whereas libssh is a relatively new effort to provided ssh and sftp functions in a library. Will -- Will Stephenson, KDE Developer, openSUSE Boosters Team SUSE LINUX Products GmbH - Nürnberg - AG Nürnberg - HRB 16746 - GF: Markus Rex -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Wed, May 19, 2010 at 6:47 AM, Will Stephenson <wstephenson@suse.de> wrote:
On Wednesday 19 May 2010 00:24:42 Boris Epstein wrote:
On Tue, May 18, 2010 at 6:05 PM, Herbert Graeber <hgraeber@opensuse.org> wrote:
Am Dienstag 18 Mai 2010, 15:59:21 schrieb Boris Epstein:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Dolpin and konqueror use kio_sftp to use the sftp protocol. kio_sftp uses the libssh library. It's likely that it is a problem of this library, which isn't part of KDE.
Herbert -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks Herbert!
You might be on to something, however, wouldn't the command line client (sftp) also be using that same library, libssh?
Things go fast using that client.
Herbert is right.
In KDE < 4.3 times, kio_sftp called the sftp command line binary. Since KDE SC 4.4, it uses libssh directly, which is more robust and (theoretically) faster. KDE already converted kio_smb from using the smbclient commandline to using libsmbclient.
However, libssh currently only supports synchronous I/O. The sftp command uses async I/O for better performance. libssh.org are working on implementing async I/O too. This is the reason that KDE sftp ioslave performance is currently worse than it has been in the past.
sftp does not use the libssh codebase. It uses the original openssh codebase, whereas libssh is a relatively new effort to provided ssh and sftp functions in a library.
Will -- Will Stephenson, KDE Developer, openSUSE Boosters Team SUSE LINUX Products GmbH - Nürnberg - AG Nürnberg - HRB 16746 - GF: Markus Rex -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks Will! Very valuable info. We are actually using Gnome if that makes a difference. Boris. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Wednesday 19 May 2010 14:44:01 Boris Epstein wrote:
On Wed, May 19, 2010 at 6:47 AM, Will Stephenson <wstephenson@suse.de> wrote:
On Wednesday 19 May 2010 00:24:42 Boris Epstein wrote:
On Tue, May 18, 2010 at 6:05 PM, Herbert Graeber <hgraeber@opensuse.org>
wrote:
Am Dienstag 18 Mai 2010, 15:59:21 schrieb Boris Epstein:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Dolpin and konqueror use kio_sftp to use the sftp protocol. kio_sftp uses the libssh library. It's likely that it is a problem of this library, which isn't part of KDE.
Herbert -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks Herbert!
You might be on to something, however, wouldn't the command line client (sftp) also be using that same library, libssh?
Things go fast using that client.
Herbert is right.
In KDE < 4.3 times, kio_sftp called the sftp command line binary. Since KDE SC 4.4, it uses libssh directly, which is more robust and (theoretically) faster. KDE already converted kio_smb from using the smbclient commandline to using libsmbclient.
However, libssh currently only supports synchronous I/O. The sftp command uses async I/O for better performance. libssh.org are working on implementing async I/O too. This is the reason that KDE sftp ioslave performance is currently worse than it has been in the past.
sftp does not use the libssh codebase. It uses the original openssh codebase, whereas libssh is a relatively new effort to provided ssh and sftp functions in a library.
Thanks Will!
Very valuable info.
We are actually using Gnome if that makes a difference.
Not overall, and definitely not to throughput. The only difference from running dolphin or any other kde application under KDE is that the kdeinit: klauncher process that spawns ioslaves is started along with the first KDE app, instead of at login. But this is an up front one time cost. Will -- Will Stephenson, KDE Developer, openSUSE Boosters Team SUSE LINUX Products GmbH - Nürnberg - AG Nürnberg - HRB 16746 - GF: Markus Rex -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Herbert Graeber wrote:
Am Dienstag 18 Mai 2010, 15:59:21 schrieb Boris Epstein:
Hello everyone,
Something very curious has just been discovered here.
We have a machine running SSH/SFTP server (a Windows XP machine if that matters). The network is a mix of 100 Mbit and 1000 Mbit ports (12.5 and 125 MB/s, respectively). When we use command-line sftp to transfer files we get decent speed (from 7 MB/s and upwards). When we use graphical interfaces on either OpenSuSE or Ubuntu (and it is the same whether it is Konqueror or Dolphin) the transfer speed drops down to no more than 2 MB/s and sometimes even to 500 KB/s. Any idea why?
Dolpin and konqueror use kio_sftp to use the sftp protocol. kio_sftp uses the libssh library. It's likely that it is a problem of this library, which isn't part of KDE.
Herbert
I believe it is a kio throughput performance issue. The following comments are more specific to smb transfers to/from Samba servers, but serve to illustrate. Quite a while back I still had a partition with my trusty old KDE 3.5.10 install present as a fallback from KDE 4.x.x. (This is no longer the case.) With the old 100Mb/s ethernet I averaged somewhere between 6-7 MB/s during smb file transfers using Konqueror in file manager mode. Perfectly nominal. I then changed my NICs and switch to 1Gb/s. On the old KDE 3.5.10 the average run of the mill was about 24MB/s throughput on file transfers. I immediately noticed very early on that the 4.x.x install averaged somewhere around 16MB/s and was lucky if it every once in a while hit 18MB/s. Could not ever get it any faster than this. Reboot back to the 3.5.10 install and it would go right back to 24MB/s. I should probably add that I have seen this identical behavior on both FreeBSD and Linux. These numbers are from a small home network using servers which do not have the throughput capabilities of real server hardware. But the delta between KDE 3.5.10 and KDE 4.x.x was real and reproducible. This speed difference between KDE 3.5.10 and KDE 4.x.x was very well defined and completely repeatable by simply booting between KDE versions. Never really tried to do anything about it and was hoping that at some point in the future others with more techn-fu would notice, complain about it, and maybe it would eventually get looked at. I'm not holding my breath, as KDE 4.x.x has been out how long now and nobody seems interested. -Mike -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (5)
-
Boris Epstein -
David C. Rankin -
Herbert Graeber -
Michael Powell -
Will Stephenson