Strange behaviour ssh on 7.1 (RSA) ?
Hi, I've found that I can no longer login using RSA challenge to an openssh server running on SuSE 7.1... Anyone else observed the same ? I've been checking and double-checking everything: filepermissions, server config, upgrading, running sshd in debug mode, but I can't find anything wrong. Of course my ssh-agent etc. is configured correctly. Needless to say, this is the only server I have problems with, out of a couple dozens, and coincidentally it is also the only machine running 7.1 Here are three sessions, the first is the wrong one, the second is for comparison only (and to reach machine #3). The third is from the other machine, back to the one that has the problem. The 'server' is running SuSE 7.1, both clients 7.0. Note that it doesn't matter which 7.0 machine I use to connect from; the sshd on the 7.1 machine invariably rejects it. "apoc": running 7.1, kernel 2.4.3 "morpheus": running 7.0 kernel 2.2.16-SMP "trinity": running 7.0 kernel 2.2.16 maarten@morpheus:~ > ssh -v apoc SSH Version OpenSSH_2.3.0p1, protocol versions 1.5/2.0. Compiled with SSL (0x0090581f). debug: Reading configuration data /etc/ssh/ssh_config debug: Applying options for * debug: Seeding random number generator debug: ssh_connect: getuid 500 geteuid 0 anon 0 debug: Connecting to apoc [x.x.x.x] port 22. debug: Seeding random number generator debug: Allocated local port 796. debug: Connection established. debug: Remote protocol version 1.99, remote software version OpenSSH_2.3.0p1 debug: no match: OpenSSH_2.3.0p1 debug: Local version string SSH-1.5-OpenSSH_2.3.0p1 debug: Waiting for server public key. debug: Received server public key (768 bits) and host key (1024 bits). debug: Host 'apoc' is known and matches the RSA host key. debug: Seeding random number generator debug: Encryption type: 3des debug: Sent encrypted session key. debug: Installing crc compensation attack detector. debug: Received encrypted confirmation. debug: Trying RSA authentication via agent with 'maarten@morpheus' debug: Server refused our key. debug: RSA authentication using agent refused. debug: Trying RSA authentication with key 'maarten@morpheus' debug: Server refused our key. debug: Doing password authentication. maarten@apoc's password: ^C maarten@morpheus:~ > ssh -v trinity SSH Version OpenSSH_2.3.0p1, protocol versions 1.5/2.0. Compiled with SSL (0x0090581f). debug: Reading configuration data /etc/ssh/ssh_config debug: Applying options for * debug: Seeding random number generator debug: ssh_connect: getuid 500 geteuid 0 anon 0 debug: Connecting to trinity [x.x.x.x] port 22. debug: Seeding random number generator debug: Allocated local port 950. debug: Connection established. debug: Remote protocol version 1.99, remote software version OpenSSH_2.3.0p1 debug: no match: OpenSSH_2.3.0p1 debug: Local version string SSH-1.5-OpenSSH_2.3.0p1 debug: Waiting for server public key. debug: Received server public key (768 bits) and host key (1024 bits). debug: Host 'trinity' is known and matches the RSA host key. debug: Seeding random number generator debug: Encryption type: 3des debug: Sent encrypted session key. debug: Installing crc compensation attack detector. debug: Received encrypted confirmation. debug: Trying RSA authentication via agent with 'maarten@morpheus' debug: Received RSA challenge from server. debug: Sending response to RSA challenge. debug: Remote: RSA authentication accepted. debug: RSA authentication accepted by server. debug: Requesting pty. debug: Requesting X11 forwarding with authentication spoofing. debug: Requesting shell. debug: Entering interactive session. Last login: Tue Apr 17 19:51:36 2001 from morpheus Have a lot of fun... maarten@trinity:~ > ssh -v apoc SSH Version OpenSSH_2.3.0p1, protocol versions 1.5/2.0. Compiled with SSL (0x0090581f). debug: Reading configuration data /etc/ssh/ssh_config debug: Seeding random number generator debug: ssh_connect: getuid 500 geteuid 0 anon 0 debug: Connecting to apoc [x.x.x.x] port 22. debug: Seeding random number generator debug: Allocated local port 1012. debug: Connection established. debug: Remote protocol version 1.99, remote software version OpenSSH_2.3.0p1 debug: no match: OpenSSH_2.3.0p1 debug: Local version string SSH-1.5-OpenSSH_2.3.0p1 debug: Waiting for server public key. debug: Received server public key (768 bits) and host key (1024 bits). debug: Host 'apoc' is known and matches the RSA host key. debug: Seeding random number generator debug: Encryption type: 3des debug: Sent encrypted session key. debug: Installing crc compensation attack detector. debug: Received encrypted confirmation. debug: Trying RSA authentication with key 'maarten@trinity' debug: Server refused our key. debug: Doing password authentication. maarten@apoc's password: ^C maarten@trinity:~ > I hope I've somehow overlooked something... If you have any pointers, Thanks in advance... Maarten
On Tue, Apr 17, 2001 at 09:18:38PM +0200, Maarten van den Berg wrote:
I've been checking and double-checking everything: filepermissions, server config, upgrading, running sshd in debug mode, but I can't find anything wrong. Of course my ssh-agent etc. is configured correctly. ...
maarten@morpheus:~ > ssh -v apoc SSH Version OpenSSH_2.3.0p1, protocol versions 1.5/2.0. ... debug: Trying RSA authentication via agent with 'maarten@morpheus' debug: Server refused our key. ...
At this point we are told that the server refused the key. So please send the corresponding server output. Use "-d -d -d" in order to increase the debug level. Best regards, Lutz PS. Running some SuSE 7.1 hosts with OpenSSH 2.3.0p1 and working RSA authentication. -- Lutz Jaenicke Lutz.Jaenicke@aet.TU-Cottbus.DE BTU Cottbus http://www.aet.TU-Cottbus.DE/personen/jaenicke/ Lehrstuhl Allgemeine Elektrotechnik Tel. +49 355 69-4129 Universitaetsplatz 3-4, D-03044 Cottbus Fax. +49 355 69-4153
On Tue, 17 Apr 2001, Lutz Jaenicke wrote:
On Tue, Apr 17, 2001 at 09:18:38PM +0200, Maarten van den Berg wrote:
I've been checking and double-checking everything: filepermissions, server config, upgrading, running sshd in debug mode, but I can't find anything wrong. Of course my ssh-agent etc. is configured correctly. ...
maarten@morpheus:~ > ssh -v apoc SSH Version OpenSSH_2.3.0p1, protocol versions 1.5/2.0. ... debug: Trying RSA authentication via agent with 'maarten@morpheus' debug: Server refused our key. ...
At this point we are told that the server refused the key. So please send the corresponding server output. Use "-d -d -d" in order to increase the debug level.
Been there, done that, can't find anything. Here follows... Script started on Wed Apr 18 01:39:23 2001 apoc:/home/maarten/APOC # killall -9 sshd apoc:/home/maarten/APOC # sshd -d -d -d -f /etc/ssh/sshd_config debug1: sshd version OpenSSH_2.3.0p1 debug1: Seeding random number generator debug1: read DSA private key done debug1: Seeding random number generator debug1: Bind to port 22 on 0.0.0.0. Server listening on 0.0.0.0 port 22. Generating 768 bit RSA key. debug1: Seeding random number generator debug1: Seeding random number generator RSA key generation complete. debug1: Server will not fork when running in debugging mode. Connection from x.x.x.x port 942 debug1: Client protocol version 1.5; client software version OpenSSH_2.3.0p1 debug1: no match: OpenSSH_2.3.0p1 debug1: Local version string SSH-1.99-OpenSSH_2.3.0p1 debug1: Sent 768 bit public key and 1024 bit host key. debug1: Encryption type: 3des debug1: Received session key; encryption turned on. debug1: Installing crc compensation attack detector. debug1: Starting up PAM with username "maarten" debug1: Attempting authentication for maarten. Failed rsa for maarten from 10.42.42.142 port 942 Failed rsa for maarten from 10.42.42.142 port 942 debug1: PAM Password authentication accepted for user "maarten" Accepted password for maarten from 10.42.42.142 port 942 debug1: PAM setting rhost to "morpheus.kijkduin" debug1: session_new: init debug1: session_new: session 0 debug1: Allocating pty. debug1: Received request for X11 forwarding with auth spoofing. debug1: x11_create_display_inet: Socket family 10 not supported debug1: fd 8 setting O_NONBLOCK debug1: fd 8 IS O_NONBLOCK debug1: channel 0: new [X11 inet listener] debug1: PAM setting tty to "/dev/pts/1" debug1: PAM establishing creds debug1: Entering interactive session. debug1: Setting controlling tty using TIOCSCTTY. debug1: fd 3 setting O_NONBLOCK debug1: fd 7 IS O_NONBLOCK debug1: server_init_dispatch_13 debug1: server_init_dispatch_15 debug1: tvp!=NULL kid 0 mili 10 debug1: tvp!=NULL kid 0 mili 10 debug1: tvp!=NULL kid 0 mili 10 Note that it says twice "Failed rsa", after which point I log in with passwd. Its a pity that at that specific point it isn't more verbose :-( Other, possibly useful, info: ### sshd_config Port 22 ListenAddress 0.0.0.0 HostKey /etc/ssh/ssh_host_key ServerKeyBits 768 LoginGraceTime 600 KeyRegenerationInterval 3600 PermitRootLogin yes IgnoreRhosts yes StrictModes yes X11Forwarding yes X11DisplayOffset 10 PrintMotd yes KeepAlive yes SyslogFacility AUTH LogLevel INFO RhostsAuthentication no RhostsRSAAuthentication no RSAAuthentication yes PasswordAuthentication yes PermitEmptyPasswords no CheckMail no Subsystem sftp /usr/lib/ssh/sftp-server MaxStartups 10:30:60 ### end sshd_config ### ls -laR .ssh/ (on client) total 76 drwxr-xr-x 2 maarten 500 4096 Apr 17 20:58 . drwxrwx--- 75 maarten 500 40960 Apr 18 02:41 .. -rw------- 1 maarten users 736 Apr 17 20:58 id_dsa -rw-r--r-- 1 maarten users 606 Apr 17 20:58 id_dsa.pub -rw------- 1 maarten 500 539 Mar 31 1999 identity -rw-rw-r-- 1 maarten 500 343 Mar 31 1999 identity.pub -rw------- 1 maarten 500 9668 Apr 17 21:31 known_hosts -rw------- 1 maarten 500 512 Sep 13 2000 random_seed ### end ls -laR ### ls -laR .ssh/ (on server) total 27 drwx------ 2 maarten users 172 Apr 17 20:53 . drwx------ 31 maarten users 5702 Apr 18 01:25 .. -rw------- 1 maarten users 334 Apr 17 20:53 authorized_keys -rw------- 1 maarten users 527 Apr 17 20:51 identity -rw-r--r-- 1 maarten users 331 Apr 17 20:51 identity.pub -rw-r--r-- 1 maarten users 671 Apr 17 20:41 known_hosts ### end ls -laR Maarten
Best regards, Lutz PS. Running some SuSE 7.1 hosts with OpenSSH 2.3.0p1 and working RSA authentication.
Thanks, now I think I know it's not a 7.1 issue. What is the exact version 'rpm -q openssh' reports ?
On Wed, Apr 18, 2001 at 02:00:39AM +0200, Maarten van den Berg wrote: ...
Been there, done that, can't find anything. Here follows...
Script started on Wed Apr 18 01:39:23 2001 ... debug1: Starting up PAM with username "maarten" debug1: Attempting authentication for maarten. Failed rsa for maarten from 10.42.42.142 port 942 Failed rsa for maarten from 10.42.42.142 port 942 ...
Note that it says twice "Failed rsa", after which point I log in with passwd. Its a pity that at that specific point it isn't more verbose :-(
Yes, that is true. It may be quiet to the client in order to not tell about possible weak points, but it should log locally. In fact, auth-rsa.c:auth_rsa() does contain several diagnostic messages, all of them in the packet_send_debug() class: ... debug1: Attempting authentication for XXXX. RSA authentication refused for kost: bad ownership or modes for '/home/aet/serv01/xxxx/'. Failed rsa for XXXX from x.x.x.x port 325 Therefore we have to look for an RSA failure _without_ debugging message. There are not many (1?), as I can see in the latest OpenSSH CVS auth-rsa.c: * The _PATH_SSH_USER_PERMITTED_KEYS (.ssh/authorized_keys) could not be found. Use strace(?) to trace sshd and see whether the file is successfully opened.
-rw------- 1 maarten 500 539 Mar 31 1999 identity -rw-rw-r-- 1 maarten 500 343 Mar 31 1999 identity.pub This PubKey is 343 and should match identity.
-rw------- 1 maarten users 334 Apr 17 20:53 authorized_keys This PubKey is 334 and should match the identity.pub. Did you edit it?
Thanks, now I think I know it's not a 7.1 issue. What is the exact version 'rpm -q openssh' reports ?
openssh-2.3.0p1-5 Best regards, Lutz -- Lutz Jaenicke Lutz.Jaenicke@aet.TU-Cottbus.DE BTU Cottbus http://www.aet.TU-Cottbus.DE/personen/jaenicke/ Lehrstuhl Allgemeine Elektrotechnik Tel. +49 355 69-4129 Universitaetsplatz 3-4, D-03044 Cottbus Fax. +49 355 69-4153
On Wednesday 18 April 2001 10:25, Lutz Jaenicke wrote:
On Wed, Apr 18, 2001 at 02:00:39AM +0200, Maarten van den Berg wrote:
....
debug1: Starting up PAM with username "maarten" debug1: Attempting authentication for maarten. Failed rsa for maarten from 10.42.42.142 port 942 Failed rsa for maarten from 10.42.42.142 port 942
....
Note that it says twice "Failed rsa", after which point I log in with passwd. Its a pity that at that specific point it isn't more verbose :-(
Yes, that is true. It may be quiet to the client in order to not tell about possible weak points, but it should log locally. In fact, auth-rsa.c:auth_rsa() does contain several diagnostic messages, all of them in the packet_send_debug() class: .... debug1: Attempting authentication for XXXX. RSA authentication refused for kost: bad ownership or modes for '/home/aet/serv01/xxxx/'. Failed rsa for XXXX from x.x.x.x port 325
I checked syslog too, but these logs are all there seems to be.
Therefore we have to look for an RSA failure _without_ debugging message. There are not many (1?), as I can see in the latest OpenSSH CVS auth-rsa.c: * The _PATH_SSH_USER_PERMITTED_KEYS (.ssh/authorized_keys) could not be found. Use strace(?) to trace sshd and see whether the file is successfully opened.
I'll try that tonight (it's my home setup)
-rw------- 1 maarten 500 539 Mar 31 1999 identity -rw-rw-r-- 1 maarten 500 343 Mar 31 1999 identity.pub
This PubKey is 343 and should match identity.
-rw------- 1 maarten users 334 Apr 17 20:53 authorized_keys
This PubKey is 334 and should match the identity.pub. Did you edit it?
Oops.! Sorry, this looks rather suspect indeed. That is because I edited out one of the two keys during testing, the only key that's there now is the one from "trinity". However, it didn't work before editing neither. To be on the safe side, I'll retry the tests tonight with the right keys of course. Sorry for not paying enough attention then.
Thanks, now I think I know it's not a 7.1 issue. What is the exact version 'rpm -q openssh' reports ?
openssh-2.3.0p1-5
Mmm. same as me... Maarten
On Wednesday 18 April 2001 14:19, Maarten van den Berg wrote:
On Wednesday 18 April 2001 10:25, Lutz Jaenicke wrote:
On Wed, Apr 18, 2001 at 02:00:39AM +0200, Maarten van den Berg wrote:
[replying to my own post] I fixed it, though I'm not real sure how. After putting both keys back in the authorized_keys file and testing, without success, I suddenly realized that I had /home exported via NFS, and that there are serious warnings in [some] ssh docs I read a while ago about exporting homedirs. So, after turning off nfsd and deleting the entry from /etc/exports, RSA auth works fine again. But here is the weird part; to check if this was reproduceable I started the NFS server again with the original /etc/exports settings, but sshd still works fine... Is this expected behaviour ? ssh + nfs weirdness ? Maarten Oh, and thanks very much for your help, Lutz :-)
Yes, that is true. It may be quiet to the client in order to not tell about possible weak points, but it should log locally. In fact, auth-rsa.c:auth_rsa() does contain several diagnostic messages, all of them in the packet_send_debug() class: .... debug1: Attempting authentication for XXXX. RSA authentication refused for kost: bad ownership or modes for '/home/aet/serv01/xxxx/'. Failed rsa for XXXX from x.x.x.x port 325
I checked syslog too, but these logs are all there seems to be.
Therefore we have to look for an RSA failure _without_ debugging message. There are not many (1?), as I can see in the latest OpenSSH CVS auth-rsa.c: * The _PATH_SSH_USER_PERMITTED_KEYS (.ssh/authorized_keys) could not be found. Use strace(?) to trace sshd and see whether the file is successfully opened.
I'll try that tonight (it's my home setup)
This PubKey is 334 and should match the identity.pub. Did you edit it?
Oops.! Sorry, this looks rather suspect indeed. That is because I edited out one of the two keys during testing, the only key that's there now is the one from "trinity". However, it didn't work before editing neither.
To be on the safe side, I'll retry the tests tonight with the right keys of course. Sorry for not paying enough attention then.
On Thu, Apr 19, 2001 at 12:40:14PM +0200, Maarten van den Berg wrote:
[replying to my own post]
I fixed it, though I'm not real sure how. After putting both keys back in the authorized_keys file and testing, without success, I suddenly realized that I had /home exported via NFS, and that there are serious warnings in [some] ssh docs I read a while ago about exporting homedirs. So, after turning off nfsd and deleting the entry from /etc/exports, RSA auth works fine again. But here is the weird part; to check if this was reproduceable I started the NFS server again with the original /etc/exports settings, but sshd still works fine...
Is this expected behaviour ? ssh + nfs weirdness ?
I have seen quit a lot of things but nothing like this. I might imagine problems when _importing_ the home directory via NFS, since root permissions are not available and sshd tends to run as root (therefore sshd changes permissions during access to the $HOME/.ssh directory). I however cannot imagine in which way the NFS _export_ of your home directory should influence the way sshd works locally. (That doesn't mean I say that I don't believe you. I just say that it is not logical and there must be a better explanation. It just doesn't make sense.) [This does not cover the security implications of using NFS, that's another topic.] Best regards, Lutz -- Lutz Jaenicke Lutz.Jaenicke@aet.TU-Cottbus.DE BTU Cottbus http://www.aet.TU-Cottbus.DE/personen/jaenicke/ Lehrstuhl Allgemeine Elektrotechnik Tel. +49 355 69-4129 Universitaetsplatz 3-4, D-03044 Cottbus Fax. +49 355 69-4153
On Thursday 19 April 2001 12:50, Lutz Jaenicke wrote:
On Thu, Apr 19, 2001 at 12:40:14PM +0200, Maarten van den Berg wrote:
[replying to my own post]
[...]
Is this expected behaviour ? ssh + nfs weirdness ?
I have seen quit a lot of things but nothing like this. I might imagine problems when _importing_ the home directory via NFS, since root permissions are not available and sshd tends to run as root (therefore sshd changes permissions during access to the $HOME/.ssh directory). I however cannot imagine in which way the NFS _export_ of your home directory should influence the way sshd works locally. (That doesn't mean I say that I don't believe you. I just say that it is not logical and there must be a better explanation. It just doesn't make sense.)
Agreed, and I did plan on doing more research anyhow why this happened. I'm still truely convinced though that my authorized_keys file was 100% correct, so I'm baffled myself...
[This does not cover the security implications of using NFS, that's another topic.]
I'm aware of that, but since this is a home LAN without any other users I disregard that issue for now. Besides, though /home is EXported, it is not IMported on the clients on mountpoint /home, but on /data. So, no real problems there. I'll change the exports in something less evil though, it must only provide many Gigs of random storage, nothing special at all. Maarten
* Lutz Jaenicke wrote on Thu, Apr 19, 2001 at 12:50 +0200:
On Thu, Apr 19, 2001 at 12:40:14PM +0200, Maarten van den Berg wrote:
Is this expected behaviour ? ssh + nfs weirdness ?
No, I don't think so.
I have seen quit a lot of things but nothing like this. I might imagine problems when _importing_ the home directory via NFS, since root permissions are not available and sshd tends to run as root (therefore sshd changes permissions during access to the $HOME/.ssh directory).
That should affect root_squash'd exports only if any. But AFAIK ssh runs pretty with NFS homes. A user needs to copy his idenity.pub into authorized_keys and can SSH to any machine that uses that NFS homes too.
[This does not cover the security implications of using NFS, that's another topic.]
Yep, by this you may "enable" ssh logins on hosts not meant for that, maybe some mailservers or so, but this would be a different thread ;) oki, Steffen -- Dieses Schreiben wurde maschinell erstellt, es trägt daher weder Unterschrift noch Siegel.
participants (3)
-
Lutz Jaenicke -
Maarten van den Berg -
Steffen Dettmer