[Bug 1180501] New: frequent sshd coredumps
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501 Bug ID: 1180501 Summary: frequent sshd coredumps Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: x86-64 OS: openSUSE Tumbleweed Status: NEW Severity: Normal Priority: P5 - None Component: Network Assignee: screening-team-bugs@suse.de Reporter: paka@opensuse.org QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- Created attachment 844783 --> http://bugzilla.opensuse.org/attachment.cgi?id=844783&action=edit sshd coredump file sshd frequently coredumps, ie: several times a day and has for several months current tw version 20210101 12:13 crash: ~ # coredumpctl info 24778 -o sshd.coredump.txt PID: 24778 (sshd) UID: 1000 (paka) GID: 100 (users) Signal: 11 (SEGV) Timestamp: Sun 2021-01-03 10:49:57 EST (1h 23min ago) Command Line: sshd: paka@notty Executable: /usr/sbin/sshd Control Group: /user.slice/user-1000.slice/session-27.scope Unit: session-27.scope Slice: user-1000.slice Session: 27 Owner UID: 1000 (paka) Boot ID: 6daaababe19d42638b365590e4f26aee Machine ID: 3384cbc37a574dcc99445aa25cd2db04 Hostname: crash Storage: /var/lib/systemd/coredump/core.sshd.1000.6daaababe19d42638b365590e4f26aee.24778.16096 88997000000.zst Message: Process 24778 (sshd) of user 1000 dumped core. Stack trace of thread 24778: #0 0x000055d628693563 cipher_free (sshd + 0x4e563) #1 0x000055d628695b67 ssh_packet_close_internal (sshd + 0x50b67) #2 0x000055d628655cbf main (sshd + 0x10cbf) #3 0x00007f2cae063152 __libc_start_main (libc.so.6 + 0x28152) #4 0x000055d62865624e _start (sshd + 0x1124e) coredump attached core.sshd.1000.6daaababe19d42638b365590e4f26aee.24778.1609688997000000.zst -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
Patrick McNeil
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c1
patrick shanahan
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c2
--- Comment #2 from patrick shanahan
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
Bj�rn Voigt
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c5
--- Comment #5 from patrick shanahan
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c6
--- Comment #6 from patrick shanahan
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c8
--- Comment #8 from patrick shanahan
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c9
--- Comment #9 from patrick shanahan
Thanks for the core dump. This seems to be happening in a few locations, but I haven't been able to repro it here yet, not even with valgrind.
The dump makes it looks like a sshcipher_ctx struct has been partially overwritten with garbage. In cipher_free() it crashes on this line:
if ((cc->cipher->flags & CFLAG_CHACHAPOLY) != 0) {
...because cc->cipher points to a bad location (but it's not NULL).
Questions:
1) Do you know more precisely when this started happening? I'm suspecting patches added to openssh or openssl this autumn.
sorry, no, but has been for 3 or 4 or more months
2) Could you run sshd like this (as root):
/usr/sbin/sshd -Dddd -p 2048
Then from a different shell session, connect to it like this:
ssh localhost -p 2048
...and trigger the crash, then attach the sshd debug output here?
I will but I do not know what triggers the crash. I will start the session you describe and leave it open until I observe another crash, then report. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c10
--- Comment #10 from Cristian Rodr��guez
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c15
--- Comment #15 from patrick shanahan
Questions:
1) Do you know more precisely when this started happening? I'm suspecting patches added to openssh or openssl this autumn.
2) Could you run sshd like this (as root):
/usr/sbin/sshd -Dddd -p 2048
Then from a different shell session, connect to it like this:
ssh localhost -p 2048
...and trigger the crash, then attach the sshd debug output here?
I am running as you requested "sshd -Dddd -p 2048" and have an open xterm instance accessing it. I have experience multiple coredumps during this time but the particular instance you requested has not failed or lost connection. I do not even notice the coredumps, no ssh instance seems to fail or it automatically reconnects w/o notice. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c16
--- Comment #16 from patrick shanahan
Questions:
1) Do you know more precisely when this started happening? I'm suspecting patches added to openssh or openssl this autumn.
2) Could you run sshd like this (as root):
/usr/sbin/sshd -Dddd -p 2048
Then from a different shell session, connect to it like this:
ssh localhost -p 2048
...and trigger the crash, then attach the sshd debug output here?
I am running as you requested "sshd -Dddd -p 2048" and have an open xterm instance accessing it.
I have experience multiple coredumps during this time but the particular instance you requested has not failed or lost connection.
I do not even notice the coredumps, no ssh instance seems to fail or it automatically reconnects w/o notice.
/usr/sbin/sshd -Dddd -p 2048 PID#16987 still running following coredumps last day & 1/2 http://wahoo.no-ip.org/core.sshd.1000.174b488671e343b18ba4e5e8599ab700.12492... http://wahoo.no-ip.org/core.sshd.1000.174b488671e343b18ba4e5e8599ab700.14708... http://wahoo.no-ip.org/core.sshd.1000.174b488671e343b18ba4e5e8599ab700.14902... http://wahoo.no-ip.org/core.sshd.1000.174b488671e343b18ba4e5e8599ab700.25481... http://wahoo.no-ip.org/core.sshd.1000.174b488671e343b18ba4e5e8599ab700.31200... http://wahoo.no-ip.org/core.sshd.1000.174b488671e343b18ba4e5e8599ab700.31200... http://wahoo.no-ip.org/core.sshd.1000.174b488671e343b18ba4e5e8599ab700.3202.... -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c18
--- Comment #18 from patrick shanahan
Thanks, Patrick. Did you try this with the new packages?
Submitreq: https://build.opensuse.org/request/show/861779
Not in Factory yet, but should be available from the network project.
currently: openssh-server-8.4p1-270.1.x86_64 openssh-common-8.4p1-270.1.x86_64 openssh-8.4p1-270.1.x86_64 openssh-debuginfo-8.4p1-270.1.x86_64 openssh-helpers-8.4p1-270.1.x86_64 openssh-server-debuginfo-8.4p1-270.1.x86_64 openssh-clients-8.4p1-270.1.x86_64 are they sufficient? pls excuse lack of knowledge :) -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c31
patrick shanahan
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501
http://bugzilla.opensuse.org/show_bug.cgi?id=1180501#c32
Hans Petter Jansson
openssh-8.4p1-3.1.x86_64
I have gone 4 days without an sshd coredump
must be fixed/corrected.
may be closed?
tks
Thanks for checking. Yes, we're considering this fixed. The security team will close the bug. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply@suse.com