[Bug 268074] New: Segfault in Xvnc
https://bugzilla.novell.com/show_bug.cgi?id=268074 Summary: Segfault in Xvnc Product: openSUSE 10.2 Version: Final Platform: x86-64 OS/Version: openSUSE 10.2 Status: NEW Severity: Major Priority: P5 - None Component: X.Org AssignedTo: sndirsch@novell.com ReportedBy: pcjc2@cam.ac.uk QAContact: sndirsch@novell.com xorg-x11-Xvnc (SuSE package version 7.1-33.3) I've been trying to get a thin-client setup with Gnome's GDM and Xdmx as a proxy X server. When invoked with the command line (security disabled for testing): /usr/bin/Xvnc :1 -query localhost SecurityTypes=None -ac -depth 24 After a short time using the session, I get a crash: Program received signal SIGSEGV, Segmentation fault. 0x0000000000522697 in XserverDesktop::deferredUpdateTimerCallback () (gdb) bt #0 0x0000000000522697 in XserverDesktop::deferredUpdateTimerCallback () #1 0x0000000000539173 in rfb::RawEncoder::create () #2 0x0000000000539260 in rfb::RawEncoder::create () #3 0x0000000000521b66 in XserverDesktop::deferredUpdateTimerCallback () #4 0x0000000000519d89 in FatalError () #5 0x00000000004283c1 in NoopDDA () #6 0x0000000000509cad in XETrapCloseDown () #7 0x000000000042456b in dixDestroyPixmap () #8 0x0000000000434be5 in NotImplemented () #9 0x00002aee4eb51ae4 in __libc_start_main () from /lib64/libc.so.6 #10 0x0000000000414919 in __gxx_personality_v0 () #11 0x00007fff5db5fc68 in ?? () #12 0x0000000000000000 in ?? () Xvnc doesn't seem prone to crashing without Xdmx as a client, however its fairly obvious that it shouldn't segfault. Googling revealed that the only similar crash reported with VNC can be caused by running "Eclipse" on Xvnc. (The stack trace isn't identical, but does contain dixDestroyPixmap shortly before it crahes). http://www.mail-archive.com/vnc-list@realvnc.com/msg23302.html Regards, Peter Clifton -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mhopf@novell.com, eich@novell.com Status|NEW |NEEDINFO Info Provider| |pcjc2@cam.ac.uk ------- Comment #1 from sndirsch@novell.com 2007-04-25 12:44 MST ------- Could you describe in moredetail thethin-client setup with Gnome's GDM and Xdmx as a proxy X server? I need a way to reproduce this issue. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #2 from pcjc2@cam.ac.uk 2007-04-25 13:14 MST ------- The thin client setup I was looking to emulate is described at: http://www.gnome.org/~markmc/guadec-2005-remotely-useful/img38.html However thats not really describing how to do it. Its reproducable without fiddling with the GDM config. (I'm now reporoducing on my Ubuntu laptop as well) Trying to get this down as simple as possible:
From a gnome-terminal:
gdb --args Xvnc :1 SecurityTypes=None -ac -depth 24 (then run of course)
From another gnome-terminal:
Xdmx :2 -ac -display :1
From another gnome-terminal:
export DISPLAY=:2 metacity & gnome-terminal &
From yet another gnome-terminal:
vncviewer :1 You should now have a VNC viewer showing a basic X11 herringbone background, and a gnome-terminal, window manged by metacity. start typing on the gnome-terminal On my SuSE box, I was typing echo $DISPLAY, and it crashed nearly every time. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 pcjc2@cam.ac.uk changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Info Provider|pcjc2@cam.ac.uk | ------- Comment #3 from pcjc2@cam.ac.uk 2007-04-25 13:21 MST ------- It seems to reliably crash just typing the $ character! On my Ubuntu box (I know how to rebuild packages on that), I compiled Xvnc with more debug options, and got this backtrace: (gdb) bt #0 0x08188bdd in XserverDesktop::keyEvent (this=0x81ed9d0, keysym=36, down=true) at XserverDesktop.cc:836 #1 0x081a9269 in rfb::VNCSConnectionST::keyEvent () #2 0x081aba32 in rfb::SMsgReader::readKeyEvent () #3 0x081a656a in rfb::SMsgReaderV3::readMsg () #4 0x081a5f1f in rfb::SConnection::processMsg () #5 0x081a90b8 in rfb::VNCSConnectionST::processMessages () #6 0x0819a0d5 in rfb::VNCServerST::processSocketEvent () #7 0x08199880 in rfb::VNCServerST::processSocketEvent () #8 0x0818825e in XserverDesktop::wakeupHandler (this=0x81ed9d0, fds=0x81e0480, nfds=1) at XserverDesktop.cc:605 #9 0x081801ac in vncWakeupHandler (data=0x0, nfds=1, readmask=0x81e0480) at vncExtInit.cc:260 #10 0x0806e1c9 in WakeupHandler (result=1, pReadmask=0x81e0480) at dixutils.c:472 #11 0x0816e549 in WaitForSomething (pClientsReady=0xbfc38720) at WaitFor.c:247 #12 0x0806a2fd in Dispatch () at dispatch.c:388 #13 0x0807b227 in main (argc=8, argv=0xbfc38c44, envp=Cannot access memory at address 0xd ) at main.c:450 (gdb) This is substantially different to the other backtrace though, so perhaps either its due to a different version of Xvnc, or we have more than one bug. Let me know if you can reproduce either. A reproduction more close to my original case would be to enable XDMCP in GDM, and start Xdmx with: Xdmx :2 -ac -display :1 -query localhost I'd then log into a gnome session, and it wouldn't be long before it crashed. (Again, gnome-terminal being a good way to speed that up). Thanks, Peter C. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074
------- Comment #4 from pcjc2@cam.ac.uk 2007-04-25 14:28 MST -------
Sorry to keep spamming - further preliminary debug info.
Can you reproduce either the crash on SuSE, or similar BT to this one? I'm
afraid my knowledge of VNC internals and X11 is very very limited, so don't
understand what this is likely to be caused by.
Depending on how / why this segfault is happening, I presume there could be
risk of arbitrary code execution under the PID of the Xvnc server?
Thanks for your help,
Peter
#0 0x08188bdd in XserverDesktop::keyEvent (this=0x81ed9f8, keysym=36,
down=true) at XserverDesktop.cc:836
#1 0x081a9269 in rfb::VNCSConnectionST::keyEvent ()
#2 0x081aba32 in rfb::SMsgReader::readKeyEvent ()
#3 0x081a656a in rfb::SMsgReaderV3::readMsg ()
#4 0x081a5f1f in rfb::SConnection::processMsg ()
#5 0x081a90b8 in rfb::VNCSConnectionST::processMessages ()
#6 0x0819a0d5 in rfb::VNCServerST::processSocketEvent ()
#7 0x08199880 in rfb::VNCServerST::processSocketEvent ()
#8 0x0818825e in XserverDesktop::wakeupHandler (this=0x81ed9f8,
fds=0x81e0480, nfds=1) at XserverDesktop.cc:605
#9 0x081801ac in vncWakeupHandler (data=0x0, nfds=1, readmask=0x81e0480)
at vncExtInit.cc:260
#10 0x0806e1c9 in WakeupHandler (result=1, pReadmask=0x81e0480)
at dixutils.c:472
#11 0x0816e549 in WaitForSomething (pClientsReady=0xbffce2b0) at WaitFor.c:247
#12 0x0806a2fd in Dispatch () at dispatch.c:388
#13 0x0807b227 in main (argc=8, argv=0xbffce7d4, envp=Cannot access memory at
address 0xd
) at main.c:450
(gdb) l
831 }
832 void press() {
833 KeyClassPtr keyc = dev->key;
834 if (!(keyc->state & (1<
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Major |Normal ------- Comment #5 from sndirsch@novell.com 2007-04-27 03:54 MST ------- I can reproduce this issue thanks to your detailed description of the Xvnc/Xdmx setup. But due to the exotic Xserver chain setup (Xdmx on top of Xvnc!) I lower the severity to NORMAL. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #6 from pcjc2@cam.ac.uk 2007-04-27 04:11 MST ------- I've found that you don't need a gnome-terminal to reproduce either.. Just typing $ over the root Xdmx window on the VNC server will cause this. It is also reproducable in the RealVNC binary as downloaded from realvnc.com. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #7 from pcjc2@cam.ac.uk 2007-04-27 04:57 MST ------- Created an attachment (id=135762) --> (https://bugzilla.novell.com/attachment.cgi?id=135762&action=view) Patch masking the segfult problem (unsure if this is the correct fix) I've got a simple patch which works around the segfault issue, but not being familiar with the X11 internals, I'm not sure if it masks a real bug elsewhere, or if this is indeed the correct way to fix things. It seemed to me on inspection that all other code-paths using (KeyClassPtr)->modifierKeyMap do so from within a loop counting (KeyClassPtr)->maxKeysPerModifier, so when that value is zero, the former structure is never accessed. I noted from gdb, msxKeysPerModifier is zero in the case where I observed the crash. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #135762|text/x-patch |text/plain mime type| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 pcjc2@cam.ac.uk changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #135762|0 |1 is obsolete| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #8 from pcjc2@cam.ac.uk 2007-04-27 05:10 MST ------- Created an attachment (id=135772) --> (https://bugzilla.novell.com/attachment.cgi?id=135772&action=view) Patch masking the segfult problem (unsure if this is the correct fix) Patch masking the segfult problem (unsure if this is the correct fix) ** This time, the diff isn't broken! ** I've got a simple patch which works around the segfault issue, but not being familiar with the X11 internals, I'm not sure if it masks a real bug elsewhere, or if this is indeed the correct way to fix things. It seemed to me on inspection that all other code-paths using (KeyClassPtr)->modifierKeyMap do so from within a loop counting (KeyClassPtr)->maxKeysPerModifier, so when that value is zero, the former structure is never accessed. I noted from gdb, msxKeysPerModifier is zero in the case where I observed the crash. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #135772|text/x-patch |text/plain mime type| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 - None |P2 - High ------- Comment #9 from sndirsch@novell.com 2007-04-28 04:34 MST ------- The patch looks correct to me. I'll apply it. Thanks! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED ------- Comment #10 from sndirsch@novell.com 2007-04-28 04:41 MST ------- Fixed for STABLE/Factory. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #12 from eich@novell.com 2007-04-28 23:05 MST ------- Looking at the xkb code but also core keyboard keyc->modifierKeyMap can theoretically be NULL. I'll investigate if this is really the case. A couple of ErrorF's should help to shed some light on this. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 eich@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |REOPENED Resolution|FIXED | ------- Comment #13 from eich@novell.com 2007-04-28 23:17 MST ------- Reopen to assign to myself. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 eich@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|sndirsch@novell.com |eich@novell.com Status|REOPENED |ASSIGNED -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #14 from eich@novell.com 2007-04-30 19:45 MST ------- I cannot reproduce this problem on openSUSE 10.2. Xvnc on there seems to be built without xkb support as -DNO_HW_ONLY_EXTS is hardcoded in the Makefile. Peter: can you do a 'xdpyinfo' on Xvnc (display :1) and look if the string XKEYBOARD is in the output? However even when I enable it I cannot see the problem. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #15 from pcjc2@cam.ac.uk 2007-05-01 01:36 MST ------- I don't see XKEYBOARD, nor do I see it in the binary RealVNC where I can also reproduce the problem. Is it possible that any patches / fixes have meant the version in SuSE 10.2 has changed? (I presume my patch wasn't applied after I questioned whether it fixes the issue or just masks it). Where is the $ key on your keyboard? Mine has it at "Shift" + "4". I wounder if how we input the character over the Xdmx window makes a difference here. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 ------- Comment #16 from eich@novell.com 2007-05-01 06:50 MST ------- OK, got it now ;) Was on the wrong track. The bug is triggered when pressing a modifier key like "Shift" together with another key. Xdmx calls SetModifierMappings and passes an empty list to erase all modifiers. RealVNC is wrong about the assumtion that maxKeysPerModifier may never be 0. I can't find anything in the protocol specs that would prohibit an empty (zero length) list and the code handling this request can certainly deal with an empty list. So the patch is correct and should be applied. Peter, thanks for the patch! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 eich@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|eich@novell.com |sndirsch@novell.com Status|ASSIGNED |NEW -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=268074 sndirsch@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED ------- Comment #17 from sndirsch@novell.com 2007-05-02 08:48 MST ------- Thanks for investigation. I've already applied the patch. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
participants (1)
-
bugzilla_noreply@novell.com