[Bug 425480] New: e1000e fails to load
https://bugzilla.novell.com/show_bug.cgi?id=425480 Summary: e1000e fails to load Product: openSUSE 11.1 Version: Factory Platform: Other OS/Version: Other Status: NEW Severity: Blocker Priority: P5 - None Component: Kernel AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: stbinner@novell.com QAContact: qa@suse.de Found By: Development Updated today my work station after two or three weeks to current Factory kernel and since then the onboard network card doesn't show up anymore: e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k2 e1000e: Copyright (c) 1999-2008 Intel Corporation. e1000e 0000:00:19.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20 e1000e 0000:00:19.0: setting latency timer to 64 input: PC Speaker as /devices/platform/pcspkr/input/input3 0000:00:19.0: 0000:00:19.0: The NVM Checksum Is Not Valid e1000e 0000:00:19.0: PCI INT A disabled e1000e: probe of 0000:00:19.0 failed with error -5 Booted an openSUSE 11.0 installation and same issue there now too. Some BIOS/checksum got broken? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User stbinner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c1
--- Comment #1 from Stephan Binner
https://bugzilla.novell.com/show_bug.cgi?id=425480
User stbinner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c2
--- Comment #2 from Stephan Binner
https://bugzilla.novell.com/show_bug.cgi?id=425480
User stbinner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c4
Stephan Binner
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c5
Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c6
--- Comment #6 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c7
Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c10
--- Comment #10 from John Ronciak
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c11
Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c13
--- Comment #13 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c14
Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User stbinner@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c15
--- Comment #15 from Stephan Binner
you did only install a new kernel, you did not install Beta1, correct ?
Correct, just "zypper dup" + reboot. openSUSE 11.1 Beta 1 didn't exist yet when I reported that bug. :-) 29: PCI 19.0: 0200 Ethernet controller [Created at pci.318] UDI: /org/freedesktop/Hal/devices/pci_8086_294c Unique ID: kpGf.nWnnnRlG_JE SysFS ID: /devices/pci0000:00/0000:00:19.0 SysFS BusID: 0000:00:19.0 Hardware Class: network Model: "Intel 82566DC-2 Gigabit Network Connection" Vendor: pci 0x8086 "Intel Corporation" Device: pci 0x294c "82566DC-2 Gigabit Network Connection" SubVendor: pci 0x8086 "Intel Corporation" SubDevice: pci 0x0000 Revision: 0x02 Memory Range: 0x92200000-0x9221ffff (rw,non-prefetchable) Memory Range: 0x92224000-0x92224fff (rw,non-prefetchable) I/O Ports: 0x3400-0x341f (rw) IRQ: 216 (no events) Module Alias: "pci:v00008086d0000294Csv00008086sd00000000bc02sc00i00" Driver Info #0: Driver Status: e1000e is active Driver Activation Cmd: "modprobe e1000e" Config Status: cfg=no, avail=yes, need=no, active=unknown -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c16
--- Comment #16 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c17
--- Comment #17 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c18
--- Comment #18 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c19
--- Comment #19 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User bob@muhlenberg.edu added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c20
Bob Mahar
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c21
Jesse Brandeburg
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c22
Jiri Kosina
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c23
--- Comment #23 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c24
--- Comment #24 from John Ronciak
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c25
Jesse Brandeburg
https://bugzilla.novell.com/show_bug.cgi?id=425480
User bob@muhlenberg.edu added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c26
--- Comment #26 from Bob Mahar
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c27
--- Comment #27 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c28
--- Comment #28 from Jesse Brandeburg
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c29
--- Comment #29 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c30
--- Comment #30 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User wstephenson@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c31
Will Stephenson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User seife@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c32
Stefan Seyfried
https://bugzilla.novell.com/show_bug.cgi?id=425480
User martin.wilck@fujitsu-siemens.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c33
Martin Wilck
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c34
--- Comment #34 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c35
--- Comment #35 from John Ronciak
https://bugzilla.novell.com/show_bug.cgi?id=425480
User andi-nbz@firstfloor.org added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c36
--- Comment #36 from Andi N Kleen
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c37
--- Comment #37 from John Ronciak
https://bugzilla.novell.com/show_bug.cgi?id=425480
User andi-nbz@firstfloor.org added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c38
--- Comment #38 from Andi N Kleen
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c39
--- Comment #39 from Jiri Kosina
Those would first need to fix their mac addresses to try again right? Also is there other vital information in that EEPROM?
By the way the current driver doesn't get even bound to the card that has wrong EEPROM CRC, right? So it's even not possible to easily fix its contents up using ethtool from within default installation. Karsten already patched and built the kernel so that it binds the driver to the card even in cases of broken EEPROM checksum. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
LTC BugProxy
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c40
--- Comment #40 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c41
--- Comment #41 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c42
--- Comment #42 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c43
--- Comment #43 from Jesse Brandeburg
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c44
--- Comment #44 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c45
Egbert Eich
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c46
--- Comment #46 from Egbert Eich
Jesse, What happens with those that have already a broken eeprom?
Point is: if it's related to #57976 writing back the eeprom doesn't work. At the time I attributed this to preproduction hardware. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User bob@muhlenberg.edu added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c47
--- Comment #47 from Bob Mahar
Point is: if it's related to #57976 writing back the eeprom doesn't work. At the time I attributed this to preproduction hardware.
I don't have access to that one... what was the issue? Can you elaborate? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c48
--- Comment #48 from John Ronciak
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c49
--- Comment #49 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c50
--- Comment #50 from Egbert Eich
(In reply to comment #46 from Egbert Eich)
Point is: if it's related to #57976 writing back the eeprom doesn't work. At the time I attributed this to preproduction hardware.
I don't have access to that one... what was the issue? Can you elaborate?
In bug #57976 an SPI type eeprom which seemed to still hold valid content (at least not 0xff) but a bogus checksum could not be restored as no matter to which byte offset a value was written it always ended up at offset 0 or 1. However looking at comment #49 it doesn't seem to be related as in this case the checksum could be fixed. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c51
--- Comment #51 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c52
--- Comment #52 from Jiri Kosina
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c53
--- Comment #53 from Jiri Kosina
If, however, userspace is corrupting the memory region (most probably X.Org), then this protection is rendered useless, but it still is worth trying so that we can potentially rule out either userspace or kernelspace code completely.
In fact, testing whether booting the system only in text-mode (so that xorg won't be started at all) also triggers the bug or not would also be a valuable test. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c54
--- Comment #54 from John Ronciak
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c55
--- Comment #55 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c56
--- Comment #56 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User seife@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c57
--- Comment #57 from Stefan Seyfried
In fact, testing whether booting the system only in text-mode (so that xorg won't be started at all) also triggers the bug or not would also be a valuable test.
It is, unfortunately, not that easy. I have rebooted my machine (hp 2510p) with e1000e 17 times since Sep 15 with 2.6.27-rc5-git9+ Kernels (always pretty recent STABLE) and I did not encounter any problems. So it is pretty hard to prove the absence of this bug. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c58
--- Comment #58 from Jiri Kosina
It is, unfortunately, not that easy. I have rebooted my machine (hp 2510p) with e1000e 17 times since Sep 15 with 2.6.27-rc5-git9+ Kernels (always pretty recent STABLE) and I did not encounter any problems. So it is pretty hard to prove the absence of this bug.
And did this machine expose the problem at least once previously? Apparently not all systems having e1000e hardware are being hit by the issue, either only specific product IDs are affected, or it might be chipset-dependent, etc. Also, please do not forget to back up contents of your EEPROM before you start playing with this :) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c59
--- Comment #59 from Jesse Brandeburg
I could write the MAC address with ethtool but now the driver do not load completely insmod hangs for about a minute and then it disable the IRQ. After this here is no eth1 this are the dmesg: e1000e: Intel(R) PRO/1000 Network Driver - 0.2.0 e1000e: Copyright (c) 1999-2007 Intel Corporation. ACPI: PCI Interrupt 0000:00:19.0[A] -> GSI 20 (level, low) -> IRQ 20 PCI: Setting latency timer of device 0000:00:19.0 to 64
at this point (without trying to activate the device) does ethtool -e still work? I would assume not.
ACPI: PCI interrupt for device 0000:00:19.0 disabled
I looked at your ethregs dump (thank you!!!) and in the EECD register, bit 8 is not set, indicating the valid bits in the eeprom are not set. bit 9 is set indicating the hardware tried to read the eeprom. bit 22 is only valid if bit 8 and 9 is set, but it would indicate which of the two eeprom banks had a valid signature. I'm curious if the other bank on the eeprom might still be okay. I'll have to figure out tomorrow if we can switch to the other bank. I may be able to get you some internal tools since this is an intel board, I'll have to see what is available. BTW this is the first desktop machine I've heard of that reported the problem. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User seife@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c60
--- Comment #60 from Stefan Seyfried
And did this machine expose the problem at least once previously? Apparently not all systems having e1000e hardware are being hit by the issue, either only specific product IDs are affected, or it might be chipset-dependent, etc.
I am not sure. See comment #32. It might of course also just have been a broken joint on the mainboard.
Also, please do not forget to back up contents of your EEPROM before you start playing with this :)
If it hits me the same as last time, this won't help :) (and yes, i backed it up) (In reply to comment #59 from Jesse Brandeburg)
BTW this is the first desktop machine I've heard of that reported the problem. Regarding the desktop: it also has intel integrated graphics, and it had recurring problems with the graphics driver (lockups) before the ethernet broke. Maybe that's one common factor.
-- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c63
--- Comment #63 from Egbert Eich
Intel has just posted patches to lkml [1] [2] [3] that mark the memory mapped EEPROM region as read-only. Therefore if the EEPROM is garbled by any bug in kernel code, after these patches are applied, the EEPROM would no longer be overwritten, and stack trace would be dumped instead, which will hopefully point to the code that is corrupting the memory.
This is indeed a valuable test.
If, however, userspace is corrupting the memory region (most probably X.Org), then this protection is rendered useless, but it still is worth trying so that we can potentially rule out either userspace or kernelspace code completely.
Not necessarily. If X overwrites this memory from user space, yes. However if it is overwritten from kernel space (by DRM - either from the Xserver or from a DRM client) we will be able to catch it. For now I would rule out user space. From user space the Xserver cannot access memory unless it is explicitly mapped. You can find out the mapped memory ranges from /proc/<pid>/maps. It would be instructive to know which ranges show up there on the affected machines and compare them to an lspci -v output. I may have missed this, but I have not seen any analysis which access method is used on the affected systems to write to the EEPROM. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c64
--- Comment #64 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c65
--- Comment #65 from Jiri Kosina
For now I would rule out user space. From user space the Xserver cannot access memory unless it is explicitly mapped. You can find out the mapped memory ranges from /proc/<pid>/maps. It would be instructive to know which ranges show up there on the affected machines and compare them to an lspci -v output.
It could be some temporary mapping that goes away after a while, so that it doesn't show in /proc/<pid>/maps permanently, but yes, this of course can be tried. Could please someone, who has access to affected hardware, provide output of cat /proc/`pidof Xorg`/maps lspci -v commands, so that we can see if there is possibly some lethal overlap? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c66
--- Comment #66 from Jiri Kosina
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c67
--- Comment #67 from Egbert Eich
It could be some temporary mapping that goes away after a while, so that it doesn't show in /proc/<pid>/maps permanently, but yes, this of course can be tried.
The Xserver itself doesn't have any temporary mappings. It could be buffers requested from DRM which (depending on the implementation) could be requested and discarded during runtime of the server. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User renato_yamane@yahoo.com.br added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c68
--- Comment #68 from Renato Yamane
https://bugzilla.novell.com/show_bug.cgi?id=425480
User okir@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c69
--- Comment #69 from Olaf Kirch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User okir@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c70
--- Comment #70 from Olaf Kirch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User renato_yamane@yahoo.com.br added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c71
--- Comment #71 from Renato Yamane
https://bugzilla.novell.com/show_bug.cgi?id=425480
User seife@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c72
Stefan Seyfried
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c73
--- Comment #73 from Jiri Kosina
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c74
--- Comment #74 from Egbert Eich
Olaf, I really don't fell confortable to test Kernel 2.6.27-rc if it can damaged my ethernet device, so I don't know if my hardware is affected, but my video card is:
01:00.0 VGA compatible controller: nVidia Corporation Quadro NVS 140M (rev a1) Subsystem: Lenovo Device 20d8 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Definitely not. And if this system does show to be affected it would point away from the gfx driver. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c75
--- Comment #75 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c76
--- Comment #76 from Jiri Kosina
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c77
--- Comment #77 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c78
--- Comment #78 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jpallen@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c80
--- Comment #80 from Jared Allen
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jpallen@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c81
--- Comment #81 from Jared Allen
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c82
--- Comment #82 from Jiri Kosina
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c83
--- Comment #83 from Jesse Brandeburg
https://bugzilla.novell.com/show_bug.cgi?id=425480
User bob@muhlenberg.edu added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c84
--- Comment #84 from Bob Mahar
I think the problem with T60p is different (it has a different lan chip 82573 with a real eeprom (not NVM based) that should not be able to be corrupted in the same manner as 82566/82567.
Oh, if it were only that simple. The T60p has the 82573L c.f. the Intel docs... http://download.intel.com/design/network/products/LAN/manuals/316080.pdf See section 2.3... "The 82573E/82573V/82573L supports both FLASH memory and EEPROM; however, only one device can be connected at a time (not both). So while the 8257x's for the most part SEEPROMs, the -E,V,L suffixed part could go either way - oh joy! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User martin.wilck@fujitsu-siemens.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c85
--- Comment #85 from Martin Wilck
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c86
--- Comment #86 from Jiri Kosina
Forgive me this dumb question - if this is due to an accidental overwrite with random data (DMA),
It's not DMA but MMIO. The data are not that random, it is really all 0xff.
why would the EEPROM contain only FFs afterwards? Can't we infer something from > that?
Well, we weren't able to use this to identify source of the corruption so far. We have patches that could help to point to the guilty, but first we need reliable way to restore the EEPROM contents, otherwise the debugging is almost impossible. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User okir@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c87
--- Comment #87 from Olaf Kirch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User autobugz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c87
--- Comment #87 from Olaf Kirch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c88
--- Comment #88 from Andreas Jaeger
https://bugzilla.novell.com/show_bug.cgi?id=425480
User okir@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c89
Olaf Kirch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jpallen@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c90
--- Comment #90 from Jared Allen
https://bugzilla.novell.com/show_bug.cgi?id=425480
User okir@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c91
--- Comment #91 from Olaf Kirch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c92
--- Comment #92 from Egbert Eich
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c93
--- Comment #93 from Egbert Eich
One more question to Intel.
There's a question whether the NVM we're talking about here is actually larger, and is used by components other than the e1000e. If for instance the video BIOS maps all of the NVM and, due to some bug, scribbles over parts of it that include the e1000e's config space - is there a way to verify this?
I don't think this is the case: the driver only maps the POSTed copy of the VBIOS. This is copied into RAM at POST time (to the 0xC-segment). This copy is then made read only. This copy is (should be) entirely independent of the EEPROM containing the PCI ROM. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User eich@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c94
--- Comment #94 from Egbert Eich
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c95
--- Comment #95 from Jesse Brandeburg
Jesse, I'm setting this bug as NEEDINFO to you. The biggest roadblock right now is our inability to bring those dead NICs back to life. Without this, we cannot proceed with testing, and we are somewhat reluctant to try this ourselves, as it seems someone at RedHat has bricked a laptop this way.
We'll get you a utility today to help with this, and at the same time we're working on a quick hack to the driver to take in an ethtool eeprom dump and push it back to the NVM. We hope to have that done and working today.
We tried a BIOS update on one of the affected laptops, but this didn't help. And since we weren't aware of the problem in advance, we don't have an ethregs dump of these.
so it depends on whether the BIOS version has the LAN part included. Some bios versions do, and some do not. I know that in particular there were a couple versions of the bioses for the X60/T60 line that had LAN NVM updates.
So can you please get someone from Intel to help us with restoring the NVM to a working condition? Thanks a lot!
We are working on it, a couple of different avenues. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c96
--- Comment #96 from Jesse Brandeburg
There's a question whether the NVM we're talking about here is actually larger, and is used by components other than the e1000e. If for instance the video BIOS maps all of the NVM and, due to some bug, scribbles over parts of it that include the e1000e's config space - is there a way to verify this?
the NVM in question is a single part that the entire machine (VGA, BIOS, LAN, Manageability, AHCI, etc) all use. I couldn't tell you how to verify if something else is mapping over the top of the LAN area of the NVM. The only reports I've heard are that the LAN NVM is corrupted. If you managed to corrupt the BIOS area, the machine wouldn't boot. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c97
--- Comment #97 from Jesse Brandeburg
Another question came up: does this happen on both 64 and 32 bit installations?
At this point we don't know. At least one reported I worked with was running 32 bit. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User seife@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c98
--- Comment #98 from Stefan Seyfried
The only reports I've heard are that the LAN NVM is corrupted. If you managed to corrupt the BIOS area, the machine wouldn't boot.
Helmut Schaa has an HP 2510p that lost some of its display modes after a hard X crash on an early 2.6.27-rc kernel (it now no longer knows that it has a 1280x800 panel but thinks that it only has 1024x768, the BIOS screen is in the upper left corner instead of centered on the screen). Even though we don't know that this is the same problem, it shows that sh*t happens. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jbarnes@virtuousgeek.org added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c99
--- Comment #99 from Jesse Barnes
https://bugzilla.novell.com/show_bug.cgi?id=425480
User renato_yamane@yahoo.com.br added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c100
--- Comment #100 from Renato Yamane
but first we need reliable way to restore the EEPROM contents, otherwise the debugging is almost impossible.
A strange comment in Ubuntu bug Report that, maybe, can help: "...I have resolved on my hp 8510w with an old image of windows, and my network card is reborn..." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/263555/comments/75 Anyone have dual-boot (Windows) and can try this? Best regards, Renato -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c101
--- Comment #101 from John Ronciak
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c102
--- Comment #102 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c103
--- Comment #103 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c104
--- Comment #104 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
Jesse Brandeburg
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c105
Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c106
--- Comment #106 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c107
Jiri Kosina
Guys, I have an HP 8510w which is experiencing some interesting behaviour. I believe I may have had a graphics corruption first, though I don't recall if the problems started directly afterward. I'm definately running the e1000e driver, the machine has an NVIDIA Quadro FX570M (Mobile Version). The first thing I noticed was the Intel Boot agent in the BIOS reports the following;
Initializing Intel (R) Boot Agent GE v1.2.45 PXE-E05: The LAN adapter's confirguration is corrupted or has not been initialized. The Boot Agent cannot continue.
Quentin, could you please post a lspci output from the affected machine? If you are experiencing the problem on a system that doesn't have intel graphics chip at all, you'd be the first one whatsoever, and this would really change the direction of our debugging efforts -- currently the main suspect is intel graphics driver in X.org, which apparently couldn't be blamed in such case. In addition to that, could you please attach your /etc/X11/xorg.conf? Thanks. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c108
--- Comment #108 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c109
--- Comment #109 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c110
--- Comment #110 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c111
--- Comment #111 from Jiri Kosina
Done
Thanks. So apparently, you are really the first one, to my knowledge, who reports the problem on ICH chipset, but with no Intel graphics chip at all. This really seems to rule out the xorg graphics driver issue in my eyes. Could you please boot a "Kernel Of The Day" from ftp://ftp.suse.com/pub/projects/kernel/kotd/HEAD/ This kernel contains a load of fixes for the e1000e driver. It is unfortunately not currently able to bring your network card back to life, but it will output a EEPROM contents dump into 'dmesg' output even if the contents are corrupt. Could you please attach this output then? This will allow us to verify whether you are really hitting the very same problem. Thanks. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c112
--- Comment #112 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User s.puch@web.de added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c113
Stefan Puch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c114
--- Comment #114 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c115
--- Comment #115 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c116
--- Comment #116 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c117
--- Comment #117 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c118
Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User mmeeks@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c119
--- Comment #119 from Michael Meeks
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c120
--- Comment #120 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User okir@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c121
--- Comment #121 from Olaf Kirch
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c122
--- Comment #122 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c123
--- Comment #123 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jesse.brandeburg@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c124
--- Comment #124 from Jesse Brandeburg
https://bugzilla.novell.com/show_bug.cgi?id=425480
User renato_yamane@yahoo.com.br added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c125
--- Comment #125 from Renato Yamane
https://bugzilla.novell.com/show_bug.cgi?id=425480
User jkosina@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c126
--- Comment #126 from Jiri Kosina
Fixed? http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=...
Yes, that's workaround that prevents the corruption of the EEPROM contents, but it doesn't fix the real problem, just prevents bad things from happening when the bug triggers. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c127
--- Comment #127 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User kkeil@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c128
--- Comment #128 from Karsten Keil
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c130
--- Comment #130 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User quentin.jackson@exclamation.co.nz added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c131
--- Comment #131 from Quentin Jackson
https://bugzilla.novell.com/show_bug.cgi?id=425480
User john.ronciak@intel.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c132
--- Comment #132 from John Ronciak
---------- Forwarded message ---------- From: Steven Rostedt
Date: Wed, Oct 15, 2008 at 3:21 PM Subject: [PATCH -stable] disable CONFIG_DYNAMIC_FTRACE due to possible memory corruption on module unload To: LKML , stable@kernel.org Cc: Linus Torvalds , Andrew Morton , Arjan van de Ven , gregkh@suse.de, jesse.brandeburg@intel.com, Thomas Gleixner , Ingo Molnar While debugging the e1000e corruption bug with Intel, we discovered today that the dynamic ftrace code in mainline is the likely source of this bug.
For the stable kernel we are providing the only viable fix patch: labeling CONFIG_DYNAMIC_FTRACE as broken. (see the patch below)
We will follow up with a backport patch that contains the fixes. But since the fixes are not a one liner, the safest approach for now is to disable the code in question.
The cause of the bug is due to the way the current code in mainline handles dynamic ftrace. When dynamic ftrace is turned on, it also turns on CONFIG_FTRACE which enables the -pg config in gcc that places a call to mcount at every function call. With just CONFIG_FTRACE this causes a noticeable overhead. CONFIG_DYNAMIC_FTRACE works to ease this overhead by dynamically updating the mcount call sites into nops.
The problem arises when we trace functions and modules are unloaded. The first time a function is called, it will call mcount and the mcount call will call ftrace_record_ip. This records the calling site and stores it in a preallocated hash table. Later on a daemon will wake up and call kstop_machine and convert any mcount callers into nops.
The evolution of this code first tried to do this without the kstop_machine and used cmpxchg to update the callers as they were called. But I was informed that this is dangerous to do on SMP machines if another CPU is running that same code. The solution was to do this with kstop_machine.
We still used cmpxchg to test if the code that we are modifying is indeed code that we expect to be before updating it - as a final line of defense.
But on 32bit machines, ioremapped memory and modules share the same address space. When a module would load its code into memory and execute some code, that would register the function.
On module unload, ftrace incorrectly did not zap these functions from its hash (this was the bug). The cmpxchg could have saved us in most cases (via luck) - but with ioremap-ed memory that was exactly the wrong thing to do - the results of cmpxchg on device memory are undefined. (and will likely result in a write)
The pending .28 ftrace tree does not have this bug anymore, as a general push towards more robustness of code patching, this is done differently: we do not use cmpxchg and we do a WARN_ON and turn the tracer off if anything deviates from its expected state. Furthermore, patch sites are statically identified during build time so there's no runtime discovery of dynamic code areas anymore, and no room for code unmaps to cause the hash to become out of date.
We believe the fragility of dynamic patching has been sufficiently addressed in the development code via the static patching method, but further suggestions to make it more robust are welcome.
Signed-off-by: Steven Rostedt
Acked-by: Ingo Molnar Acked-by: Thomas Gleixner --- kernel/trace/Kconfig | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) Index: linux-compile.git/kernel/trace/Kconfig =================================================================== --- linux-compile.git.orig/kernel/trace/Kconfig 2008-10-02 10:18:49.000000000 -0400 +++ linux-compile.git/kernel/trace/Kconfig 2008-10-15 17:29:34.000000000 -0400 @@ -103,7 +103,8 @@ config CONTEXT_SWITCH_TRACER all switching of tasks.
config DYNAMIC_FTRACE - bool "enable/disable ftrace tracepoints dynamically" + bool "enable/disable ftrace tracepoints dynamically (BROKEN)" + depends on BROKEN depends on FTRACE depends on HAVE_DYNAMIC_FTRACE default y
-- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
-- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=425480
User gregkh@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=425480#c133
Greg Kroah-Hartman
participants (1)
-
bugzilla_noreply@novell.com