[Bug 573311] New: 'xm create' fails with pci-passthrough if two PCI devices under the same bridge; patch/workaround provided
http://bugzilla.novell.com/show_bug.cgi?id=573311 http://bugzilla.novell.com/show_bug.cgi?id=573311#c0 Summary: 'xm create' fails with pci-passthrough if two PCI devices under the same bridge; patch/workaround provided Classification: openSUSE Product: openSUSE 11.2 Version: Final Platform: All OS/Version: openSUSE 11.2 Status: NEW Severity: Normal Priority: P5 - None Component: Xen AssignedTo: jdouglas@novell.com ReportedBy: 0.bugs.only.0@gmail.com QAContact: qa@suse.de Found By: --- Blocker: --- User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.0) Gecko/20100115 SUSE/3.6.0-145.1 Firefox/3.6 i've rpm -qa | grep -i xen-3 xen-3.4.1_19718_05-1.1.x86_64 i'm trying to passthrough a PCI Eth NIC to a DomU @ grub, module /vmlinuz-xen ... guestdev=0000:04:06.0 reassign_resources ... and. test.cfg ... pci = [ '04:06.0' ] ... @ xm create test.cfg i get, Error: pci: 0000:04:07.0 must be co-assigned to the same guest with 0000:04:06.0 04:06.0 & 04:07.0 are two PCI devices under the same bridge lspci -vt | egrep "06\.0|07\.0" +-14.4-[04]--+-06.0 Intel Corporation 82541PI Gigabit Ethernet Controller | \-07.0 Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller reading @ this thread, http://xen.markmail.org/search/?q=disable+flr#query:disableflr+page:1+mid:7d... application of a "disable FLR" patch, http://xen.markmail.org/download.xqy?id=7dbm675e4x4dawec&number=1 works around the problem. after patch & cold reboot, @ Dom0, xm create test.cfg @ Domu lspci 00:01.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) Reproducible: Always Steps to Reproduce: 1. 2. 3. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=573311 http://bugzilla.novell.com/show_bug.cgi?id=573311#c1 --- Comment #1 from mail ignored <0.bugs.only.0@gmail.com> 2010-01-23 18:14:36 UTC --- nice to have some sort of accommodation for this in code, so as to survive Xen(-tools) updates. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=573311
http://bugzilla.novell.com/show_bug.cgi?id=573311#c
Charles Arnold
http://bugzilla.novell.com/show_bug.cgi?id=573311
http://bugzilla.novell.com/show_bug.cgi?id=573311#c2
--- Comment #2 from mail ignored <0.bugs.only.0@gmail.com> 2010-01-25 16:11:38 UTC ---
worth mentioning, from the related thread @ xen-devel,
http://lists.xensource.com/archives/html/xen-devel/2010-01/msg00870.html
On Sun, Jan 24, 2010 at 5:44 PM, Weidong Han
I think this is not a bug. pls note that the devices behind PCIe-to-PCI/PCI-x bridge or Conventional PCI bridge can only be collectively assigned to a single domain, because the source-id in DMA requests is not device's bdf. Pls refer to section 3.6.1 in VT-d spec. What's more, FLR is required to make sure the device in a correct status before assignment, and some devices need to reset secondary bus to reset the device which results in all the devices behind same PCIe-to-PCI/PCI-x bridge or Conventional PCI bridge be reset. In your case, you need to hide all NICs (04:06.0 and 04:07.0) behind the same bridge, and assign them together to a guest. or you can set "pci-passthrough-strict-check no" in /etc/xen/xend-config.sxp and restart xend to loose the check in xend, but in this way you should be aware of potential issues (e.g. assigned device doesn't work).
the 'set "pci-passthrough-strict-check no" in /etc/xen/xend-config.sxp' option is a xen 4.0 option ( on my sys, atm, Xen4 Dom0 boots fine, but booting DomUs under Xen 4.0 is non-functional :-/ ) that, iiuc, effects a similart workaround to that offered above. i'm sure the technical discussion above has merit, but ferreting out the issues of FLR & diving into the VT-d spec are not exactly user-friendly. this really needs either a fix or some clear discussion/explanation ... reading @ http://www.novell.com/documentation/sles11/book_xen/?page=/documentation/sle... implies "can't be done" ... if a fix _really_ isn't doable, at least, seems reasonable to provide a functionally-equivalent workaround in xen 3x to that in xen 4x, with similar warnings/explanation in docs -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=573311 http://bugzilla.novell.com/show_bug.cgi?id=573311#c3 --- Comment #3 from mail ignored <0.bugs.only.0@gmail.com> 2010-01-27 15:07:56 PST --- fyi, (1) Re: [Xen-devel] Re: follow up to a pciback "pv pci-passthrough co-assign" http://lists.xensource.com/archives/html/xen-devel/2010-01/msg00937.html
sure. would it not be reasonable to have a similar config-file option in xen 3.x -- with similar warnings etc -- as is done in 4.x? if all that's required for a workaround is a patch similar to that above ...
thanks
The option was introduced by changeset 20179. It's small. It can be back ported to Xen 3.x if need.
It would be good to backport this.
The point here is, I think, that before the VT-d stuff you could passthrough for example each port from a dual-port NIC to different PV guests, even if it was not the recommended way.
Now at some point it stopped working for many people, and Xen started complaining that the devices need to be passed to the same guest.
I know some people are also using another port from a dual-port NIC in the dom0, and another port passthroughed to a PV guest. And they've been doing this for a long time.
(2) [Xen-devel] PV PCI passthrough - summary/status http://lists.xensource.com/archives/html/xen-devel/2009-10/msg01226.html Bug #1340: The Function Level Reset reset requires that PCI devices behind a bridge be owned by the same guest. Based on the comments, without the FLR capability in 3.2, the PCI devices worked fine. We could provie a 'flr_inhibit' flag which would revert back to 3.2 and not do an FLR? -> "[1340] 3.3.0 PCI devices ... must be co-assigned to the same guest" http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1340 -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=573311
http://bugzilla.novell.com/show_bug.cgi?id=573311#c4
Charles Arnold
participants (1)
-
bugzilla_noreply＠novell.com