I have this system setup on:
Asus Rampage 3 Extreme Motherboard with 24Gb RAM
Intel 980x Extreme Edition
PCIe slot 1 is an nVidia Quadro K610
PCIe slot 2 is an AMD R9 290x
PCIe slot 3 is an nVidia GTX 960
It looks like all my GPUs are in different groups, but no matter what I do / try I always get this error, or similar based on which GPU group I'm trying to passthrough to the VM
internal error: qemu unexpectedly closed the monitor: 2017-02-24T21:24:15.991544Z qemu-system-x86_64: -device vfio-pci,host=04:00.0,id=hostdev0,x-vga=on,bus=pci.2,addr=0x4: vfio: failed to set iommu for container: Operation not permitted2017-02-24T21:24:15.991572Z qemu-system-x86_64: -device vfio-pci,host=04:00.0,id=hostdev0,x-vga=on,bus=pci.2,addr=0x4: vfio: failed to setup container for group 172017-02-24T21:24:15.991577Z qemu-system-x86_64: -device vfio-pci,host=04:00.0,id=hostdev0,x-vga=on,bus=pci.2,addr=0x4: vfio: failed to get group 172017-02-24T21:24:15.991591Z qemu-system-x86_64: -device vfio-pci,host=04:00.0,id=hostdev0,x-vga=on,bus=pci.2,addr=0x4: Device initialization failed
And this is what is listed for my groups:
IOMMU group 0
[8086:3405] 00:00.0 Host bridge: Intel Corporation 5520/5500/X58 I/O Hub to ESI Port (rev 13)
IOMMU group 1
[8086:3408] 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13)
IOMMU group 2
[8086:3409] 00:02.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 2 (rev 13)
IOMMU group 3
[8086:340a] 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13)
IOMMU group 4
[8086:340c] 00:05.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 5 (rev 13)
IOMMU group 5
[8086:340e] 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13)
IOMMU group 6
[8086:342e] 00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 13)
[8086:3422] 00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
[8086:3423] 00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13)
[8086:3438] 00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle Registers (rev 13)
IOMMU group 7
[8086:10ce] 00:19.0 Ethernet controller: Intel Corporation 82567V-2 Gigabit Network Connection
IOMMU group 8
[8086:3a37] 00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
[8086:3a38] 00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
[8086:3a39] 00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
[8086:3a3c] 00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
IOMMU group 9
[8086:3a3e] 00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
IOMMU group 10
[8086:3a40] 00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
[8086:3a48] 00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
[197b:2363] 06:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
[197b:2363] 06:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
IOMMU group 11
[8086:3a34] 00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
[8086:3a35] 00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
[8086:3a36] 00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
[8086:3a3a] 00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
IOMMU group 12
[8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
[1106:3044] 08:02.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)
IOMMU group 13
[8086:3a16] 00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
[8086:3a22] 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
[8086:3a30] 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
IOMMU group 14
[1b4b:9123] 01:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9123 PCIe SATA 6.0 Gb/s controller (rev 11)
IOMMU group 15
[1033:0194] 02:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)
IOMMU group 16
[10de:13bb] 03:00.0 VGA compatible controller: NVIDIA Corporation GM107GL [Quadro K620] (rev a2)
[10de:0fbc] 03:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)
IOMMU group 17
[1002:67b0] 04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X]
[1002:aac8] 04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X]
IOMMU group 18
[10de:1401] 05:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1)
[10de:0fba] 05:00.1 Audio device: NVIDIA Corporation Device 0fba (rev a1)
IOMMU group 19
[8086:2c71] ff:00.0 Host bridge: Intel Corporation Device 2c71 (rev 02)
[8086:2d81] ff:00.1 Host bridge: Intel Corporation Xeon 5600 Series QuickPath Architecture System Address Decoder (rev 02)
IOMMU group 20
[8086:2d90] ff:02.0 Host bridge: Intel Corporation Xeon 5600 Series QPI Link 0 (rev 02)
[8086:2d91] ff:02.1 Host bridge: Intel Corporation Xeon 5600 Series QPI Physical 0 (rev 02)
[8086:2d92] ff:02.2 Host bridge: Intel Corporation Xeon 5600 Series Mirror Port Link 0 (rev 02)
[8086:2d93] ff:02.3 Host bridge: Intel Corporation Xeon 5600 Series Mirror Port Link 1 (rev 02)
IOMMU group 21
[8086:2d98] ff:03.0 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Registers (rev 02)
[8086:2d99] ff:03.1 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Target Address Decoder (rev 02)
[8086:2d9c] ff:03.4 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Test Registers (rev 02)
IOMMU group 22
[8086:2da0] ff:04.0 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Control (rev 02)
[8086:2da1] ff:04.1 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Address (rev 02)
[8086:2da2] ff:04.2 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Rank (rev 02)
[8086:2da3] ff:04.3 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Thermal Control (rev 02)
IOMMU group 23
[8086:2da8] ff:05.0 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Control (rev 02)
[8086:2da9] ff:05.1 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Address (rev 02)
[8086:2daa] ff:05.2 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Rank (rev 02)
[8086:2dab] ff:05.3 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Thermal Control (rev 02)
IOMMU group 24
[8086:2db0] ff:06.0 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Control (rev 02)
[8086:2db1] ff:06.1 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Address (rev 02)
[8086:2db2] ff:06.2 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Rank (rev 02)
[8086:2db3] ff:06.3 Host bridge: Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Thermal Control (rev 02)
Does anyone have any suggestions on this?