archedraft

***GUIDE*** Passthrough Entire PCI USB Controller

408 posts in this topic

6 hours ago, darrenyorston said:

Yes the card works when not passed through to a VM.

 

I think I may have solved the problem. Seems the power cable might have had a problem. I have changed out to a different Molex cable and now the ports are active. Maybe a bad connection though, the same connectors run my hot swap-able drive bays fine.

 

Now I am getting "DRIVER_IRQL_NOT_LESS_OR_EQUAL" on astkmd.sys

 

You are still stubbing the card? If you are, you could try to change the pcie slot to see if it makes a difference. 

0

Share this post


Link to post
Share on other sites

Still doing it the same as listed previously. I can't change the slot unfortunately. The Supermicro board only has two slots and the other one is occupied by my 4 port NIC. It would be a bitch to change over as the 4 port NIC is passed through to my PFSense VM.

 

Im thinking of deleting the VM and making it anew. Maybe something is wrong with it.

0

Share this post


Link to post
Share on other sites

I am thinking there is something wrong with the card. I deleted my VM and tried to reinstall a new one and I keep getting the message in the attached image. If I remove the USB card from the "Other PCI Devices" VMs will install.

 

unraid.jpg

0

Share this post


Link to post
Share on other sites

I have tried "stubbing the devices", according to "gridrunner", using the new method "vfio-pci".

 

Syslinux Configuration: 

  append vfio-pci.ids=1b73:1100 pcie_acs_override=downstream initrd=/bzroot

PCI Devices and IOMMU Groups:

IOMMU group 18
	[1b73:1100] 03:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev ff)

USB Devices:

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 03f0:5607 Hewlett-Packard 
Bus 001 Device 003: ID 045b:0209 Hitachi, Ltd 
Bus 001 Device 004: ID 1b1c:1c00 Corsair 
Bus 001 Device 005: ID 046d:c332 Logitech, Inc. 
Bus 001 Device 006: ID 1b1c:1b13 Corsair 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 002: ID 045b:0210 Hitachi, Ltd 

However, I am getting this error when I start my Windows 10 Virtual Machines.

Execution error

internal error: qemu unexpectedly closed the monitor: 

2017-05-10T08:04:34.456440Z qemu-system-x86_64: 

-device vfio-pci,host=03:00.0,id=hostdev2,bus=pci.0,addr=0x9: vfio: Error: Failed to setup INTx fd: Device or resource busy 

2017-05-10T08:04:36.333862Z qemu-system-x86_64: 

-device vfio-pci,host=03:00.0,id=hostdev2,bus=pci.0,addr=0x9: Device initialization failed

Should,  I edit my VM XML?

<domain type='kvm'>
  <name>Windows 10 VM2 (Opperational)</name>
  <uuid>0ceecee2-196e-7ba7-ef43-5cd95ee4143b</uuid>
  <description>Windows 10 Enterprise X64 1703 RS2</description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>12582912</memory>
  <currentMemory unit='KiB'>12582912</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='3'/>
    <vcpupin vcpu='1' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='1' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10 VM2 (Opperational)/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10 VM2 (Opperational)/vdisk2.img'/>
      <target dev='hdd' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/iso/windows/en_windows_10_enterprise_version_1703_updated_march_2017_x64_dvd_10189290.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/latest-virtio/virtio-win/virtio-win-0.1.135-2/virtio-win-0.1.135.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='nec-xhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:5c:f3:9a'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

0

Share this post


Link to post
Share on other sites

Posted (edited)

On 2017-5-10 at 9:07 AM, Awkward said:

I have tried "stubbing the devices", according to "gridrunner", using the new method "vfio-pci".

 

Syslinux Configuration: 


  append vfio-pci.ids=1b73:1100 pcie_acs_override=downstream initrd=/bzroot

PCI Devices and IOMMU Groups:


IOMMU group 18
	[1b73:1100] 03:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev ff)

USB Devices:


Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 03f0:5607 Hewlett-Packard 
Bus 001 Device 003: ID 045b:0209 Hitachi, Ltd 
Bus 001 Device 004: ID 1b1c:1c00 Corsair 
Bus 001 Device 005: ID 046d:c332 Logitech, Inc. 
Bus 001 Device 006: ID 1b1c:1b13 Corsair 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 002: ID 045b:0210 Hitachi, Ltd 

However, I am getting this error when I start my Windows 10 Virtual Machines.


Execution error

internal error: qemu unexpectedly closed the monitor: 

2017-05-10T08:04:34.456440Z qemu-system-x86_64: 

-device vfio-pci,host=03:00.0,id=hostdev2,bus=pci.0,addr=0x9: vfio: Error: Failed to setup INTx fd: Device or resource busy 

2017-05-10T08:04:36.333862Z qemu-system-x86_64: 

-device vfio-pci,host=03:00.0,id=hostdev2,bus=pci.0,addr=0x9: Device initialization failed

Should,  I edit my VM XML?


<domain type='kvm'>
  <name>Windows 10 VM2 (Opperational)</name>
  <uuid>0ceecee2-196e-7ba7-ef43-5cd95ee4143b</uuid>
  <description>Windows 10 Enterprise X64 1703 RS2</description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>12582912</memory>
  <currentMemory unit='KiB'>12582912</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='3'/>
    <vcpupin vcpu='1' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='1' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10 VM2 (Opperational)/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10 VM2 (Opperational)/vdisk2.img'/>
      <target dev='hdd' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/iso/windows/en_windows_10_enterprise_version_1703_updated_march_2017_x64_dvd_10189290.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/latest-virtio/virtio-win/virtio-win-0.1.135-2/virtio-win-0.1.135.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='nec-xhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:5c:f3:9a'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

 

3
 

 

Quote

 

 

Try this

append vfio-pci.ids=1b73 vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt pcie_acs_override=downstream initrd=/bzroot

Edited by gridrunner
0

Share this post


Link to post
Share on other sites

Posted (edited)

On 5/10/2017 at 6:27 PM, gridrunner said:

 

 

Try this

append vfio-pci.ids=1b73 vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt pcie_acs_override=downstream initrd=/bzroot

 

We use "vfio-pci.ids=1b73" and omitting the ":1100" since we are using "vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt".

 

Thank you for your assistance, Gridrunner, I will give that a try.

 

Update:

 

I have noticed that in the VM setting, "Other PCI Devices" no longer see the "USB controller: Fresco Logic FL1100 USB 3.0 Host Controller" as it is now integrated into the "USB Devices", which is now hot pluggable.

 

 

Edited by Awkward
Updating Progress
0

Share this post


Link to post
Share on other sites
3 hours ago, Awkward said:

 

We use "vfio-pci.ids=1b73" and omitting the ":1100" since we are using "vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt".

 

Thank you for your assistance, Gridrunner, I will give that a try.

 

Update:

 

I have noticed that in the VM setting, "Other PCI Devices" no longer see the "USB controller: Fresco Logic FL1100 USB 3.0 Host Controller" as it is now integrated into the "USB Devices", which is now hot pluggable.

 

 

 

oopps no my bad. I left out the :1100 in error

should be

append vfio-pci.ids=1b73:1100 vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt pcie_acs_override=downstream initrd=/bzroot

 

0

Share this post


Link to post
Share on other sites

Hey,

started the routine but got confused O.o

 

I am using:

USB 2.0 on board(2 ports),

USB 3.0 on board(4 ports),

USB 3.0 Header (2 ports)

 

my output:

~#lspci | grep USB
00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31)

~#lsusb
Bus 002 Device 002: ID 0951:1666 Kingston Technology DataTraveler G4
Bus 002 Device 003: ID 0781:5590 SanDisk Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 04b3:3025 IBM Corp. NetVista Full Width Keyboard
Bus 001 Device 004: ID 03f0:2b17 Hewlett-Packard LaserJet 1020
Bus 001 Device 003: ID 045e:0083 Microsoft Corp. Basic Optical Mouse
Bus 001 Device 002: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

IOMMU group 3
	[8086:a12f] 00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31)

well,

looks like the 'USB 2.0 onboard' and the 'USB 3.0 Header',

sharing the same bus (BUS 001).


I only want to passthru all the devices from Bus 001,
how do I do that?

Edited by amstel
0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.