CloseTurkey

Confirmed, reverting to 6.5 fixed my Win 10 VM issue. I only tried shutting down the VM from the web gui which caused unRAID to crash several times, haven't done any further investigation as unfortunately I don't have the time available.

I'm also experiencing this issue. I suspect its since I upgraded to the latest OS, 6.5.3. I'm going to test this theory by reverting back to 6.5. Will update shortly

ok so here is what I've found with one gpu lspci - v gives me this 04:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 980] (rev a1) (prog-if 00 [VGA controller]) Subsystem: eVga.com. Corp. GM204 [GeForce GTX 980] Flags: fast devsel, IRQ 11, NUMA node 0 Memory at f8000000 (32-bit, non-prefetchable) [disabled] Memory at b0000000 (64-bit, prefetchable) [disabled] Memory at c0000000 (64-bit, prefetchable) [disabled] I/O ports at d000 [disabled] Expansion ROM at f9000000 [disabled by cmd] Capabilities: [60] Power Management version 3 Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [78] Express Legacy Endpoint, MSI 00 Capabilities: [100] Virtual Channel Capabilities: [250] Latency Tolerance Reporting Capabilities: [258] L1 PM Substates Capabilities: [128] Power Budgeting <?> Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?> Capabilities: [900] #19 04:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1) Subsystem: eVga.com. Corp. GM204 High Definition Audio Controller Flags: bus master, fast devsel, latency 0, IRQ 10, NUMA node 0 Memory at f9080000 (32-bit, non-prefetchable) Capabilities: [60] Power Management version 3 Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [78] Express Endpoint, MSI 00 and IOMMU groups are this IOMMU group 26 [8086:8d47] 00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC Controller (rev 05) [8086:2822] 00:1f.2 RAID bus controller: Intel Corporation SATA Controller [RAID mode] (rev 05) [8086:8d22] 00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus Controller (rev 05) IOMMU group 27 [10de:13c0] 04:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 980] (rev a1) [10de:0fbb] 04:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1) IOMMU group 28 [1b21:1242] 05:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller IOMMU group 29 [1b21:1242] 06:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller Logs SeaBIOS Jul 21 11:44:29 Tower kernel: input: Logitech USB Optical Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-9/1-9.3/1-9.3:1.0/0003:046D:C077.000A/input/input14 Jul 21 11:44:29 Tower kernel: hid-generic 0003:046D:C077.000A: input,hidraw3: USB HID v1.11 Mouse [Logitech USB Optical Mouse] on usb-0000:00:14.0-9.3/input0 Jul 21 11:44:41 Tower kernel: vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem Jul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered blocking state Jul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered disabled state Jul 21 11:44:41 Tower kernel: device vnet0 entered promiscuous mode Jul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered blocking state Jul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered forwarding state Jul 21 11:44:42 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258 Jul 21 11:44:42 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900 Jul 21 11:44:42 Tower kernel: vfio-pci 0000:00:1b.0: enabling device (0000 -> 0002) Jul 21 11:44:42 Tower acpid: input device has been disconnected, fd 6 Jul 21 11:44:42 Tower acpid: input device has been disconnected, fd 7 Jul 21 11:44:44 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound Jul 21 11:44:49 Tower kernel: usb 1-9.3: reset low-speed USB device number 11 using xhci_hcd Jul 21 11:44:50 Tower kernel: usb 1-9.3: reset low-speed USB device number 11 using xhci_hcd Jul 21 11:44:50 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcd Jul 21 11:44:51 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcd Jul 21 11:44:51 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound VM Logs 2017-07-21 18:44:41.766+0000: starting up libvirt version: 2.4.0, qemu version: 2.7.1, hostname: Tower LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10 Gaming,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-Windows 10 Gaming/master-key.aes' -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -m 8192 -realtime mlock=off -smp 4,sockets=1,cores=2,threads=2 -uuid 3e3a2e1e-bfa3-59e0-2bde-cab34c96cd5f -display none -no-user-config -nodefaults -chardev 'socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-2-Windows 10 Gaming/monitor.sock,server,nowait' -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,id=usb,bus=pci.0,addr=0x7 -device ahci,id=sata0,bus=pci.0,addr=0x3 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive 'file=/mnt/user/domains/Windows 10 Gaming/vdisk1.img,format=r=0x6,romfile=/mnt/user/drivers/test.dump -device vfio-pci,host=00:1b.0,id=hostdev1,bus=pci.0,addr=0x8 -device usb-host,hostbus=1,hostaddr=11,id=hostdev2,bus=usb.0,port=1 -device usb-host,hostbus=1,hostaddr=6,id=hostdev3,bus=usb.0,port=2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -msg timestamp=on Domain id=2 is tainted: high-privileges Domain id=2 is tainted: host-cpu char device redirected to /dev/pts/0 (label charserial0) LOGS OVMF Jul 21 11:52:33 Tower kernel: usb 1-9.3: USB disconnect, device number 13 Jul 21 11:52:34 Tower kernel: usb 1-9.3: new low-speed USB device number 14 using xhci_hcd Jul 21 11:52:34 Tower kernel: input: Logitech USB Optical Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-9/1-9.3/1-9.3:1.0/0003:046D:C077.0010/input/input20 Jul 21 11:52:34 Tower kernel: hid-generic 0003:046D:C077.0010: input,hidraw3: USB HID v1.11 Mouse [Logitech USB Optical Mouse] on usb-0000:00:14.0-9.3/input0 Jul 21 11:52:57 Tower kernel: vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem Jul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered blocking state Jul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered disabled state Jul 21 11:52:57 Tower kernel: device vnet0 entered promiscuous mode Jul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered blocking state Jul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered forwarding state Jul 21 11:52:59 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258 Jul 21 11:52:59 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900 Jul 21 11:52:59 Tower kernel: vfio-pci 0000:00:1b.0: enabling device (0000 -> 0002) Jul 21 11:52:59 Tower acpid: input device has been disconnected, fd 6 Jul 21 11:52:59 Tower acpid: input device has been disconnected, fd 7 Jul 21 11:53:06 Tower kernel: usb 1-9.3: reset low-speed USB device number 14 using xhci_hcd Jul 21 11:53:07 Tower kernel: usb 1-9.3: reset low-speed USB device number 14 using xhci_hcd Jul 21 11:53:08 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcd Jul 21 11:53:08 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcd Jul 21 11:53:10 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound Jul 21 11:53:10 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound VM Logs 2017-07-21 18:52:57.121+0000: starting up libvirt version: 2.4.0, qemu version: 2.7.1, hostname: Tower LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-Windows 10/master-key.aes' -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/988a76c3-d21b-e56b-8e8c-a0f7c78f1c24_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 20480 -realtime mlock=off -smp 8,sockets=1,cores=4,threads=2 -uuid 988a76c3-d21b-e56b-8e8c-a0f7c78f1c24 -display none -no-user-config -nodefaults -chardev 'socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-Windows 10/monitor.sock,server,nowait' -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,i,nowait' -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x6,romfile=/mnt/user/drivers/test.dump -device vfio-pci,host=00:1b.0,id=hostdev1,bus=pci.0,addr=0x8 -device usb-host,hostbus=1,hostaddr=14,id=hostdev2,bus=usb.0,port=1 -device usb-host,hostbus=1,hostaddr=6,id=hostdev3,bus=usb.0,port=2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -msg timestamp=on Domain id=3 is tainted: high-privileges Domain id=3 is tainted: host-cpu char device redirected to /dev/pts/0 (label charserial0) 2017-07-21T18:54:12.473689Z qemu-system-x86_64: libusb_release_interface: -4 [NO_DEVICE] I performed a cat of the rom with the GTX980 in the second pci slot which worked successfully (attached). I then tested that rom by editing the xml on 2 seperate vms. one with SeaBIOS and the other OVMF, both of which gave me a black screen. Ive tried this with the GTX 980 in the primary PCI slot and the secondary slot with the same results. The vms tested above do work with passthrough when there are 2 GFX cards in the rig, when GT 710 is in the primary PCI slot and theGTX 980 in the secondary . Sorry for the lengthy post. TL;DR Cannot get to work with 1 GPU and rom dump. Will work with 2 GPU's GTX980test.dump

Hi gridrunner, Managed to get passthrough working tonight, the frustration got too much last night and i purchased a GT 710 on prime now to arrive today ? Still not sure what was causing the passthrough to fail with a single gpu. Tried editing several vbios from techpowerup with no success and I'm wondering if it was the vbios or the actual passthrough? I'm not really sure where to go with diagnosing this issue but so far I've taken a dump straight from the GTX 980 so I may take out the GT 710 - I jumped the gun here, dump size is 4 bytes and try passthrough with a single card again, it would be nice to just have the one in the rig EVGA GeForce GTX 980 SC ACX2.0 Not sure how to get my IOMMU groups and device, I'm still pretty new to all of this Thanks, i'll contact them in the next couple days. Now the real work starts now I'm partially over that hurdle

No luck so far, I'm getting the error log below. VM starts up and stuck with black screen LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-Windows 10/master-key.aes' -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/943beee4-6867-a88e-a068-e5aed337923e_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 21504 -realtime mlock=off -smp 8,sockets=1,cores=4,threads=2 -uuid 943beee4-6867-a88e-a068-e5aed337923e -display none -no-user-config -nodefaults -chardev 'socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-Windows 10/monitor.sock,server,nowait' -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,i0,server,nowait' -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x5,romfile=/mnt/user/drivers/EVGA.GTX980.4096.rom -device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x6 -device usb-host,hostbus=1,hostaddr=6,id=hostdev2,bus=usb.0,port=2 -device usb-host,hostbus=1,hostaddr=7,id=hostdev3,bus=usb.0,port=3 -device usb-host,hostbus=1,hostaddr=2,id=hostdev4,bus=usb.0,port=4 -device usb-host,hostbus=1,hostaddr=5,id=hostdev5,bus=usb.0,port=1.1 -device usb-host,hostbus=1,hostaddr=3,id=hostdev6,bus=usb.0,port=1.2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on Domain id=1 is tainted: high-privileges Domain id=1 is tainted: host-cpu char device redirected to /dev/pts/0 (label charserial0)

Hi @gridrunner I feel so stupid, i watched it on my iPhone when you posted it and inadvertently watched the posts original video again. Come to think of it i did think it was strange to post the same video twice ?? Thanks for your prompt response, i will try the rom you attached tomorrow and let you know how i get on.

I'm having no luck getting my gtx 980 passed through. Im struggling to locate a second card perform a vbios dump like you demonstrated in the video. I've even put together an old rig only to find out it doesn't support IOMMU. So i've ended up trying multiple different vbios from the techpowerup.com to no avail. I'm really at a loss here and pulling my hair out, my free trial runs out in a few days and i'd really like to get my windows vm with gpu passthrough working so i can continue with my project. I've posted my machine spec and xml below in case I've done something glaringly obviously wrong, any help appreciated, thanks. Machine Spec: Model: Custom M/B: ASUSTeK COMPUTER INC. - X99-DELUXE II CPU: Intel® Core™ i7-6800K CPU @ 3.40GHz HVM: Enabled IOMMU: Enabled Cache: 384 kB, 1536 kB, 15360 kB Memory: 32 GB (max. installable capacity 512 GB) Network: bond0: fault-tolerance (active-backup), mtu 1500 eth0: 1000 Mb/s, full duplex, mtu 1500 eth1: not connected Kernel: Linux 4.9.30-unRAID x86_64 XML <domain type='kvm'> <name>Windows 10</name> <uuid>943beee4-6867-a88e-a068-e5aed337923e</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>22020096</memory> <currentMemory unit='KiB'>22020096</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> <vcpupin vcpu='4' cpuset='6'/> <vcpupin vcpu='5' cpuset='7'/> <vcpupin vcpu='6' cpuset='8'/> <vcpupin vcpu='7' cpuset='9'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/943beee4-6867-a88e-a068-e5aed337923e_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='4' threads='2'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win10_1607_English_x64.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.126-2.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk1.img'/> <target dev='hdc' bus='sata'/> <boot order='1'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <controller type='usb' index='0' model='nec-xhci'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:ea:e7:f3'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/Drivers/gtx980-141103.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc52e'/> </source> <address type='usb' bus='0' port='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0b05'/> <product id='0x180a'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x2433'/> <product id='0xb200'/> </source> <address type='usb' bus='0' port='3'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </memballoon> </devices> </domain>

Does anyone have a gtx 980 rom dump? Or is there any way to get a rom dump in my setup without a 2nd temporary GPU?