mikeyosm Posted December 10, 2016 Share Posted December 10, 2016 HI All I am trying to isolate some random very brief pops/crackles in sound using a Windows 10 VM. I have isolated it down to disk activity but not sure how I can fix it. Scenario/Setup UNRAID - 6.3rc6 VM1 - Windows Server 2012 - image located on SSD1 VM2 - Windows 10 latest patches (realtek assigned) - image located on NVME01 Onboard Realtek Audio using latest 2.80 realtek drivers With next to 0 disk activity on either VM audio plays fine on VM2. As soon as I stress test using ASSD disk bench tool on either VM, the sound playing on VM2 exhibits random pops/crackles. I can reproduce this every time. I am isolating the CPU cores in pairs and using emulatorpin cpuset on both VMs. At first I thought it could be related to the nvme drive my vm2 sits on, but then I did the same stress test on vm1 which is on a seperate SSD. Any ideas what might be causing this? thanks! Quote Link to comment
mikeyosm Posted December 11, 2016 Author Share Posted December 11, 2016 Well I have been doing some testing and research and it appears that it's definitely related to virtio/scsi. I can generate audio stutter/crackle by using ATTO Disk Bencmark, setting the transfer size to 12MB to 60MB and a total length 256MB. A transfer size below 12MB does not generate much if any crackle/pops in the audio so I'm curious why 12MB+ does? My VM image is in RAW format and I'm not using any disk 'tweaks' in the XML. Any ideas people? <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/disks/VM01/MOHOME/MOHOME.img'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> Quote Link to comment
SpaceInvaderOne Posted December 11, 2016 Share Posted December 11, 2016 How have you setup the isolation of cpus? Are they isolated from unraid in the syslinux or just paired in the xml? Which cores does each vm use for emulatorpin? How many vcpus does each vm have? which cpu do you have? Are you passing through a gpu on these vms. If so do you get same issue with the sound from the gpu? I see you have your disk as an assigned device here. Have you thought of passing through the whole disk/a partition to the vm and not using a vdisk? my disk xml looks like this (i passthough a partition on an unassigned disk) <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source dev='/dev/disk/by-id/ata-ST32000542AS_5XW1HXX5-part2'/> <target dev='hdd' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> Quote Link to comment
mikeyosm Posted December 11, 2016 Author Share Posted December 11, 2016 How have you setup the isolation of cpus? Are they isolated from unraid in the syslinux or just paired in the xml? Which cores does each vm use for emulatorpin? How many vcpus does each vm have? which cpu do you have? Are you passing through a gpu on these vms. If so do you get same issue with the sound from the gpu? I see you have your disk as an assigned device here. Have you thought of passing through the whole disk/a partition to the vm and not using a vdisk? my disk xml looks like this (i passthough a partition on an unassigned disk) <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source dev='/dev/disk/by-id/ata-ST32000542AS_5XW1HXX5-part2'/> <target dev='hdd' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> Firstly, answers to your q's.... How have you setup the isolation of cpus? isolcpus=0-8,10-18 (leaving 9,19 as paired cores for UNRAID) Are they isolated from unraid in the syslinux or just paired in the xml? syslinux - emulatorpin didn't help much so i left it out Which cores does each vm use for emulatorpin? VM1 (0-3,10-13) VM2 (4-5,14-15) How many vcpus does each vm have? 8 cores, 4 cores which cpu do you have? xeon 2660v3 Are you passing through a gpu on these vms. If so do you get same issue with the sound from the gpu? Yes, didn't try GPU HDMI sound. With regards to partition passthrough, can I have UNRAID using partition1 and assign partition2 to a VM? Or does the nvme drive need to be completely unassigned from UNRAID to passthrough a partition? Quote Link to comment
SpaceInvaderOne Posted December 11, 2016 Share Posted December 11, 2016 How have you setup the isolation of cpus? Are they isolated from unraid in the syslinux or just paired in the xml? Which cores does each vm use for emulatorpin? How many vcpus does each vm have? which cpu do you have? Are you passing through a gpu on these vms. If so do you get same issue with the sound from the gpu? I see you have your disk as an assigned device here. Have you thought of passing through the whole disk/a partition to the vm and not using a vdisk? my disk xml looks like this (i passthough a partition on an unassigned disk) <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source dev='/dev/disk/by-id/ata-ST32000542AS_5XW1HXX5-part2'/> <target dev='hdd' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> Firstly, answers to your q's.... How have you setup the isolation of cpus? isolcpus=0-8,10-18 (leaving 9,19 as paired cores for UNRAID) Are they isolated from unraid in the syslinux or just paired in the xml? syslinux - emulatorpin didn't help much so i left it out Which cores does each vm use for emulatorpin? VM1 (0-3,10-13) VM2 (4-5,14-15) How many vcpus does each vm have? 8 cores, 4 cores which cpu do you have? xeon 2660v3 Are you passing through a gpu on these vms. If so do you get same issue with the sound from the gpu? Yes, didn't try GPU HDMI sound. With regards to partition pass-through, can I have UNRAID using partition1 and assign partition2 to a VM? Or does the nvme drive need to be completely unassigned from UNRAID to pass-through a partition? Ok, so you have a 2660v3 that has 10 cores. I would split it like this. The first 2 cores are for unRAID. But also pinned for the emulation calls 0 9 unRAID/emulatorpin 1 10 unRAID/emulatorpin Next 4 cores for vm 1 2 11 vm1 isolcpu 3 12. vm1 isolcpu 4 13. vm1 isolcpu 5 14. vm1 isolcpu Next 4 cores for vm2 6 15. vm2 isolcpu 7 16. vm2 isolcpu 8 17. vm2 isolcpu 9 18. vm2 isolcpu So as well the VMS will both have the first 2 cores used for emulatorpin <emulatorpin cpuset=‘0,1,9,10'/> vm1 <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset=‘2’/> <vcpupin vcpu='1' cpuset=‘3’/> <vcpupin vcpu='2' cpuset=‘4’/> <vcpupin vcpu='3' cpuset=‘5’/> <vcpupin vcpu='4' cpuset=’11’/> <vcpupin vcpu='5' cpuset='12’/> <vcpupin vcpu='6' cpuset='13’/> <vcpupin vcpu='7' cpuset=’14’/> <emulatorpin cpuset=‘0,1,9,10'/> </cputune> vm2 <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset=‘6’/> <vcpupin vcpu='1' cpuset=‘7’/> <vcpupin vcpu='2' cpuset=‘8’/> <vcpupin vcpu='3' cpuset=‘9’/> <vcpupin vcpu='4' cpuset=’15’/> <vcpupin vcpu='5' cpuset='16’/> <vcpupin vcpu='6' cpuset='17’/> <vcpupin vcpu='7' cpuset=’18’/> <emulatorpin cpuset=‘0,1,9,10'/> </cputune> First, try not using isolcpu in syslinux at all see how it is without. Then try with isolcpu as below append isolcpus=2-9,11-18 initrd=/bzroot If using partition pass-through, the drive must be unassigned to the array. You can split the drive into, say, 2 partitions. Format the first as xfs. On here you could put vdisks etc and/or share it using unassigned devices plugin. The other partition format as NTFS and use for windows. The way I use mine is to have an 80gig vdisk (on my SSD cache) for the os drive on the VM. Then I pass-through the partition 1tb (on unassigned 2tb mechanical HDD)to the VM where I store all my game installs. This way I keep a backup of the 80 gig vdisk. If I have a windows problem I can just then replace the 80 gig vdisk with the backup and don't have to reinstall all the games. The other partition on the passed through drive I have formatted as xfs and I store vdisk images there, of VMS that I don't often use Quote Link to comment
mikeyosm Posted December 11, 2016 Author Share Posted December 11, 2016 Thanks for the very thorough response dude. However, the dashboard in Unraid says my pairs are 1,11 2,12 etc. This goes against your suggestions for CPU pinning? Quote Link to comment
SpaceInvaderOne Posted December 11, 2016 Share Posted December 11, 2016 opps sorry my bad. Can you just cut and paste the pairings here and i will re do it Quote Link to comment
mikeyosm Posted December 11, 2016 Author Share Posted December 11, 2016 cpu 0 / 10 cpu 1 / 11 cpu 2 / 12 cpu 3 / 13 cpu 4 / 14 cpu 5 / 15 cpu 6 / 16 cpu 7 / 17 cpu 8 / 18 cpu 9 / 19 Quote Link to comment
SpaceInvaderOne Posted December 12, 2016 Share Posted December 12, 2016 cpu 0 / 10 cpu 1 / 11 cpu 2 / 12 cpu 3 / 13 cpu 4 / 14 cpu 5 / 15 cpu 6 / 16 cpu 7 / 17 cpu 8 / 18 cpu 9 / 19 Ok thanks lets try that again! So try splitting the 10 cores like this The first 2 cores are for unRAID. But also pinned for the emulation calls 0 10 unRAID/emulatorpin 1 11 unRAID/emulatorpin Next 4 cores for vm 1 2 12 vm1 isolcpu 3 13 vm1 isolcpu 4 14 vm1 isolcpu 5 15. vm1 isolcpu Next 4 cores for vm2 6 16. vm2 isolcpu 7 17 vm2 isolcpu 8 18 vm2 isolcpu 9 19. vm2 isolcpu So as well the VMS will both have the first 2 cores used for emulatorpin <emulatorpin cpuset=‘0,1,10,11'/> vm1 <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset=‘2’/> <vcpupin vcpu='1' cpuset=‘3’/> <vcpupin vcpu='2' cpuset=‘4’/> <vcpupin vcpu='3' cpuset=‘5’/> <vcpupin vcpu='4' cpuset=’12’/> <vcpupin vcpu='5' cpuset='13’/> <vcpupin vcpu='6' cpuset='14’/> <vcpupin vcpu='7' cpuset=’15’/> <emulatorpin cpuset=‘0,1,10,11’/> </cputune> vm2 <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset=‘6’/> <vcpupin vcpu='1' cpuset=‘7’/> <vcpupin vcpu='2' cpuset=‘8’/> <vcpupin vcpu='3' cpuset=‘9’/> <vcpupin vcpu='4' cpuset=’16’/> <vcpupin vcpu='5' cpuset='17’/> <vcpupin vcpu='6' cpuset='18’/> <vcpupin vcpu='7' cpuset=’19’/> <emulatorpin cpuset=‘0,1,10,11’/> </cputune> First, try not using isolcpu in syslinux at all see how it is without. Then try with isolcpu as below append isolcpus=2-9,12-19 initrd=/bzroot Quote Link to comment
mikeyosm Posted December 12, 2016 Author Share Posted December 12, 2016 Thanks. I implemented all of your suggestions and I still get slight pops/static during audio whenever there is significant disk access from ATTO benchmark. In real world use, there is hardly any noticeable audio static during gaming and general media/web stuff. It wasn't until I thought of stress testing the virtio scsi capability that I came to the conclusion it was that affecting the audio stream in my VM. I think i will go down the route of a dedicated nvme drive for the vm and pass it through. That should fix it. Out of curiosity, please try ATTO bench as per my original post/instructions and see what happens with your audio stream (assuming you have a VM as a disk image that you can test against). Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.