[GUIDE] Virtualizing unRAID on Proxmox 3.1

[mangocrysis]

I am about to embark on this journey. When you move unraid to a VM, how do you move all the docker data? Like my current running dockers and their settings? I am not moving my current cache drive. Do I just restore my appdata backup? Is that sufficient? 

 

I am also curious if it's possible to pass through a LVM-thin from the PVE host to unraid to share data as I'd like to have a shared drive between unraid and pve on top of the HBA i'm passing through to unraid. 

 

I could pass through a drive and make it available as a NFS from within unraid and then access it via proxmox but that kind of feels more complicated than it needs to be.

[s.Oliver]

11 hours ago, mangocrysis said:

I am about to embark on this journey. When you move unraid to a VM, how do you move all the docker data? Like my current running dockers and their settings? I am not moving my current cache drive. Do I just restore my appdata backup? Is that sufficient?

well, i used the cache ssd in passthrough mode for several reasons: speed and avoidance of write amplification. for a fast/responsive system, unRAID needs a fast device for it's docker storage and i didn't want to use a block device on another storage in another system. so this has the advantage of (in case it's necessary) to simply boot from the usb-stick natively in unRAID and have everything working (because cache device was passed through).

 

11 hours ago, mangocrysis said:

I am also curious if it's possible to pass through a LVM-thin from the PVE host to unraid to share data as I'd like to have a shared drive between unraid and pve on top of the HBA i'm passing through to unraid.

might work, but keep in mind you need to have a filesystem which is built for multi-/user access, or risk corruption of data. 

 

11 hours ago, mangocrysis said:

I could pass through a drive and make it available as a NFS from within unraid and then access it via proxmox but that kind of feels more complicated than it needs to be.

more reliable for me (than NFS) was a simple SMB-share from unRAID, configured as storage inside PROXMOX. it's fast and simply works.

Edited February 23, 2023 by s.Oliver

[mangocrysis]

Quote

more reliable for me (than NFS) was a simple SMB-share from unRAID, configured as storage inside PROXMOX. it's fast and simply works.

Thank you for your response and that's good to know and I think that's what I am going to do. NFS has been nothing but trouble. Especially with the mover messing with hardlinks and such. "Stale file handles" when the mover runs are the biggest problem. 

 

Quote

well, i used the cache ssd in passthrough mode for several reasons: speed and avoidance of write amplification. for a fast/responsive system, unRAID needs a fast device for it's docker storage and i didn't want to use a block device on another storage in another system. so this has the advantage of (in case it's necessary) to simply boot from the usb-stick natively in unRAID and have everything working (because cache device was passed through).

I didn't mean I was going to have a remote cache. I have trouble passing through the NVMe to the unraid VM because it's using a PCIe adapter. It keeps kernal panicking on me so I had to remove it. I have my HBA passed through without any issues.

 

My latest plan is to just passthrough the NVMe disk as a virtual device (iscsi2) to the VM and use that as the cache. From what I've read it's not much of a performance hit vs direct passthrough.

[s.Oliver]

12 hours ago, mangocrysis said:

I didn't mean I was going to have a remote cache. I have trouble passing through the NVMe to the unraid VM because it's using a PCIe adapter. It keeps kernal panicking on me so I had to remove it. I have my HBA passed through without any issues.

or you could use (if available) a free HBA-port to an sata SSD (albeit choose a good one) as an alternative for the NVMe.

 

12 hours ago, mangocrysis said:

My latest plan is to just passthrough the NVMe disk as a virtual device (iscsi2) to the VM and use that as the cache. From what I've read it's not much of a performance hit vs direct passthrough.

well, best option if direct passthrough isn't working/available. did you check your BIOS for updates? sometime those problems you mentioned are fixed later on.

[dodgypast]

I got it working pretty well.

 

It doesn't seem like passing hardware through to an UNRAID VM, and passing those through to a VM in UNRAID is feasible however I did get my Intel iGPU passed through to UNRAID and have my plex docker in UNRAID use that for transcoding which was a huge win. That was the only real issue, every other UNRAID docker and the VMs that didn't rely on hardware pass through worked fine. Drives just worked by passing through my SATA controllers for my storage and the nvme drive my cache is on. ( Proxmox is installed on an nvme drive )

 

It took a fair bit of trial and errror but here is the config file for my unraid VM in proxmox:

bios: ovmf

boot: order=usb0

cores: 20

cpu: host

efidisk0: local-lvm:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M

hostpci0: 0000:02:00,pcie=1

hostpci3: 0000:00:02,pcie=1,x-vga=1

hostpci4: 0000:00:17.0,pcie=1hostpci5: 0000:03:00.0,pcie=1

machine: q35

memory: 64000

meta: creation-qemu=7.1.0,ctime=1677658184

name: UNRAID

net0: e1000=BE:10:40:4C:9C:2E,bridge=vmbr0,firewall=1

numa: 0

onboot: 1

ostype: l26

scsihw: virtio-scsi-single

smbios1: uuid=d6b4046f-fef5-43bd-b70b-10aa383dd291

sockets: 1

usb0: host=0781:5567

vmgenid: c10f6ea6-ade4-4a61-9bb8-10045a7c2329

 

[modem7]

1 hour ago, dodgypast said:

I got it working pretty well.

 

It doesn't seem like passing hardware through to an UNRAID VM, and passing those through to a VM in UNRAID is feasible however I did get my Intel iGPU passed through to UNRAID and have my plex docker in UNRAID use that for transcoding which was a huge win. That was the only real issue, every other UNRAID docker and the VMs that didn't rely on hardware pass through worked fine. Drives just worked by passing through my SATA controllers for my storage and the nvme drive my cache is on. ( Proxmox is installed on an nvme drive )

 

It took a fair bit of trial and errror but here is the config file for my unraid VM in proxmox:

bios: ovmf

boot: order=usb0

cores: 20

cpu: host

efidisk0: local-lvm:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M

hostpci0: 0000:02:00,pcie=1

hostpci3: 0000:00:02,pcie=1,x-vga=1

hostpci4: 0000:00:17.0,pcie=1hostpci5: 0000:03:00.0,pcie=1

machine: q35

memory: 64000

meta: creation-qemu=7.1.0,ctime=1677658184

name: UNRAID

net0: e1000=BE:10:40:4C:9C:2E,bridge=vmbr0,firewall=1

numa: 0

onboot: 1

ostype: l26

scsihw: virtio-scsi-single

smbios1: uuid=d6b4046f-fef5-43bd-b70b-10aa383dd291

sockets: 1

usb0: host=0781:5567

vmgenid: c10f6ea6-ade4-4a61-9bb8-10045a7c2329

 

Why on earth would you try running a VM/Docker inside unraid? 

 

Nested Virtualisation is going to bite you at some point. 

 

If you're running unraid as a vm inside proxmox, you create vm's on proxmox, not in unraid. 

 

Your best bet for stability in this particular scenario is to run unraid as a pure nas VM, create something like an Ubuntu VM in proxmox for your hardware passthrough and docker needs (or LXC's), passing just your HBA through to the unraid VM. 

Edited March 26, 2023 by modem7

[dodgypast]

1 hour ago, modem7 said:

Your best bet for stability in this particular scenario is to run unraid as a pure nas VM, create something like an Ubuntu VM in proxmox for your hardware passthrough and docker needs (or LXC's), passing just your HBA through to the unraid VM. 

Guess what type of environment I'm building right now.

 

But since I do not have infinite spare time and I'm also running a service for my family, my priority wasn't immediate perfection. It was how to migrate my services from unraid to proxmox with the minimum of downtime.

 

I wanted to put the information out there for others who may find themselves in a similar situation.

[willmx]

On 3/26/2023 at 7:11 AM, modem7 said:

Your best bet for stability in this particular scenario is to run unraid as a pure nas VM, create something like an Ubuntu VM in proxmox for your hardware passthrough and docker needs (or LXC's), passing just your HBA through to the unraid VM. 

If I go this route, using Unraid purely for its NAS capabilities without running anything in docker and using the Ubuntu VM for docker instead, is there any point at all in having cache pools in Unraid?

 

I have run multiple Proxmox servers for several years and am familiar with all of the ins and outs, and I currently run a bunch of docker containers in an Ubuntu VM on two different Proxmox hosts.

 

But I’m completely new to Unraid and attracted to its NAS functionality since I’ve found myself in a position with a bunch of mixed drives.

So I plan on doing HBA passthrough and then sharing via SMB/NFS the Unraid shares with one or more Linux VMs in Proxmox. There’s no reason to run docker in Unraid for my use case, as I prefer my existing docker compose setup and will be setting up a K8s lab in the Linux VMs as well.

 

I suppose I won’t need cache pools in Unraid, right?

 

Are there any tips and tricks surrounding the NICs in the VMs? To get virtual 10GbE between them, maybe adjust the MTU as well?

 

I will also be accessing the Unraid SMB/NFS shares from other hosts on my mixed 1/2.5/10 GbE LAN.

Edited April 5, 2023 by willmx
Trim the quote

[modem7]

36 minutes ago, willmx said:

If I go this route, using Unraid purely for its NAS capabilities without running anything in docker and using the Ubuntu VM for docker instead, is there any point at all in having cache pools in Unraid?

 

I have run multiple Proxmox servers for several years and am familiar with all of the ins and outs, and I currently run a bunch of docker containers in an Ubuntu VM on two different Proxmox hosts.

 

But I’m completely new to Unraid and attracted to its NAS functionality since I’ve found myself in a position with a bunch of mixed drives.

So I plan on doing HBA passthrough and then sharing via SMB/NFS the Unraid shares with one or more Linux VMs in Proxmox. There’s no reason to run docker in Unraid for my use case, as I prefer my existing docker compose setup and will be setting up a K8s lab in the Linux VMs as well.

 

I suppose I won’t need cache pools in Unraid, right?

 

Are there any tips and tricks surrounding the NICs in the VMs? To get virtual 10GbE between them, maybe adjust the MTU as well?

 

I will also be accessing the Unraid SMB/NFS shares from other hosts on my mixed 1/2.5/10 GbE LAN.

The way I'll be doing it is:

 

2-3x Cache pools. 

1x for usenet downloads (which then gets moved by *arr to the appropriate shares which won't be cached) 

1x for Plex DB (btrfs mirrored set to cache only) 

1x (maybe) for scratch. 

 

If I can get away with keeping everything on the array (+ cache) rather than some things on a VM, I will. But the reality is I'm in the same boat as yourself atm, so I'm playing with options and preparing the infrastructure design. 

[modem7]

@willmx - It might be worthwhile checking this to see if this works: 

https://wiki.unraid.net/Manual/VM_Guest_Support#Unraid_as_a_VM

Edited April 24, 2023 by modem7

[spants]

I use unRaid as a vm in Proxmox now. 

 

unRaid has all my docker technology covered.

Proxmox has all my VMs (HomeAssitant, opnsense) covered

Proxmox is also used for LXC - primarily Tailscale

[Exceed]

many thanks for this great guide

[Zippi]

Hi everyone,

i read the whole thread.....
i installed an unRAID vm on ProxMox as a test.
How can i enabled IOMMU in unRAID?

I understand that in previous versions of ProxMox it was perhaps possible but now with version 8 it is no longer possible?

I know it doesn't make sense but I wanted to try switching a GPU to a nested Win11 VM as a test via unRAID

On ProxMox host is all OK.....i think.....

It's possible?