Phoenix19

Virtualizing Unraid in XenServer

Recommended Posts

Hi All,

 

I am a newbie to unRAID. 

 

I was able to setup unRAID with no problem via the normal step by step instructions.I see there are some posts  that I read that seem to help with installing XenServer ,but they seem outdated. 

 

I was looking for advice for how to setup XenServer and then setup unRAID as a VM.

 

I also wanted to have a test lab for other os's such as Server 2012 and wanted to play around with different virtual os's. 

 

I know you can setup VM's via the unRAID gui with unRAID version 6.3.2. 

 

 

The posts that  I read  are -

 

https://lime-technology.com/unraid-6-virtualization-update/

https://forums.lime-technology.com/topic/29082-guide-virtualizing-unraid-in-xenserver-62-updated-to-504/

 

 

Also there are posts that seem to say that KVM is better than XenServer. Just like anything it might come down to a matter of opinion.

 

 

I'm sure both can do what I'm looking for ,but I'm looking for a stable option.

 

 

 

 

 

It appears that most of if not all of my hardware for the server I bought  is supported to install XenServer.

 

Here are some of the specs for the server I have -

 

 

 

Motherboard: Supermicro - X8DTU
Processor: Intel® Xeon® CPU E5645 @ 2.40GHz
HVM: Enabled
IOMMU: Enabled
Cache: L1-Cache = 384 kB (max. capacity 384 kB)
  L2-Cache = 1536 kB (max. capacity 1536 kB)
  L3-Cache = 12288 kB (max. capacity 12288 kB)
Memory: 68 GB (max. installable capacity 192 GB)

 

Network: bond0: fault-tolerance (active-backup), mtu 1500
  eth0: 1000Mb/s, full duplex, mtu 1500

 

 

Please advise .

 

Thank you.

 

Share this post


Link to post
Share on other sites

Hi,

If i remember correctly, i was able to run unRAID as VM on XenServer host in my test server some years ago, however i decide to stay on Vmvare ESXi cos of features and better knowledge about ESXi ..

keep in mind, unRAID is supported on bare-metal only..

Share this post


Link to post
Share on other sites

Thanks for getting back to me.

 

 

If you don't mind me asking , If it is bare- metal only ,why are you using it as a virtual machine or is there another reason you setup unRAID as a VM instead of the main host os ?

 

Thanks. 

 

 

Share this post


Link to post
Share on other sites
12 minutes ago, Phoenix19 said:

Thanks for getting back to me.

 

 

If you don't mind me asking , If it is bare- metal only ,why are you using it as a virtual machine or is there another reason you setup unRAID as a VM instead of the main host os ?

 

Thanks. 

I'm using unRAID since version 5, that version i think was supported as VM too in ESXi. 

since version 6 unRAID added KVM virtualization layer, since this version they said in various posts - bare-metal only..

 

For me, ESXi looks more stable, more flexible, more feature rich solution. i do not need restart it for very long time since it rocks very stable(for me for months without restart).. if you run unRAID as Host and need to restart it then all VMs needs to be powered off. this is main reason why i'm still on ESXi, but there are more other reasons too. there are some issues with USB pass-trough on some hardware(you can look at another threads on this forum) with some versions of unRAID. it affects me too, this is why i'm still on unRAID  version 6.1.9. I upgraded my ESXi Host to most recent version 6.5 two week ago, and looks like i can upgrade unRAID to latest version too - tried on my test server and it just works :) it's in my to-do list since upgrade 6.1->6.2 needs some more actions to be taken.

Share this post


Link to post
Share on other sites

So with the latest version that I have 6.3.2 ,will I be able to install ESXI 6.5 and create an unRAID VM and take advantage of the array ?

 

Is that the original issue as to where it couldn't virtualize or pass through the hdd's and that's why some recommendations are to use bare-metal only ?

 

 

 Hopefully this doesn't get confusing too fast , I can see how it might .

 

I would like to accomplish various tasks -

 

1.) I would like to setup a test lab where I can get comfortable with a VM os such as ESXI,KVM or Xen,etc.

2.) Utilize the unRAID OS and use it how I am now as a network attached storage.

3.) With my test lab I would install Windows Server 2012 or another Linux distro possibly

4.) Eventually set this up a media server. 

 

 

 

I appreciate you giving your insight.I'm very excited about this and want to continue to learn and have fun doing this.

 

Thanks. :D

 

 

 

Share this post


Link to post
Share on other sites

As for bare-metal - i think they just say: we support bare-metal only cos we have our own hyper-visor (KVM).

in general, you will be able to setup ESXi with unRAID as VM i think. You wrote some hardware info above, can you please provide more details? exact motherboard number (we have very similar motherboards by the way, see my signature), sata cards you have.. how you plan to connect your HDDs/SSDs?

 

if you wanna more information about ESXi and unRAID, you can try to read this old thread: 

many of us are inspired from this one including me :) there are very detailed instructions and how too's. versions are old, but in general changes are so minimal..

(here is the link above from the old forum - formatting looks much better http://lime-technology.com/oldforum/index.php?topic=14695.0 )

Share this post


Link to post
Share on other sites

Sorry it took so long to get back to you.  I have the unRAID trail version for now.

 

In regards to connecting the hard drives ,they are connected to the jbod controller. When you ask how I plan to connect my HDDs/SSD, do you mean in ESXI ?

 

If  it would be in ESXI, I'm not sure I'm a little new to ESXI.I have used it a little for work.

 

 

I was just starting to look at the links.

 

My specs are below. I attached my raid array config as well. 

 

 

Server Chassis/ Case

Supermicro 2u SC825TQ-R720UB

Back Plane

BPN-SAS-825TQ (8x 3.5" SAS/SATA Hot-swappable Drive Bays)

Motherboard

X8DTU-F Motherboard

CPU Processor

Dual Intel Xeon E5645 Six core 

RAM Memory

64GB DDR3 ECC REG Memory (8 x 8GB)

Hard Drives/ Caddies

None. come with 8 3.5" Caddies

RAID Controller

8 Ports are connected to LSI 9211-8i 6GB/S HBA JBOD IT-MODE controller 

IPMI Management

Onboard dedicate IPMI 2.0 Management port
Intel X520-DA2 dual 10GB SFP+ (no optics) Expansion Card PCI-E Installed
Onboard Dual 1GBE NICS 

Power Supply

Dual Supermicro 720W REDUNDANT POWER SUPPLYPWS-721P-1R Gold Certified

RAILS

come with Rail kit

EXTRA

Slim DVD Rom drive, SuperMicro RSC-R2UU-UA3E8+ Left Slot Riser Card PCI Express x8

 

 

 

 

 

 

unRAID system: unRAID server Trial, version 6.3.2
   
Motherboard: Supermicro - X8DTU
Processor: Intel® Xeon® CPU E5645 @ 2.40GHz
HVM: Enabled
IOMMU: Enabled
Cache: L1-Cache = 384 kB (max. capacity 384 kB)
  L2-Cache = 1536 kB (max. capacity 1536 kB)
  L3-Cache = 12288 kB (max. capacity 12288 kB)
Memory: 68 GB (max. installable capacity 192 GB)
  BANK0 = 8192 MB, 1333 MHz
  BANK1 = 8192 MB, 1333 MHz
  BANK2 = 8192 MB, 1333 MHz
  BANK4 = 8192 MB, 1333 MHz
  BANK6 = 8192 MB, 1333 MHz
  BANK7 = 8192 MB, 1333 MHz
  BANK8 = 8192 MB, 1333 MHz
  BANK10 = 8192 MB, 1333 MHz
  ROM0 = 4096 kB, 33 MHz
Network: bond0: fault-tolerance (active-backup), mtu 1500
  eth0: 1000Mb/s, full duplex, mtu 1500
  eth1: not connected
  eth2: not connected
  eth3: not connected

 

 

 

Tower specs.JPG

Share this post


Link to post
Share on other sites

Your Specs looks very well.

and about drives - if you go ESXi route, then you install ESXi host on addition USB. and you will need some additional hdd/ssd(connect them to MB ports in ESXi Case) for your VMs in either case. to create unRAID VM, you don't need additional hdd, just use plopKExec (google is your friend) to boot from unRAID USB stick. and pass-trough your HBA to unRAID VM and you are done :)   

Share this post


Link to post
Share on other sites

I'm still trying to go through the ATLAS Virtual documentation. I have been kind of busy working on other stuff. I purchased another hdd. I bought a wd 4 tb black drive.

 

 

 

I know you've read the article so I don't want to repeat what you already read ,but I'm trying to gather what I need to use for my own build.:) You sound patient. Hopefully you will be after my 100 questions. Lol.

 

 

I see in the article the guy talks about having 2 SSD drives for ESXI Datastore Drives. I only have one 128 gb cheap SSD drive that I can use. He mentioned that running ssd's in ESXI will wear them out and a fast rate. Then he mentioned about getting certain drives that have a chipset  will utilize the advanced garbage collection and is designed for this.

 

Do I need to get any special ssd drive/drives ?

 

 

I already downloaded  ESXI Version 6 update 2 and created a  bootable esxi 8gb  flash drive 

 

You mentioned I would have to connect additional hdd/ssd drives to my internal sata ports ,correct ? Is this for the passthrough or is this for the datastore ?

 

 

 

I'm googling and using the ATLAS article as a  base ,but I'm still trying to prepare for everything .

 

 

Thanks. :)

 

 

 

 

 

 

 

 

 

Share this post


Link to post
Share on other sites
3 hours ago, Phoenix19 said:

I'm still trying to go through the ATLAS Virtual documentation. I have been kind of busy working on other stuff. I purchased another hdd. I bought a wd 4 tb black drive.

This is ok drive for esxi Datastore.

 

3 hours ago, Phoenix19 said:

I see in the article the guy talks about having 2 SSD drives for ESXI Datastore Drives. I only have one 128 gb cheap SSD drive that I can use. He mentioned that running ssd's in ESXI will wear them out and a fast rate. Then he mentioned about getting certain drives that have a chipset  will utilize the advanced garbage collection and is designed for this.

 

Do I need to get any special ssd drive/drives ?

i think you can start with with your 4TB WD black. but of course, SSD for datastore will be much faster. And according to garbage collection - I've been reading about many SSDs and about they garbage collections and didn't find one answer how to choose. yes, ESXi does not have Trim, and then SSD need to have some builtin garbage collection. i have and Plextor M5Pro and Samsung Evo as my datastore sdd's, and they perform both very well.

 

3 hours ago, Phoenix19 said:

I already downloaded  ESXI Version 6 update 2 and created a  bootable esxi 8gb  flash drive

i would recommend to go with Esxi 6.5 to avoid potential USB pass-trough problem, but maybe on your hardware this will be OK with version 6 too. if you created bootable instalation flash, then you need another flash on which to Install. you can install without any other HDDs/SSDs connected. just boot your Install flash and follow onscreen instructions.  

 

3 hours ago, Phoenix19 said:

You mentioned I would have to connect additional hdd/ssd drives to my internal sata ports ,correct ? Is this for the passthrough or is this for the datastore ?

connect to Motherboard ports only SSDs/HDDs for datastore, cos you cant pass-through MB sata ports.

Share this post


Link to post
Share on other sites

So this might sound like a stupid question . It's in regards to power.

 

I know one way for sure that I can provide power to the 4tb datastore drive ,but wanted to see if you ever heard of or have tried this.

 

So there are three molex psu cables ,two of the molex cables plug right into the Back Plane. The third one isn't plugged into anything. I know that I could buy a molex to sata adapter cable.

I was wondering if I could connect the 4tb hard into one of the hard drive bays ,use the power connector that is already provided from the back plane ,then disconnect the sata cable that is coming from the  HBA JBOD controller and connect a sata cable that will be connected to one of the onboard sata ports and the other end into the sata port where the 4tb drive is connected ?

 

Unless this is the way that it should be connected ??

 

 

 

Hopefully I'm not making this more difficult than it should be.Lol.

 

 

Thanks.

Share this post


Link to post
Share on other sites
7 hours ago, Phoenix19 said:

I was wondering if I could connect the 4tb hard into one of the hard drive bays ,use the power connector that is already provided from the back plane ,then disconnect the sata cable that is coming from the  HBA JBOD controller and connect a sata cable that will be connected to one of the onboard sata ports and the other end into the sata port where the 4tb drive is connected ?

You can do this way, but this depends on your hardware.. for example, mine every 4 bays slots are connected with one SFF cable to HBA card. so, then i need breakout cables to split SFF connection to SATA. how is connect your gays to HBA, which data cable? 

Share this post


Link to post
Share on other sites

 

Esxi was able to see the 4 tb drive.

 

There is a SFF cable connected in every port in the back plane which then goes to the HBA card.So there is a total of 8 drive bars and 8 connectors. I just disconnected one of the SFF cables and connected a sata data cable from the drive bay that the 4 tb wd black drive is in and connected the other end to one of the onboard sata ports. I posted  pictures of the inside of server. The black sata data cable is the one that is connected to my 4 tb wd black drive.

 

I was able to setup the passthrough 

 

 

This whole process is  definitely an adventure.I was able to install Esxi 6.5  . I then followed the instructions that ATLAS had to download PLOP,WIN SCP and Putty. I was able to boot into unRAID and login to the previous setup.

 

However it doesn't detect any of my drives. I was looking at one article that the guy had a similar issue where it wasn't seeing the license file and he had to rebuild the unRAID usb stick with a clean version. I'm going to try that ,but I don't know if that would be why I'm having issues with unRAID not seeing the drives.

 

 

 

I didn't understand the part below . I was able to open my passthru.map file and added the false entry ,but I wasn't able to find the volumes folder where the vmx file was at. I posted my pass through file.

 

I also was able to toggle passthrough the  LSI HBA card  in vmware as well.

 

 

 

 

Applying the MV8 Hack


With "Remote Tech Support" enabled, use WinSCP to connect to ESXi, and add there two lines to the /etc/vmware/passthru.map file:
ev6Xlm.png
Quote
# Marvell Technologies, Inc. MV64460/64461/64462 System Controller, Revision B
11ab  6485  d3d0     false
 
EE0ozm.png

Now open your VM's .vmx file and change this:
Quote
pciPassthru0.present = "TRUE"
pciPassthru0.deviceId = "6485"
pciPassthru0.vendorId = "11ab"
pciPassthru0.systemId = "4dfc27f9-93be-d5c1-9198-00259027d9d8"
pciPassthru0.id = "01:00.0"
 

to this:
 
Quote
pciPassthru0.present = "TRUE"
pciPassthru0.msiEnabled = "FALSE"
pciPassthru0.deviceId = "6485"
pciPassthru0.vendorId = "11ab"
pciPassthru0.systemId = "4dfc27f9-93be-d5c1-9198-00259027d9d8"
pciPassthru0.id = "01:00.0"
 
tfYoym.png
The catch is force the use of IOAPIC mode with the "pciPassthru0.msiEnabled = 'FALSE'" statement.


Reboot the hypervisor and start your unRAID VM!

Good luck.
 

* Note, the pciPassthru0 # could be different depending the number of cards you have passed though. For my last rebuild, it was pciPassthru3 

 

 

Supermicro 1.jpg

Supermicro2.jpg

Supermicro3.jpg

Supermicro4.jpg

passthru file for vmware.JPG

Share this post


Link to post
Share on other sites

First, congrats you on able to build up your ESXi and setup unRAID VM! :)

 

13 minutes ago, Phoenix19 said:

Applying the MV8 Hack

are you sure, you need this ? you wrote you have LSI 9211-8i card, but this hack is needed for supermicro cards..

 

when you boot up your unRAID, are you able to see your drives in drive slots choose under Array Devices?

Share this post


Link to post
Share on other sites

Thats a good question. That makes sense. I don't have the Marvell ,but I thought it was a generic option to put in.

 

 

I swear everytime something goes right ,something else goes wrong with this process. So I did a clean install on unRAID and now when I login to unRAID I can't use the mouse ,but I can use the tab key.

 

 

After doing the clean install it is now having me register the trail key. I will register it later and let you know if it shows the drives.

 

 

Thanks !!!

 

Share this post


Link to post
Share on other sites
14 hours ago, Phoenix19 said:

After doing the clean install it is now having me register the trail key. I will register it later and let you know if it shows the drives.

in theory, there is no need for fresh install when you going from Esxi to bare-metal or vice versa. just choose which USB to boot and you are on ESXi or bare-metal :) 

Share this post


Link to post
Share on other sites

I have already performed a clean install on it,so at this point I have to register it. 

 

Before I did a clean install on the flash drive , I installed the drives for the array. The drives had a message that said they were missing. I then shut down the  VM, then restarted the vmware server and then logged back into vmware and then started the unRAID vm up again and they still weren't showing up.

I think this might have to do with the pass-through options possibly.  Also I can't move the mouse around to click on anything. It appears as though I can use the tab key and go up and down,but still no mouse.

 

I just found out that I'm not getting a network connection within unRAID.

 

I tried to copy and paste the register key ,but it gave me an error. 

 

I then tried to go out to several websites such as google and yahoo ,but wasn't able to. I then restarted and then went to the command line unRAID OS.

 

I then get Tower Login: Device "eth0" does not exist.

 

When I run the ifconfig eth0 I get error fetching interface information:device not found.

 

I tried to edit the syslinux.cfg file ,but that didn't work.

 

Sorry for laying all of this information out on you. It seems like it is more work to try to get unraid to work through vm.Lol

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Share this post


Link to post
Share on other sites
4 hours ago, Phoenix19 said:

I have already performed a clean install on it,so at this point I have to register it. 

 

Before I did a clean install on the flash drive , I installed the drives for the array. The drives had a message that said they were missing. I then shut down the  VM, then restarted the vmware server and then logged back into vmware and then started the unRAID vm up again and they still weren't showing up.

I think this might have to do with the pass-through options possibly.  Also I can't move the mouse around to click on anything. It appears as though I can use the tab key and go up and down,but still no mouse.

 

I just found out that I'm not getting a network connection within unRAID.

 

I tried to copy and paste the register key ,but it gave me an error. 

 

I then tried to go out to several websites such as google and yahoo ,but wasn't able to. I then restarted and then went to the command line unRAID OS.

 

I then get Tower Login: Device "eth0" does not exist.

 

When I run the ifconfig eth0 I get error fetching interface information:device not found.

 

I tried to edit the syslinux.cfg file ,but that didn't work.

 

Sorry for laying all of this information out on you. It seems like it is more work to try to get unraid to work through vm.Lol

Please, post a picture of unRAID VM Settings.

Share this post


Link to post
Share on other sites

I attached the settings to my unRAID vm and also the pci pass through devices that come up .

 

I was reading other forums talking about how there AMD(Advanced Micro Devices) network card wouldn't work in vmware ,but they got it to work in virtualbox by changing the chip set to Intel .

 

 

 

I also see the speed difference between having a datastore on a hdd and sdd. The boot up of unraid is slow. 

 

 

There is probably something that I didn't do right  , such as selecting the wrong pass-through device or not selecting the correct one. hopefully and maybe you'll be able to see it.

 

 

 

 

 

 

 

 

Screenshot of unRAID VM settings 1.JPG

Screenshot of unRAID VM settings 2.JPG

vmware unraid pci passthrough screenshot 1.JPG

vmware unraid pci passthrough screenshot 2.JPG

Share this post


Link to post
Share on other sites

Well I have network connectivity now. I changed the network adapter in the vm to use the E1000  adapter type.

 

 

I still can't actually move the mouse around. I thought it might be a lagging issue ,but I have moved the mouse around to see if the mouse arrow moves ,but I waited about 10 to 15 minutes and it never moves. I can move around a little by using the tab key.

Share this post


Link to post
Share on other sites

So now I tried to see if I could now assign the drives are plugged into the backplane ,but it doesn't show up with any of them. I think it could be that I may need to add the LSI PCI device. Not sure if that will allow the drives to be detected.

 

I post added screenshots of the system devices and also the pci device option I was 

System devices 2.JPG

Sytem devices unraid.JPG

Array not coming up unraid vm.JPG

PCI device unraid.JPG

Share this post


Link to post
Share on other sites

So I seem to keep digging deeper and playing around. As I do so I seem to be answering my own questions.Lol. :)

 

 

So now I got unRAID to see the array drives. I still don't know why the mouse emulation isn't working . That is very bothersome.

 

I'm going to see how it goes with assigning the drives and see if it will work out for me.At least the drives are showing up. I'll see how it goes from there.

 

 

Let me know about the mouse emulation ,if you see anything in any of the screenshots that I may need to change.

 

 

 

 

 

 

 

Share this post


Link to post
Share on other sites

Congrats to you to get this up and running, a few more tips from me:

- use VMXNET3 as your network adapter, this enables 10Gbit networking inside ESXi between VMs

- use plopKExec to speed up boot times - https://www.plop.at/en/plopkexec/full.html

- are you booting GUI version of unRAID? i think no need of that. just boot non-gui version and then connect to your unRAID from LAN from another device by http://<your-unRAID-ip>/

- every ESXi VM, including unRAID needs VMWare tools installed inside it. for unRAID use this one: https://forums.lime-technology.com/topic/36603-open-vm-tools-for-unraid-6/ 

good luck, if you have more questions, just ask :)

 

Share this post


Link to post
Share on other sites

and about your PCI devices - why you are passing so many devices? for unRAID you just need LSI 2008 to pass-trough. if you pass-trough a device, it's not visible to ESXi host, and then you need to add it to particular VM and then this device is used only for this VM. Virtual network from ESXi works just fine, so does USB attaching to VM too.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.