ATLAS My Virtualized unRAID server


Recommended Posts

So I'm getting close to purchasing some hardware. Because I currently live in Dubai, and have to order equipment from US and get it freight forwarded (Dubai's crap for this sort of thing), I am trying to cover all the bases and get it right first time. Much easier than trying to return/get new hardware at a later date.

The problem you are going to run into is if you get a piece part that does not work and is DOA

Link to comment

I know, but I don't see that I have much choice. I can get most parts here, it is just the mobo I think. Also I found a user with M1015 here on the forums that will sell me a card and even flash it before sending it, so that is low risk.

 

Thanks for your advice, it is a lot clearer now.

Link to comment

From your earlier recommendation I am thinking of getting the X8Sil instead of the board you are using. I understand this makes it easier to get 32GB memory now, but I'm also sacrificing the onboard SATA3 connections, which I was thinking I could use for some fast SSD drives for running VMs. Do you think this is a reasonable tradeoff and would you still recommend the X8SIL instead of X9SCM?

I like my X9SCM just fine. I was just putting it out there there that the X8SIL can run ESXi with 32GiB of RAM now (for a hefty price), while the X9SCM is lacking 8GiB RAM sticks. Honestly, for casual use, the 16 Gib is OK.

(...)

ESXi tends to run out of RAM before CPU.

Again, for home use, unless you are going crazy, 16GiB Is fine.

 

As far as using SSD for a datastore, I am sort of an oddball that I use SSD's for my datastore.

 

It is worth mentioning in this context, that ESXi 5 identifies SSDs natively and uses them for fast swap space when it runs out of RAM. That also alleviates the need for more than 16GB RAM, in particular given a quick SSD on SATA3 on the X9SCM board with the fast processors it supports  ;D

Link to comment

Be aware the M1015 needs to be reflashed, you can not do it on a lot of motherboards including the x9scm (or x8sil?).

 

The SuperMicro X8SI-Family of boards supports flashing the re-badged LSI Controllers. Hopefully I can get around to putting together what everyone posted, Motherboards that do and do not work on my LSI post. To be clear once flashed they pretty much work on any MB.

 

Link to comment

I thought you had m1015's. if you have MV8's, 5B13 might be good for you. it is a bit early to tell if you went beta.

I think i confused your build with Nia's.

5B6a would be my recommendation then if you do not run macs.

 

You are sticking to 4.7 so.. that is irreverent.

 

I just created a new win7 VM. I'll see if i can update that booting tip later today

 

 

about update.

I wrote a quick esx setup (NTP) and update (patches) running only through the cli, so it is pretty simple.

do you want me to post it here so you can include it ?

 

Cheers,

R

 

anything that you feel that can help others. even I come back to the build notes when I forget something

 

ok.. I did not want to hijack your topic.

I will setup a clean computer and another esx and will come up with some upgrade path for the esx by monday.

I will try to make as many screenshot as possible.

 

Cheers,

R

Link to comment

It is worth mentioning in this context, that ESXi 5 identifies SSDs natively and uses them for fast swap space when it runs out of RAM. That also alleviates the need for more than 16GB RAM, in particular given a quick SSD on SATA3 on the X9SCM board with the fast processors it supports  ;D

 

Yes it is good to mention that SSD's are usable in ESXi 5 as cache to help with the limited ram.

It may be slower then RAM, but it is a lot cheaper and seems to be effective.

 

 

Link to comment

Johnm,

Don't these ssd drives have on controller garbage cleanup therefore negating TRIM support somewhat?  I know my MacBook has an ssd but apple doesn't support it for non apple approved SSD but I've read it doesn't matter cause my Vertex 2 takes care of it on the controller.

 

Yes if you get one with garbage collection I believe so.

 

I know the  the Kingston pro v+ (Plus) do have Trim and Garbage collection from the Bios so it is not controlled by the OS.

 

The SSD's I am using in here were for pure performance and not longevity. I was planning on RAIDing them anyways.

As SSD's advance and become cheaper. I am sure I will upgrade to bigger, better, faster. the old derives can always go into other toys i have like HTPC's or netbooks.

Link to comment

I thought you had m1015's. if you have MV8's, 5B13 might be good for you. it is a bit early to tell if you went beta.

I think i confused your build with Nia's.

5B6a would be my recommendation then if you do not run macs.

 

You are sticking to 4.7 so.. that is irreverent.

 

I just created a new win7 VM. I'll see if i can update that booting tip later today

 

 

 

about update.

I wrote a quick esx setup (NTP) and update (patches) running only through the cli, so it is pretty simple.

do you want me to post it here so you can include it ?

 

Cheers,

R

 

anything that you feel that can help others. even I come back to the build notes when I forget something

 

ok.. I did not want to hijack your topic.

I will setup a clean computer and another esx and will come up with some upgrade path for the esx by monday.

I will try to make as many screenshot as possible.

 

Cheers,

R

 

No problem. if you have questions or suggestions or corrections to anything I posted, it is more then welcome. not only does it help you, it helps the next person also.

Link to comment
  • 2 weeks later...

Sorry if thsi was asked anywhere else in the thread...

 

Do you plan on creating (or have you created) a datastore on your unRAID array?  If so, what kind of performace hit did you take?  I would love to do this simply for teh sake of not having to buy/add additonal drives...especially SSDs.

 

Thanks,

 

John

Link to comment

Good question.

 

I could do the same on my cache drive, however I'm not sure if the cache drive will be accessible in ESXi, given that it's passed through to unRaid.

 

By the way, thanks for recommending the Tyan mobo. After reading a few reviews of this, and seeing the ESXi compliant NICs, I've gone ahead and bought it from Newegg. Waiting for it to get to Dubai.

Link to comment

Yes, you will need a datastore drive.

 

For unraid, it does not have to be anything fancy since you will only save the configuration on it and plop.

but... I'll assume you want to run other guests. Why else would you use ESXi? if that is so. the faster the drive the better.

 

you don't need and SSD. i just chose that route for the high IO i need.

 

 

Link to comment

Yes, you will need a datastore drive.

 

For unraid, it does not have to be anything fancy since you will only save the configuration on it and plop.

but... I'll assume you want to run other guests. Why else would you use ESXi? if that is so. the faster the drive the better.

 

you don't need and SSD. i just chose that route for the high IO i need.

 

 

I went SSD for my XP vm because I did not want anything to feel "slow".  An ssd to put VM's on that you will be using regularly is a good choice.

Link to comment

Minor issue with Atlas..

 

I'll quote myself from another thread..

My power has gone out twice in the last 3 days. my script came through for me the first time and shutdown all my guests cleanly.

however it failed the second time. My fail over "timeout" in unMENU did kick in and at least shut down unRAID. just not ESXi.

 

 

I realized after it failed that my windows box was not listening. i had not logged into it since the previous days outage.

I since set the windows script guest to auto log in on reboot. now it picks up every time.

 

If you do copy my power script. set the PC that is handling the script to auto log in.  Otherwise it might not be listening if you power cycled your ESXi box...

 

I'll add that to the instructions.

Link to comment

For those running the X9SCM with ESXi.

the driver for ESXi 5 for the Intel 82579 has been released. well hacked...

Instructions and download are here (jump to page 4 for ESXi 5)

 

it looks like it might only work with a "new install" at this point. I did not investigate it at all yet.

Even if that is the case, a reinstall will take a few minutes only. just remember to pull your drives first.

 


 

I also picked up a new toy for this box.. I got a supermicro 4in1 2.5" drive caddy for half price.

I'll move all of my datastore drives to that.

 

I was planning on just bolting it to the inside wall of my case..

but now I am seriously thinking about modding my chassis to make this fit sticking out side or the back.

 

Here is a cheap (not quite to scale) photoshop mockup of a possible idea i had in my brain.

YplqZ.png

 

(yes it has to be easy to remove to get at the PCIe slots.)

Link to comment

usually when new egg puts an item on open box.. they only have 1.. in this case, i bought it myself :P

 

i know they only had one because i tried for 2 and it said insufficient quantity, then i switched back to 1 and got the same error. but i hit checkout anyways..

 

 

Is there a cable for that SM drive caddy, that ends with a mini-SAS connector (instead of the supplied (4) SATA connectors)?

that is the flaw in this particular 4in1, the sas plug. there is a sff-8484 to sff-8087 cable out there, that should work for you.

 

ideally i should get one with sata plugs. I would have liked to use 3 bays to the Mobo for ESxi and 1 to my MV8 in unraid for the cache drive SSD. such is life. they do sell sata joiners.. i doubt I'd trust it in a server.

 

If you are running a raid with 2.5" drives. this would be a good 4in1 because of the sas cables and the better quality sleds/backplanes then an off the shelf item.

 

 

With norco shipping the wrong backplane, it actually works to my advantage in this build..

I can now split a row between ESXi and unraid.

This is a way for me to fit my parity drive into the pullouts..

top row = urParity*open*DatastoreBackup*WHS2011

 

 

I also need to find a cheap dual or quad port esata bracket (like the one here http://www.satacables.com/html/sata-pci-brackets.html) for my esata backups

 

Link to comment

As promised a little while back.

really sorry for the delay... I am really bad at documentation / layout... so feel free to take my procedure and reformat it.

 

 

This is a very quick procedure to update your ESX(i) installation.

[glow=yellow,2,300]

be careful :

- during the installation the server will be switched to maintenance mode

- after the installation the machine will need to be rebooted

this means that all your machine must be off before applying the patches. (or moved to another host if you have a cluster)

not following this basic rule might lead to unwanted results.[/glow]

 

preparation phase :

 

1. if not done yet install vSphere remote command line (aka as vSphere cli) - do not confuse it with the powercli

- browse to : http:\\esx_ip

- on the right side under For Administrators download windows version.

- install it.

 

2. download the packages you want to install

- from VMWARE

.... go to http://www.vmware.com/patchmgr/download.portal

.... select the ESX or ESXi version you want to update. (make sure you are downloading the correct version.. I spent 2 days wondering why packages were not applied because I downloaded the ESX version instead of the ESXi !!!)

.... save the package on your HD

- from any other provider

.... any other zip package provided to you can be installed this way.

.... if you have a doubt simply open the zip file... if it contains a .vib file + a metadata.zip you are fine.

 

 

3. run the cli and set a couple of variables

- run the cli from your start menu.

- a the command prompt enter all this (hint : copy paste does work in the cli)

.      . obviously you need to edit and enter the the information related to your system...

.      . be careful with the IP.. you do not want to mess with the wrong ESX host.

.      . I advise you to have a simple path... with no space or any special character..(i.e. d:\downloads\patches_ESX)

set VI_SERVER=esx_ip

set VI_USERNAME=root

set VI_PASSWORD=password

set VI_PATCH_PATH=path_were_the_patches_are_saved

 

 

 

4. Check the connectivity and the variables

. before going further I like to run this command to check if the values I entered are right

- in the cli enter : vicfg-hostops.pl --operation info

. check the results... and if you are happy move on. otherwise check the variables

 

[glow=red,2,300]WARNING WARNING WARNING

 

You should shutdown all your VMs before proceeding to the next step

 

WARNING WARNING WARNING [/glow]

 

5. entering the maintenance mode

- in the cli enter : vicfg-hostops.pl --operation enter

. you should get a success message

 

6. the update itself

- in the cli enter : vihostupdate.pl -i -b %VI_PATCH_PATH%\patchfilename.zip

. the update is applied

. at the time of the writing the latest patch for ESXi 4.1 is : update-from-esxi4.1-4.1_update02.zip (aka build 502767)

. the cli line would then be : vihostupdate.pl -i -b %VI_PATCH_PATH%\update-from-esxi4.1-4.1_update02.zip

- you can chain the updates

- if you have multiple updates you can put them all in the patch folder and run this command in the cli : 

. .... for /f %f in ('dir /b /ogen %VI_PATCH_PATH%\*.zip') do vihostupdate.pl -i -b %VI_PATCH_PATH%\%f

 

7. reboot the ESX host

- in the cli enter : vicfg-hostops.pl --operation reboot

 

8. after the reboot you need to exit the maintenance mode prior to starting any VM

- if your cli is still open simply run : vicfg-hostops.pl --operation exit

- otherwise in the vSphere client right-click on the host / exit maintenance mode

 

 

that's it you are done with the update.

 

I hope this helps someone,

Cheers,

R

 

Link to comment

Johnm, what (if any) backups do you perform for your datastore(s)???

 

I had a look through ESXi and i couldn't see anything built in, but i could be wrong.

 

I will have 2x500GB Blacks in a Mirror for my main datastore and would like to use another 500GB as a backup drive but don't know how to go about doing it...

Link to comment

Johnm, what (if any) backups do you perform for your datastore(s)???

 

I had a look through ESXi and i couldn't see anything built in, but i could be wrong.

 

I will have 2x500GB Blacks in a Mirror for my main datastore and would like to use another 500GB as a backup drive but don't know how to go about doing it...

 

pantner,

 

I cannot talk for Johnm, and as a matter of fact I would love to read his answer on that.

but if this can help...

 

you are totally right there is no visible backup process on the ESX itself.

 

I am going to widen the subject and bring a few things I can think of on the table.

 

I believe backing up the datastore as an entity would not make sense as per say because quite a few files are useless and would not need backup (namely all the swap files)

 

Backing up the ESX installation. could make sense, although you have to keep in mind that a clean esx installation takes only a few seconds, specially in our scenarios were we are not building very complex system... it would only be a matter of reconnecting the drives with the datastore after the install (remember Johnm's comment on unplugging all the drives before installing ESX) and then add the VMs from the datastore to the inventory

if your setup is more complex you can look into a vmware fling called inventory snapshot... http://labs.vmware.com/flings/inventorysnapshot.. it works pretty well.

 

 

for VMs backup many options are available, but you first have to define what and how you want to backup things.

1. full machine backup

-- it means you would backup all the disk (system + data)

-- solutions like that goes two ways :

----- a. snapshot / backup vmdk files

----- b. turn off the machine / backup vmdk files

---------if you are a free tool user and do not mind doing some reading look for ghettovcb

---------it is a script which is being worked on by the vmware community for nearly three years.. but you have to be able to handle a few command lines... (http://communities.vmware.com/docs/DOC-8760)

-- one thing is important : some services/application do not support snapshot.. this option can work as a miracle, but make sure you test it first

 

2. data only

-- you can then rely on any backup system

 

 

Cheers,

R

 

 

 

Link to comment

As promised a little while back.

really sorry for the delay... I am really bad at documentation / layout... so feel free to take my procedure and reformat it.

[snip]

hey thanks for that.

 

that was very detailed and handy.

 

that is pretty much the way i upgraded to 4.1u1 and the way i upgraded  ESXi 5 for the Intel 82579.

 

I'll have to document that nic install with step for step for others.

 

for my upgrade from 4 to 5, I just used the CD install using IPMI to moint the iso and selected "upgrade".

 

Your like the open box king!  ;D

Congrats.

I just have the "supermicro open box" store as a bookmark and tend to check whenever i can.

 

 

 

Johnm, what (if any) backups do you perform for your datastore(s)???

 

I had a look through ESXi and i couldn't see anything built in, but i could be wrong.

 

I will have 2x500GB Blacks in a Mirror for my main datastore and would like to use another 500GB as a backup drive but don't know how to go about doing it...

Well was using Ghetto VBbackup when i was running 4.1.

I sort of picked up a spare lenience for Vranger from work.

My second box is still running Ghetto.

 

I was going to do a writeup on Ghetto. while dong the write up, I corrupted a guests doing the restore.

I am sure it was because i was trying to restore a 4.1u1 guest backup over a 5.0 install.. this was before ghetto supported 5.0

 

They have since upgraded the script for 5.0. I can probably test that out again and let work have that license back.. ( i know they need it back before year end).

 

maybe i can do that this week. i have a lot on my personal to-do list right now.

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.