Virginity Smashed!


Recommended Posts

Good evening everyone!

 

Well after about a month of scouring the forums and basically googling the shit out of anything Linux/unRAID/HDD/HBA related, I finally, almost, have my first unRAID build completed.

 

So I was fortunate enough to win a few ebay auctions, ending up with some good quality kit at little to no price, so before I go though my experience I will list out the core components of my build!

 

Fractal Design Arc XL 2 case

ASRock Rack EP2C612D16C-4L motherboard

256GB (16x16GB) 2133 DDR4 ECC RAM

2 x Intel Xeon E5-2660v3 (10 core, HT, 25MB L3 cache, 9.6GTs QPI each) air cooled by Noctua NH-U12DXi4  HSFs

AMD Firepro W5100

AMD Firepro W7100

2 x HP H240 HBA

8 x SG 4TB 6GBs SAS HDDs

4 x Pliant 400GB 6GBs SAS SSDs

1 x SanDisk UltraFit USB3 16GB

1 x Corsair HX850i PSU

1 x Corsair ML-140 Pro fan

 

Oh, and, of course, an unRAID Pro licence.

 

So I initially received an HP micro server (N40) free of charge from a friend. I was  going to install RockStor but installation kept failing, so I decided to use FreeNAS. However as soon as I saw the UI, I turned tail and ran! Finally I ended up with a trial version of unRAID.

 

Once I had it up and running, PLEX installed etc. I thought "Brilliant, finally a Linux Distro where you can do almost everything via the GUI". However I quickly ran into two problems.

1. Plex ran like a dog on the crappy AMD Turions - transcoding was almost out of the question.

2. Everything else also struggled for headrooom on the Turion.

 

So, I decided to go all out.

 

Initially everything went quite well. I did have an issue with the MoBo - it was SSI EEB and the case only had mounts for E-ATX. (however I decided that the risk to the MoBo for the two missing mounts was worth it), and trying to update the HP H240 HBAs was a bit of a nightmare - they had arrived pre-configured for RAID, and it took me a while to find a copy of HPSSA that would run offline and allow me to change them to HBA mode. Interestingly, it had one of the RAID logical drives set as boot, and once I had changed it to HBA, I was completely unable to deselect it. changing back to raid had no effect. In the end I just updated the firmware to 6.06 and that seemed to do the job.

 

Things started to get hairy when I decided to just connect it all up, start-up unRAID and see what happened. It was Horrific. Millions of errors, HDD marked as failed. I was gutted, but fortunately not undeterred. I quickly re-booted into a Mint environment, and proceeded to completely erase (partitions and all), all the drives. I formatted the SSDs as BTRFS, the 4TBs as XFS, then run a full SMART check on them, which came back clear, and also cheered me up in that the SSDs had only 135 hours, and the 4TB only 2700 hours use!

 

I then decided to give it another all-in go. Fatal mistake! Millions of errors, failed disk again unreadable and un-mountable disks, I was nearly crying.

 

However after many, many, repeated bouts of GParting, DDing, SMARTCTLing and various other "Jesus Christ help me my HDD are failing" commands I realised that the reason all these tools were telling me my drives were fine, was probably because my drives were, in fact, fine. This left me feeling slightly pissed off. $129 on a pro license and I can't even get the damned thing to start up Grrrr.

 

But, like I mentioned earlier, I am made of stronger stuff, so after about 13 hours 43 minutes of fruitless googling, I decided to start over from the beginning. Whilst in BIOS, I noticed that the HBAs were running at around 87 and 113 degrees C. Oops. That might explain it (it certainly explains the extra Corsair MLL-140 pro I bought to blow some extra air on them!), unfortunately it did not change anything.

 

After a few more hours of equally fruitless googling, I came to the conclusion that either the HBAs were faulty or the cables (Mini-SAS to 22P SAS with 15P SATA power on the rear) weren't fit for purpose. I almost bought another "approved" HBA, but fortunately didn't. I decided to try and add the drives a couple at a time, in case the cables were a bit wonky, and the combined data loading on them was causing some sort of crosstalk/interference.

 

After deleting all partitions and partition tables, I simply created a new partition table (GPT) on each drive, formatted the Data drives as XFS, the Cache drives as BTRFS and the Parity drives I just formatted as Cleared.

 

After adding one drive as Parity, one drive as Data and one drive as cache, everything started with no errors or problems, I then stopped the array, added another Parity drive, two data drives and a further Cache drive. Once that had completed successfully, I then stopped the array and added the three remaining data drives and the final two cache drives. (Currently sat at 91.8% clearing).

 

YAY! I now feel confident enough that it is going to work to write this post.

 

Now, the real question.

 

40 Cores, 1/4TB of RAM, 24TB of dual parity protected storage, 800GB of SSD cache and a pair of FirePros. What am I going to do with it?

 

Just running Plex and mucking about seems like such a waste now :)

 

 

 

 

 

 

Link to comment

Congrats! Curious how much all those eBay auctions cost you! That is quite a haul! 

 

Only item that might be a small problem is the USB 3 flash drive. Put it in a USB 2 slot and should be fine. And if it gives trouble, switch to a USB 2. Anything over about 4G is wasted, and that factors room to grow.

 

I have converted my workstation to an unRaid VM with passthrough video and USB. Works great and no one would ever know it was a VM. You could have quite a number of VMs, including a media player. You can set up PLEX to transcode to RAM. I think you have enough xD.

 

Enjoy finding ways to use your array! 

Link to comment

Thanks SSD,

 

The Case, Power Supply, MoBo and HSF/Case fans and SSD mounts I bought at for around 599 total, The drives and processors I got for substantially less :)

So far USB drive has been fine, the native slot on the MoBo is a USB 3, but I will keep an eye open for errors. I am aware that 16G is overkill, but I figured that if I want to re-use the drive later in its life, then bigger is better, especially as the cost was nominal.

 

Think I am going to start with a PF Sense VM, to aggregate my Virgin Fibre (200/12) and ADSL (9/.8 - just for backup as the Virgin has been unreliable, fast, but unreliable). I will feed that out to the Nighthawk for the rest of the house.

 

Was undecided about having a VM or a docker for Plex, but reading through the forums, and now that you can assign cores to dockers, I will probably go with the docker for that, and for the associated media programs ;) .

Wasn't aware that you could set Plex up to transcode in RAM, but now that I am I will do a bit of research on that and incorporate that too. at least I will use up a few extra Gig of RAM then.

 

Only other things I want to do is set up (UNL) EVE on a VM (because trying to get a Hypervisor running a VM, runnning a Hypervisor running VMs is just too good an opportunity to pass up, and maybe get a couple of VMs running for the kids, with the GFX passed through to the VMs and the KVM remoted via IP, although that, again will probable take a little research!

 

Oh and I wonder how it would cope with a VM dedicated to mining with almost all resources passed to it...... :)

 

Link to comment
  • 1 month later...
5 hours ago, DZMM said:

pictures?

 

Pictures will follow... Once I get the damed thing working the way I want LOL.

Kept having numerous issues with the H240 HBAs under unRAID (they seemd fine under Kali). So I dropped the SAS disks from the Array completely and reverted to SATA disks using the built in controller. While this has enabled me to get it up and running, and start moving content. I am running on only 1 4TB parity, and 8.5TB data (1x4TB, 4x1TB and 500GB).

 

I have tried each HBA on its own, with every combination of the 4 SSDs on each port, but although the HBA appears to boot ok, the disks are never visible within unRAID.

 

Next step will be to boot back into HPSSA and check there are no issues present, before booting into another distro and ensuring that the disks are actually up and working.

 

In the meantime, heres a clip of one of my favourite views... ;)

 

2018-01-04.png

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.