In need of new ECC-capable mobo and CPU - help!

[t33j4y]

So, came home to a dead server today :-(

 

My trusty  AsRock C2750-D4I has given up, so I need to find a replacement pretty quick.

 

- I would like to re-use my ECC DDR3 (2x Kingston kvr1333d3e9s/8g) and would appreciate a board with 10+ SATA ports (aternatively, I do have a RocketRaid SATA card in a drawer which could be utilized of there are fewer ports)

- At least two ports must be 6Gbps for the SSD cache disks.

- The server is housed in a Node 804 which I also want to re-use

- IPMI is not a harsh requirement but it is very nice to have...

- I do like low power consumption, so if that could be met I'd be a happy camper

 

Server runs Plex (not much in terms of transcoding, max one stream at a time), SabNzbd, SickBeard, Crashplan. I do run dual parity disks.

 

I have looked at Supermicro X10SL7-F (mATX) and E3C224D4I-14S. I am unsure if the latter which is extended mini-ITX will fit my Node 804. Also unsure what CPU to pair it with.

 

Any input is appreciated as I need to figure out parts and order this weekend :-)

 

Cheers,

 

T.

Edited November 10, 2017 by t33j4y

[tdallen]

Supermicro  and ASRock are the motherboards to be looking at for DDR3 ECC RAM support.  The darn chips are getting expensive, though, since they are a few generations old.  Would you want a new chip, or willing to consider used?

[t33j4y]

Have no problem going for second-hand :-)

 

I realize that with the price tag some older stuff seems to end up with, I might end up being financially better off by skipping the ideas about re-using the RAM and instead going for a newer generation.

 

But since I don't have high performance requirements I might get away with some second-hand stuff :-)

 

I am a bit partial to SuperMicro since the IPMI on AsRock ended up not-working very well due to AsRock not maintaining the security certificate. I could power cycle and monitor voltages and such, but the KVM over IP was non-functional due to Java not accepting the security certificate :-/  This might be better with newer AsRock boards. The feature set in terms of SATA is better out of the box, though. Price wise, here in DK they're mostly a tie.

[tdallen]

Here in the US the only Xeon E3 v3 which is reasonably priced new is the E3-1220v3.  But it's a 4 core part without hyperthreading.  It's more powerful than what you had but most of the E3 line has hyperthreading.  I'd be inclined to find something like an E3-1245v3 or 1275v3 on eBay.  Keep in mind that the Xeon E3 xyz5 chips have onboard graphics while the xyz0 chips do not.  Take a look at what you can find, anyway, and then compare to a newer 1151 setup.  You could still get the same board you had here in the US, by the way.

[t33j4y]

Thanks - I'll look around for an E3 xyz5 variant :-)

 

I've had some time to research what might have put my AsRock board out of commission, and it seems that they have a tendency to die of due to a BIOS bug where use of a watchdog timer triggers writing to a flash circuit, wearing it down prematurely. Even though AsRock has fixed it now, I've kind of lost faith in the mobo...

 

[tdallen]

Some server boards come with built in VGA, anticipating that your CPU may not.  So it's worth evaluating the board and CPU together. 

[t33j4y]

So, I've kind of narrowed things down to two options.

 

Option 1:

Hang on to ECC DDR3-1333 RAM (2x 8GB)

Buy Supermicro X10SL7-F with 6xSATA and 8xSAS and cross flash to IT mode.

Buy Xeon E3-1246v3

Rough cost: USD 630 (I converted from euros)

 

Option 2:

Sell off existing ECC DDR3 RAM

Buy Supermicro X11SSM with 6xSATA and add my RocketRaid card (4xSATA)

Buy 2x Kingston ValueRAM ECC DDR4-2133 (2x 8GB)

Buy Xeon E3-1245v5

Rough cost: USD 850

 

Any thoughts, suggestions as to what is the better option? I don't think there is much difference in terms of perceived performance in everyday use, so the argument for the newer setup would be that in a couple of years, I would be able to pick up a used top shelf CPU at a reasonable price, whereas the option 1 is already some generations older and will probably strand once something goes *poof*

 

Cheers,

 

T.

 

Edited November 11, 2017 by t33j4y

[t33j4y]

There is of course also the option of buying the C2550-D4I (slower version of the board I had) and accepting the crappy IPMI access.

 

Would only be around USD200. Just seems kind of .. meh.

[tdallen]

You'd be reasonably close to top shelf with the E3-1245v5 since the Coffee Lake CPUs require a new chipset.  Have you read about the Sky Lake / Kaby Lake hyperthreading issue, though?

 

 

 

[t33j4y]

I hadn't read about it until you linked to it in the thread in UCD, so thank you for making me aware. Thankfully, the X11SSM is one of the boards that have an updated BIOS out that fixes the issue.

 

I have just pulled the trigger on the X11SSM as there was a favorable offer from a retailer that had it in stock (most had a weeks delivery time, and I'd like to get up and running soon)

 

So now I have to decide on a CPU - I am leaning towards the E3-1245 v6 as it is priced the same as v5. Performance wise they're probably neck to neck, but it is one generation newer and now that I'm refreshing and not reusing I might as well go for the latest one.

 

I'm thinking of only getting  8GB to start with, though. Don't think that I'll be needing more with my usage pattern.

 

 

[tdallen]

23 minutes ago, t33j4y said:

I am leaning towards the E3-1245 v6 as it is priced the same as v5. Performance wise they're probably neck to neck, but it is one generation newer

Hmm, where's a thumb's up emoji when you need one?   That's what I'd do.

[dmacias]

My 2nd ASRock C2750-D4I is having trouble. Still running though. The BIOS can't see any sensors and the BMC webpage is missing several sensors e.g. 3v, 5v, 12, MB Temp. This happened to 1st board too. I RMA'd it then bought another C2750. I sold the RMA when I got it back. I'm just gonna RMA this one and sell it when I get it back too but move on to a Supermicro board. Just remember there is a 3 year warranty on the ASRock boards.

 

So I decided to use my back up server and part it out. It has a Supermicro X10SLL-F ($109) and 8GB ECC that I'll use. And I bought a LSI 9207-8i ($65). I'm gonna buy a E-1230 v3 or 1240 v3.($150-$160).  I got everything on eBay.

 

I saw you went with the X11 but my vote would be to reuse your ram in an SM X10 LGA1150.

[t33j4y]

dmacias, thank you for your input.

 

Yes, unfortunately the C2750-D4I seems prone to dying after a couple of years service. Mine was bought in April 2015 so it is within the 3 year warranty you mention. Im just not sure that it is 3 years here in Europe but I will try to reach out to the retailer to get it RMA'ed. If succesful, I too will sell it off. If they offer a refund, even better. Got that on a Seasonic (of all PSUs...) last month.

 

Unfortunately there is not really much to find on eBay (available in my area), so I am looking at buying new parts since I can't wait too long for the right offer to come along. The X10SL7-F I had my eye on turned out to be more expensive than the X11 I ordered (by approx USD 50) and the E3-1245v3 is the same price as the E3-1245v6.

 

So in the end, there is not very much (for me) to save by trying to reuse my RAM. If I can sell my DDR3 ECC at a modest price, the difference between X10-based or X11-based will be USD30-40. I consider that quite ok for getting a couple of generations newer kit.

 

 

 

[t33j4y]

To wrap this up, I ended up purchasing these parts for the new build

 

Supermicro X11SSM

Kingston Server Premier RAM ECC DDR4-2133 (2x 8GB) - (KVR24E17S8/8MA)

Xeon E3-1230v6 w Noctua NH-L9x65 cooler

Dell Perc H310 HBA

 

So now I can lean back and wait for things to start arriving.

 

Thank you to all for input!
 

[JorgeB]

12 minutes ago, t33j4y said:

Supermicro X11SSM

Kingston Server Premier RAM ECC DDR4-2133 (2x 8GB) - (KVR24E17S8/8MA)

Xeon E3-1230v6 w Noctua NH-L9x65 cooler

Dell Perc H310 HBA

 

Good board, have some myself, you'll need to use the tape mode for the H310 to work with this board.

[t33j4y]

27 minutes ago, johnnie.black said:

 

Good board, have some myself, you'll need to use the tape mode for the H310 to work with this board.

 

Wasn't aware of that - seems straightforward though.

 

http://yannickdekoeijer.blogspot.co.uk/2012/04/modding-dell-perc-6-sas-raidcontroller.html?m=1

 

Thanks again!

[t33j4y]

So, some days later and a lot of both grief and relief has passed. This is gonna be longish but might hold some interesting lessons for others with the X11SSM.

BMC and BIOS are at 1.37 and 2.0c (board was delivered with 2.0b, so I upgraded). H310 has been modded with electrical tape to cover B5 and B6.

1. H310 HBA cross flashing is in itself not troublesome. Creating the right bootable USB for it , is. I could not run the flash tool executables without "misfiring" in weird error messages like stuff about mouse driver not being loaded (why the hell this would be important for a DOS shell, running CLI programs is beyond me), memory not available etc. Finally got it working using HP USB Disk Storage Format Tool and Win98 files downloaded from allbootdisks.com and succesfully cross flashed the H310. I feel kind of lucky that I succeeded because when I rebooted using the same USB boot key half a day later, I could not get any of the tools to start (I was back to the "mouse driver missing" type of errors). Seems very hit or miss (and also made me realize how much I had forgotten about DOS config.sys of olden days - which made me sad, but that's another story :-) )

 

2. Because I wanted to make absolutely sure that everything worked as expected, I installed a trial version of unRaid on a different USB key and set mobo to boot legacy USB key only. and plugged in two extra disks I had lying around. Both on mobo controller. Booted and let unRaid start in non-GUI mode. Accessed via web interface, formatted disks, created share, put files on it. Powered down, moved data disk to H310, restarted. Everything went swimmingly, disk was recognized, files were present. Great stuff.

 

3. I hooked everything up - cache disks on mobo controller and the rest on the H310. Plugged in my USB key with licensed unRaid. Booted into unRaid non-GUI mode. Did not get an IP for unRaid (this may be due to another issue - see note below). Strange, but it seemed like there was no link at all.

 

4. Rebooted to try and acquire DHCP. Now unRaid would not boot and got stuck in an endless loop where bzroot and bzimage would load, and then board reset. Ok, back to last known good config - the test setup. unhooked all disks from mobo and mini-SAS connectors from H310. Swapped to test USB key. Reboot. Same thing. Loads bzimage and bzroot, then resets. But this setup just worked an hour ago!

 

5. Searched on forums and found other X11SSM owners who cannot boot into unRaid non-GUI mode either. They can boot a plethora of other stuff (including Linux distros) without issues at all, but not unRaid. But if they boot to unRaid GUI instead of unRaid non-GUI, it works. No one knows why. It's not even a problem on all X11SSM boards - only some. And some have reported that if they load optimized defaults for BIOS and touch nothing in the BIOS setup, they can boot unRaid non-GUI (resetting to optimized defaults didn't work for me though). So, I tried booting into unRaid GUI and lo and behold, it worked like a charm!

 

6. Ok, so back to hooking up all disks and the licensed unRaid USB key. Boot in unRaid GUI, change config.sys so that unRaid GUI is now the default (for unattended reboots) and everything worked as expected. Disk positions were intact from my original C2750D4I. Am now running parity check at > 150MB/s which is a 40% improvement from my old server :-)

I would still like to boot into unRaid non-GUI instead of GUI (personal preference) but can live with this.

 

Note: It may be that one of the network cables is iffy and that between the stage of "test setup" (bullet 2) and then "hooking everything up" (bullet 3) I might have swapped the iffy one to NIC #0 thereby not getting an IP address. However, since both network cables were inserted, I would have thought that unRaid would fall back to NIC #1. When I swapped them, and did not put a cable in NIC#1, I got an IP address. So I need to figure out if I have an iffy cable or if the board does not like to have two cables attached. But I have a feeling that there is more quirkiness to this board that I have yet to see.