Anybody planning a Ryzen build?

[Charlie22911]

On 11/19/2017 at 3:05 PM, -Daedalus said:

Thought I'd chime in here. Haven't read the thread (yet) as it's huge, but:

 

1950X / X399 Taichi build.

 

On 6.3.5 at the moment. Have been for a day or two now. Originally I was having lots of problems with a HBA, so I ended up leaving SVM disabled. Ran for over a day with no problems. Turned SVM on today and everything's back up, with the only change being I can't see my CPU usage on the dashboard (though it works in Stats). I should mention I've changed no settings at all, other than enabling, SR-IOV, SVM, and IOMMU in BIOS.

 

Now the problems with 6.4

 

If I boot into 6.4 with BIOS defaults, my HBA isn't detected.

 

If I then enable IOMMU, my HBA is detected, but my logs are flooded with:

 

 

I'm hesitant to start my array in this condition, so I've gone back to 6.3.5, which seems (weirdly), to be working completely fine so far. I'm not using passthrough for any of my VMs, so perhaps that's why. I also haven't disabled C-States.

 

Does anyone have anything I can try? As I said, I haven't read through the thread yet, so there could well be something simple I'm missing.

server-diagnostics-20171118-1501.zip

server-diagnostics-20171119-1730.zip

 

I also have the 1950x and x399 Taichi, with two LSI 9211-8i with latest IT mode FW. The latest BIOS causes one of the adapters to not be detected in my build, but I'm sure it is unrelated to unRAID since it also fails to complete POST intermittently with the second adapter installed.

I have all VM related features enabled in BIOS along with global c-states with no issues though, currently up 5 days after the bios experiment.

 

One strange issue I just can't tackle though is running an Ubuntu VM. I have pfSense (BSD) and Windows 10 VMs up fine, but as soon as I try going though the install process with Ubuntu the entire unRAID server becomes unresponsive, and I am unable to retrieve logs.

[John_M]

6 hours ago, Charlie22911 said:

The latest BIOS causes one of the adapters to not be detected in my build, but I'm sure it is unrelated to unRAID since it also fails to complete POST intermittently with the second adapter installed.

 

Does each card work ok if you don't install the other? Have you tried swapping the cards over or putting them in different slots? You could re-flash the HBA IT firmware without the BIOS as it isn't actually needed by unRAID.

 

6 hours ago, Charlie22911 said:

One strange issue I just can't tackle though is running an Ubuntu VM. I have pfSense (BSD) and Windows 10 VMs up fine, but as soon as I try going though the install process with Ubuntu the entire unRAID server becomes unresponsive, and I am unable to retrieve logs.

 

If you need help with this you should start a new thread in the KVM support section, including your diagnostics. It sounds as though it might be something as simple as a clash of IP addresses/bad netmask/overlapping subnets though.

[-Daedalus]

7 hours ago, Charlie22911 said:

 

I also have the 1950x and x399 Taichi, with two LSI 9211-8i with latest IT mode FW. The latest BIOS causes one of the adapters to not be detected in my build, but I'm sure it is unrelated to unRAID since it also fails to complete POST intermittently with the second adapter installed.

I have all VM related features enabled in BIOS along with global c-states with no issues though, currently up 5 days after the bios experiment.

 

Similar to what John_M asked, if you try with one card, and it's detected, do you see your logs full of the same errors I do? You don't always see it by opening the live log, so I usually go Tools -> System Log.

 

What slots have you tried the card in? Because I've tried it in all of them, same thing. I have a 9207-8i on the way as well. I'll give things a go with that at some point, but the exact same behavior was present with an IT-flashed H310 as well, which makes me think it's not the card.

[Charlie22911]

The cards work fine together before flashing the newest bios as well as individually on the newest bios. I’ve tried moving them around to each of the 4 x16 slots with the same results, almost seems like a resource conflict; that’s a bit above my ability to troubleshoot though.

I created a support ticket with Asrock the same day I made that post and haven’t heard anything back aside from their automatic support confirmation email. For now I’ve flashed back to the previous bios and I’m up and running fine.

 

As for the VM hard locking, I should have mentioned that it includes the CLI on the physical machine as well. Everything including num/caps/scroll lock lights on the keyboard are even unresponsive. I’ll create a new thread on this though for troubleshooting when I get a chance.

[-Daedalus]

Cheers. BIOS 2.00 was released not too long ago for the Taichi. I'm going to try that this weekend with RC15e (released today) assuming you don't beat me to it. If it's still not working I'll flash 1.70 and see what happens.

[Charlie22911]

10 hours ago, -Daedalus said:

Cheers. BIOS 2.00 was released not too long ago for the Taichi. I'm going to try that this weekend with RC15e (released today) assuming you don't beat me to it. If it's still not working I'll flash 1.70 and see what happens.

 

Just tried 2.0, same issue. Invoking the LSI\Avago bios causes a system lock, unraid is missing all disks on the second controller; back to 1.7 I go.

I *still* have not heard from Asrock support, I'll be submitting a new ticket referencing the new 2.0 bios.

Edited December 6, 2017 by Charlie22911

[njrwc]

I'm using 6.4.0_rc15e on a  Ryzen 7 1700, 16GB ram, msi gaming pro x370 motherboard. I run 2 VMs (both ubuntu linux), and a few dockers (plex, duckdns, crashplan pro).  My Cstate is disabled and bios is updated.

 

Things seem to run ok, but the computer won't stay on more than a day--typically it crashes shortly after the parity check finishes!

 

How can I tell if I need to  be more patient to wait for unraid/linux to catch up or if there is something wrong w/ my hardware?

Thanks!

 

Ron 

 

[Prokie]

23 hours ago, njrwc said:

I'm using 6.4.0_rc15e on a  Ryzen 7 1700, 16GB ram, msi gaming pro x370 motherboard. I run 2 VMs (both ubuntu linux), and a few dockers (plex, duckdns, crashplan pro).  My Cstate is disabled and bios is updated.

 

Things seem to run ok, but the computer won't stay on more than a day--typically it crashes shortly after the parity check finishes!

 

How can I tell if I need to  be more patient to wait for unraid/linux to catch up or if there is something wrong w/ my hardware?

Thanks!

 

Ron 

 

Have you tried this fix that worked for gridrunner at 9:43?

 

[methanoid]

Since Ryzen has NPT patch and C-states fixed does this also mean Threadripper is okay for unRAID inc VMs with passthrough graphics cards?

[mikeyosm]

6 minutes ago, methanoid said:

Since Ryzen has NPT patch and C-states fixed does this also mean Threadripper is okay for unRAID inc VMs with passthrough graphics cards?

I bought a TR board then read all sorts of issues with IOMMU and general PCIe passthrough even after all the fixes to UNRAID and KVM. So, now runnign Ryzen and happy with the performance and stability with uptimes of weeks rather than days.

[david279]

Threadripper has a whole other set of problem. GPU pass thru etc.

Sent from my SM-G955U using Tapatalk

[njrwc]

I tried the rcu_nocbs option in the syslinux config, but it seems like that option was removed from the latest kernel. (https://bugzilla.kernel.org/show_bug.cgi?id=196683)

[limetech]

4 hours ago, njrwc said:

I tried the rcu_nocbs option in the syslinux config, but it seems like that option was removed from the latest kernel. (https://bugzilla.kernel.org/show_bug.cgi?id=196683)

 

Continue reading in that thread.  It was replaced with "rcu_nocbs=0-N" where N is the number of threads (eg, see comment 16).

 

[MadeOfCard]

 
Continue reading in that thread.  It was replaced with "rcu_nocbs=0-N" where N is the number of threads (eg, see comment 16).
 

Where N is the number of thread minus 1


Sent from my iPhone using Tapatalk

[limetech]

2 hours ago, MadeOfCard said:

Where N is the number of thread minus 1

 

You passed the test

[MadeOfCard]

 
You passed the test

Thanks, arrays start at zero guys.

Been keeping close eyes on this thread for a while. Waiting when the time is right to upgrade to Ryzen. Seems like the time is nigh. Very excited. Thanks for all the work.


Sent from my iPhone using Tapatalk

[njrwc]

Quote

 

My problem was much stupider than that! I added the rcu_nocbs option to the sysconfig-- but missed the non-gui boot section!  OPPS!

 

Maybe I need those new glasses afterall!  

 

Thanks for all the help!

[H8Edge]

https://www.reddit.com/r/Amd/comments/7gp1z7/threadripper_kvm_gpu_passthru_testers_needed/

 

 

In case anyone hasn't seen this..  Things are looking up..

 

New to Unraid still and my 1080 Ti has just been sitting in my case looking pretty for 2 months..

Anyone try out the patch yet?

[david279]

NPT Patch for ryzen went in a couple of RC versions ago so it works without issue now. Threadripper has a different set of problems. On threadripper passing thru GPU's still don't work properly if at all. That patch seems pretty ugly right now hopefully they clean it up.

 

Sent from my SM-G955U using Tapatalk

 

 

 

 

[methanoid]

2 minutes ago, david279 said:

NPT Patch for ryzen went in a couple of RC versions ago so it works without issue now. Threadripper has a different set of problems. On threadripper passing thru GPU's still don't work properly if at all. That patch seems pretty ugly right now hopefully they clean it up.

 

But apparently all fine with ESXi ... damn!

[JohanSF]

I have a ryzen 1700, running rc14, using the rcu_nocbs=0-15 workaround. I upgraded to bios 3.30 (https://www.asrock.com/MB/AMD/AB350M Pro4/index.us.asp#BIOS) a week ago and my system has been running stable since that. I might just be lucky but a full 7 days of stability is a new record for me!

Also - I have global c-states enabled in bios.

[joohn]

4 hours ago, JohanSF said:

I have a ryzen 1700, running rc14, using the rcu_nocbs=0-15 workaround. I upgraded to bios 3.30 (https://www.asrock.com/MB/AMD/AB350M Pro4/index.us.asp#BIOS) a week ago and my system has been running stable since that. I might just be lucky but a full 7 days of stability is a new record for me!

Also - I have global c-states enabled in bios.

 

JohanSF, I am going to buy exactly the same setup (Mobo+CPU) as you have. Do you use GPU pass through? If yes, what is your experience with htpc and gaming use cases?

[JohanSF]

11 hours ago, joohn said:

 

JohanSF, I am going to buy exactly the same setup (Mobo+CPU) as you have. Do you use GPU pass through? If yes, what is your experience with htpc and gaming use cases?

I don't run any vm's and don't use it for gaming.

 

I designed my build to:

• have a small form factor
• have a good amount of hdd space and room for 2 more hdd's in the case later on
• have very low power consumption
• be quiet (everything that is spinning must be high quality to ensure good, quiet and sturdy ball bearings)
• run plex, sonarr, radarr, and some other dockers depending on what I want to spend time fiddling with

https://pcpartpicker.com/list/XrHkyf

 

I know I don't need unRaid Pro, but I wanted to support lime-technology  

 

Another night just passed with the server idling and it's still up, I might just pop a bottle of champagne soon! Apart from the stability issue that everyone is facing, I think this is an amazing build. It's actually overpowered in regards to cpu and ram but I just decided to go all-in.
 

Edited December 14, 2017 by JohanSF

[joohn]

Thanks for response. If I wouldn't go into VMs, I would just stick to Skylake Pentium since I am using maximum one PLEX stream and standard Unraid functionalities for family data sharing. Currently I have 2 core Celeron 1007u which is great piece of hardware - it handles Windows VM, Linux VM and Plex - rock solid, months of uptime. Of course performance of VMs is low, but enough for my needs. It struggles with transcoding 1080p streams, so this is my main reason to upgrade. And if put money into setup, I would like something future proof, which would also replace wife's gaming PC. Ryzen is fantastic solution, but I am still concerned with pass through performance, stability and choosing the right board (definitely matx). Can you guys post some setups which are proven to work near bare metal in gaming vms with latest RC version (after npt fixes)?

 

 

[mikeyosm]

Just now, joohn said:

Thanks for response. If I wouldn't go into VMs, I would just stick to Skylake Pentium since I am using maximum one PLEX stream and standard Unraid functionalities for family data sharing. Currently I have 2 core Celeron 1007u which is great piece of hardware - it handles Windows VM, Linux VM and Plex - rock solid, months of uptime. Of course performance of VMs is low, but enough for my needs. It struggles with transcoding 1080p streams, so this is my main reason to upgrade. And if put money into setup, I would like something future proof, which would also replace wife's gaming PC. Ryzen is fantastic solution, but I am still concerned with pass through performance, stability and choosing the right board (definitely matx). Can you guys post some setups which are proven to work near bare metal in gaming vms with latest RC version (after npt fixes)?

 

 

 

I'm using RC15 on Asus Crosshair VI Hero, 32GB Corsair veng mem, 1700x Ryzen, Samsung 950 pro (in passthrough) and a plethora of HDDs.

I manage to stay stable for days/weeks  whilst running a W10 VM with 1050ti/audio/nvme passthrough, Windows server 2012 VM, Plex and various dockers.

Plex transcodes 4k no issues, W10 gaming is very close the bare metal. Very very pleased thus far.