Pauven

unraid-tunables-tester.sh - A New Utility to Optimize unRAID md_* Tunables

719 posts in this topic

@Pauven any chance of getting the last version you had in private beta last year posted publicly so we can better tune our 6.2 / 6.3 systems?

 

 

0

Share this post


Link to post
Share on other sites

Posted (edited)

43 minutes ago, BRiT said:

@Pauven any chance of getting the last version you had in private beta last year posted publicly so we can better tune our 6.2 / 6.3 systems?

 

 

Hey BRiT and all the rest of the gang.

 

Apologies for my absence.  I had some health issues crop up unexpected back in September, and it took me out of commission for a while.  Plus I've been pretty busy in the new year.

 

I would like to resume work on the v4 script.  I recall I had ideas back in September, getting ready to completely revamp the testing method.  What I had coded just wasn't working right.  Of course, half a year later, I don't recall what my ideas were.  But fortunately, I have a habit of sharing my ideas in writing, so I think I can get back to where I was in September by re-reading my old posts.

 

Hopefully my absence brought back fond memories of Tom disappearing for long periods of time between unRAID releases...  xD

 

Probably best I don't make any promises of dates just yet.  It'll take me a while to get my bearings and remember how to code again.

 

Once I get a good enough v4, I think it would be a great idea to get it into a git repository so the community can maintain it.  None of ever know when life attacks.  Even better, hopefully it can become a true plugin.

 

-Paul

 

 

Edited by Pauven
3

Share this post


Link to post
Share on other sites
2 hours ago, StevenD said:

 

I have dual Xeons, and my parity check/build times are slower.  Its even more noticeable now that I'm migrating to 8TB drives.

 

Interesting.   I had assumed that my slower times were due to the relatively slow E6300 -- although candidly it should still be plenty fast enough to compute parity in real time (especially on my single parity system -- I have two E6300-based setups, one with single; one with dual parity ... and both have become notably slower with the newest versions).

 

But I'm surprised a dual Xeon setup would have this issue.    What model Xeons?

0

Share this post


Link to post
Share on other sites

Posted (edited)

1 minute ago, garycase said:

 

Interesting.   I had assumed that my slower times were due to the relatively slow E6300 -- although candidly it should still be plenty fast enough to compute parity in real time (especially on my single parity system -- I have two E6300-based setups, one with single; one with dual parity ... and both have become notably slower with the newest versions).

 

But I'm surprised a dual Xeon setup would have this issue.    What model Xeons?

 

E5-2670s. Im running under ESX, but Ive passed thru plenty of cores. 

 

I also ran one parity build on baremetal the other day and it was the same speed.

 

my parity checks were over 100MB/s 

Edited by StevenD
0

Share this post


Link to post
Share on other sites

Definitely surprised -- dual 2670's score 18353 on PassMark -- my Pentium E6300 scores 1701.   So clearly you have PLENTY of "horsepower".   I'd have not anticipated ANY slowdown in parity checks with that setup.   I presume you haven't made any changes in the disk controllers, memory, or anything else that might have impacted this.

 

0

Share this post


Link to post
Share on other sites
5 minutes ago, garycase said:

Definitely surprised -- dual 2670's score 18353 on PassMark -- my Pentium E6300 scores 1701.   So clearly you have PLENTY of "horsepower".   I'd have not anticipated ANY slowdown in parity checks with that setup.   I presume you haven't made any changes in the disk controllers, memory, or anything else that might have impacted this.

 

 

Nope.  My setup has been the same for about a year.  

 

I have Pauvens alpha script, but it didnt give me any improvement, so I went back to default.

 

i did realize the other day that both on my controllers are on CPU1, so im goes ng to open up my box and move one of the controllers to another slot controlled by CPU0. 

0

Share this post


Link to post
Share on other sites
29 minutes ago, StevenD said:

i did realize the other day that both on my controllers are on CPU1, so im goes ng to open up my box and move one of the controllers to another slot controlled by CPU0.

 

While that may have a bit of impact; I assume it's been like that all along; so it still doesn't explain why the checks are taking longer with the newer versions.   Until you posted, I had simply assumed it was the higher CPU demands of the v6 releases -- but clearly your system doesn't support that conclusion.

 

0

Share this post


Link to post
Share on other sites
16 hours ago, Pauven said:

Hey BRiT and all the rest of the gang.

 

Apologies for my absence.  I had some health issues crop up unexpected back in September, and it took me out of commission for a while.  Plus I've been pretty busy in the new year.

 

I would like to resume work on the v4 script.  I recall I had ideas back in September, getting ready to completely revamp the testing method.  What I had coded just wasn't working right.  Of course, half a year later, I don't recall what my ideas were.  But fortunately, I have a habit of sharing my ideas in writing, so I think I can get back to where I was in September by re-reading my old posts.

 

Hopefully my absence brought back fond memories of Tom disappearing for long periods of time between unRAID releases...  xD

 

Probably best I don't make any promises of dates just yet.  It'll take me a while to get my bearings and remember how to code again.

 

Once I get a good enough v4, I think it would be a great idea to get it into a git repository so the community can maintain it.  None of ever know when life attacks.  Even better, hopefully it can become a true plugin.

 

-Paul

 

 

 

Great to see you back!   :D

 

I suspect once you hop back on the bike, it will all come back pretty quickly.  But take your time, as I'm sure you have learned, some things are more important than us!

 

I also suspect you will have lots of willing helpers, ready to test anything you want tested.

 

It's always good to make anything that's of value to the community readily accessible and editable by the community.  But we will still need and want you in charge of it!   ;)

0

Share this post


Link to post
Share on other sites

Welcome back Paul.   Hope the health issue has been fully resolved.    My wife has had 6 surgeries in the past 3 years, so I can definitely relate ... and I had a nearly fatal bout with pneumonia about 3 years ago that was pretty scary.

 

But definitely good to see you're "back in the saddle" -- and as Rob noted, I'm sure once you start pedaling, the skills will come back quickly.   Definitely agree, however, that it's good you documented things in writing => there have been MANY times I've wished I had done a better job of that :D

0

Share this post


Link to post
Share on other sites
On 3/4/2017 at 0:18 PM, Pauven said:

Hey BRiT and all the rest of the gang.

 

Apologies for my absence.  I had some health issues crop up unexpected back in September, and it took me out of commission for a while.  Plus I've been pretty busy in the new year.

 

I would like to resume work on the v4 script.  I recall I had ideas back in September, getting ready to completely revamp the testing method.  What I had coded just wasn't working right.  Of course, half a year later, I don't recall what my ideas were.  But fortunately, I have a habit of sharing my ideas in writing, so I think I can get back to where I was in September by re-reading my old posts.

 

Hopefully my absence brought back fond memories of Tom disappearing for long periods of time between unRAID releases...  xD

 

Probably best I don't make any promises of dates just yet.  It'll take me a while to get my bearings and remember how to code again.

 

Once I get a good enough v4, I think it would be a great idea to get it into a git repository so the community can maintain it.  None of ever know when life attacks.  Even better, hopefully it can become a true plugin.

 

-Paul

 

 

 

Welcome back @Pauven Hope you are recovered/recovering well.

 

0

Share this post


Link to post
Share on other sites

Thanks for all the happy greetings and well wishes, much appreciated!  Squid, I especially liked the Welcome Back theme song!  Careful, though, it has me wanting to binge watch the whole series, and I'm easily distracted... :)

 

Recovery is a work in progress, but so is life.  I'm certainly feeling good enough now to try and tackle this challenge.  I really would like to get it knocked out pretty quick, as I've got a long list backlogged projects I need to get restarted.

 

On topic, my parity check times have stayed constant through 6.2 to 6.3.2, all within a few seconds of each other.  I think I read some of you have experienced slowdowns related to different releases.  Is anyone aware of Lime-Tech making any tuneables related changes over the past 6-months?

 

Thanks again!

-Paul

1

Share this post


Link to post
Share on other sites

My Parity checks on the system in my sig have been the same all the way back to December 2015... 111.2 MB/s in 7h 29m 38s give or take 30 seconds... apart from once at the end of June 2016 when it was 103.5 MB/s in 8h 3m 20s. No idea what tuneable was different that month.

0

Share this post


Link to post
Share on other sites

My parity checks took notably longer going from v5 to v6; and have slowed down a good bit more as I moved from 6.1 to 6.2, and then even more going to 6.3.2.    I had assumed this was due to the higher CPU demands of the v6 releases -- and that seems to fit well with most other folks who have reported on this -- those with higher powered systems tend to have stable times; whereas those with older systems (like the two systems I have on 6.3.2) have seen slowdowns.

 

The exception to that is Steven D's dual-Xeon system, as discussed earlier in this thread.   I have no idea why his checks are taking longer.

 

Both of my v6 systems are Pentium E6300's.   Note that I also have an Atom D525 system, but it slowed down so much with v6 I decided to simply leave it on v5 ... where it performs perfectly.

 

I also have a test system with an i7-4770, which has no issues at all with parity check speeds >:(

... but that also gets redone a good bit, depending on just what I'm using it for on any given day.

 

I've been tempted for at least a year to build a high-end Xeon-based setup to replace one of my older systems, but they work so well, and I have SO many computers around the house already, that I really can't talk myself into it (at least not yet).

 

0

Share this post


Link to post
Share on other sites

Good to have you back Pauven... Was just installing unRAID again after some months away and wanted to see if the script was still valid for latest unRAID versions...

0

Share this post


Link to post
Share on other sites

Hey gang,

 

Just wanted to drop a quick update.

 

For those of you that have been reading the "Anybody planning a Ryzen build" message thread, you've realized that I've purchased a Ryzen system to upgrade my server.

 

You would also know that there is a problem with Ryzen + unRAID, a combination that causes random crashes.

 

For the past 2+ weeks, I've been troubleshooting these issues and trying to find a solution.  This has consumed all my free time.

 

In addition, my unRAID server is offline due to my testing.

 

Long story short, I got sidetracked and haven't been able to do anything with the Tunables Tester script.

 

I plan to wrap up my Ryzen testing this week, regardless of the outcome, and worst case I will get the old motherboard back in place to get my server back up.  Then I will resume development of this script.

 

Sorry for the delay after the big tease.

 

-Paul

1

Share this post


Link to post
Share on other sites
2 minutes ago, Pauven said:

Sorry for the delay after the big tease.

 

No problem, good luck, hope you get those issues sorted.

0

Share this post


Link to post
Share on other sites

Thanks Paul, good luck with the development of both the Ryzen and the Tunable Script


Sent from my iPhone using Tapatalk

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.