unraid-tunables-tester.sh - A New Utility to Optimize unRAID md_* Tunables


Recommended Posts

43 minutes ago, BRiT said:

@Pauven any chance of getting the last version you had in private beta last year posted publicly so we can better tune our 6.2 / 6.3 systems?

 

 

Hey BRiT and all the rest of the gang.

 

Apologies for my absence.  I had some health issues crop up unexpected back in September, and it took me out of commission for a while.  Plus I've been pretty busy in the new year.

 

I would like to resume work on the v4 script.  I recall I had ideas back in September, getting ready to completely revamp the testing method.  What I had coded just wasn't working right.  Of course, half a year later, I don't recall what my ideas were.  But fortunately, I have a habit of sharing my ideas in writing, so I think I can get back to where I was in September by re-reading my old posts.

 

Hopefully my absence brought back fond memories of Tom disappearing for long periods of time between unRAID releases...  xD

 

Probably best I don't make any promises of dates just yet.  It'll take me a while to get my bearings and remember how to code again.

 

Once I get a good enough v4, I think it would be a great idea to get it into a git repository so the community can maintain it.  None of ever know when life attacks.  Even better, hopefully it can become a true plugin.

 

-Paul

 

 

Edited by Pauven
  • Upvote 3
Link to comment
2 hours ago, StevenD said:

 

I have dual Xeons, and my parity check/build times are slower.  Its even more noticeable now that I'm migrating to 8TB drives.

 

Interesting.   I had assumed that my slower times were due to the relatively slow E6300 -- although candidly it should still be plenty fast enough to compute parity in real time (especially on my single parity system -- I have two E6300-based setups, one with single; one with dual parity ... and both have become notably slower with the newest versions).

 

But I'm surprised a dual Xeon setup would have this issue.    What model Xeons?

Link to comment
1 minute ago, garycase said:

 

Interesting.   I had assumed that my slower times were due to the relatively slow E6300 -- although candidly it should still be plenty fast enough to compute parity in real time (especially on my single parity system -- I have two E6300-based setups, one with single; one with dual parity ... and both have become notably slower with the newest versions).

 

But I'm surprised a dual Xeon setup would have this issue.    What model Xeons?

 

E5-2670s. Im running under ESX, but Ive passed thru plenty of cores. 

 

I also ran one parity build on baremetal the other day and it was the same speed.

 

my parity checks were over 100MB/s 

Edited by StevenD
Link to comment

Definitely surprised -- dual 2670's score 18353 on PassMark -- my Pentium E6300 scores 1701.   So clearly you have PLENTY of "horsepower".   I'd have not anticipated ANY slowdown in parity checks with that setup.   I presume you haven't made any changes in the disk controllers, memory, or anything else that might have impacted this.

 

Link to comment
5 minutes ago, garycase said:

Definitely surprised -- dual 2670's score 18353 on PassMark -- my Pentium E6300 scores 1701.   So clearly you have PLENTY of "horsepower".   I'd have not anticipated ANY slowdown in parity checks with that setup.   I presume you haven't made any changes in the disk controllers, memory, or anything else that might have impacted this.

 

 

Nope.  My setup has been the same for about a year.  

 

I have Pauvens alpha script, but it didnt give me any improvement, so I went back to default.

 

i did realize the other day that both on my controllers are on CPU1, so im goes ng to open up my box and move one of the controllers to another slot controlled by CPU0. 

Link to comment
29 minutes ago, StevenD said:

i did realize the other day that both on my controllers are on CPU1, so im goes ng to open up my box and move one of the controllers to another slot controlled by CPU0.

 

While that may have a bit of impact; I assume it's been like that all along; so it still doesn't explain why the checks are taking longer with the newer versions.   Until you posted, I had simply assumed it was the higher CPU demands of the v6 releases -- but clearly your system doesn't support that conclusion.

 

Link to comment
16 hours ago, Pauven said:

Hey BRiT and all the rest of the gang.

 

Apologies for my absence.  I had some health issues crop up unexpected back in September, and it took me out of commission for a while.  Plus I've been pretty busy in the new year.

 

I would like to resume work on the v4 script.  I recall I had ideas back in September, getting ready to completely revamp the testing method.  What I had coded just wasn't working right.  Of course, half a year later, I don't recall what my ideas were.  But fortunately, I have a habit of sharing my ideas in writing, so I think I can get back to where I was in September by re-reading my old posts.

 

Hopefully my absence brought back fond memories of Tom disappearing for long periods of time between unRAID releases...  xD

 

Probably best I don't make any promises of dates just yet.  It'll take me a while to get my bearings and remember how to code again.

 

Once I get a good enough v4, I think it would be a great idea to get it into a git repository so the community can maintain it.  None of ever know when life attacks.  Even better, hopefully it can become a true plugin.

 

-Paul

 

 

 

Great to see you back!   :D

 

I suspect once you hop back on the bike, it will all come back pretty quickly.  But take your time, as I'm sure you have learned, some things are more important than us!

 

I also suspect you will have lots of willing helpers, ready to test anything you want tested.

 

It's always good to make anything that's of value to the community readily accessible and editable by the community.  But we will still need and want you in charge of it!   ;)

Link to comment

Welcome back Paul.   Hope the health issue has been fully resolved.    My wife has had 6 surgeries in the past 3 years, so I can definitely relate ... and I had a nearly fatal bout with pneumonia about 3 years ago that was pretty scary.

 

But definitely good to see you're "back in the saddle" -- and as Rob noted, I'm sure once you start pedaling, the skills will come back quickly.   Definitely agree, however, that it's good you documented things in writing => there have been MANY times I've wished I had done a better job of that :D

Link to comment
On 3/4/2017 at 0:18 PM, Pauven said:

Hey BRiT and all the rest of the gang.

 

Apologies for my absence.  I had some health issues crop up unexpected back in September, and it took me out of commission for a while.  Plus I've been pretty busy in the new year.

 

I would like to resume work on the v4 script.  I recall I had ideas back in September, getting ready to completely revamp the testing method.  What I had coded just wasn't working right.  Of course, half a year later, I don't recall what my ideas were.  But fortunately, I have a habit of sharing my ideas in writing, so I think I can get back to where I was in September by re-reading my old posts.

 

Hopefully my absence brought back fond memories of Tom disappearing for long periods of time between unRAID releases...  xD

 

Probably best I don't make any promises of dates just yet.  It'll take me a while to get my bearings and remember how to code again.

 

Once I get a good enough v4, I think it would be a great idea to get it into a git repository so the community can maintain it.  None of ever know when life attacks.  Even better, hopefully it can become a true plugin.

 

-Paul

 

 

 

Welcome back @Pauven Hope you are recovered/recovering well.

 

Link to comment

Thanks for all the happy greetings and well wishes, much appreciated!  Squid, I especially liked the Welcome Back theme song!  Careful, though, it has me wanting to binge watch the whole series, and I'm easily distracted... :)

 

Recovery is a work in progress, but so is life.  I'm certainly feeling good enough now to try and tackle this challenge.  I really would like to get it knocked out pretty quick, as I've got a long list backlogged projects I need to get restarted.

 

On topic, my parity check times have stayed constant through 6.2 to 6.3.2, all within a few seconds of each other.  I think I read some of you have experienced slowdowns related to different releases.  Is anyone aware of Lime-Tech making any tuneables related changes over the past 6-months?

 

Thanks again!

-Paul

  • Upvote 1
Link to comment

My parity checks took notably longer going from v5 to v6; and have slowed down a good bit more as I moved from 6.1 to 6.2, and then even more going to 6.3.2.    I had assumed this was due to the higher CPU demands of the v6 releases -- and that seems to fit well with most other folks who have reported on this -- those with higher powered systems tend to have stable times; whereas those with older systems (like the two systems I have on 6.3.2) have seen slowdowns.

 

The exception to that is Steven D's dual-Xeon system, as discussed earlier in this thread.   I have no idea why his checks are taking longer.

 

Both of my v6 systems are Pentium E6300's.   Note that I also have an Atom D525 system, but it slowed down so much with v6 I decided to simply leave it on v5 ... where it performs perfectly.

 

I also have a test system with an i7-4770, which has no issues at all with parity check speeds >:(

... but that also gets redone a good bit, depending on just what I'm using it for on any given day.

 

I've been tempted for at least a year to build a high-end Xeon-based setup to replace one of my older systems, but they work so well, and I have SO many computers around the house already, that I really can't talk myself into it (at least not yet).

 

Link to comment
  • 2 weeks later...

Hey gang,

 

Just wanted to drop a quick update.

 

For those of you that have been reading the "Anybody planning a Ryzen build" message thread, you've realized that I've purchased a Ryzen system to upgrade my server.

 

You would also know that there is a problem with Ryzen + unRAID, a combination that causes random crashes.

 

For the past 2+ weeks, I've been troubleshooting these issues and trying to find a solution.  This has consumed all my free time.

 

In addition, my unRAID server is offline due to my testing.

 

Long story short, I got sidetracked and haven't been able to do anything with the Tunables Tester script.

 

I plan to wrap up my Ryzen testing this week, regardless of the outcome, and worst case I will get the old motherboard back in place to get my server back up.  Then I will resume development of this script.

 

Sorry for the delay after the big tease.

 

-Paul

  • Upvote 1
Link to comment
  • 4 weeks later...
  • 3 weeks later...
  • 2 weeks later...
On 5/6/2017 at 9:05 PM, BRiT said:

I guess living on the bleeding edge hardware Ryzen for the past 2 months has taken all the time away from a public release?

 

That has definitely been a factor, yes, but just one of many.

 

I have started development of the next UTT release, but have not got to the testing stage yet.  I also have a solid road map for coding my changes.

 

For the moment, I've had to halt progress to work on another project.

 

What could possibly be more important than a new unRAID Tunables Tester?  Pinball, of course!  

 

Okay, okay, I get it, I have messed up priorities.  I'll try to explain.

 

For those of you in the Southeast (USA), the 4th annual "Southern Fried Gameroom Expo" (SFGE) is June 9-11 in Atlanta.  Tons of pinball and arcade machines.  You should go if you can.

 

A few years ago I built my own pinball machine from parts, wrote my own software and used off the shelf, non-pinball circuitry combined with a few circuit boards of my own design.  It came out great.  I took it to SFGE 3 years ago, and it did really well.

 

After that show, I designed my own control circuit boards to replace the off the shelf parts I had used.  My stuff is both more capable and cheaper.  I also had picked up a Williams Black Knight playfield that had been basically trashed, with the intention of building a machine around it and adding my own circuit boards and software to it.

 

Nothing like waiting until the last minute, but I've finally started the build.  With the show 3 weeks from today, I'm completely focused on building this pinball machine.  My goal is to take it to SFGE.  Hopefully I make it, but it is a massive project.  The good news is that if I don't make the show, I've got another year to get it finished...

 

If you interested in reading more, you can check out my blog.  Currently all my posts are from my previous build, Modern Firepower, 3 years ago, but I have a lot of new posts I'm working on detailing the new build:  The Black Knight Rises.

 

http://pinballchameleon.blogspot.com/

 

And right after the show, I'm going on vacation.

 

 

TL;DR:  unRAID Tunables Tester development is on hold until the end of June.  :$

 

Sorry.

 

-Paul

  • Upvote 1
Link to comment

Paul => NEAT hobby !!   I watched the partial interview in your blog and was really looking forward to the details of the "innards" ... but the interview ended (i.e. camera ran out of storage) just as you took it apart) >:(

 

There was enough to get a good idea of the complexity of the project, however.

 

Good luck with finishing your Black Knight in time for this year's expo.

 

Did you ever market your own "pinball construction kit" that you referred to in the blog ??

Link to comment
50 minutes ago, garycase said:

Paul => NEAT hobby !!   I watched the partial interview in your blog and was really looking forward to the details of the "innards" ... but the interview ended (i.e. camera ran out of storage) just as you took it apart) >:(

 

There was enough to get a good idea of the complexity of the project, however.

 

Good luck with finishing your Black Knight in time for this year's expo.

 

Did you ever market your own "pinball construction kit" that you referred to in the blog ??

 

Thanks guys!

 

I think the camera running out of storage is exactly what happened!  Cody (of 908 Pinball Zine) was filming and was very disappointed.  He had caught me at the very end of the show, after I had pretty much packed up.  I showed him under the hood, but he never actually got to see it turned on... and I couldn't be motivated to do so since his storage was full.  We'll go for round 2 in a few weeks.  

 

If you want to see under the hood of Modern Firepower, here's a good post:  http://pinballchameleon.blogspot.com/2013/03/the-modern-firepower-pinball-project.html

 

That was before the playfield was installed, but otherwise it runs pretty much complete (just needs 3 cables to playfield:  USB, 5V/12V power and 50V power).  If you compare the wiring of my playfield with a typical pinball playfield, this is a HUGE improvement.  Black Knight Rises will be even better!

 

While I haven't gone to market, I have developed my own kit, and it is in Alpha Testing (Black Knight is the Alpha test).  I have a new IO Controller for the LED's and Switches, both superior and cheaper to anything else I could find on the market (except for what maybe Stern uses internally).  I have a new version of my Power Driver, shrunken to the size of a business card yet more capable (and Arduino/Pi compatible to boot).  I've developed my own pinball LED lighting solution, though I think I need another revision to make it smaller.

 

I'm really proud of my PCB designs.  I went from through-hole 1980's style circuit boards to surface mount SMT designs, ready for mass production - but a PITA to hand assemble, especially with resistors the size of fleas!!! I can't wait to show it all off, only a handful of people have seen anything I've done.

 

And the crown jewel is my software.  I've been making it better so much better.  

 

There's a guy in the Atlanta area that makes custom pinball machines, and I have him lined up as a Beta Tester.  Once that's done, then I'm ready to try and market it.

 

Oh, and the biggest problem I had with Modern Firepower is that it looked too good!  Most people walked by and didn't realize how unique and special it was in a sea of pinball machines.  This time, I'm making my cabinet look very different, hopefully to draw more attention to Black Knight Rises.

 

-Paul

 

 

  • Upvote 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.