unraid-tunables-tester.sh - A New Utility to Optimize unRAID md_* Tunables


Recommended Posts

I cancelled that and ran the short test using v4.0b3.  The baseline with the new numbers was still a bit lower than I would have expected, but no as bad as before.  Actually, *all* of the numbers were slightly lower than before (none of them beat 148), but perhaps that is the difference between a 30 second test and a 5 minute test. Or maybe I should have rebooted between tests?

 

30 second tests are very inaccurate.  During development, I don't have time to wait around for 30 second tests (especially when running 80+ of them for a test, adds up), so I cheat and run 3 second tests.  The results are hilarious.

 

In the old 2.2 version of the script, going from memory so I may be wrong, I think the normal test length was 3 minutes.  I had tried to set it to the shortest amount of time that produced an accurate answer.

 

With unRAID v6.2, I found that 3 minutes wasn't long enough anymore, so I tested 4 minutes and finally settled on 5 minutes.  Of course, this was all developed on my server, and every server is unique.  I've seen some server results posted here that, surprisingly, had semi-accurate results for 30 seconds.  I imagine the opposite is true, and that for some servers (scottc's, for example), 5 minutes isn't long enough.

 

Anyway, long story longer, the goal of the 30 second test isn't to give accurate results, but rather to determine if changing the values does anything at all to the server.  From what I'm seeing, it does seem to provide that answer.

 

I rebooted and started a full parity check.  We'll see how that compares.

 

Update...  Wow, it started off around 160 MB/s, but I'm 20 minutes in and it has consistently been 170-175!  Can't wait to see the final numbers :)

 

Me too!  Hopefully you used the numbers from the Normal length test, and not the Short.

 

-Paul

Link to comment

Quick question:  On unRAID v5, I used unMenu to handle installing my extras like Screen.  Now on 6.x, I don't use unMenu anymore.  I looked for a plugin to install screen for me, but didn't see one. 

 

Did I just miss it?  Any guidance?

 

unRAID 6 NerdPack

 

But please suggest screen, don't force.

 

Cool, thanks!  Quick question, does the NerdPack install all of the utilities, or can you pick and choose?

 

I've already got burned by a plugin adding a symlink, I don't think I'd want the beta version of lshw installed since I'm using it in my script.

 

Totally agree on suggest vs force.

 

-Paul

Link to comment

Quick question:  On unRAID v5, I used unMenu to handle installing my extras like Screen.  Now on 6.x, I don't use unMenu anymore.  I looked for a plugin to install screen for me, but didn't see one. 

 

Did I just miss it?  Any guidance?

 

unRAID 6 NerdPack

 

But please suggest screen, don't force.

 

Cool, thanks!  Quick question, does the NerdPack install all of the utilities, or can you pick and choose?

 

I've already got burned by a plugin adding a symlink, I don't think I'd want the beta version of lshw installed since I'm using it in my script.

 

Totally agree on suggest vs force.

 

-Paul

You have to select which packages to install.

Link to comment

So, I ran the short test on 4b3 before running the normal test overnight. I got very different results than the previous short test.

 

Yes, for two reasons.  Short test is too short to be accurate.  You could run it a dozen times and get differing results each time.  Also, I made big changes to the Short test in this beta version.

 

The goal of the Short test is to give you a preview of your server's behavior to changing the Tunables before committing 10+ hours to actual testing.

 

The Normal test should be mostly accurate.

 

-Paul

Link to comment

Overnight did a normal test for Tower7, previous short test report for this server is here.

 

 

 

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 00:11:11 BST 2016

                         Run on server: Tower7

                  Normal Automatic Parity Sync Test


Current Values:  md_num_stripes=4096, md_sync_window=2048, md_sync_thresh=2000
                 Global nr_requests=8
                    sdc nr_requests=8
                    sdd nr_requests=8
                    sde nr_requests=8
                    sdf nr_requests=8
                    sdg nr_requests=8


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  | 106 |   4096  |  2048  |   8  |  2000  | 182.4 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    | 185.9 MB/s 
  2   |     1536    |     768     |     128     |      384    | 184.6 MB/s 
  3   |     1536    |     768     |       8     |      767    | 184.7 MB/s 
  4   |     1536    |     768     |       8     |      384    | 184.7 MB/s 

Fastest vals were nr_reqs=128 and sync_thresh=99% of sync_window at 185.9 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 13 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  19 |    768  |   384  | 128  |   383  | 184.2 |   192  | 187.6 
   2  |  23 |    896  |   448  | 128  |   447  | 184.6 |   224  | 183.1 
   3  |  26 |   1024  |   512  | 128  |   511  | 183.7 |   256  | 184.6 
   4  |  29 |   1152  |   576  | 128  |   575  | 181.3 |   288  | 184.7 
   5  |  33 |   1280  |   640  | 128  |   639  | 187.4 |   320  | 184.7 
   6  |  36 |   1408  |   704  | 128  |   703  | 184.4 |   352  | 185.0 
   7  |  39 |   1536  |   768  | 128  |   767  | 183.0 |   384  | 181.8 
   8  |  43 |   1664  |   832  | 128  |   831  | 183.2 |   416  | 184.5 
   9  |  46 |   1792  |   896  | 128  |   895  | 187.5 |   448  | 181.2 
  10  |  49 |   1920  |   960  | 128  |   959  | 182.9 |   480  | 184.6 
  11  |  53 |   2048  |  1024  | 128  |  1023  | 183.8 |   512  | 182.0 
  12  |  56 |   2176  |  1088  | 128  |  1087  | 181.9 |   544  | 184.7 
  13  |  59 |   2304  |  1152  | 128  |  1151  | 184.7 |   576  | 182.5 

--- FULLY AUTOMATIC TEST PASS 1b (Rough - 5 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |   3 |    128  |    64  | 128  |    63  | 184.3 |    32  | 159.9 
   2  |   6 |    256  |   128  | 128  |   127  | 184.5 |    64  | 168.0 
   3  |   9 |    384  |   192  | 128  |   191  | 184.4 |    96  | 181.0 
   4  |  13 |    512  |   256  | 128  |   255  | 184.5 |   128  | 189.5 
   5  |  16 |    640  |   320  | 128  |   319  | 184.5 |   160  | 184.6 

--- Targeting Fastest Result of md_sync_window 256 bytes for Final Pass ---


--- FULLY AUTOMATIC nr_requests TEST 2 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |      512    |     256     |     128     |      255    | 184.4 MB/s 
  2   |      512    |     256     |     128     |      128    | 184.3 MB/s 
  3   |      512    |     256     |       8     |      255    | 184.7 MB/s 
  4   |      512    |     256     |       8     |      128    | 184.1 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 184.7 MB/s

This nr_requests value will be used for the next test.



--- FULLY AUTOMATIC TEST PASS 2 (Fine - 33 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |   6 |    256  |   128  |   8  |   127  | 184.2 |    64  | 166.5 
   2  |   7 |    272  |   136  |   8  |   135  | 184.4 |    68  | 167.9 
   3  |   7 |    288  |   144  |   8  |   143  | 184.4 |    72  | 166.7 
   4  |   7 |    304  |   152  |   8  |   151  | 182.9 |    76  | 175.3 
   5  |   8 |    320  |   160  |   8  |   159  | 184.9 |    80  | 171.5 
   6  |   8 |    336  |   168  |   8  |   167  | 185.0 |    84  | 172.0 
   7  |   9 |    352  |   176  |   8  |   175  | 184.4 |    88  | 171.2 
   8  |   9 |    368  |   184  |   8  |   183  | 184.4 |    92  | 173.7 
   9  |   9 |    384  |   192  |   8  |   191  | 184.4 |    96  | 175.1 
  10  |  10 |    400  |   200  |   8  |   199  | 186.8 |   100  | 180.9 
  11  |  10 |    416  |   208  |   8  |   207  | 187.4 |   104  | 182.7 
  12  |  11 |    432  |   216  |   8  |   215  | 186.2 |   108  | 182.6 
  13  |  11 |    448  |   224  |   8  |   223  | 184.4 |   112  | 184.4 
  14  |  12 |    464  |   232  |   8  |   231  | 184.5 |   116  | 187.8 
  15  |  12 |    480  |   240  |   8  |   239  | 184.5 |   120  | 184.8 
  16  |  12 |    496  |   248  |   8  |   247  | 183.5 |   124  | 184.6 
  17  |  13 |    512  |   256  |   8  |   255  | 183.8 |   128  | 183.3 
  18  |  13 |    528  |   264  |   8  |   263  | 184.5 |   132  | 187.3 
  19  |  14 |    544  |   272  |   8  |   271  | 188.1 |   136  | 184.8 
  20  |  14 |    560  |   280  |   8  |   279  | 184.8 |   140  | 184.4 
  21  |  14 |    576  |   288  |   8  |   287  | 183.9 |   144  | 184.5 
  22  |  15 |    592  |   296  |   8  |   295  | 184.5 |   148  | 187.5 
  23  |  15 |    608  |   304  |   8  |   303  | 184.7 |   152  | 184.5 
  24  |  16 |    624  |   312  |   8  |   311  | 184.4 |   156  | 184.7 
  25  |  16 |    640  |   320  |   8  |   319  | 185.9 |   160  | 182.1 
  26  |  17 |    656  |   328  |   8  |   327  | 187.4 |   164  | 184.5 
  27  |  17 |    672  |   336  |   8  |   335  | 187.6 |   168  | 184.5 
  28  |  17 |    688  |   344  |   8  |   343  | 184.4 |   172  | 184.6 
  29  |  18 |    704  |   352  |   8  |   351  | 184.5 |   176  | 184.7 
  30  |  18 |    720  |   360  |   8  |   359  | 184.6 |   180  | 187.4 
  31  |  19 |    736  |   368  |   8  |   367  | 186.5 |   184  | 184.6 
  32  |  19 |    752  |   376  |   8  |   375  | 186.1 |   188  | 184.5 
  33  |  19 |    768  |   384  |   8  |   383  | 184.5 |   192  | 184.7 

The results below do NOT include the Basline test of current values.

The Fastest Sync Speed tested was md_sync_window=272 at 188.1 MB/s
     Tunable (md_num_stripes): 544
     Tunable (md_sync_window): 272
     Tunable (md_sync_thresh): 271
     Tunable (nr_requests): 8
This will consume 14 MB with md_num_stripes=544, 2x md_sync_window.
This is 92MB less than your current utilization of 106MB.

The Thriftiest Sync Speed tested was md_sync_window=64 at 184.3 MB/s
     Tunable (md_num_stripes): 128
     Tunable (md_sync_window): 64
     Tunable (md_sync_thresh): 63
     Tunable (nr_requests): 8
This will consume 3 MB with md_num_stripes=128, 2x md_sync_window.
This is 103MB less than your current utilization of 106MB.

The Recommended Sync Speed is md_sync_window=208 at 187.4 MB/s
     Tunable (md_num_stripes): 416
     Tunable (md_sync_window): 208
     Tunable (md_sync_thresh): 207
     Tunable (nr_requests): 8
This will consume 10 MB with md_num_stripes=416, 2x md_sync_window.
This is 96MB less than your current utilization of 106MB.

NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Completed: 10 Hrs 17 Min 54 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  Tower7
              unRAID version 6.2.0-rc4
                   md_num_stripes=4096
                   md_sync_window=2048
                   md_sync_thresh=2000
                   nr_requests=8 (Global Setting)
                   sbNumDisks=6
              CPU: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz
              RAM: 32GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path            Device       Class      Description
=======================================================
/0/100/6/0          scsi1        storage    ASC-1405 Unified Serial HBA
/0/100/6/0/0.1.0    /dev/sdk     disk       120GB KINGSTON SV300S3
/0/100/6/0/0.2.0    /dev/sdl     disk       120GB KINGSTON SV300S3
/0/100/6/0/0.3.0    /dev/sdm     disk       120GB KINGSTON SV300S3
/0/100/6/0/0.0.0    /dev/sdj     disk       120GB KINGSTON SV300S3
/0/100/1c/0                      storage    ASM1062 Serial ATA Controller
/0/100/1f.2                      storage    6 Series/C200 Series Chipset Family SATA AHCI Controller
/0/1                scsi0        storage    
/0/1/0.0.0          /dev/sda     disk       7864MB DataTraveler 2.0
/0/1/0.0.0/0        /dev/sda     disk       7864MB 
/0/2                scsi2        storage    
/0/2/0.0.0          /dev/sdb     disk       512GB TS512GSSD370S
/0/3                scsi3        storage    
/0/3/0.0.0          /dev/sdc     disk       3TB TOSHIBA DT01ACA3
/0/8                scsi4        storage    
/0/8/0.0.0          /dev/sdd     disk       3TB TOSHIBA DT01ACA3
/0/9                scsi5        storage    
/0/9/0.0.0          /dev/sde     disk       3TB TOSHIBA DT01ACA3
/0/a                scsi6        storage    
/0/a/0.0.0          /dev/sdf     disk       3TB TOSHIBA DT01ACA3
/0/b                scsi7        storage    
/0/b/0.0.0          /dev/sdg     disk       3TB TOSHIBA DT01ACA3
/0/c                scsi8        storage    
/0/c/0.0.0          /dev/sdh     disk       180GB INTEL SSDSC2CT18
/0/d                scsi9        storage    
/0/d/0.0.0          /dev/sdi     disk       500GB TOSHIBA MK5055GS

Array Devices:
    Disk0 sdc is a Parity drive named parity
    Disk1 sdd is a Data drive named disk1
    Disk2 sde is a Data drive named disk2
    Disk3 sdf is a Data drive named disk3
    Disk4 sdg is a Data drive named disk4

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:       32991160     5420312    25549684      430264     2021164    26694448
Low:       32991160     7441476    25549684
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

 

 

Note the much more consistent results, I'm now running a parity check with the fastest values found, I don't expect much speed improvement as this is a very simple server, but if it remains similar as before it means I had exaggeratedly high values and was wasting a lot of RAM.

 

Awesome, you've got a semi-bizarro server!  The script triggered test 1b, to test values lower than stock unRAID values.  Awesome.

 

When focusing on the md_sync_thresh=50%Max (rightmost column), and if you put the 1b results before the 1a results, you can see a gradual climb from 160 MB/s at 64 to a peak around 188 MB/s in the 320-384 range, and from there it tapers off to about 182 MB/s.

 

When looking at the md_sync_thresh=Max-1 column (3rd column from right), it seems nearly maxed out regardless of the other values set.

 

Can't wait to hear the parity check results.  Sure hope we saved you some memory with no sacrifice in speed.  Fingers crossed, it might even be faster...

 

Suggestion: Since the short test is now much faster, how about doubling (or even tripling) each sample time, I believe this would make the short test much more accurate, helping each user decide if it's worth doing the 10 hour normal test.

 

I see what you're saying.  I'm trying to depreciate the Short test, make it quick and easy to get out of the way, but you make a good point.  I'm not sure there is a right answer here, at least not on the current path.

 

I'm beginning to think that the Short Auto test needs to be replaced by a different test altogether.  Perhaps just the nr_requests test, which for me is illustrating better than anything else if the server responds to changing the Tunables.  In my head, I'm thinking of running the nr_requests test 3 times, once at stock unRAID md_sync_window, again at 768, and again at 1536. 

 

With 5 minute length tests, the above would complete in 1 hour.  With 2.5 minute tests, just 30 minutes.

 

I'm also thinking of adding in a calculation that compares the fastest to the slowest, and if it is beyond a certain threshold, recommend running the Normal Auto.

 

But as of right now, these are just ideas.  I'm certainly open to suggestions.

 

P.S.: could there be a difference in how speeds are reported between the script and unRAID, like one is using MiB/s and other MB/s? I noticed this with the original script, reported unRAID speed is usually about 8% higher than the script, not that it really matters, since the point is finding the best reported speed, but it can look like the script results are slower when in fact they're not.

 

That is a very interesting idea!!!  To calculate the speeds, I using 1024 based notation, not 1000.

 

I just mocked up the impact in Excel, and a 100 MB/s speed with the current 1024 notation becomes 102.4 MB/s.  Slightly higher, but not the 8% you're looking for, just 2.4%.

 

There's always the possibility that Lime-Tech doesn't have the math exactly right, though I'd be surprised if that was the case.  Perhaps my math is bad too...  ;D

 

There's another variable to keep in mind too:  the unRAID value is the average from the beginning of the parity check.  My scripts value doesn't start at the beginning, but rather waits a few seconds for things to settle down, before it begins measuring.  Any early spikes or slowdowns in performance that occur in the first 15 seconds are being filtered out by my script, but not by unRAID.

 

-Paul

Link to comment

Finally back at the computer.  Short test results for Server A

*******************************************************************************
       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 07:07:43 EDT 2016

                         Run on server: Server_A

                   Short Automatic Parity Sync Test


Current Values:  md_num_stripes=1280, md_sync_window=384, md_sync_thresh=192
                 Global nr_requests=128
                    sdg nr_requests=128
                    sdl nr_requests=128
                    sdc nr_requests=128
                    sde nr_requests=128
                    sdf nr_requests=128
                    sdh nr_requests=128
                    sdk nr_requests=128
                    sdi nr_requests=128
                    sdm nr_requests=128
                    sdj nr_requests=128
                    sdd nr_requests=128


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s
-------------------------------------------------------
   1  |  63 |   1280  |   384  | 128  |   192  |  61.8

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 60sec Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  76.2 MB/s
  2   |     1536    |     768     |     128     |      384    |  81.6 MB/s
  3   |     1536    |     768     |       8     |      767    |  95.3 MB/s
  4   |     1536    |     768     |       8     |      384    |  76.4 MB/s

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 95.3 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 4 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s
------------------------------------------------------------------------
   1  |  37 |    768  |   384  |   8  |   383  |  54.3 |   192  |  43.8
   2  |  63 |   1280  |   640  |   8  |   639  |  74.3 |   320  |  64.5
   3  |  88 |   1792  |   896  |   8  |   895  | 102.3 |   448  |  90.0
   4  | 113 |   2304  |  1152  |   8  |  1151  | 115.2 |   576  | 115.1

--- FULLY AUTOMATIC TEST PASS 1c (Rough - 5 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s
------------------------------------------------------------------------
   1  | 120 |   2432  |  1216  |   8  |  1215  | 118.6 |   608  | 116.1
   2  | 145 |   2944  |  1472  |   8  |  1471  | 115.9 |   736  | 103.2
   3  | 170 |   3456  |  1728  |   8  |  1727  | 115.4 |   864  | 111.9
   4  | 195 |   3968  |  1984  |   8  |  1983  | 117.1 |   992  | 108.9
   5  | 221 |   4480  |  2240  |   8  |  2239  | 116.0 |  1120  | 110.5

--- END OF SHORT AUTO TEST FOR DETERMINING IF YOU SHOULD RUN THE NORMAL AUTO ---

If the speeds changed with different values you should run a NORMAL AUTO test.

Completed: 0 Hrs 14 Min 29 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  Server_A
              unRAID version 6.2.0-rc4
                   md_num_stripes=1280
                   md_sync_window=384
                   md_sync_thresh=192
                   nr_requests=128 (Global Setting)
                   sbNumDisks=12
              CPU: AMD A8-6600K APU with Radeon(tm) HD Graphics
              RAM: 12GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path            Device       Class       Description
========================================================
/0/100/3/0          scsi3        storage     88SE9485 SAS/SATA 6Gb/s controller
/0/100/3/0/0.5.0    /dev/sdk     disk        4TB ST4000DM000-1F21
/0/100/3/0/0.6.0    /dev/sdl     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.7.0    /dev/sdm     disk        3TB ST3000DM001-9YN1
/0/100/3/0/0.0.0    /dev/sdf     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.1.0    /dev/sdg     disk        4TB ST4000DM000-1F21
/0/100/3/0/0.2.0    /dev/sdh     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.3.0    /dev/sdi     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.4.0    /dev/sdj     disk        3TB ST3000DM001-1ER1
/0/100/11                        storage     FCH SATA Controller [AHCI mode]
/0/1                scsi0        storage
/0/1/0.0.0          /dev/sda     disk        7756MB DataTraveler SE9
/0/1/0.0.0/0        /dev/sda     disk        7756MB
/0/2                scsi1        storage
/0/2/0.0.0          /dev/sdb     disk        240GB Corsair Force LE
/0/3                scsi2        storage
/0/3/0.0.0          /dev/sdc     disk        3TB ST3000DM001-9YN1
/0/4                scsi4        storage
/0/4/0.0.0          /dev/sdd     disk        3TB WDC WD30EZRX-00S
/0/5                scsi5        storage
/0/5/0.0.0          /dev/sde     disk        3TB WDC WD30EZRX-00M

Array Devices:
    Disk0 sdg is a Parity drive named parity
    Disk1 sdl is a Data drive named disk1
    Disk2 sdc is a Data drive named disk2
    Disk3 sde is a Data drive named disk3
    Disk4 sdf is a Data drive named disk4
    Disk5 sdh is a Data drive named disk5
    Disk6 sdk is a Data drive named disk6
    Disk7 sdi is a Data drive named disk7
    Disk8 sdm is a Data drive named disk8
    Disk9 sdj is a Data drive named disk9
    Disk10 sdd is a Data drive named disk10

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:       12182180     1699264     9952928      384640      529988     9827008
Low:       12182180     2229252     9952928
High:             0           0           0
Swap:             0           0           0

Starting the Normal Test

TLDR The values being returned I don't believe are entirely accurate.  IE:  The baseline test done at the start of the script comes up with 61.8MB/s  and yet my average speed on a complete parity check is 98.6MB/s (and checking dynamix for progress during the first half hour or so always comes up with ~140MB/s

 

Correct, the values are not accurate, but they do show a nice linear progression of speeds from 44 MB/s at low values to 115 MB/s at high values.  That alone is all the confirmation you need that you should run a Normal length test.  Your server responds well to changing the tunables values.

 


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 4 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s
------------------------------------------------------------------------
   1  |  37 |    768  |   384  |   8  |   383  |  54.3 |   192  |  43.8
   2  |  63 |   1280  |   640  |   8  |   639  |  74.3 |   320  |  64.5
   3  |  88 |   1792  |   896  |   8  |   895  | 102.3 |   448  |  90.0
   4  | 113 |   2304  |  1152  |   8  |  1151  | 115.2 |   576  | 115.1

 

Thanks for sharing!

Paul

Link to comment

Server B results (upgraded it to 6.2 just for this, will have to revert back to 6.1.9 due to CA required to run on both 6.2 and 6.1)

I wouldn't be opposed to you running this on 6.1.9.  I'm curious how it would behave.  No idea if it is compatible, though, so no pressure.

 

I appreciate the extra effort!!!

 

 

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 07:22:39 EDT 2016

                         Run on server: Server_B

                   Short Automatic Parity Sync Test


Current Values:  md_num_stripes=1280, md_sync_window=384, md_sync_thresh=192
                 Global nr_requests=128
                    sdd nr_requests=128
                    sdf nr_requests=128
                    sdb nr_requests=128
                    sdc nr_requests=128
                    sde nr_requests=128
                    sdg nr_requests=128
                    sdl nr_requests=128
                    sdh nr_requests=128
                    sdi nr_requests=128
                    sdj nr_requests=128
                    sdk nr_requests=128
                    sdm nr_requests=128


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  68 |   1280  |   384  | 128  |   192  |  37.7 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 60sec Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  94.6 MB/s 
  2   |     1536    |     768     |     128     |      384    |  54.3 MB/s 
  3   |     1536    |     768     |       8     |      767    |  96.4 MB/s 
  4   |     1536    |     768     |       8     |      384    |  53.4 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 96.4 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 4 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  40 |    768  |   384  |   8  |   383  |  80.7 |   192  |  35.7 
   2  |  68 |   1280  |   640  |   8  |   639  |  97.9 |   320  |  49.4 
   3  |  95 |   1792  |   896  |   8  |   895  |  89.6 |   448  |  57.8 
   4  | 122 |   2304  |  1152  |   8  |  1151  |  96.4 |   576  |  71.4 

--- FULLY AUTOMATIC TEST PASS 1c (Rough - 5 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 129 |   2432  |  1216  |   8  |  1215  | 105.7 |   608  |  75.0 
   2  | 156 |   2944  |  1472  |   8  |  1471  |  97.8 |   736  |  99.0 
   3  | 184 |   3456  |  1728  |   8  |  1727  |  99.6 |   864  |  98.4 
   4  | 211 |   3968  |  1984  |   8  |  1983  |  92.2 |   992  |  99.2 
   5  | 238 |   4480  |  2240  |   8  |  2239  |  97.4 |  1120  |  97.8 

--- END OF SHORT AUTO TEST FOR DETERMINING IF YOU SHOULD RUN THE NORMAL AUTO ---

If the speeds changed with different values you should run a NORMAL AUTO test.

Completed: 0 Hrs 14 Min 25 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  Server_B
              unRAID version 6.2.0-rc4
                   md_num_stripes=1280
                   md_sync_window=384
                   md_sync_thresh=192
                   nr_requests=128 (Global Setting)
                   sbNumDisks=13
              CPU: AMD Sempron(tm) 3850 APU with Radeon(tm) R3
   91xx Config
              RAM: 4GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path              Device     Class       Description
========================================================
/0/100/2.1/0          scsi11     storage     SAS1068E PCI-Express Fusion-MPT SAS
/0/100/2.1/0/0.4.0    /dev/sdj   disk        1TB ST31000333AS
/0/100/2.1/0/0.5.0    /dev/sdk   disk        2TB Hitachi HDS72202
/0/100/2.1/0/0.6.0    /dev/sdl   disk        2TB WDC WD20EFRX-68E
/0/100/2.1/0/0.7.0    /dev/sdm   disk        1TB ST31000528AS
/0/100/2.1/0/0.0.0    /dev/sdf   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.1.0    /dev/sdg   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.2.0    /dev/sdh   disk        1TB Hitachi HDT72101
/0/100/2.1/0/0.3.0    /dev/sdi   disk        1TB ST31000524AS
/0/100/2.2/0                     storage     88SE9123 PCIe SATA 6.0 Gb/s controller
/0/100/11                        storage     FCH SATA Controller [AHCI mode]
/0/1                  scsi0      storage     
/0/1/0.0.0            /dev/sda   disk        31GB USB Flash Drive
/0/1/0.0.0/0          /dev/sda   disk        31GB 
/0/2                  scsi1      storage     
/0/2/0.0.0            /dev/sdb   disk        2TB ST2000DM001-1CH1
/0/2/0.0.0/0          /dev/sdb   disk        2TB 
/0/3                  scsi2      storage     
/0/3/0.0.0            /dev/sdc   disk        3TB ST3000DM001-1CH1
/0/3/0.0.0/0          /dev/sdc   disk        3TB 
/0/4                  scsi3      storage     
/0/4/0.0.0            /dev/sdd   disk        3TB ST3000DM001-1CH1
/0/5                  scsi4      storage     
/0/5/0.0.0            /dev/sde   disk        2TB ST2000DL003-9VT1
/0/6                  scsi10     storage     

Array Devices:
    Disk0 sdd is a Parity drive named parity
    Disk1 sdf is a Data drive named disk1
    Disk2 sdb is a Data drive named disk2
    Disk3 sdc is a Data drive named disk3
    Disk4 sde is a Data drive named disk4
    Disk5 sdg is a Data drive named disk5
    Disk6 sdl is a Data drive named disk6
    Disk7 sdh is a Data drive named disk7
    Disk8 sdi is a Data drive named disk8
    Disk9 sdj is a Data drive named disk9
    Disk10 sdk is a Data drive named disk10
    Disk11 sdm is a Data drive named disk11

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:        3961340      146748     3415092      356832      399500     3254348
Low:        3961340      546248     3415092
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

 

Your Short Auto results look very promising.  Can't wait to the the result of the Normal Auto.

 

One suggestion.  Do a TODOS on the resulting txt file.  Make things easy for people to read the resulting text file.

 

Thank you thank you thank you.  I tried to use unix2dos, but found it wasn't installed and gave up.  I was frustrated with the idea of having to modify all the echo statements to get the new lines.

 

Also not sure if reported yet, but if you save the values, its saving null values which kinda messes things up

 

      Current in disk.cfg   |      New Setting
    -----------------------------------------------
     md_num_stripes="1280"  | md_num_stripes="0"
     md_sync_window="384"   | md_sync_window="0"
     md_sync_thresh="192"   | md_sync_thresh=""
     nr_requests="128"      | nr_requests="8"

 

Good catch, that's my mistake.  I pulled the Fastest/Thriftiest/Recommended calculations out of the Short Auto test, but failed to pull the ability to Apply or SAVE the values from the Short Auto run. 

 

You definitely should NOT be saving or using values from the Short test, it's just not accurate enough.

 

I'll fix for the next beta.

 

Thanks,

Paul

Link to comment

Server B results (upgraded it to 6.2 just for this, will have to revert back to 6.1.9 due to CA required to run on both 6.2 and 6.1)

I wouldn't be opposed to you running this on 6.1.9.  I'm curious how it would behave.  No idea if it is compatible, though, so no pressure.

 

I appreciate the extra effort!!!

 

Doesn't work because mdcmd fails.  (You get some error like invalid -lt on lines 1175, etc etc etc)

After the normal tests are done, I'll revert and let you know the exact error.  As it stands now, the script is ONLY 6.2

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 07:22:39 EDT 2016

                         Run on server: Server_B

                   Short Automatic Parity Sync Test


Current Values:  md_num_stripes=1280, md_sync_window=384, md_sync_thresh=192
                 Global nr_requests=128
                    sdd nr_requests=128
                    sdf nr_requests=128
                    sdb nr_requests=128
                    sdc nr_requests=128
                    sde nr_requests=128
                    sdg nr_requests=128
                    sdl nr_requests=128
                    sdh nr_requests=128
                    sdi nr_requests=128
                    sdj nr_requests=128
                    sdk nr_requests=128
                    sdm nr_requests=128


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  68 |   1280  |   384  | 128  |   192  |  37.7 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 60sec Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  94.6 MB/s 
  2   |     1536    |     768     |     128     |      384    |  54.3 MB/s 
  3   |     1536    |     768     |       8     |      767    |  96.4 MB/s 
  4   |     1536    |     768     |       8     |      384    |  53.4 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 96.4 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 4 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  40 |    768  |   384  |   8  |   383  |  80.7 |   192  |  35.7 
   2  |  68 |   1280  |   640  |   8  |   639  |  97.9 |   320  |  49.4 
   3  |  95 |   1792  |   896  |   8  |   895  |  89.6 |   448  |  57.8 
   4  | 122 |   2304  |  1152  |   8  |  1151  |  96.4 |   576  |  71.4 

--- FULLY AUTOMATIC TEST PASS 1c (Rough - 5 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 129 |   2432  |  1216  |   8  |  1215  | 105.7 |   608  |  75.0 
   2  | 156 |   2944  |  1472  |   8  |  1471  |  97.8 |   736  |  99.0 
   3  | 184 |   3456  |  1728  |   8  |  1727  |  99.6 |   864  |  98.4 
   4  | 211 |   3968  |  1984  |   8  |  1983  |  92.2 |   992  |  99.2 
   5  | 238 |   4480  |  2240  |   8  |  2239  |  97.4 |  1120  |  97.8 

--- END OF SHORT AUTO TEST FOR DETERMINING IF YOU SHOULD RUN THE NORMAL AUTO ---

If the speeds changed with different values you should run a NORMAL AUTO test.

Completed: 0 Hrs 14 Min 25 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  Server_B
              unRAID version 6.2.0-rc4
                   md_num_stripes=1280
                   md_sync_window=384
                   md_sync_thresh=192
                   nr_requests=128 (Global Setting)
                   sbNumDisks=13
              CPU: AMD Sempron(tm) 3850 APU with Radeon(tm) R3
   91xx Config
              RAM: 4GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path              Device     Class       Description
========================================================
/0/100/2.1/0          scsi11     storage     SAS1068E PCI-Express Fusion-MPT SAS
/0/100/2.1/0/0.4.0    /dev/sdj   disk        1TB ST31000333AS
/0/100/2.1/0/0.5.0    /dev/sdk   disk        2TB Hitachi HDS72202
/0/100/2.1/0/0.6.0    /dev/sdl   disk        2TB WDC WD20EFRX-68E
/0/100/2.1/0/0.7.0    /dev/sdm   disk        1TB ST31000528AS
/0/100/2.1/0/0.0.0    /dev/sdf   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.1.0    /dev/sdg   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.2.0    /dev/sdh   disk        1TB Hitachi HDT72101
/0/100/2.1/0/0.3.0    /dev/sdi   disk        1TB ST31000524AS
/0/100/2.2/0                     storage     88SE9123 PCIe SATA 6.0 Gb/s controller
/0/100/11                        storage     FCH SATA Controller [AHCI mode]
/0/1                  scsi0      storage     
/0/1/0.0.0            /dev/sda   disk        31GB USB Flash Drive
/0/1/0.0.0/0          /dev/sda   disk        31GB 
/0/2                  scsi1      storage     
/0/2/0.0.0            /dev/sdb   disk        2TB ST2000DM001-1CH1
/0/2/0.0.0/0          /dev/sdb   disk        2TB 
/0/3                  scsi2      storage     
/0/3/0.0.0            /dev/sdc   disk        3TB ST3000DM001-1CH1
/0/3/0.0.0/0          /dev/sdc   disk        3TB 
/0/4                  scsi3      storage     
/0/4/0.0.0            /dev/sdd   disk        3TB ST3000DM001-1CH1
/0/5                  scsi4      storage     
/0/5/0.0.0            /dev/sde   disk        2TB ST2000DL003-9VT1
/0/6                  scsi10     storage     

Array Devices:
    Disk0 sdd is a Parity drive named parity
    Disk1 sdf is a Data drive named disk1
    Disk2 sdb is a Data drive named disk2
    Disk3 sdc is a Data drive named disk3
    Disk4 sde is a Data drive named disk4
    Disk5 sdg is a Data drive named disk5
    Disk6 sdl is a Data drive named disk6
    Disk7 sdh is a Data drive named disk7
    Disk8 sdi is a Data drive named disk8
    Disk9 sdj is a Data drive named disk9
    Disk10 sdk is a Data drive named disk10
    Disk11 sdm is a Data drive named disk11

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:        3961340      146748     3415092      356832      399500     3254348
Low:        3961340      546248     3415092
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

 

Your Short Auto results look very promising.  Can't wait to the the result of the Normal Auto.

 

One suggestion.  Do a TODOS on the resulting txt file.  Make things easy for people to read the resulting text file.

 

Thank you thank you thank you.  I tried to use unix2dos, but found it wasn't installed and gave up.  I was frustrated with the idea of having to modify all the echo statements to get the new lines.

 

Also not sure if reported yet, but if you save the values, its saving null values which kinda messes things up

 

      Current in disk.cfg   |      New Setting
    -----------------------------------------------
     md_num_stripes="1280"  | md_num_stripes="0"
     md_sync_window="384"   | md_sync_window="0"
     md_sync_thresh="192"   | md_sync_thresh=""
     nr_requests="128"      | nr_requests="8"

 

Good catch, that's my mistake.  I pulled the Fastest/Thriftiest/Recommended calculations out of the Short Auto test, but failed to pull the ability to Apply or SAVE the values from the Short Auto run. 

 

You definitely should NOT be saving or using values from the Short test, it's just not accurate enough.

 

I'll fix for the next beta.

 

Thanks,

Paul

Link to comment

Normal test completed.

 


       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Thu Aug 25 20:22:39 CDT 2016

                         Run on server: nas

                  Normal Automatic Parity Sync Test


Current Values:  md_num_stripes=768, md_sync_window=384, md_sync_thresh=50
                 Global nr_requests=128
                    sdo nr_requests=128
                    sdn nr_requests=128
                    sdm nr_requests=128
                    sdl nr_requests=128
                    sdh nr_requests=128
                    sdk nr_requests=128
                    sdj nr_requests=128
                    sdi nr_requests=128
                    sdg nr_requests=128
                    sdf nr_requests=128
                    sde nr_requests=128
                    sdd nr_requests=128
                    sdc nr_requests=128


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  43 |    768  |   384  | 128  |    50  | 129.7 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    | 138.6 MB/s 
  2   |     1536    |     768     |     128     |      384    | 130.5 MB/s 
  3   |     1536    |     768     |       8     |      767    |  90.1 MB/s 
  4   |     1536    |     768     |       8     |      384    | 118.4 MB/s 

Fastest vals were nr_reqs=128 and sync_thresh=99% of sync_window at 138.6 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 13 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  43 |    768  |   384  | 128  |   383  | 120.3 |   192  |  90.2 
   2  |  51 |    896  |   448  | 128  |   447  |  80.7 |   224  | 104.1 
   3  |  58 |   1024  |   512  | 128  |   511  | 127.2 |   256  | 144.5 
   4  |  65 |   1152  |   576  | 128  |   575  | 122.7 |   288  |  92.6 
   5  |  73 |   1280  |   640  | 128  |   639  |  87.5 |   320  |  78.1 
   6  |  80 |   1408  |   704  | 128  |   703  | 124.2 |   352  | 125.1 
   7  |  87 |   1536  |   768  | 128  |   767  | 101.1 |   384  |  88.0 
   8  |  95 |   1664  |   832  | 128  |   831  |  61.8 |   416  |  39.6 
   9  | 102 |   1792  |   896  | 128  |   895  | 100.2 |   448  |  84.8 
  10  | 109 |   1920  |   960  | 128  |   959  |  82.6 |   480  |  80.0 
  11  | 117 |   2048  |  1024  | 128  |  1023  |  78.6 |   512  |  81.2 
  12  | 124 |   2176  |  1088  | 128  |  1087  |  80.3 |   544  |  86.8 
  13  | 131 |   2304  |  1152  | 128  |  1151  |  83.4 |   576  |  85.5 

--- Targeting Fastest Result of md_sync_window 512 bytes for Final Pass ---


--- FULLY AUTOMATIC nr_requests TEST 2 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1024    |     512     |     128     |      511    |  83.2 MB/s 
  2   |     1024    |     512     |     128     |      256    | 121.3 MB/s 
  3   |     1024    |     512     |       8     |      511    | 101.7 MB/s 
  4   |     1024    |     512     |       8     |      256    | 123.3 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=50% of sync_window at 123.3 MB/s

This nr_requests value will be used for the next test.



--- FULLY AUTOMATIC TEST PASS 2 (Fine - 33 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  43 |    768  |   384  |   8  |   383  |  79.0 |   192  |  71.7 
   2  |  44 |    784  |   392  |   8  |   391  |  95.2 |   196  | 135.2 
   3  |  45 |    800  |   400  |   8  |   399  |  92.6 |   200  |  75.5 
   4  |  46 |    816  |   408  |   8  |   407  | 101.6 |   204  | 125.9 
   5  |  47 |    832  |   416  |   8  |   415  | 113.7 |   208  | 115.3 
   6  |  48 |    848  |   424  |   8  |   423  |  77.3 |   212  |  63.8 
   7  |  49 |    864  |   432  |   8  |   431  |  98.6 |   216  |  97.3 
   8  |  50 |    880  |   440  |   8  |   439  |  90.9 |   220  |  66.8 
   9  |  51 |    896  |   448  |   8  |   447  |  70.9 |   224  |  41.6 
  10  |  52 |    912  |   456  |   8  |   455  |  61.8 |   228  |  48.7 
  11  |  53 |    928  |   464  |   8  |   463  |  76.8 |   232  |  44.1 
  12  |  54 |    944  |   472  |   8  |   471  |  71.3 |   236  |  68.3 
  13  |  54 |    960  |   480  |   8  |   479  |  77.9 |   240  |  54.1 
  14  |  55 |    976  |   488  |   8  |   487  |  70.6 |   244  |  56.5 
  15  |  56 |    992  |   496  |   8  |   495  |  92.0 |   248  |  83.1 
  16  |  57 |   1008  |   504  |   8  |   503  |  88.0 |   252  |  58.2 
  17  |  58 |   1024  |   512  |   8  |   511  |  77.3 |   256  |  70.7 
  18  |  59 |   1040  |   520  |   8  |   519  |  98.8 |   260  | 139.4 
  19  |  60 |   1056  |   528  |   8  |   527  |  97.0 |   264  |  74.2 
  20  |  61 |   1072  |   536  |   8  |   535  |  81.2 |   268  | 114.5 
  21  |  62 |   1088  |   544  |   8  |   543  |  77.8 |   272  |  87.9 
  22  |  63 |   1104  |   552  |   8  |   551  |  69.7 |   276  |  68.3 
  23  |  64 |   1120  |   560  |   8  |   559  |  68.9 |   280  |  74.4 
  24  |  64 |   1136  |   568  |   8  |   567  |  72.9 |   284  |  70.1 
  25  |  65 |   1152  |   576  |   8  |   575  |  72.6 |   288  |  62.3 
  26  |  66 |   1168  |   584  |   8  |   583  |  73.4 |   292  |  67.3 
  27  |  67 |   1184  |   592  |   8  |   591  |  81.2 |   296  |  83.4 
  28  |  68 |   1200  |   600  |   8  |   599  |  81.1 |   300  |  79.1 
  29  |  69 |   1216  |   608  |   8  |   607  |  81.6 |   304  |  83.9 
  30  |  70 |   1232  |   616  |   8  |   615  |  81.2 |   308  |  81.5 
  31  |  71 |   1248  |   624  |   8  |   623  |  68.7 |   312  |  81.6 
  32  |  72 |   1264  |   632  |   8  |   631  |  78.5 |   316  |  77.9 
  33  |  73 |   1280  |   640  |   8  |   639  |  83.6 |   320  |  79.9 

The results below do NOT include the Basline test of current values.

The Fastest Sync Speed tested was md_sync_window=520 at 139.4 MB/s
     Tunable (md_num_stripes): 1040
     Tunable (md_sync_window): 520
     Tunable (md_sync_thresh): 260
     Tunable (nr_requests): 8
This will consume 59 MB with md_num_stripes=1040, 2x md_sync_window.
This is 16MB more than your current utilization of 43MB.

The Thriftiest Sync Speed tested was md_sync_window=392 at 135.2 MB/s
     Tunable (md_num_stripes): 784
     Tunable (md_sync_window): 392
     Tunable (md_sync_thresh): 196
     Tunable (nr_requests): 8
This will consume 44 MB with md_num_stripes=784, 2x md_sync_window.
This is 1MB more than your current utilization of 43MB.

The Recommended Sync Speed is md_sync_window=520 at 139.4 MB/s
     Tunable (md_num_stripes): 1040
     Tunable (md_sync_window): 520
     Tunable (md_sync_thresh): 260
     Tunable (nr_requests): 8
This will consume 59 MB with md_num_stripes=1040, 2x md_sync_window.
This is 16MB more than your current utilization of 43MB.

NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Completed: 10 Hrs 8 Min 38 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  nas
              unRAID version 6.2.0-rc4
                   md_num_stripes=768
                   md_sync_window=384
                   md_sync_thresh=50
                   nr_requests=128 (Global Setting)
                   sbNumDisks=14
              CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
  Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
  CPU
  CPU
  CPU
  CPU
  CPU
  CPU
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
  CPU [empty]
              RAM: 32GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path             Device     Class      Description
======================================================
/0/100/7.1                      storage    82371AB/EB/MB PIIX4 IDE
/0/100/15/0          scsi3      storage    SAS1068 PCI-X Fusion-MPT SAS
/0/100/15/0/0.0.0    /dev/sda   disk       209MB Virtual disk
/0/100/16/0          scsi2      storage    SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/100/16/0/0.0.0    /dev/sdb   disk       512GB Crucial_CT512M55
/0/100/16/0/0.1.0    /dev/sdc   disk       4TB HGST HDN724040AL
/0/100/16/0/0.2.0    /dev/sdd   disk       4TB Hitachi HDS72404
/0/100/16/0/0.3.0    /dev/sde   disk       4TB HGST HDN724040AL
/0/100/16/0/0.4.0    /dev/sdf   disk       4TB Hitachi HDS72404
/0/100/16/0/0.5.0    /dev/sdg   disk       4TB HGST HDS724040AL
/0/100/17/0          scsi4      storage    SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/100/17/0/0.3.0    /dev/sdk   disk       4TB HGST HDS724040AL
/0/100/17/0/0.4.0    /dev/sdl   disk       4TB HGST HDS724040AL
/0/100/17/0/0.5.0    /dev/sdm   disk       4TB HGST HDS724040AL
/0/100/17/0/0.6.0    /dev/sdn   disk       4TB HGST HDS724040AL
/0/100/17/0/0.7.0    /dev/sdo   disk       4TB Hitachi HDS72404
/0/100/17/0/0.0.0    /dev/sdh   disk       4TB HGST HDS724040AL
/0/100/17/0/0.1.0    /dev/sdi   disk       4TB Hitachi HDS72404
/0/100/17/0/0.2.0    /dev/sdj   disk       4TB HGST HDS724040AL
/0/1                 scsi5      storage    
/0/1/0.0.0           /dev/sdp   disk       15GB Reader     SD/MS
/0/1/0.0.0/0         /dev/sdp   disk       15GB 
/0/1/0.0.1           /dev/sdq   disk       Reader  MicSD/M2
/0/1/0.0.1/0         /dev/sdq   disk       

Array Devices:
    Disk0 sdo is a Parity drive named parity
    Disk1 sdn is a Data drive named disk1
    Disk2 sdm is a Data drive named disk2
    Disk3 sdl is a Data drive named disk3
    Disk4 sdh is a Data drive named disk4
    Disk5 sdk is a Data drive named disk5
    Disk6 sdj is a Data drive named disk6
    Disk7 sdi is a Data drive named disk7
    Disk8 sdg is a Data drive named disk8
    Disk9 sdf is a Data drive named disk9
    Disk10 sde is a Data drive named disk10
    Disk11 sdd is a Data drive named disk11
    Disk12 sdc is a Data drive named disk12

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:       32950372     4713036    25198628      415548     3038708    27065324
Low:       32950372     7751744    25198628
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

[/cpde]

 

I'm not liking these results.  There's not much consistency, even at 5 minutes.

 

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    | 138.6 MB/s 
  2   |     1536    |     768     |     128     |      384    | 130.5 MB/s 

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s
------------------------------------------------------------------------
   7  |  87 |   1536  |   768  | 128  |   767  | 101.1 |   384  |  88.0

 

Tests 1 & 2 were from the nr_requests test, and ran for 10 minutes each (so more accurate):

 

Test 7 tested the exact same values from tests 1 and 2, just at 5 minutes instead of 10, with downward swings of 37.5 MB/s and 42.5 MB/s.

 

That's not accurate at all.

 

scottc had similar results, I recommend you read my post to him:  https://lime-technology.com/forum/index.php?topic=29009.msg493338#msg493338

 

Thanks,

Paul

Link to comment

Doesn't work because mdcmd fails.  (You get some error like invalid -lt on lines 1175, etc etc etc)

After the normal tests are done, I'll revert and let you know the exact error.  As it stands now, the script is ONLY 6.2

 

 

Okay, wow.  I appreciate the info.

 

What's the general consensus, do I need to target 6.1.9 too?  I understand the reservations to upgraded to a remotely authorized version, but I'm hopeful 6.2 final is close at hand.

 

Paul

Link to comment

Doesn't work because mdcmd fails.  (You get some error like invalid -lt on lines 1175, etc etc etc)

After the normal tests are done, I'll revert and let you know the exact error.  As it stands now, the script is ONLY 6.2

 

 

Okay, wow.  I appreciate the info.

 

What's the general consensus, do I need to target 6.1.9 too?  I understand the reservations to upgraded to a remotely authorized version, but I'm hopeful 6.2 final is close at hand.

 

Paul

Its your script - you decide.  v2.2 with the mods I posted does work fine on 6.1  I have no choice but to downgrade the one server if only to verify CA compatibility with 6.1 for at least 6mths after 6.2 goes final.
Link to comment

Check finished, original settings:

 

Duration: 5 hours, 32 minutes, 38 seconds. Average speed: 150.3 MB/s

 

New settings, using ~90% less RAM:

 

Duration: 5 hours, 33 minutes, 11 seconds. Average speed: 150.1 MB/s

 

 

As to the difference in speed reported, see this server as an example, script reports max speed at parity check start of ~185MB/s, unRAID reports ~200MB/s during the first 5% of the check:

 

pc1.png

 

pc2.png

 

 

Link to comment

Just a silly question which I'm sure has been considered and taken care of...

 

Before the very first test starts, does the script have a step to manually spin up each and every drive to make sure drive spin-up doesn't impact the measured test?

 

Not a silly question at all.

 

Yes, new in v4, I spin up each idle drive, one-by-one with a 2 second delay inbetween, so it is a staggered spin-up.

 

That said, this was just a cosmetic feature.  Because I don't start measuring from the beginning of a parity check, but rather 15+ seconds into it, the impact of the automatic spin-up was already negated in the results.

 

I didn't like how the script appeared to hang during an automatic spin-up, every time it made me think something was wrong.  With my new spin-up code, it never hangs and you can always tell what the script is doing.

 

-Paul

Link to comment

Check finished, original settings:

 

Duration: 5 hours, 32 minutes, 38 seconds. Average speed: 150.3 MB/s

 

New settings, using ~90% less RAM:

 

Duration: 5 hours, 33 minutes, 11 seconds. Average speed: 150.1 MB/s

 

That's awesome!

 

I guess the questing is what IS the real impact of all that memory usage? Because if it doesn't really matter, 99% seems good. If it does, then 95% seems like the place to draw the line. Either way, awesome work.

 

Hey jumperalex, not to call you out, but wanted to make sure you saw this.  The tool isn't just about throwing big numbers and using more RAM.  Some servers need more, but some need less.  This tool simply discovers what works.  While that doesn't negate your question, which is very valid when the values that work consume tons of RAM, it's certainly nice to see results that are so good we don't even have to question them.

 

As to the difference in speed reported, see this server as an example, script reports max speed at parity check start of ~185MB/s, unRAID reports ~200MB/s during the first 5% of the check:

 

pc1.png

 

pc2.png

 

 

I get similar results.  My tests never report higher than 132 MB/s, yet unRAID is reporting 147 MB/s.  That's an 11% variance, much bigger than simply a MiB vs MB conversion factor.  I'm really not sure where it is coming from.

 

Interestingly, the variance you have is close to 11% too.

 

Here's mine:

 

Elapsed time:		1 minute	
Current position:	10.1 GB (0.3 %)	
Estimated speed:	145.0 MB/sec	
Estimated finish:	5 hours, 44 minutes

 

Elapsed time:		5 minutes	
Current position:	43.9 GB (1.5 %)	
Estimated speed:	147.1 MB/sec	
Estimated finish:	5 hours, 35 minutes

 

Elapsed time:	10 minutes	
Current position:	92.2 GB (3.1 %)	
Estimated speed:	144.4 MB/sec	
Estimated finish:	5 hours, 36 minutes

 

 

Here's an overview of the math & logic in the script:

 

First I determine at what point I will begin the stopwatch:

TestStartPos=$(( ArraySize / (500000 / TestLen) )) 

 

On my server, ArraySize=2,930,266,532 (3TB).  For a 5 minute test (TestLen=300) then I don't start measuring until the Parity Check hits 1,758,160 (that's about 1.75 GB).  On my server, it takes about 14 seconds to get to that point.

 

Next I start and monitor the parity check, waiting for it to reach the TestStartPos:

### Start a Non-Correcting Parity Check ###
mdcmd check NOCORRECT

### Get the Current Position of the Parity Check ###
Position=`mdcmd status | egrep mdResyncPos | sed 's/.*=\(.*\)/\1/'`

### Wait until the Current Position reaches the Test Starting Position ###
while [ "$Position" -lt "$TestStartPos" ] 
do
	Position=`mdcmd status | egrep mdResyncPos | sed 's/.*=\(.*\)/\1/'`
	echo -ne "\rTest $XSpacer$X$T - md_sync_window=$SyncSpacer$TestSyncWindow - Finding Test Range..." # - Stopwatch: 0.00s - Current Position: "$Position
done

 

Notice that there is no "sleep" statement in there, so there is a possibility that frequent mdcmd status checks are slowing down the speed.  I'm pretty sure this is spiking CPU usage, as this is how the original version of this script worked, before I changed over to the stopwatch method.

 

That said, technically, I'm not measuring anything yet, so even if this is spiking CPU and slowing speeds, is it having any actual impact?

 

Once the parity check reaches the TestStartPos, I take measurements and start the countdown:

StopWatch=$TestLen
### Test Range Entered - Log the Start Time ###
StartTime=$(date +%s.%N) #$SECONDS

### Display the countdown timer while the Parity Sync runs ###
while [ $StopWatch -gt 0 ] 
do
echo -ne "\rTest $XSpacer$X$T - md_sync_window=$SyncSpacer$TestSyncWindow - Test Range Entered - Time Remaining: "$StopWatch"s " # - Current Position: "$Position
sleep 1s
let "StopWatch-=1"
done 

 

I grab the StartTime so I get the exact moment I started the actual measurement portion of the test.  I then begin the countdown, which is easy on the CPU because the script is primarily sleeping during this portion, allowing the parity check to run unhindered.

 

When the countdown completes, I take the final measurements:

### Test Range Exited - Log the End Time ###
Position=`mdcmd status | egrep mdResyncPos | sed 's/.*=\(.*\)/\1/'`		
EndTime=$(date +%s.%N) #$SECONDS

### Cancel the Parity Check ###
mdcmd nocheck

 

Notice that I grab the Position first, then grab the End Time, then end the parity check.  Perhaps I could grab the End Time before grabbing the Position, but I doubt the lag time in grabbing the Postion is enough to cause a huge swing in MB/s calculations.

 

Finally, I do the math:

### Calculate the test duration and resulting speed ###
RealDuration=`echo "$EndTime $StartTime" | awk '{ printf( "%0.3f\n", ($1 - $2) )}'` 

BytesProcessed=$(( Position - TestStartPos ))

RealSyncSpeed=`echo "$BytesProcessed $RealDuration" | awk '{ printf( "%0.1f\n", (($1 / $2) / 1024) )}'`

 

The duration is the actual start time minus the actual end time, to 3 decimal places.  Should be pretty accurate.

 

Bytes Processed is the Position the parity check got to when the test ended, minus the TestStartPos since I don't measure from 0.  These are unconverted unRAID #'s, so should be pretty accurate.

 

Finally I get the MB/s by dividing the actual Bytes Processed (again these seem to be KB and not Bytes) by the actual elapsed test time, and then divide that result by 1024 to convert from KB/s to MB/s.

 

I don't know what Lime-Tech is doing, but best I can tell, my logic is pretty sound.  If there's a problem, I'm just not seeing it.

 

If I divide by 1000 instead of 1024, that only gives me an increase of 2.4%, far too low to fix the 11% variance.

 

Perhaps the BytesProcessed, being in KB, has already been divided by 1024 from the true Bytes, in which case I could do ByteProcess*(1024/1000) to modify that conversion, and also use 1000 when I convert to MB/s. 

 

But that would only raise the MB/s by about 4.9%, still far less than the 11% variance.

 

If someone sees something I'm missing, please help me out.  Perhaps Lime-Tech can give us some insight.

 

-Paul

 

Link to comment

One suggestion.  Do a TODOS on the resulting txt file.  Make things easy for people to read the resulting text file.

 

Thank you thank you thank you.  I tried to use unix2dos, but found it wasn't installed and gave up.  I was frustrated with the idea of having to modify all the echo statements to get the new lines.

 

 

I tried to use todos, and it never did anything.  It always acted like it wanted more parameters.

 

I gave up on todos, but found another solution, a simple sed command to do the conversion.  Works a treat:

 

sed -i -e 's/\r*$/\r/' $ReportFile 

Link to comment

Well, I don't know what to make out of these results either.  Yes, this is a short test, but the numbers seem rather low.  This time, I booted bare metal, instead of in ESXi.  I will run a full test tonight.

 

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 16:25:23 CDT 2016

                         Run on server: nas

                   Short Automatic Parity Sync Test


Current Values:  md_num_stripes=1040, md_sync_window=520, md_sync_thresh=260
                 Global nr_requests=8
                    sdq nr_requests=8
                    sdp nr_requests=8
                    sdo nr_requests=8
                    sdn nr_requests=8
                    sdj nr_requests=8
                    sdm nr_requests=8
                    sdl nr_requests=8
                    sdk nr_requests=8
                    sdi nr_requests=8
                    sdh nr_requests=8
                    sdf nr_requests=8
                    sde nr_requests=8
                    sdg nr_requests=8


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  59 |   1040  |   520  |   8  |   260  |  61.8 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 60sec Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  73.5 MB/s 
  2   |     1536    |     768     |     128     |      384    |  63.3 MB/s 
  3   |     1536    |     768     |       8     |      767    |  83.3 MB/s 
  4   |     1536    |     768     |       8     |      384    |  60.8 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 83.3 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 4 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  43 |    768  |   384  |   8  |   383  |  66.2 |   192  |  34.9 
   2  |  73 |   1280  |   640  |   8  |   639  |  74.5 |   320  |  49.0 
   3  | 102 |   1792  |   896  |   8  |   895  |  75.8 |   448  |  68.6 
   4  | 131 |   2304  |  1152  |   8  |  1151  | 103.3 |   576  |  88.8 

--- FULLY AUTOMATIC TEST PASS 1c (Rough - 5 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 139 |   2432  |  1216  |   8  |  1215  |  78.1 |   608  |  76.5 
   2  | 168 |   2944  |  1472  |   8  |  1471  |  75.3 |   736  |  79.9 
   3  | 197 |   3456  |  1728  |   8  |  1727  |  77.0 |   864  |  82.6 
   4  | 226 |   3968  |  1984  |   8  |  1983  | 102.3 |   992  |  77.2 
   5  | 256 |   4480  |  2240  |   8  |  2239  |  87.6 |  1120  |  85.9 

--- END OF SHORT AUTO TEST FOR DETERMINING IF YOU SHOULD RUN THE NORMAL AUTO ---

If the speeds changed with different values you should run a NORMAL AUTO test.

Completed: 0 Hrs 15 Min 2 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  nas
              unRAID version 6.2.0-rc4
                   md_num_stripes=1040
                   md_sync_window=520
                   md_sync_thresh=260
                   nr_requests=8 (Global Setting)
                   sbNumDisks=14
              CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
      Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
              RAM: System Memory
         System Memory

Outputting lshw information for Drives and Controllers:

H/W path            Device     Class          Description
=========================================================
/0/100/1f.2                    storage        C600/X79 series chipset 6-Port SATA AHCI Controller
/0/2/0              scsi1      storage        SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/2/0/0.0.0        /dev/sdd   disk           512GB Crucial_CT512M55
/0/2/0/0.1.0        /dev/sde   disk           4TB Hitachi HDS72404
/0/2/0/0.2.0        /dev/sdf   disk           4TB HGST HDN724040AL
/0/2/0/0.3.0        /dev/sdg   disk           4TB HGST HDN724040AL
/0/2/0/0.4.0        /dev/sdh   disk           4TB Hitachi HDS72404
/0/2/0/0.5.0        /dev/sdi   disk           4TB HGST HDS724040AL
/0/3/0              scsi8      storage        SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/3/0/0.1.0        /dev/sdk   disk           4TB Hitachi HDS72404
/0/3/0/0.2.0        /dev/sdl   disk           4TB HGST HDS724040AL
/0/3/0/0.3.0        /dev/sdm   disk           4TB HGST HDS724040AL
/0/3/0/0.4.0        /dev/sdn   disk           4TB HGST HDS724040AL
/0/3/0/0.5.0        /dev/sdo   disk           4TB HGST HDS724040AL
/0/3/0/0.6.0        /dev/sdp   disk           4TB HGST HDS724040AL
/0/3/0/0.7.0        /dev/sdq   disk           4TB Hitachi HDS72404
/0/3/0/0.0.0        /dev/sdj   disk           4TB HGST HDS724040AL
/0/68               scsi0      storage        
/0/68/0.0.0         /dev/sda   disk           15GB Reader     SD/MS
/0/68/0.0.0/0       /dev/sda   disk           15GB 
/0/68/0.0.1         /dev/sdb   disk           Reader  MicSD/M2
/0/68/0.0.1/0       /dev/sdb   disk           
/0/69               scsi2      storage        
/0/69/0.0.0         /dev/sdc   disk           60GB INTEL SSDSC2CT06

Array Devices:
    Disk0 sdq is a Parity drive named parity
    Disk1 sdp is a Data drive named disk1
    Disk2 sdo is a Data drive named disk2
    Disk3 sdn is a Data drive named disk3
    Disk4 sdj is a Data drive named disk4
    Disk5 sdm is a Data drive named disk5
    Disk6 sdl is a Data drive named disk6
    Disk7 sdk is a Data drive named disk7
    Disk8 sdi is a Data drive named disk8
    Disk9 sdh is a Data drive named disk9
    Disk10 sdf is a Data drive named disk10
    Disk11 sde is a Data drive named disk11
    Disk12 sdg is a Data drive named disk12

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:      132071772     4478572   124860888      406796     2732312   126232912
Low:      132071772     7210884   124860888
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

Link to comment

Well, I don't know what to make out of these results either.  Yes, this is a short test, but the numbers seem rather low.  This time, I booted bare metal, instead of in ESXi.  I will run a full test tonight.

 

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 16:25:23 CDT 2016

                         Run on server: nas

                   Short Automatic Parity Sync Test


Current Values:  md_num_stripes=1040, md_sync_window=520, md_sync_thresh=260
                 Global nr_requests=8
                    sdq nr_requests=8
                    sdp nr_requests=8
                    sdo nr_requests=8
                    sdn nr_requests=8
                    sdj nr_requests=8
                    sdm nr_requests=8
                    sdl nr_requests=8
                    sdk nr_requests=8
                    sdi nr_requests=8
                    sdh nr_requests=8
                    sdf nr_requests=8
                    sde nr_requests=8
                    sdg nr_requests=8


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  59 |   1040  |   520  |   8  |   260  |  61.8 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 60sec Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  73.5 MB/s 
  2   |     1536    |     768     |     128     |      384    |  63.3 MB/s 
  3   |     1536    |     768     |       8     |      767    |  83.3 MB/s 
  4   |     1536    |     768     |       8     |      384    |  60.8 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 83.3 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 4 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  43 |    768  |   384  |   8  |   383  |  66.2 |   192  |  34.9 
   2  |  73 |   1280  |   640  |   8  |   639  |  74.5 |   320  |  49.0 
   3  | 102 |   1792  |   896  |   8  |   895  |  75.8 |   448  |  68.6 
   4  | 131 |   2304  |  1152  |   8  |  1151  | 103.3 |   576  |  88.8 

--- FULLY AUTOMATIC TEST PASS 1c (Rough - 5 Sample Points @ 30sec Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 139 |   2432  |  1216  |   8  |  1215  |  78.1 |   608  |  76.5 
   2  | 168 |   2944  |  1472  |   8  |  1471  |  75.3 |   736  |  79.9 
   3  | 197 |   3456  |  1728  |   8  |  1727  |  77.0 |   864  |  82.6 
   4  | 226 |   3968  |  1984  |   8  |  1983  | 102.3 |   992  |  77.2 
   5  | 256 |   4480  |  2240  |   8  |  2239  |  87.6 |  1120  |  85.9 

--- END OF SHORT AUTO TEST FOR DETERMINING IF YOU SHOULD RUN THE NORMAL AUTO ---

If the speeds changed with different values you should run a NORMAL AUTO test.

Completed: 0 Hrs 15 Min 2 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  nas
              unRAID version 6.2.0-rc4
                   md_num_stripes=1040
                   md_sync_window=520
                   md_sync_thresh=260
                   nr_requests=8 (Global Setting)
                   sbNumDisks=14
              CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
      Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
              RAM: System Memory
         System Memory

Outputting lshw information for Drives and Controllers:

H/W path            Device     Class          Description
=========================================================
/0/100/1f.2                    storage        C600/X79 series chipset 6-Port SATA AHCI Controller
/0/2/0              scsi1      storage        SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/2/0/0.0.0        /dev/sdd   disk           512GB Crucial_CT512M55
/0/2/0/0.1.0        /dev/sde   disk           4TB Hitachi HDS72404
/0/2/0/0.2.0        /dev/sdf   disk           4TB HGST HDN724040AL
/0/2/0/0.3.0        /dev/sdg   disk           4TB HGST HDN724040AL
/0/2/0/0.4.0        /dev/sdh   disk           4TB Hitachi HDS72404
/0/2/0/0.5.0        /dev/sdi   disk           4TB HGST HDS724040AL
/0/3/0              scsi8      storage        SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/3/0/0.1.0        /dev/sdk   disk           4TB Hitachi HDS72404
/0/3/0/0.2.0        /dev/sdl   disk           4TB HGST HDS724040AL
/0/3/0/0.3.0        /dev/sdm   disk           4TB HGST HDS724040AL
/0/3/0/0.4.0        /dev/sdn   disk           4TB HGST HDS724040AL
/0/3/0/0.5.0        /dev/sdo   disk           4TB HGST HDS724040AL
/0/3/0/0.6.0        /dev/sdp   disk           4TB HGST HDS724040AL
/0/3/0/0.7.0        /dev/sdq   disk           4TB Hitachi HDS72404
/0/3/0/0.0.0        /dev/sdj   disk           4TB HGST HDS724040AL
/0/68               scsi0      storage        
/0/68/0.0.0         /dev/sda   disk           15GB Reader     SD/MS
/0/68/0.0.0/0       /dev/sda   disk           15GB 
/0/68/0.0.1         /dev/sdb   disk           Reader  MicSD/M2
/0/68/0.0.1/0       /dev/sdb   disk           
/0/69               scsi2      storage        
/0/69/0.0.0         /dev/sdc   disk           60GB INTEL SSDSC2CT06

Array Devices:
    Disk0 sdq is a Parity drive named parity
    Disk1 sdp is a Data drive named disk1
    Disk2 sdo is a Data drive named disk2
    Disk3 sdn is a Data drive named disk3
    Disk4 sdj is a Data drive named disk4
    Disk5 sdm is a Data drive named disk5
    Disk6 sdl is a Data drive named disk6
    Disk7 sdk is a Data drive named disk7
    Disk8 sdi is a Data drive named disk8
    Disk9 sdh is a Data drive named disk9
    Disk10 sdf is a Data drive named disk10
    Disk11 sde is a Data drive named disk11
    Disk12 sdg is a Data drive named disk12

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:      132071772     4478572   124860888      406796     2732312   126232912
Low:      132071772     7210884   124860888
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

 

Short test isn't accurate enough to worry about the numbers.  All you should do is look for the behavior that the numbers appear to change in response to different Tunables values.

 

Additionally, there is a possiblity that bare metal behaves differently than virtualized.  I'm curious to see the Normal results.

 

Paul

Link to comment

My normal test results:

 

Server_A

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 08:47:57 EDT 2016

                         Run on server: Server_A

                  Normal Automatic Parity Sync Test


Current Values:  md_num_stripes=1280, md_sync_window=384, md_sync_thresh=192
                 Global nr_requests=128
                    sdg nr_requests=128
                    sdl nr_requests=128
                    sdc nr_requests=128
                    sde nr_requests=128
                    sdf nr_requests=128
                    sdh nr_requests=128
                    sdk nr_requests=128
                    sdi nr_requests=128
                    sdm nr_requests=128
                    sdj nr_requests=128
                    sdd nr_requests=128


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  63 |   1280  |   384  | 128  |   192  |  99.5 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  74.2 MB/s 
  2   |     1536    |     768     |     128     |      384    | 112.6 MB/s 
  3   |     1536    |     768     |       8     |      767    | 111.7 MB/s 
  4   |     1536    |     768     |       8     |      384    | 111.7 MB/s 

Fastest vals were nr_reqs=128 and sync_thresh=50% of sync_window at 112.6 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 13 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  37 |    768  |   384  | 128  |   383  |  48.5 |   192  | 106.4 
   2  |  44 |    896  |   448  | 128  |   447  |  53.0 |   224  | 103.1 
   3  |  50 |   1024  |   512  | 128  |   511  |  59.9 |   256  | 102.2 
   4  |  56 |   1152  |   576  | 128  |   575  |  64.2 |   288  | 102.3 
   5  |  63 |   1280  |   640  | 128  |   639  |  70.4 |   320  |  98.7 
   6  |  69 |   1408  |   704  | 128  |   703  |  74.4 |   352  |  98.8 
   7  |  75 |   1536  |   768  | 128  |   767  |  77.4 |   384  |  99.5 
   8  |  82 |   1664  |   832  | 128  |   831  |  81.3 |   416  | 100.9 
   9  |  88 |   1792  |   896  | 128  |   895  |  84.6 |   448  | 100.3 
  10  |  94 |   1920  |   960  | 128  |   959  |  87.7 |   480  | 102.8 
  11  | 101 |   2048  |  1024  | 128  |  1023  |  88.0 |   512  | 104.5 
  12  | 107 |   2176  |  1088  | 128  |  1087  |  92.8 |   544  | 105.7 
  13  | 113 |   2304  |  1152  | 128  |  1151  |  94.4 |   576  | 105.1 

--- FULLY AUTOMATIC TEST PASS 1b (Rough - 5 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |   6 |    128  |    64  | 128  |    63  |  12.0 |    32  | 111.7 
   2  |  12 |    256  |   128  | 128  |   127  |  21.7 |    64  | 115.9 
   3  |  18 |    384  |   192  | 128  |   191  |  29.2 |    96  | 114.6 
   4  |  25 |    512  |   256  | 128  |   255  |  36.2 |   128  | 107.5 
   5  |  31 |    640  |   320  | 128  |   319  |  43.8 |   160  | 107.7 

--- Targeting Fastest Result of md_sync_window 128 bytes for Final Pass ---


--- FULLY AUTOMATIC nr_requests TEST 2 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |      256    |     128     |     128     |      127    |  21.3 MB/s 
  2   |      256    |     128     |     128     |       64    | 113.8 MB/s 
  3   |      256    |     128     |       8     |      127    |  66.8 MB/s 
  4   |      256    |     128     |       8     |       64    | 116.1 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=50% of sync_window at 116.1 MB/s

This nr_requests value will be used for the next test.



--- FULLY AUTOMATIC TEST PASS 2 (Fine - 33 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |   0 |     16  |    8  |   8  |     7  |   6.5 |     4  |  30.5 
   2  |   1 |     32  |    16  |   8  |    15  |  12.2 |     8  |  60.4 
   3  |   2 |     48  |    24  |   8  |    23  |  17.0 |    12  |  75.2 
   4  |   3 |     64  |    32  |   8  |    31  |  22.7 |    16  |  85.6 
   5  |   3 |     80  |    40  |   8  |    39  |  27.8 |    20  |  94.0 
   6  |   4 |     96  |    48  |   8  |    47  |  31.3 |    24  |  91.7 
   7  |   5 |    112  |    56  |   8  |    55  |  35.7 |    28  |  99.6 
   8  |   6 |    128  |    64  |   8  |    63  |  37.2 |    32  | 101.0 
   9  |   7 |    144  |    72  |   8  |    71  |  40.3 |    36  | 101.5 
  10  |   7 |    160  |    80  |   8  |    79  |  41.6 |    40  |  97.6 
  11  |   8 |    176  |    88  |   8  |    87  |  44.6 |    44  |  95.9 
  12  |   9 |    192  |    96  |   8  |    95  |  48.6 |    48  | 103.3 
  13  |  10 |    208  |   104  |   8  |   103  |  50.6 |    52  | 104.2 
  14  |  11 |    224  |   112  |   8  |   111  |  53.7 |    56  |  99.0 
  15  |  11 |    240  |   120  |   8  |   119  |  56.7 |    60  | 106.3 
  16  |  12 |    256  |   128  |   8  |   127  |  59.2 |    64  | 100.0 
  17  |  13 |    272  |   136  |   8  |   135  |  61.4 |    68  |  94.7 
  18  |  14 |    288  |   144  |   8  |   143  |  62.0 |    72  |  93.8 
  19  |  15 |    304  |   152  |   8  |   151  |  64.5 |    76  |  99.7 
  20  |  15 |    320  |   160  |   8  |   159  |  68.1 |    80  |  98.8 
  21  |  16 |    336  |   168  |   8  |   167  |  68.9 |    84  | 100.8 
  22  |  17 |    352  |   176  |   8  |   175  |  70.8 |    88  | 100.0 
  23  |  18 |    368  |   184  |   8  |   183  |  73.6 |    92  | 103.6 
  24  |  18 |    384  |   192  |   8  |   191  |  77.6 |    96  | 100.2 
  25  |  19 |    400  |   200  |   8  |   199  |  77.8 |   100  | 105.1 
  26  |  20 |    416  |   208  |   8  |   207  |  77.0 |   104  |  99.7 
  27  |  21 |    432  |   216  |   8  |   215  |  79.8 |   108  | 104.9 
  28  |  22 |    448  |   224  |   8  |   223  |  80.3 |   112  | 100.3 
  29  |  22 |    464  |   232  |   8  |   231  |  81.6 |   116  |  99.6 
  30  |  23 |    480  |   240  |   8  |   239  |  81.4 |   120  |  94.0 
  31  |  24 |    496  |   248  |   8  |   247  |  85.5 |   124  | 104.1 
  32  |  25 |    512  |   256  |   8  |   255  |  86.5 |   128  |  98.5 

The results below do NOT include the Basline test of current values.

The Fastest Sync Speed tested was md_sync_window=320 at 107.7 MB/s
     Tunable (md_num_stripes): 640
     Tunable (md_sync_window): 320
     Tunable (md_sync_thresh): 160
     Tunable (nr_requests): 8
This will consume 31 MB with md_num_stripes=640, 2x md_sync_window.
This is 32MB less than your current utilization of 63MB.

The Thriftiest Sync Speed tested was md_sync_window=96 at 103.3 MB/s
     Tunable (md_num_stripes): 192
     Tunable (md_sync_window): 96
     Tunable (md_sync_thresh): 48
     Tunable (nr_requests): 8
This will consume 9 MB with md_num_stripes=192, 2x md_sync_window.
This is 54MB less than your current utilization of 63MB.

The Recommended Sync Speed is md_sync_window=320 at 107.7 MB/s
     Tunable (md_num_stripes): 640
     Tunable (md_sync_window): 320
     Tunable (md_sync_thresh): 160
     Tunable (nr_requests): 8
This will consume 31 MB with md_num_stripes=640, 2x md_sync_window.
This is 32MB less than your current utilization of 63MB.

NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Completed: 11 Hrs 30 Min 46 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  Server_A
              unRAID version 6.2.0-rc4
                   md_num_stripes=1280
                   md_sync_window=384
                   md_sync_thresh=192
                   nr_requests=128 (Global Setting)
                   sbNumDisks=12
              CPU: AMD A8-6600K APU with Radeon(tm) HD Graphics
              RAM: 12GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path            Device       Class       Description
========================================================
/0/100/3/0          scsi5        storage     88SE9485 SAS/SATA 6Gb/s controller
/0/100/3/0/0.5.0    /dev/sdk     disk        4TB ST4000DM000-1F21
/0/100/3/0/0.6.0    /dev/sdl     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.7.0    /dev/sdm     disk        3TB ST3000DM001-9YN1
/0/100/3/0/0.0.0    /dev/sdf     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.1.0    /dev/sdg     disk        4TB ST4000DM000-1F21
/0/100/3/0/0.2.0    /dev/sdh     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.3.0    /dev/sdi     disk        3TB ST3000DM001-1CH1
/0/100/3/0/0.4.0    /dev/sdj     disk        3TB ST3000DM001-1ER1
/0/100/11                        storage     FCH SATA Controller [AHCI mode]
/0/1                scsi0        storage     
/0/1/0.0.0          /dev/sda     disk        7756MB DataTraveler SE9
/0/1/0.0.0/0        /dev/sda     disk        7756MB 
/0/2                scsi1        storage     
/0/2/0.0.0          /dev/sdb     disk        240GB Corsair Force LE
/0/3                scsi2        storage     
/0/3/0.0.0          /dev/sdc     disk        3TB ST3000DM001-9YN1
/0/4                scsi3        storage     
/0/4/0.0.0          /dev/sdd     disk        3TB WDC WD30EZRX-00S
/0/5                scsi4        storage     
/0/5/0.0.0          /dev/sde     disk        3TB WDC WD30EZRX-00M

Array Devices:
    Disk0 sdg is a Parity drive named parity
    Disk1 sdl is a Data drive named disk1
    Disk2 sdc is a Data drive named disk2
    Disk3 sde is a Data drive named disk3
    Disk4 sdf is a Data drive named disk4
    Disk5 sdh is a Data drive named disk5
    Disk6 sdk is a Data drive named disk6
    Disk7 sdi is a Data drive named disk7
    Disk8 sdm is a Data drive named disk8
    Disk9 sdj is a Data drive named disk9
    Disk10 sdd is a Data drive named disk10

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:       12182180     1059756    10481732      368392      640692    10418612
Low:       12182180     1700448    10481732
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

 

Link to comment

Server B

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 08:15:57 EDT 2016

                         Run on server: Server_B

                  Normal Automatic Parity Sync Test


Current Values:  md_num_stripes=1280, md_sync_window=384, md_sync_thresh=192
                 Global nr_requests=128
                    sdd nr_requests=128
                    sdf nr_requests=128
                    sdb nr_requests=128
                    sdc nr_requests=128
                    sde nr_requests=128
                    sdg nr_requests=128
                    sdl nr_requests=128
                    sdh nr_requests=128
                    sdi nr_requests=128
                    sdj nr_requests=128
                    sdk nr_requests=128
                    sdm nr_requests=128


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  68 |   1280  |   384  | 128  |   192  |  59.3 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  95.3 MB/s 
  2   |     1536    |     768     |     128     |      384    |  80.5 MB/s 
  3   |     1536    |     768     |       8     |      767    |  95.9 MB/s 
  4   |     1536    |     768     |       8     |      384    |  80.1 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 95.9 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 13 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  40 |    768  |   384  |   8  |   383  |  82.5 |   192  |  61.3 
   2  |  47 |    896  |   448  |   8  |   447  |  91.0 |   224  |  63.2 
   3  |  54 |   1024  |   512  |   8  |   511  |  90.2 |   256  |  66.0 
   4  |  61 |   1152  |   576  |   8  |   575  |  97.7 |   288  |  65.7 
   5  |  68 |   1280  |   640  |   8  |   639  |  98.8 |   320  |  67.0 
   6  |  75 |   1408  |   704  |   8  |   703  | 100.0 |   352  |  68.0 
   7  |  81 |   1536  |   768  |   8  |   767  |  98.3 |   384  |  69.6 
   8  |  88 |   1664  |   832  |   8  |   831  |  98.5 |   416  |  69.9 
   9  |  95 |   1792  |   896  |   8  |   895  |  99.6 |   448  |  70.8 
  10  | 102 |   1920  |   960  |   8  |   959  |  93.0 |   480  |  72.1 
  11  | 109 |   2048  |  1024  |   8  |  1023  |  96.1 |   512  |  72.9 
  12  | 115 |   2176  |  1088  |   8  |  1087  | 102.0 |   544  |  78.4 
  13  | 122 |   2304  |  1152  |   8  |  1151  | 101.4 |   576  |  81.7 

--- FULLY AUTOMATIC TEST PASS 1c (Rough - 17 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 129 |   2432  |  1216  |   8  |  1215  | 105.0 |   608  |  84.5 
   2  | 136 |   2560  |  1280  |   8  |  1279  | 102.3 |   640  |  98.7 
   3  | 143 |   2688  |  1344  |   8  |  1343  | 102.7 |   672  | 103.5 
   4  | 150 |   2816  |  1408  |   8  |  1407  | 103.0 |   704  | 102.2 
   5  | 156 |   2944  |  1472  |   8  |  1471  | 102.5 |   736  | 104.9 
   6  | 163 |   3072  |  1536  |   8  |  1535  | 103.2 |   768  | 100.6 
   7  | 170 |   3200  |  1600  |   8  |  1599  | 101.7 |   800  | 102.7 
   8  | 177 |   3328  |  1664  |   8  |  1663  | 102.5 |   832  | 105.0 
   9  | 184 |   3456  |  1728  |   8  |  1727  | 105.8 |   864  | 101.1 
  10  | 191 |   3584  |  1792  |   8  |  1791  | 100.3 |   896  | 103.7 
  11  | 197 |   3712  |  1856  |   8  |  1855  |  99.7 |   928  |  89.4 
  12  | 204 |   3840  |  1920  |   8  |  1919  |  91.0 |   960  |  95.8 
  13  | 211 |   3968  |  1984  |   8  |  1983  | 100.4 |   992  | 101.9 
  14  | 218 |   4096  |  2048  |   8  |  2047  |  99.0 |  1024  |  89.7 
  15  | 225 |   4224  |  2112  |   8  |  2111  |  90.4 |  1056  |  88.2 
  16  | 231 |   4352  |  2176  |   8  |  2175  |  90.1 |  1088  |  93.1 
  17  | 238 |   4480  |  2240  |   8  |  2239  |  95.9 |  1120  |  95.7 

--- Targeting Fastest Result of md_sync_window 1728 bytes for Final Pass ---


--- FULLY AUTOMATIC nr_requests TEST 2 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     3456    |    1728     |     128     |     1727    |  93.9 MB/s 
  2   |     3456    |    1728     |     128     |      864    |  81.0 MB/s 
  3   |     3456    |    1728     |       8     |     1727    |  75.2 MB/s 
  4   |     3456    |    1728     |       8     |      864    |  73.9 MB/s 

Fastest vals were nr_reqs=128 and sync_thresh=99% of sync_window at 93.9 MB/s

This nr_requests value will be used for the next test.



--- FULLY AUTOMATIC TEST PASS 2 (Fine - 33 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 170 |   3200  |  1600  | 128  |  1599  |  78.3 |   800  |  74.5 
   2  | 171 |   3216  |  1608  | 128  |  1607  |  79.2 |   804  |  83.6 
   3  | 172 |   3232  |  1616  | 128  |  1615  |  82.3 |   808  |  81.7 
   4  | 173 |   3248  |  1624  | 128  |  1623  |  77.0 |   812  |  80.3 
   5  | 173 |   3264  |  1632  | 128  |  1631  |  82.4 |   816  |  84.0 
   6  | 174 |   3280  |  1640  | 128  |  1639  |  81.4 |   820  |  88.6 
   7  | 175 |   3296  |  1648  | 128  |  1647  |  90.3 |   824  |  87.9 
   8  | 176 |   3312  |  1656  | 128  |  1655  |  88.6 |   828  |  84.7 
   9  | 177 |   3328  |  1664  | 128  |  1663  |  85.4 |   832  |  81.5 
  10  | 178 |   3344  |  1672  | 128  |  1671  |  84.7 |   836  |  86.3 
  11  | 179 |   3360  |  1680  | 128  |  1679  |  88.9 |   840  |  87.5 
  12  | 179 |   3376  |  1688  | 128  |  1687  |  90.3 |   844  |  92.6 
  13  | 180 |   3392  |  1696  | 128  |  1695  |  84.5 |   848  |  79.9 
  14  | 181 |   3408  |  1704  | 128  |  1703  |  77.5 |   852  |  77.2 
  15  | 182 |   3424  |  1712  | 128  |  1711  |  80.0 |   856  |  74.6 
  16  | 183 |   3440  |  1720  | 128  |  1719  |  79.2 |   860  |  82.5 
  17  | 184 |   3456  |  1728  | 128  |  1727  |  81.5 |   864  |  86.1 
  18  | 185 |   3472  |  1736  | 128  |  1735  |  70.1 |   868  |  77.3 
  19  | 185 |   3488  |  1744  | 128  |  1743  |  79.8 |   872  |  76.7 
  20  | 186 |   3504  |  1752  | 128  |  1751  |  76.3 |   876  |  80.9 
  21  | 187 |   3520  |  1760  | 128  |  1759  |  75.4 |   880  |  80.0 
  22  | 188 |   3536  |  1768  | 128  |  1767  |  91.5 |   884  |  77.6 
  23  | 189 |   3552  |  1776  | 128  |  1775  |  65.5 |   888  |  67.9 
  24  | 190 |   3568  |  1784  | 128  |  1783  |  73.6 |   892  |  73.0 
  25  | 191 |   3584  |  1792  | 128  |  1791  |  72.3 |   896  |  81.2 
  26  | 191 |   3600  |  1800  | 128  |  1799  |  89.8 |   900  |  94.0 
  27  | 192 |   3616  |  1808  | 128  |  1807  |  92.5 |   904  |  85.3 
  28  | 193 |   3632  |  1816  | 128  |  1815  |  81.9 |   908  |  75.4 
  29  | 194 |   3648  |  1824  | 128  |  1823  |  73.6 |   912  |  82.1 
  30  | 195 |   3664  |  1832  | 128  |  1831  |  80.1 |   916  |  79.1 
  31  | 196 |   3680  |  1840  | 128  |  1839  |  84.5 |   920  |  90.9 
  32  | 196 |   3696  |  1848  | 128  |  1847  |  79.2 |   924  |  80.0 
  33  | 197 |   3712  |  1856  | 128  |  1855  |  76.7 |   928  |  74.1 

The results below do NOT include the Basline test of current values.

The Fastest Sync Speed tested was md_sync_window=1216 at 105.0 MB/s
     Tunable (md_num_stripes): 2432
     Tunable (md_sync_window): 1216
     Tunable (md_sync_thresh): 1215
     Tunable (nr_requests): 128
This will consume 129 MB with md_num_stripes=2432, 2x md_sync_window.
This is 61MB more than your current utilization of 68MB.

The Thriftiest Sync Speed tested was md_sync_window=704 at 100.0 MB/s
     Tunable (md_num_stripes): 1408
     Tunable (md_sync_window): 704
     Tunable (md_sync_thresh): 703
     Tunable (nr_requests): 128
This will consume 75 MB with md_num_stripes=1408, 2x md_sync_window.
This is 7MB more than your current utilization of 68MB.

The Recommended Sync Speed is md_sync_window=1216 at 105.0 MB/s
     Tunable (md_num_stripes): 2432
     Tunable (md_sync_window): 1216
     Tunable (md_sync_thresh): 1215
     Tunable (nr_requests): 128
This will consume 129 MB with md_num_stripes=2432, 2x md_sync_window.
This is 61MB more than your current utilization of 68MB.

NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Completed: 12 Hrs 44 Min 2 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  Server_B
              unRAID version 6.2.0-rc4
                   md_num_stripes=1280
                   md_sync_window=384
                   md_sync_thresh=192
                   nr_requests=128 (Global Setting)
                   sbNumDisks=13
              CPU: AMD Sempron(tm) 3850 APU with Radeon(tm) R3
   91xx Config
              RAM: 4GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path              Device     Class       Description
========================================================
/0/100/2.1/0          scsi11     storage     SAS1068E PCI-Express Fusion-MPT SAS
/0/100/2.1/0/0.4.0    /dev/sdj   disk        1TB ST31000333AS
/0/100/2.1/0/0.5.0    /dev/sdk   disk        2TB Hitachi HDS72202
/0/100/2.1/0/0.6.0    /dev/sdl   disk        2TB WDC WD20EFRX-68E
/0/100/2.1/0/0.7.0    /dev/sdm   disk        1TB ST31000528AS
/0/100/2.1/0/0.0.0    /dev/sdf   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.1.0    /dev/sdg   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.2.0    /dev/sdh   disk        1TB Hitachi HDT72101
/0/100/2.1/0/0.3.0    /dev/sdi   disk        1TB ST31000524AS
/0/100/2.2/0                     storage     88SE9123 PCIe SATA 6.0 Gb/s controller
/0/100/11                        storage     FCH SATA Controller [AHCI mode]
/0/1                  scsi0      storage     
/0/1/0.0.0            /dev/sda   disk        31GB USB Flash Drive
/0/1/0.0.0/0          /dev/sda   disk        31GB 
/0/2                  scsi1      storage     
/0/2/0.0.0            /dev/sdb   disk        2TB ST2000DM001-1CH1
/0/2/0.0.0/0          /dev/sdb   disk        2TB 
/0/3                  scsi2      storage     
/0/3/0.0.0            /dev/sdc   disk        3TB ST3000DM001-1CH1
/0/3/0.0.0/0          /dev/sdc   disk        3TB 
/0/4                  scsi3      storage     
/0/4/0.0.0            /dev/sdd   disk        3TB ST3000DM001-1CH1
/0/5                  scsi4      storage     
/0/5/0.0.0            /dev/sde   disk        2TB ST2000DL003-9VT1
/0/6                  scsi10     storage     

Array Devices:
    Disk0 sdd is a Parity drive named parity
    Disk1 sdf is a Data drive named disk1
    Disk2 sdb is a Data drive named disk2
    Disk3 sdc is a Data drive named disk3
    Disk4 sde is a Data drive named disk4
    Disk5 sdg is a Data drive named disk5
    Disk6 sdl is a Data drive named disk6
    Disk7 sdh is a Data drive named disk7
    Disk8 sdi is a Data drive named disk8
    Disk9 sdj is a Data drive named disk9
    Disk10 sdk is a Data drive named disk10
    Disk11 sdm is a Data drive named disk11

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:        3961340      177772     3042704      649288      740864     2904704
Low:        3961340      918636     3042704
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

Link to comment

Server B

       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 08:15:57 EDT 2016

                         Run on server: Server_B

                  Normal Automatic Parity Sync Test


Current Values:  md_num_stripes=1280, md_sync_window=384, md_sync_thresh=192
                 Global nr_requests=128
                    sdd nr_requests=128
                    sdf nr_requests=128
                    sdb nr_requests=128
                    sdc nr_requests=128
                    sde nr_requests=128
                    sdg nr_requests=128
                    sdl nr_requests=128
                    sdh nr_requests=128
                    sdi nr_requests=128
                    sdj nr_requests=128
                    sdk nr_requests=128
                    sdm nr_requests=128


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  68 |   1280  |   384  | 128  |   192  |  59.3 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  95.3 MB/s 
  2   |     1536    |     768     |     128     |      384    |  80.5 MB/s 
  3   |     1536    |     768     |       8     |      767    |  95.9 MB/s 
  4   |     1536    |     768     |       8     |      384    |  80.1 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 95.9 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 13 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  40 |    768  |   384  |   8  |   383  |  82.5 |   192  |  61.3 
   2  |  47 |    896  |   448  |   8  |   447  |  91.0 |   224  |  63.2 
   3  |  54 |   1024  |   512  |   8  |   511  |  90.2 |   256  |  66.0 
   4  |  61 |   1152  |   576  |   8  |   575  |  97.7 |   288  |  65.7 
   5  |  68 |   1280  |   640  |   8  |   639  |  98.8 |   320  |  67.0 
   6  |  75 |   1408  |   704  |   8  |   703  | 100.0 |   352  |  68.0 
   7  |  81 |   1536  |   768  |   8  |   767  |  98.3 |   384  |  69.6 
   8  |  88 |   1664  |   832  |   8  |   831  |  98.5 |   416  |  69.9 
   9  |  95 |   1792  |   896  |   8  |   895  |  99.6 |   448  |  70.8 
  10  | 102 |   1920  |   960  |   8  |   959  |  93.0 |   480  |  72.1 
  11  | 109 |   2048  |  1024  |   8  |  1023  |  96.1 |   512  |  72.9 
  12  | 115 |   2176  |  1088  |   8  |  1087  | 102.0 |   544  |  78.4 
  13  | 122 |   2304  |  1152  |   8  |  1151  | 101.4 |   576  |  81.7 

--- FULLY AUTOMATIC TEST PASS 1c (Rough - 17 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 129 |   2432  |  1216  |   8  |  1215  | 105.0 |   608  |  84.5 
   2  | 136 |   2560  |  1280  |   8  |  1279  | 102.3 |   640  |  98.7 
   3  | 143 |   2688  |  1344  |   8  |  1343  | 102.7 |   672  | 103.5 
   4  | 150 |   2816  |  1408  |   8  |  1407  | 103.0 |   704  | 102.2 
   5  | 156 |   2944  |  1472  |   8  |  1471  | 102.5 |   736  | 104.9 
   6  | 163 |   3072  |  1536  |   8  |  1535  | 103.2 |   768  | 100.6 
   7  | 170 |   3200  |  1600  |   8  |  1599  | 101.7 |   800  | 102.7 
   8  | 177 |   3328  |  1664  |   8  |  1663  | 102.5 |   832  | 105.0 
   9  | 184 |   3456  |  1728  |   8  |  1727  | 105.8 |   864  | 101.1 
  10  | 191 |   3584  |  1792  |   8  |  1791  | 100.3 |   896  | 103.7 
  11  | 197 |   3712  |  1856  |   8  |  1855  |  99.7 |   928  |  89.4 
  12  | 204 |   3840  |  1920  |   8  |  1919  |  91.0 |   960  |  95.8 
  13  | 211 |   3968  |  1984  |   8  |  1983  | 100.4 |   992  | 101.9 
  14  | 218 |   4096  |  2048  |   8  |  2047  |  99.0 |  1024  |  89.7 
  15  | 225 |   4224  |  2112  |   8  |  2111  |  90.4 |  1056  |  88.2 
  16  | 231 |   4352  |  2176  |   8  |  2175  |  90.1 |  1088  |  93.1 
  17  | 238 |   4480  |  2240  |   8  |  2239  |  95.9 |  1120  |  95.7 

--- Targeting Fastest Result of md_sync_window 1728 bytes for Final Pass ---


--- FULLY AUTOMATIC nr_requests TEST 2 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     3456    |    1728     |     128     |     1727    |  93.9 MB/s 
  2   |     3456    |    1728     |     128     |      864    |  81.0 MB/s 
  3   |     3456    |    1728     |       8     |     1727    |  75.2 MB/s 
  4   |     3456    |    1728     |       8     |      864    |  73.9 MB/s 

Fastest vals were nr_reqs=128 and sync_thresh=99% of sync_window at 93.9 MB/s

This nr_requests value will be used for the next test.



--- FULLY AUTOMATIC TEST PASS 2 (Fine - 33 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  | 170 |   3200  |  1600  | 128  |  1599  |  78.3 |   800  |  74.5 
   2  | 171 |   3216  |  1608  | 128  |  1607  |  79.2 |   804  |  83.6 
   3  | 172 |   3232  |  1616  | 128  |  1615  |  82.3 |   808  |  81.7 
   4  | 173 |   3248  |  1624  | 128  |  1623  |  77.0 |   812  |  80.3 
   5  | 173 |   3264  |  1632  | 128  |  1631  |  82.4 |   816  |  84.0 
   6  | 174 |   3280  |  1640  | 128  |  1639  |  81.4 |   820  |  88.6 
   7  | 175 |   3296  |  1648  | 128  |  1647  |  90.3 |   824  |  87.9 
   8  | 176 |   3312  |  1656  | 128  |  1655  |  88.6 |   828  |  84.7 
   9  | 177 |   3328  |  1664  | 128  |  1663  |  85.4 |   832  |  81.5 
  10  | 178 |   3344  |  1672  | 128  |  1671  |  84.7 |   836  |  86.3 
  11  | 179 |   3360  |  1680  | 128  |  1679  |  88.9 |   840  |  87.5 
  12  | 179 |   3376  |  1688  | 128  |  1687  |  90.3 |   844  |  92.6 
  13  | 180 |   3392  |  1696  | 128  |  1695  |  84.5 |   848  |  79.9 
  14  | 181 |   3408  |  1704  | 128  |  1703  |  77.5 |   852  |  77.2 
  15  | 182 |   3424  |  1712  | 128  |  1711  |  80.0 |   856  |  74.6 
  16  | 183 |   3440  |  1720  | 128  |  1719  |  79.2 |   860  |  82.5 
  17  | 184 |   3456  |  1728  | 128  |  1727  |  81.5 |   864  |  86.1 
  18  | 185 |   3472  |  1736  | 128  |  1735  |  70.1 |   868  |  77.3 
  19  | 185 |   3488  |  1744  | 128  |  1743  |  79.8 |   872  |  76.7 
  20  | 186 |   3504  |  1752  | 128  |  1751  |  76.3 |   876  |  80.9 
  21  | 187 |   3520  |  1760  | 128  |  1759  |  75.4 |   880  |  80.0 
  22  | 188 |   3536  |  1768  | 128  |  1767  |  91.5 |   884  |  77.6 
  23  | 189 |   3552  |  1776  | 128  |  1775  |  65.5 |   888  |  67.9 
  24  | 190 |   3568  |  1784  | 128  |  1783  |  73.6 |   892  |  73.0 
  25  | 191 |   3584  |  1792  | 128  |  1791  |  72.3 |   896  |  81.2 
  26  | 191 |   3600  |  1800  | 128  |  1799  |  89.8 |   900  |  94.0 
  27  | 192 |   3616  |  1808  | 128  |  1807  |  92.5 |   904  |  85.3 
  28  | 193 |   3632  |  1816  | 128  |  1815  |  81.9 |   908  |  75.4 
  29  | 194 |   3648  |  1824  | 128  |  1823  |  73.6 |   912  |  82.1 
  30  | 195 |   3664  |  1832  | 128  |  1831  |  80.1 |   916  |  79.1 
  31  | 196 |   3680  |  1840  | 128  |  1839  |  84.5 |   920  |  90.9 
  32  | 196 |   3696  |  1848  | 128  |  1847  |  79.2 |   924  |  80.0 
  33  | 197 |   3712  |  1856  | 128  |  1855  |  76.7 |   928  |  74.1 

The results below do NOT include the Basline test of current values.

The Fastest Sync Speed tested was md_sync_window=1216 at 105.0 MB/s
     Tunable (md_num_stripes): 2432
     Tunable (md_sync_window): 1216
     Tunable (md_sync_thresh): 1215
     Tunable (nr_requests): 128
This will consume 129 MB with md_num_stripes=2432, 2x md_sync_window.
This is 61MB more than your current utilization of 68MB.

The Thriftiest Sync Speed tested was md_sync_window=704 at 100.0 MB/s
     Tunable (md_num_stripes): 1408
     Tunable (md_sync_window): 704
     Tunable (md_sync_thresh): 703
     Tunable (nr_requests): 128
This will consume 75 MB with md_num_stripes=1408, 2x md_sync_window.
This is 7MB more than your current utilization of 68MB.

The Recommended Sync Speed is md_sync_window=1216 at 105.0 MB/s
     Tunable (md_num_stripes): 2432
     Tunable (md_sync_window): 1216
     Tunable (md_sync_thresh): 1215
     Tunable (nr_requests): 128
This will consume 129 MB with md_num_stripes=2432, 2x md_sync_window.
This is 61MB more than your current utilization of 68MB.

NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Completed: 12 Hrs 44 Min 2 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  Server_B
              unRAID version 6.2.0-rc4
                   md_num_stripes=1280
                   md_sync_window=384
                   md_sync_thresh=192
                   nr_requests=128 (Global Setting)
                   sbNumDisks=13
              CPU: AMD Sempron(tm) 3850 APU with Radeon(tm) R3
   91xx Config
              RAM: 4GiB System Memory

Outputting lshw information for Drives and Controllers:

H/W path              Device     Class       Description
========================================================
/0/100/2.1/0          scsi11     storage     SAS1068E PCI-Express Fusion-MPT SAS
/0/100/2.1/0/0.4.0    /dev/sdj   disk        1TB ST31000333AS
/0/100/2.1/0/0.5.0    /dev/sdk   disk        2TB Hitachi HDS72202
/0/100/2.1/0/0.6.0    /dev/sdl   disk        2TB WDC WD20EFRX-68E
/0/100/2.1/0/0.7.0    /dev/sdm   disk        1TB ST31000528AS
/0/100/2.1/0/0.0.0    /dev/sdf   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.1.0    /dev/sdg   disk        2TB ST2000DM001-1CH1
/0/100/2.1/0/0.2.0    /dev/sdh   disk        1TB Hitachi HDT72101
/0/100/2.1/0/0.3.0    /dev/sdi   disk        1TB ST31000524AS
/0/100/2.2/0                     storage     88SE9123 PCIe SATA 6.0 Gb/s controller
/0/100/11                        storage     FCH SATA Controller [AHCI mode]
/0/1                  scsi0      storage     
/0/1/0.0.0            /dev/sda   disk        31GB USB Flash Drive
/0/1/0.0.0/0          /dev/sda   disk        31GB 
/0/2                  scsi1      storage     
/0/2/0.0.0            /dev/sdb   disk        2TB ST2000DM001-1CH1
/0/2/0.0.0/0          /dev/sdb   disk        2TB 
/0/3                  scsi2      storage     
/0/3/0.0.0            /dev/sdc   disk        3TB ST3000DM001-1CH1
/0/3/0.0.0/0          /dev/sdc   disk        3TB 
/0/4                  scsi3      storage     
/0/4/0.0.0            /dev/sdd   disk        3TB ST3000DM001-1CH1
/0/5                  scsi4      storage     
/0/5/0.0.0            /dev/sde   disk        2TB ST2000DL003-9VT1
/0/6                  scsi10     storage     

Array Devices:
    Disk0 sdd is a Parity drive named parity
    Disk1 sdf is a Data drive named disk1
    Disk2 sdb is a Data drive named disk2
    Disk3 sdc is a Data drive named disk3
    Disk4 sde is a Data drive named disk4
    Disk5 sdg is a Data drive named disk5
    Disk6 sdl is a Data drive named disk6
    Disk7 sdh is a Data drive named disk7
    Disk8 sdi is a Data drive named disk8
    Disk9 sdj is a Data drive named disk9
    Disk10 sdk is a Data drive named disk10
    Disk11 sdm is a Data drive named disk11

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:        3961340      177772     3042704      649288      740864     2904704
Low:        3961340      918636     3042704
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***

 

Thanks for posting your results.  I do have some concerns with what I'm seeing.

 

First is a lack of consistency (or accuracy) at lower run times.

 

These two nr_requests tests (10 minutes each):

 Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed
---------------------------------------------------------------------------
  3   |     1536    |     768     |       8     |      767    |  95.9 MB/s
  4   |     1536    |     768     |       8     |      384    |  80.1 MB/s 

 

Are the data test points in these 2 md_sync_window tests in Test 7 (5 minutes each)

 

 Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s
------------------------------------------------------------------------
   7  |  81 |   1536  |   768  |   8  |   767  |  98.3 |   384  |  69.6

 

The high md_sync_threst tests (at 767) were actually pretty good, but the 50% test (at 384) were different by > 10 MB/s.

 

The first pass is looking pretty good, I'm even okay with the fastest values it found:

 Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s
------------------------------------------------------------------------
   9  | 184 |   3456  |  1728  |   8  |  1727  | 105.8 |   864  | 101.1 

 

But then it retested those values in the nr_requests test (Pass 2), with very different results:

 Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed
---------------------------------------------------------------------------
  1   |     3456    |    1728     |     128     |     1727    |  93.9 MB/s
  2   |     3456    |    1728     |     128     |      864    |  81.0 MB/s
  3   |     3456    |    1728     |       8     |     1727    |  75.2 MB/s
  4   |     3456    |    1728     |       8     |      864    |  73.9 MB/s 

 

All of a sudden, the server is producing speeds > 25 MB/s slower!  This is a longer test (10 minutes), so perhaps we were getting falsely high values at 5 minutes?  I'm really not sure.  But it is from here things really go downhill.  Instead of using nr_requests=8, which it used from the first pass, it switches to 128 because that is now testing faster.  But the measured speeds in Pass 2 are all lower than what we were seeing on Pass 1.

 

This does reveal a small cosmetic bug in my code, because Pass 2 returns lower results, the fastest speed came from Pass 1.  While my Fastest/Thriftiest/Recommended report correctly found this, it incorrectly reported the nr_requests value to use with those values:

The Fastest Sync Speed tested was md_sync_window=1216 at 105.0 MB/s
     Tunable (md_num_stripes): 2432
     Tunable (md_sync_window): 1216  <--This was from Pass 1, not Pass 2 (good)
     Tunable (md_sync_thresh): 1215
     Tunable (nr_requests): 128      <--But Pass 1 was nr_requests=8, not 128 (bad)   

 

I charted the results.  Red/Blue are Pass 1, Green/Purple are Pass 2 with a lot of sample points in a small range.  I liked the semi-curve Pass 1 was producing, but when the nr_requests was switched to 128 for Pass 2, the results looked pretty bad:

 

D4aAkVKP-84-JEyi-snpic0VBHz6bzpbPg6TNk1O_OYu2Q8_v5X6VJlov1-aFL0AH1eJfK0ZAEJjhAzwwe12EhzY2mjJmSu55MOvOL1q2uKwMIxE_W3KKKh7Vvggt5R3ZuplSkt1iSmXbJ1RlR6c1lv9R0oV_gMQ0hUHDTOZUD3MvmpaY6LR6v2Z3tw_UzQgMtmyxx_nHSi0vomM-5iaiPRD0qSnx9qLog1vjKNCLcZ85mRi8llkqCDOYL9JKNgdvRXWwf-XNii2OtUyfS1W0PjyC4w_lUmtei0-Juat423TmHh6dncPwxAtOtoQDqcS1sV0qRRk3zUP-xw3MAWXHmux5BBgYui4fMDyVHQ6h60W5U-81WMPRYowk7uvbGjFF-zmMGXlOATJ4P3S_Z5JCOslUpf-Fkv6QH3_aJpB2jg3AqnL0v065sFKpwfqg_DSQtw-crO2DoPOAakK4hQbTocOITn2PLaBHT5cywC_pmRUJPVGMgN8HlUWvfl7VzUqVEBCPMXDrZf6lCIOHw7iIAs9D1bjWm3SrrCvMkn9LP9kobMKw5028D-hy364u1GGh-IAyHk-Y9RbvJkitPgcE1xQaCLMBms=w1432-h945-no

 

Thoughts?

Paul

 

Link to comment

Normal test finished on bare metal

 


       unRAID Tunables Tester v4.0b3 by Pauven (for unRAID v6.2)

        Tunables Report produced Fri Aug 26 22:16:52 CDT 2016

                         Run on server: nas

                  Normal Automatic Parity Sync Test


Current Values:  md_num_stripes=1040, md_sync_window=520, md_sync_thresh=260
                 Global nr_requests=8
                    sdq nr_requests=8
                    sdp nr_requests=8
                    sdo nr_requests=8
                    sdn nr_requests=8
                    sdj nr_requests=8
                    sdm nr_requests=8
                    sdl nr_requests=8
                    sdk nr_requests=8
                    sdi nr_requests=8
                    sdh nr_requests=8
                    sdf nr_requests=8
                    sde nr_requests=8
                    sdg nr_requests=8


--- INITIAL BASELINE TEST OF CURRENT VALUES (1 Sample Point @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s 
-------------------------------------------------------
   1  |  59 |   1040  |   520  |   8  |   260  | 146.2 

--- FULLY AUTOMATIC nr_requests TEST 1 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |     1536    |     768     |     128     |      767    |  62.0 MB/s 
  2   |     1536    |     768     |     128     |      384    |  79.2 MB/s 
  3   |     1536    |     768     |       8     |      767    |  79.6 MB/s 
  4   |     1536    |     768     |       8     |      384    | 105.8 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=50% of sync_window at 105.8 MB/s

This nr_requests value will be used for the next test.


--- FULLY AUTOMATIC TEST PASS 1a (Rough - 13 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  43 |    768  |   384  |   8  |   383  |  92.0 |   192  | 149.8 
   2  |  51 |    896  |   448  |   8  |   447  | 108.0 |   224  |  81.3 
   3  |  58 |   1024  |   512  |   8  |   511  |  77.1 |   256  | 133.1 
   4  |  65 |   1152  |   576  |   8  |   575  | 102.6 |   288  | 149.3 
   5  |  73 |   1280  |   640  |   8  |   639  |  82.1 |   320  |  82.9 
   6  |  80 |   1408  |   704  |   8  |   703  |  78.9 |   352  |  98.0 
   7  |  87 |   1536  |   768  |   8  |   767  | 131.3 |   384  | 125.8 
   8  |  95 |   1664  |   832  |   8  |   831  |  89.4 |   416  |  75.9 
   9  | 102 |   1792  |   896  |   8  |   895  |  75.8 |   448  | 102.8 
  10  | 109 |   1920  |   960  |   8  |   959  |  89.9 |   480  |  94.2 
  11  | 117 |   2048  |  1024  |   8  |  1023  |  93.8 |   512  |  79.8 
  12  | 124 |   2176  |  1088  |   8  |  1087  | 132.4 |   544  | 132.8 
  13  | 131 |   2304  |  1152  |   8  |  1151  | 125.5 |   576  | 108.0 

--- FULLY AUTOMATIC TEST PASS 1b (Rough - 5 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |   7 |    128  |    64  |   8  |    63  |  71.8 |    32  | 111.1 
   2  |  14 |    256  |   128  |   8  |   127  | 114.6 |    64  | 102.2 
   3  |  21 |    384  |   192  |   8  |   191  |  63.9 |    96  |  84.6 
   4  |  29 |    512  |   256  |   8  |   255  |  87.2 |   128  |  74.2 
   5  |  36 |    640  |   320  |   8  |   319  |  83.2 |   160  |  59.4 

--- Targeting Fastest Result of md_sync_window 384 bytes for Final Pass ---


--- FULLY AUTOMATIC nr_requests TEST 2 (4 Sample Points @ 10min Duration)---

Test | num_stripes | sync_window | nr_requests | sync_thresh |   Speed 
---------------------------------------------------------------------------
  1   |      768    |     384     |     128     |      383    |  65.8 MB/s 
  2   |      768    |     384     |     128     |      192    |  72.3 MB/s 
  3   |      768    |     384     |       8     |      383    |  69.4 MB/s 
  4   |      768    |     384     |       8     |      192    |  96.4 MB/s 

Fastest vals were nr_reqs=8 and sync_thresh=50% of sync_window at 96.4 MB/s

This nr_requests value will be used for the next test.



--- FULLY AUTOMATIC TEST PASS 2 (Fine - 33 Sample Points @ 5min Duration)---

Test | RAM | stripes | window | reqs | thresh |  MB/s | thresh |  MB/s 
------------------------------------------------------------------------
   1  |  29 |    512  |   256  |   8  |   255  | 104.5 |   128  | 134.5 
   2  |  30 |    528  |   264  |   8  |   263  |  72.6 |   132  |  69.4 
   3  |  31 |    544  |   272  |   8  |   271  |  97.5 |   136  | 121.8 
   4  |  32 |    560  |   280  |   8  |   279  |  99.8 |   140  |  84.3 
   5  |  32 |    576  |   288  |   8  |   287  |  57.6 |   144  |  54.9 
   6  |  33 |    592  |   296  |   8  |   295  |  72.1 |   148  |  36.6 
   7  |  34 |    608  |   304  |   8  |   303  |  60.2 |   152  |  27.2 
   8  |  35 |    624  |   312  |   8  |   311  |  59.4 |   156  |  27.3 
   9  |  36 |    640  |   320  |   8  |   319  |  54.9 |   160  |  27.6 
  10  |  37 |    656  |   328  |   8  |   327  |  42.5 |   164  |  28.8 
  11  |  38 |    672  |   336  |   8  |   335  |  34.8 |   168  |  24.4 
  12  |  39 |    688  |   344  |   8  |   343  |  35.2 |   172  |  35.2 
  13  |  40 |    704  |   352  |   8  |   351  |  62.9 |   176  |  32.9 
  14  |  41 |    720  |   360  |   8  |   359  |  63.6 |   180  |  36.3 
  15  |  42 |    736  |   368  |   8  |   367  |  71.6 |   184  |  66.2 
  16  |  43 |    752  |   376  |   8  |   375  |  76.5 |   188  |  35.0 
  17  |  43 |    768  |   384  |   8  |   383  |  52.9 |   192  |  30.8 
  18  |  44 |    784  |   392  |   8  |   391  |  62.3 |   196  |  33.5 
  19  |  45 |    800  |   400  |   8  |   399  |  73.2 |   200  |  38.5 
  20  |  46 |    816  |   408  |   8  |   407  |  62.1 |   204  |  56.6 
  21  |  47 |    832  |   416  |   8  |   415  |  75.6 |   208  |  55.7 
  22  |  48 |    848  |   424  |   8  |   423  |  58.2 |   212  |  46.9 
  23  |  49 |    864  |   432  |   8  |   431  |  78.1 |   216  |  54.9 
  24  |  50 |    880  |   440  |   8  |   439  |  67.2 |   220  |  49.0 
  25  |  51 |    896  |   448  |   8  |   447  |  62.3 |   224  |  58.9 
  26  |  52 |    912  |   456  |   8  |   455  |  77.5 |   228  | 146.1 
  27  |  53 |    928  |   464  |   8  |   463  |  97.8 |   232  |  81.0 
  28  |  54 |    944  |   472  |   8  |   471  |  67.4 |   236  |  56.9 
  29  |  54 |    960  |   480  |   8  |   479  |  83.1 |   240  |  54.4 
  30  |  55 |    976  |   488  |   8  |   487  |  66.9 |   244  |  38.1 
  31  |  56 |    992  |   496  |   8  |   495  |  64.0 |   248  |  58.5 
  32  |  57 |   1008  |   504  |   8  |   503  |  79.0 |   252  |  54.0 
  33  |  58 |   1024  |   512  |   8  |   511  |  74.8 |   256  |  69.2 

The results below do NOT include the Basline test of current values.

The Fastest Sync Speed tested was md_sync_window=576 at 149.3 MB/s
     Tunable (md_num_stripes): 1152
     Tunable (md_sync_window): 576
     Tunable (md_sync_thresh): 288
     Tunable (nr_requests): 8
This will consume 65 MB with md_num_stripes=1152, 2x md_sync_window.
This is 6MB more than your current utilization of 59MB.

The Thriftiest Sync Speed tested was md_sync_window=456 at 146.1 MB/s
     Tunable (md_num_stripes): 912
     Tunable (md_sync_window): 456
     Tunable (md_sync_thresh): 228
     Tunable (nr_requests): 8
This will consume 52 MB with md_num_stripes=912, 2x md_sync_window.
This is 7MB less than your current utilization of 59MB.

The Recommended Sync Speed is md_sync_window=576 at 149.3 MB/s
     Tunable (md_num_stripes): 1152
     Tunable (md_sync_window): 576
     Tunable (md_sync_thresh): 288
     Tunable (nr_requests): 8
This will consume 65 MB with md_num_stripes=1152, 2x md_sync_window.
This is 6MB more than your current utilization of 59MB.

NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Completed: 11 Hrs 22 Min 47 Sec.


NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.


System Info:  nas
              unRAID version 6.2.0-rc4
                   md_num_stripes=1040
                   md_sync_window=520
                   md_sync_thresh=260
                   nr_requests=8 (Global Setting)
                   sbNumDisks=14
              CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
      Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
              RAM: System Memory
         System Memory

Outputting lshw information for Drives and Controllers:

H/W path            Device     Class          Description
=========================================================
/0/100/1f.2                    storage        C600/X79 series chipset 6-Port SATA AHCI Controller
/0/2/0              scsi1      storage        SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/2/0/0.0.0        /dev/sdd   disk           512GB Crucial_CT512M55
/0/2/0/0.1.0        /dev/sde   disk           4TB Hitachi HDS72404
/0/2/0/0.2.0        /dev/sdf   disk           4TB HGST HDN724040AL
/0/2/0/0.3.0        /dev/sdg   disk           4TB HGST HDN724040AL
/0/2/0/0.4.0        /dev/sdh   disk           4TB Hitachi HDS72404
/0/2/0/0.5.0        /dev/sdi   disk           4TB HGST HDS724040AL
/0/3/0              scsi8      storage        SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
/0/3/0/0.1.0        /dev/sdk   disk           4TB Hitachi HDS72404
/0/3/0/0.2.0        /dev/sdl   disk           4TB HGST HDS724040AL
/0/3/0/0.3.0        /dev/sdm   disk           4TB HGST HDS724040AL
/0/3/0/0.4.0        /dev/sdn   disk           4TB HGST HDS724040AL
/0/3/0/0.5.0        /dev/sdo   disk           4TB HGST HDS724040AL
/0/3/0/0.6.0        /dev/sdp   disk           4TB HGST HDS724040AL
/0/3/0/0.7.0        /dev/sdq   disk           4TB Hitachi HDS72404
/0/3/0/0.0.0        /dev/sdj   disk           4TB HGST HDS724040AL
/0/68               scsi0      storage        
/0/68/0.0.0         /dev/sda   disk           15GB Reader     SD/MS
/0/68/0.0.0/0       /dev/sda   disk           15GB 
/0/68/0.0.1         /dev/sdb   disk           Reader  MicSD/M2
/0/68/0.0.1/0       /dev/sdb   disk           
/0/69               scsi2      storage        
/0/69/0.0.0         /dev/sdc   disk           60GB INTEL SSDSC2CT06

Array Devices:
    Disk0 sdq is a Parity drive named parity
    Disk1 sdp is a Data drive named disk1
    Disk2 sdo is a Data drive named disk2
    Disk3 sdn is a Data drive named disk3
    Disk4 sdj is a Data drive named disk4
    Disk5 sdm is a Data drive named disk5
    Disk6 sdl is a Data drive named disk6
    Disk7 sdk is a Data drive named disk7
    Disk8 sdi is a Data drive named disk8
    Disk9 sdh is a Data drive named disk9
    Disk10 sdf is a Data drive named disk10
    Disk11 sde is a Data drive named disk11
    Disk12 sdg is a Data drive named disk12

Outputting free low memory information...

              total        used        free      shared  buff/cache   available
Mem:      132071772     5972928   122539640      415652     3559204   124705276
Low:      132071772     9532132   122539640
High:             0           0           0
Swap:             0           0           0


                      *** END OF REPORT ***


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.