[Plugin] CA Auto Turbo Write Mode


Recommended Posts

CA Auto Turbo Write Mode

 

A simple plugin which will automatically enable or disable Turbo Write Mode (aka reconstruct write mode) based upon the number of drives which are currently spinning.

 

Turbo Write Mode can significantly increase the write speed to your protected array (my system it doubles it).  See This Post for a discussion about it.  Note that unRaid's setting of Automatic does not enable / disable turbo mode based upon the number of drives spinning.  unRaid's setting of "Automatic" simply disables turbo write mode.

 

CA Auto Turbo Mode also has a feature where you can enable or disable Turbo Mode on a set schedule.  For instance to always to have it enabled when mover runs, etc.

 

Full help text is available within the plugin

 

Install it via the Apps tab (Community Applications).  Either search for turbo or you can also find it within the Additional CA Modules section

 

Untitled_zpspuf0yf4n.thumb.PNG.b139c59835c54732b68167b5e6d665be.PNG

 

 

CA Auto Turbo Mode also has a feature where you can enable or disable Turbo Mode on a set schedule.  For instance to always to have it enabled when mover runs, etc.

Edited by Squid
  • Like 2
  • Upvote 7
Link to comment

Is there any merit to run the settings as a % opposed to how many drives are up and spinning?  6 Drives installed and 3 spinning is the same as 50%

Automatic really becomes automatic so we don't have to keep adjusting the amount of drives when we add or merge drives and let the system do what it does. 

 

Just some Rambling food for thought before I give it a go sometime. 

  • Like 1
Link to comment
19 minutes ago, kizer said:

Is there any merit to run the settings as a % opposed to how many drives are up and spinning?  6 Drives installed and 3 spinning is the same as 50%

Automatic really becomes automatic so we don't have to keep adjusting the amount of drives when we add or merge drives and let the system do what it does. 

lol  I would say no because its more work on my part vs something you can easily figure out for yourself.     B|

 

Now, if adding / removing drives was a very common occurrence then I would say OK.  

 

It is really nothing to add, but IMO the value is only ever going to get set to be 0-1-2 drives  anyways.  And if the write requires one of those 2 drives to be spun up to write, then it'll switch into turbo mode anyways.

 

As an aside, the default polling time is 300 seconds.  I've set it to be 1 sec during development, and no stuttering or anything happened on my system while playing back.

 

Further FYI, for accuracy, I directly issue the hdparm -C command rather than relying on what unRaid thinks is spinning because it has its own polling time.

 

Further testing notes:  While copying a 100g test file, I switched unRaid back and forth from turbo to normal (and vice versa) about 100 times and compared md5's of the source and destination.  unRaid handled no problems constant switching of the write mode and didn't corrupt the file at all.  :D

 

 

Edited by Squid
Link to comment
8 hours ago, Squid said:

As an aside, the default polling time is 300 seconds.  I've set it to be 1 sec during development, and no stuttering or anything happened on my system while playing back.

 

Further FYI, for accuracy, I directly issue the hdparm -C command rather than relying on what unRaid thinks is spinning because it has its own polling time.

 

Further testing notes:  While copying a 100g test file, I switched unRaid back and forth from turbo to normal (and vice versa) about 100 times and compared md5's of the source and destination.  unRaid handled no problems constant switching of the write mode and didn't corrupt the file at all.  :D

 

Really appreciate the testing.  It answers questions I had, as to whether the constant polling could impact performance.  Sounds well implemented.

Link to comment
  Really appreciate the testing.  It answers questions I had, as to whether the constant polling could impact performance.  Sounds well implemented.

 

Like anything else in the world ymmv though the real key thing to keep in mind is that if you have a situation such as docker apps running without a cache drive is that once the plugin switches to turbo mode it will always stay in turbo mode because writes will be constant and continuous.

 

This week's update will add in scheduling to force turbo on or off at certain times of the day...

 

Sent from my SM-T560NU using Tapatalk

 

 

 

 

 

Link to comment

EDIT: Added to the GUI built-in support for scheduling turbo mode on or off at certain times of the day

 

Rather than add in a GUI for scheduling of turbo on or off, (which would entail the need to support multiple / expandable schedules), here's a script for user.scripts which will enable / disable turbo mode, and stop / restart this plugin if it was running.  It will also handle overlapping schedules when running multiple versions of it via user.scripts and only restart the plugin after all scheduled tasks have completed.  There was also an update pumped out to the plugin to display when it is being overriden by this script.

 

We'll address a proper GUI for this instead of leveraging the user.scripts plugin depending upon demand....

 

Set whatever cron schedule you want for the script.  (Anything but array stop / start)

 

 

At the completion of the script, this plugin will either be restarted, or unRaid's settings for turbo mode will be reactivated if the plugin wasn't running.

#!/usr/bin/php
<?PHP
#description=Script to enable or disable CA auto turbo mode
#backgroundOnly=true

$enableTurboMode = true;              # set to false if you want the schedule to disable turbo mode
$duration = 60;                       # set to number of minutes to keep turbo mode enabled / disabled
                                      # ie: how long to disable the auto script for

exec("mkdir -p /tmp/ca.tmp/otherPIDs");                                    
$myPID = getmypid();
file_put_contents("/tmp/ca.tmp/otherPIDs/$myPID",$myPID);
                                    
if ( is_file("/tmp/ca.turbo/PID") ) {
  $scriptPID = file_get_contents("/tmp/ca.turbo/PID");
  posix_kill($scriptPID,SIGKILL);    # kill the running auto script
  unlink("/tmp/ca.turbo/PID");
  $runningFlag = true;
}

$mode = $enableTurboMode ? "1" : "0";
$status['spundown'] = "Unknown";
$status['mode'] = $enableTurboMode ? "turbo" : "normal";
$status['override'] = true;

file_put_contents("/tmp/ca.turbo/status.json",json_encode($status, JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT));

exec("/usr/local/sbin/mdcmd set md_write_method $mode");

sleep($duration * 60);
unlink("/tmp/ca.tmp/otherPIDs/$myPID");


# Check if any other overlapping schedules happen to still be running.
# Don't restart ca auto turbo if a schedule is still active
# If no other active schedules, either restart CA Auto Turbo or restore unRaid's settings

$dirContents = array_diff(@scandir("/tmp/ca.tmp/otherPIDs/"),array(".",".."));

if (empty($dirContents) ) {
  unlink("/tmp/ca.turbo/status.json");
  if ( $runningFlag ) {
    $descriptorspec = array(
      0 => array("pipe", "r"),
      1 => array("pipe", "w"),
      2 => array("file", "/tmp/error-output.txt", "a")
    );
    proc_open("/usr/local/emhttp/plugins/ca.turbo/scripts/auto_turbo.php",$descriptorspec,$pipes);
  } else {
    $unRaidSettings = parse_ini_file("/boot/config/disk.cfg");
    echo "Restoring unRaid settings";
    exec("/usr/local/sbin/mdcmd set md_write_method {$unRaidSettings['md_write_method']}");
  }
}

?>

 

Edited by Squid
  • Upvote 1
Link to comment
2 minutes ago, Squid said:

At least the script is done...  I just hate HTML....  I don't suppose you'd just let me package user.scripts with this?  ;-)

absolutely not! :) seriously though i personally dont mind, a script is fine but a ui would be finer :-).

Link to comment
3 hours ago, yitzi said:

Is there a suggested number for "Disks Allowed To Be Spun Down Before Invoking Turbo Mode:"? I have 3 Cache SSD's Disks, 6 Data HDD and 1 Parity HDD. Current spindown time is set to 30 minutes

 

Thanks.

Set it to however many disks you want to spin up when a write happens (turbo mode all disks have to be spinning)...  Really up to you and no recommended setting

 

Link to comment

Hi @Squid

 

There seems to be a bug with the debug logging in the CA Turbo Write script.

 

If I enable Automatic Turbo Mode with Additional Debugging Logging, I get this in my log and the script crashes after the first drive:

Mar 23 19:23:12 Tower root: Starting Auto Turbo
Mar 23 19:23:12 Tower root: #012/dev/sdi:#012 drive state is: active/idle

But if I comment out this line in the logger function of ca.turbo/include/helpers.php:

  if ($debug) {
    # echo "$string\n";
  }

then the script runs to completion:

Mar 23 19:23:44 Tower root: Starting Auto Turbo
Mar 23 19:23:44 Tower root: #012/dev/sdi:#012 drive state is: active/idle
Mar 23 19:23:44 Tower root: #012/dev/sdj:#012 drive state is: active/idle
Mar 23 19:23:44 Tower root: #012/dev/sdk:#012 drive state is: standby
Mar 23 19:23:44 Tower root: Total Spundown: 1
Mar 23 19:23:44 Tower root: Entering Normal Mode
Mar 23 19:23:44 Tower kernel: mdcmd (409): set md_write_method 0

I have seen this on two systems, both running 6.3.2

Link to comment

wnd_cab754844c1460473a61e8fc91eb67e1.jpg

 

Added in schedules.

 

  • Schedules are set via a standard cron entry to force on or off turbo mode as requested.
  • Turbo / Normal mode will stay set until a set time frame has expired, after which either Auto Turbo mode will be re-enabled (or unRaid's settings will take over if auto turbo is not enabled)
  • In the event of an overlapping schedule, a warning will be logged, and the later schedule will take precedence

 

* if you were using the script a few posts up to implement scheduling via user.scripts, then discontinue utilizing it -> changes have been made

 

@binhex - Added scheduling not because I particularly see a need for it (but you may, who knows), but needed to work out how to do unlimited multiple schedules for another idea rolling around in my head)

Edited by Squid
  • Upvote 1
Link to comment

On two occasions of shutting down my main server and test server I've gotten a message on the console.

stopping_svcs ca.turbo line 14 unknown...

I didn't get the message exactly right because it flew by so fast, but you get the idea.  There wasn't anything in the log that indicated any issue.

 

I don't think it's necessarily a big deal, but might be an indication of a gremlin in hiding.

Link to comment
13 minutes ago, dlandon said:

On two occasions of shutting down my main server and test server I've gotten a message on the console.


stopping_svcs ca.turbo line 14 unknown...

I didn't get the message exactly right because it flew by so fast, but you get the idea.  There wasn't anything in the log that indicated any issue.

 

I don't think it's necessarily a big deal, but might be an indication of a gremlin in hiding.

It'll be a harmless message.  Assuming you got the line number correct, then its a message from the kill of the PID.

 

I cannot replicate it at all, but there are safe guards in place to ensure that only a single copy of the script ever gets executed at a time in case something somewhere does get confused.

Link to comment

This specific plugin does not display the grey/green "Done" button at the bottom of the popup window after it updates. Doesn't seem to effect anything, and all my other plugin updates have the done button, so maybe it's a small syntax thing? I don't think it's my machine, it does it on 2 different servers.

Link to comment
17 minutes ago, jonathanm said:

This specific plugin does not display the grey/green "Done" button at the bottom of the popup window after it updates. Doesn't seem to effect anything, and all my other plugin updates have the done button, so maybe it's a small syntax thing? I don't think it's my machine, it does it on 2 different servers.

Minor side effect of the fact that the installation routine is done in php instead of bash for various reasons.   It doesn't affect anything and when it says plugin installed you can close the window.   It is on my to do list but not at the top of my priorities

Link to comment

Very cool! A lot of people will really benefit from this!!

 

But, thought I would explain some potential usage pitfalls.

 

This feature does not just take advantage of time other drives happen to be spun up, by its very nature time drives are spinning up will elongated, and in some cases, drives will be prevented from spinning down. Basically the spindown timer countdown on all drives will be reset after any write to the array. And until the array is quiet (no writes) long enough for drives to spindown, turbo write is going to stay on.

 

If you spin up your array - within 5 minutes (with default settings above) turbo write mode will be enabled. You would not know how long you have to wait until turbo kicked in (1 second to 5 minutes). If a transfer were in progress and you said - hey, it'd be nice if turbo were on, you could spin up the array and in a few minutes it would engage. If you want instant, though, this would not give you instant. A shorter interval would help, but there may be negative implications to checking it very frequently.

 

And if you manually spin down at least one disk (assuming disks allowed to be spun down is 0), turbo write mode is disabled within that same time interval. This would not be foolproof. If you spin down some disk(s), and then before the interval, a new array write occurs, the disk(s) would spin up again (since turbo write is still enabled) and the write is performed, and turbo write would not be turned off. As mentioned, having a docker image on an array disk would all but guarantee your array would never get out of turbo-write mode with this plug in, even if you are manually spinning down disks. You would not want to create a situation where your disks spinning up and down frequently for no useful purpose.

 

After a parity check, fresh boot, or if there are frequent writes to the array, array drives may never spin down unless there is manual intervention. Using the feature to force turbo write off (probably late evening and after mover runs) would help. But using this plugin will result in more drives spinning more of the time (not just during faster writes). Those running power efficient arrays might find their electric bill inching north.

 

A person might think a shorter spindown interval might be in order so that turbo write would get disabled "naturally" more of the time. Problem is, shorter spindown intervals will result in drives going through multiple thermal cycles (through your normal usage pattern), which has been identified as a bad thing (maybe the worst thing) for drive health. I have my spindown interval set to 5 hours which largely prevents multiple spinups per day. Balancing settings to allow drives to spin down to shut off turbo write vs setting to reduce frequent spinup/spindown cycles would be tricky.

 

Since my experience is that normal writes are fast enough most of the time, and I'd prefer drives to stay spin down most of the time, I'd prefer to just have an icon on my desktop that turns turbo on for a time interval. That way, if I am waiting for a write to finish, I can quickly turn on turbo write and it will turn off automatically. User scripts with remote SSH execution could be used to set this up. But that is me - I think this plugin would be useful for many unRAID users, and hoping the info above might help get it configured optimally.

  • Upvote 1
Link to comment
2 minutes ago, bjp999 said:

Very cool! A lot of people will really benefit from this!!

 

But, thought I would explain some potential usage pitfalls.

 

This feature does not just take advantage of time other drives happen to be spun up, by its very nature it will elongated and in some cases prevent, drives from spinning down. Basically the spindown timer countdown on all drives will be reset after any write to the array. And until the array is quiet (no writes) long enough for drives to spindown, turbo write is going to stay on.

 

If you spin up your array - within 5 minutes (with default settings above) turbo write mode will be enabled. You would not know how long you have to wait until turbo kicked in (1 second to 5 minutes). If a transfer were in progress and you said - hey, it'd be nice if turbo were on, you could spin up the array and in a few minutes it would engage. If you want instant, though, this would not give you instant. A shorter interval would help, but there may be negative implications to checking it very frequently.

 

And if you manually spin down at least one disk (assuming disks allowed to be spun down is 0), turbo write mode is disabled within that same time interval. This would not be foolproof. If you spin down some disk(s), and then before the interval, a new array write occurs, the disk(s) would spin up again (since turbo write is still enabled) and the write is performed, and turbo write would not be turned off. As mentioned, having a docker image on an array disk would all but guarantee your array would never get out of turbo-write mode with this plug in, even if you are manually spinning down disks. You would not want to create a situation where your disks spinning up and down frequently for no useful purpose.

 

After a parity check, fresh boot, or if there are frequent writes to the array, array drives may never spin down unless there is manual intervention. Using the feature to force turbo write off (probably late evening and after mover runs) would help. But using this plugin will result in more drives spinning more of the time (not just during faster writes). Those running power efficient arrays might find their electric bill inching north.

 

A person might think a shorter spindown interval might be in order so that turbo write would get disabled "naturally" more of the time. Problem is, shorter spindown intervals will result in drives going through multiple thermal cycles (through your normal usage pattern), which has been identified as a bad thing (maybe the worst thing) for drive health. I have my spindown interval set to 5 hours which largely prevents multiple spinups per day. Balancing settings to allow drives to spin down to shut off turbo write vs setting to reduce frequent spinup/spindown cycles would be tricky.

 

Since my experience is that normal writes are fast enough most of the time, and I'd prefer drives to stay spin down most of the time, I'd prefer to just have an icon on my desktop that turns turbo on for a time interval. That way, if I am waiting for a write to finish, I can quickly turn on turbo write and it will turn off automatically. User scripts with remote SSH execution could be used to set this up. But that is me - I think this plugin would be useful for many unRAID users, and hoping the info above might help get it configured optimally.

All your points are valid. 

 

unRaid's UI isn't particularly reliable at notifying when drives are up or down, hence the need for separate polling.  On my systems at home, I have it set to every 30 seconds.

 

Absolutely correct that if constant writes happen to the array, then at some point, turbo mode will kick in, and it will not let down until the writes cease long enough for the drives to spin down manually.  You can't manually spin down a drive in order to cancel turbo mode in the middle of writes because unRaid will wind up just spinning it back up again.

 

In my use case, the only setting for number of disks spun down is zero.  If they are all up, then I might as well enable turbo.  But, writes to my system are all handled automatically, and worst case scenario is a 10-15 gigs is written every hour or so in a single batch.  Net result is that pretty much any time I look at my system to see how it is, all drives or most drives are indeed spun down.  

 

But, if you have say VMs or Docker applications and no cache drive that they are stored on, then its a given that once turbo kicks in, it will stay kicked in due to the constant writes by either VM / Docker to the appdata.  Cache-enabled shares won't affect this.  (I don't have any cache enabled shares)

 

Even if LT should implement a true Auto mode at the md driver level, every single one of your points is still valid.  There is no perfect world unfortunately.  But the plugin does fill a whole in the unRaid eco-system...

 

YMMV

  • Like 1
Link to comment

After thought:

 

No matter what you do, you should always set the polling time to be significantly less than the spindown time for the disks (ie: my default of 5 minutes vs the default spindown of 15 minutes).  This will let you take advantage of when drives do get spundown and avoid spinning them back up again.  You have to keep in mind that even with turbo mode enabled, the disks may not spin down concurrently due to reads taking place.

 

Another thing to keep in mind with relation to all of this is how you set your spin up groups should you utilize them.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.