RobJ

Tips and Tweaks page

Recommended Posts

This thread is for feedback for a new Tips and Tweaks page on the wiki.  Feedback includes ideas, suggestions, criticism, questions, requests, discussion...

 

It's my first stab at collecting a bunch of tips, tricks, and tweaks for unRAID, to help make it work right, fix known issues, and make it work better, safer, and possibly faster.  I started by wanting a single place for users to find ideas and fixes for the current bugs with recent 6.1 and 6.2 releases.

 

Obviously, it needs to be dynamic, and change as often as necessary.  I've started with a small collection, not really knowing which are good and useful, which are suspect, which are bogus, and which need correction.  But I'm hoping the great collection of helpers here will refine it, either directly by editing the wiki page or indirectly by suggesting the changes here in this thread.  I hope that some will be removed in time, as they are incorporated into new unRAID releases.

 

Hopefully, this may also help LimeTech, as they monitor what works and what doesn't work.  When we point users with relevant issues to the page, we should request their feedback, as to what helped and what didn't.  After a short review period, I will post about it in the latest beta announcement thread.

Share this post


Link to post
Share on other sites

Very nice!  This collects the ideas that are spread around the forum that can be hard to find into one place.

Share this post


Link to post
Share on other sites

Here's one to add. Need lots of RAM. Really shines when you don't have a cache drive to speed up writes.

 

add to Go file. The number is a percentage of your RAM.

 

# set size of write cache to RAM

sysctl vm.dirty_ratio=75

Share this post


Link to post
Share on other sites

Here's one to add. Need lots of RAM. Really shines when you don't have a cache drive to speed up writes.

 

add to Go file. The number is a percentage of your RAM.

 

# set size of write cache to RAM

sysctl vm.dirty_ratio=75

 

This is a good tip, I use it myself, just a small correction, the number is a percent of your free RAM, i usually set it at 99.

 

Edit to add: This tweak should only be used on a server with a UPS.

Share this post


Link to post
Share on other sites

I've been reading up on write caching, and the various settings that affect it, and it's a complex topic!  There are 2 schools of thought, go larger and go smaller, and the prevalent one seems to be to go smaller.  The current settings were set up when memory sizes were smaller, 512MB or less, so the ratios are already allowing much larger memory amounts for write caching.  That initially  seems like a good idea and why some feel you might as well make them even larger, but there's a problem.  At some point when it maxes out the allotted space or expires (30 seconds I believe), the system *has* to flush that data out, and it takes MUCH longer to flush say 20GB than 20MB, causing enormous latencies because the flushing becomes the highest priority.

 

That's why many feel it's better to go smaller and force constant small writes, creating consistent streaming, both in the general writing and for consistent execution of other processes.  Some suggest using the byte forms instead of the ratio forms, for tighter control of write cache space.  E.g. vm.dirty_bytes and vm.dirty_background_bytes instead of vm.dirty_ratio and vm.dirty_background_ratio

 

If all you do is handle small files, then going huge is not as much of a problem.  But usually there comes a time when you want to transfer 100GB or more, something more than your write caching amount.  At that point, it's my understanding that the kernel says "whoa" to everything else, and concentrates on draining the write cache, resulting in a nearly unusable keyboard and mouse, unresponsive programs, broken media streams, etc.

 

This is all 'book larnin', what I got from Googling.  You guys have the real world experience.  What do you think?  Have you tested the various scenarios (high and low ratios, with high and low write amounts, with simultaneous streaming, etc)?

 

Some relevant reading -

* Better Linux Disk Caching - a fairly balanced intro to the controlling factors and settings

* The pernicious USB-stick stall problem

* http://unix.stackexchange.com/questions/107703/why-is-my-pc-freezing-while-im-copying-a-file-to-a-pendrive?lq=1

* http://wiki.archlinux.org/index.php/sysctl#Virtual_memory - and "Small periodic system freezes" farther down

* Toward less-annoying background writeback - an interesting article about future plans for smarter write caching, self-tuning; shows how complex this problem is

 

In addition, the USB articles above may have a fix for the system problems some have had with USB "high speed resets".  More reading -

* Linux hang on USB writes

* Change value of USB "max_sectors"

If someone with experience in udev rule making in unRAID can create a procedure from this, that would be another good tip for the page.

Share this post


Link to post
Share on other sites

Ooooh, ooooh, ooooh, I feel a plugin coming!  The programmer in me is restless right now!

 

I will put together a plugin to allow setting the tweaks in the gui once we gather enough information on what the tweaks are, what they do, and suggestions on how they should be set.

 

LT is highly discouraging any modifications to the 'go' file and a plugin could manage the settings outside the 'go' file.

 

I will create some help text to help with making the settings, what can and should be done, etc.

 

I'm open to other 'tweaks' and adjustments that make sense for this plugin.  Any suggestions for the name of the plugin?  'tips.and.tweaks'.

Share this post


Link to post
Share on other sites

Ooooh, ooooh, ooooh, I feel a plugin coming!  The programmer in me is restless right now!

 

I will put together a plugin to allow setting the tweaks in the gui once we gather enough information on what the tweaks are, what they do, and suggestions on how they should be set.

 

LT is highly discouraging any modifications to the 'go' file and a plugin could manage the settings outside the 'go' file.

 

I will create some help text to help with making the settings, what can and should be done, etc.

 

I'm open to other 'tweaks' and adjustments that make sense for this plugin.  Any suggestions for the name of the plugin?  'tips.and.tweaks'.

 

I would love to get stuff out of my Go. I think this one (support for my Areca RAID card) probably won't meet plug-in criteria since it needs to install before emhttp is launched.

 

cp /boot/60-persistent-storage.rules /lib/udev/rules.d

udevadm control --reload-rules

udevadm trigger

sleep 5

 

Would be even better if LT would add this to 6.2 :)

Share this post


Link to post
Share on other sites

Ooooh, ooooh, ooooh, I feel a plugin coming!  The programmer in me is restless right now!

 

I will put together a plugin to allow setting the tweaks in the gui once we gather enough information on what the tweaks are, what they do, and suggestions on how they should be set.

 

LT is highly discouraging any modifications to the 'go' file and a plugin could manage the settings outside the 'go' file.

 

I will create some help text to help with making the settings, what can and should be done, etc.

 

I'm open to other 'tweaks' and adjustments that make sense for this plugin.  Any suggestions for the name of the plugin?  'tips.and.tweaks'.

 

I would love to get stuff out of my Go. I think this one (support for my Areca RAID card) probably won't meet plug-in criteria since it needs to install before emhttp is launched.

 

cp /boot/60-persistent-storage.rules /lib/udev/rules.d

udevadm control --reload-rules

udevadm trigger

sleep 5

 

Would be even better if LT would add this to 6.2 :)

 

What we could do here is to set up a script dialog where you could enter custom commands in a script to be run when unraid is booted.  I would probably set up the script where the script would only run on a reboot on an event before emhttp is run.  i.e. the 'go' file is executed.

 

Other tweaks would be applied whenever 'Apply' is hit in the gui and when unraid is started.

 

I will do some research on the events, but I believe there is a services started, or something along those lines that is run before emhttp.

 

Share this post


Link to post
Share on other sites

 

cp /boot/60-persistent-storage.rules /lib/udev/rules.d

udevadm control --reload-rules

udevadm trigger

sleep 5

 

 

What is the content of that file?

 

udev is also ran by unRAID upon startup and it  should find the relevant devices.

 

Share this post


Link to post
Share on other sites

 

cp /boot/60-persistent-storage.rules /lib/udev/rules.d

udevadm control --reload-rules

udevadm trigger

sleep 5

 

 

What is the content of that file?

 

udev is also ran by unRAID upon startup and it  should find the relevant devices.

 

Here is the thread out of which all of this came. :)

 

http://lime-technology.com/forum/index.php?topic=38487.0

 

That persistent-rules file is based on unRAID v5, was it ever tried to run on unRAID v6 without copying this file?

 

FYI the rules file in v6 has been updated to support new NVMe cards as well, overwriting it with an 'old' file will loose such support.

 

 

Share this post


Link to post
Share on other sites

I've created a plugin to help in making these settings with some explanations to help with making settings.  The plugin can be found in Community Applications or install by inserting the following in the install plugin.

 

https://github.com/dlandon/tips.and.tweaks/raw/master/tips.and.tweaks.plg

Share this post


Link to post
Share on other sites

I've created a plugin to help in making these settings with some explanations to help with making settings.  The plugin can be found in Community Applications or install by inserting the following in the install plugin.

 

https://github.com/dlandon/tips.and.tweaks/raw/master/tips.and.tweaks.plg

 

getting an error about URL trying to install from CA:

plugin: installing: https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg
plugin: downloading https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg
plugin: downloading: https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg ... failed (Invalid URL / Server error response)
plugin: wget: https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg download failure (Invalid URL / Server error response)

Share this post


Link to post
Share on other sites

I've created a plugin to help in making these settings with some explanations to help with making settings.  The plugin can be found in Community Applications or install by inserting the following in the install plugin.

 

https://github.com/dlandon/tips.and.tweaks/raw/master/tips.and.tweaks.plg

 

getting an error about URL trying to install from CA:

plugin: installing: https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg
plugin: downloading https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg
plugin: downloading: https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg ... failed (Invalid URL / Server error response)
plugin: wget: https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg download failure (Invalid URL / Server error response)

Error in the template:

 

plugin: installing: https://github.com/dlandon/unassigned.devices/raw/master/tips.and.tweaks.plg

 

Works installing it manually

Share this post


Link to post
Share on other sites

Sorry, typo in the link.  Fixed now.

Won't get fixed within CA until the next appfeed scan (within 2 hours)

Share this post


Link to post
Share on other sites

Sorry, typo in the link.  Fixed now.

 

Great job!! :) Here to help with the testing phase.

 

Errors on console when plugin is run. I removed my disk RAM cache lines from go and the plug-in tells me it set them successfully :) One thing I noticed (may be error related) is the Nic buffers didn't seem to take.

 

Cannot get device pause settings: Operation not supported
/usr/local/emhttp/plugins/tips.and.tweaks/scripts/rc.tweaks : Line 46: [:=:] unary operator expected
Cannot get device ring settings: Operation not supported
Cannot get device ring settings: Operation not supported

Screenshot_1.png.94544424dc6033dfb3aae1799881d9c0.png

Share this post


Link to post
Share on other sites

It looks like your NIC is having trouble with the TSO command.

 

Go to a command line and tell me the output of this command:

 

ethtool -a eth0

 

Also:

 

/ethtool -k eth0

Share this post


Link to post
Share on other sites

 

cp /boot/60-persistent-storage.rules /lib/udev/rules.d

udevadm control --reload-rules

udevadm trigger

sleep 5

 

 

What is the content of that file?

 

udev is also ran by unRAID upon startup and it  should find the relevant devices.

 

Here is the thread out of which all of this came. :)

 

http://lime-technology.com/forum/index.php?topic=38487.0

 

That persistent-rules file is based on unRAID v5, was it ever tried to run on unRAID v6 without copying this file?

 

FYI the rules file in v6 has been updated to support new NVMe cards as well, overwriting it with an 'old' file will loose such support.

 

I commented out the lines and rebooted. See pic. LOst my drive labels insomuch that unRaid can't tell which one is which and couldn't start the array. I am putting them back in ;)

Screenshot_2.png.63359238782e1eba1b2b84c0f2e813f1.png

Share this post


Link to post
Share on other sites

It looks like your NIC is having trouble with the TSO command.

 

Go to a command line and tell me the output of this command:

 

ethtool -a eth0

 

Also:

 

/ethtool -k eth0

 

here is output for both

 

login as: root
root@tower's password:
Linux 4.1.18-unRAID.
root@Tower:~# ethtool -a eth0
Pause parameters for eth0:
Cannot get device pause settings: Operation not supported

root@Tower:~# ethtool -k eth0
Features for eth0:
rx-checksumming: on
tx-checksumming: off
        tx-checksum-ipv4: off
        tx-checksum-ip-generic: off [fixed]
        tx-checksum-ipv6: off
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: off
        tx-scatter-gather: off
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
        tx-tcp-segmentation: off [requested on]
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp6-segmentation: off [requested on]
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
hw-switch-offload: off [fixed]
root@Tower:~#

Share this post


Link to post
Share on other sites

Ok.  Your NIC does not have flow control and I need to handle that differently so you don't get an error.  I'll show "N/A" to indicate the NIC does not support flow control.

 

I don't understand the ring buffers.  Give me the output of this command:

 

ethtool -g eth0

Share this post


Link to post
Share on other sites

Ok.  Your NIC does not have flow control and I need to handle that differently so you don't get an error.  I'll show "N/A" to indicate the NIC does not support flow control.

 

I don't understand the ring buffers.  Give me the output of this command:

 

ethtool -g eth0

 

root@Tower:~# ethtool -g eth0
Ring parameters for eth0:
Cannot get device ring settings: Operation not supported

Share this post


Link to post
Share on other sites

Ok.  Your NIC does not have flow control and I need to handle that differently so you don't get an error.  I'll show "N/A" to indicate the NIC does not support flow control.

 

I don't understand the ring buffers.  Give me the output of this command:

 

ethtool -g eth0

 


Ok.  All fixed in the next version.
root@Tower:~# ethtool -g eth0
Ring parameters for eth0:
Cannot get device ring settings: Operation not supported

Share this post


Link to post
Share on other sites

I see good things happening here!  Are we ready to go live?

 

I'd like to add the plugin to the wiki page, when you feel it's ready.  In order to do that though, you'll need to set up a plugin support thread for it, that I can link to.  Then I'd like to go live with this, promote it in the beta thread and possibly elsewhere.  I'd like to introduce these as a set of new Troubleshooting tools, "Fix Common Problems" plugin (with the new Troubleshooting mode), "Tips and Tweaks" wiki page, and "Tips and Tweaks" plugin, and strongly recommend everyone try them out, but especially anyone having problems with recent versions.  Anyone see a reason why it's too early?  I'd hoped for more critiquing of the page.  And I'd hoped to add something about write caching, but I'd like more knowledgeable opinions on it.  I'm not comfortable yet with how to present the 'dirty' options.

Share this post


Link to post
Share on other sites

I'd like to hold a little bit before going live.  I'd like a bit more time to think about the 'dirty' cache settings.  The thing that bothers me is that some users will just crank up the settings thinking 'more is better'.  I need to do some more research on the settings, but I think what might be nest are some fixed settings that pre-determine the values:

 

'Default' - 10, 20 (best for systems with VMs)

'Balance' - 5,80 (in between settings to offer best of both)

'Performance' - 50,80 (best for a lot of repeating, small file I/O)

 

Anything higher that 'Default' will have some risk of data loss that has not been committed to disk.  User must have a UPS.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.