fredsherbet

Rough Guide: Improving rsync's speed (ssh encryption bottleneck)

4 posts in this topic

Hi, I thought I'd share my success getting rsync faster. I almost doubled the transfer speed.

 

I found that the bottleneck was SSH's CPU usage on the target box, due to decrypting being expensive. You can improve performance easily by using blowfish encryption, which is less secure than 3DES, but significantly less computationally expensive.

 

Put this in your rsync to make that happen:

rsync --rsh="ssh -c blowfish" ....

 

This got me from 15MB/s to about 20-25MB/s.

 

I got more gain by ditching ssh for rsh. RSH is completely insecure. Do not use it over the open Internet. If you're on a private home network, then it completely removes the SSH overhead.

 

Getting it to work on unraid took a little fiddling. You can get an rshd package from http://pkgs.org/slackware-13.1/slackware-i486/netkit-rsh-0.17-i486-1.txz.html

 

Edit

/etc/inetd.conf

to uncomment the rsh.d (starts shell) line, and run

kill -HUP `pgrep inetd`

so inetd reloads its config, and knows to listen for rsh connections, and call rshd to handle them.

 

Edit

~/.rhosts

(create it, if it doesn't exist) to list the IP addresses that are allowed to connect to the system.

 

When debugging,

tail /var/log/syslog

to see what's not working.

 

Using RSH got me from 15MB/s to 25-30MB/s.

0

Share this post


Link to post
Share on other sites

Remember any edits to /etc/inetd.conf and ~/.rhosts will be lost on a reboot.  You need to setup those edits to happen from your go script if you want to preserve your work.

0

Share this post


Link to post
Share on other sites

Try "arcfour" instead of "blowfish" as the kind of encryption.

I use it for large database files all the time.

Seems to be the fastest from my tests.

YMMV though.

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.