nick5429

EncFs plugin for unRAID v5

22 posts in this topic

I've written an EncFs plugin for unRAID v5. 

 

I'm calling this a "beta" because while it works for me, it's got a couple kinks to work out, and nobody else has tested it yet... so download it and give it a try!

Any feedback would be welcome.

 

The plugin:

  • Automatically downloads/installs/sets up all the needed packages and dependencies for EncFs
  • Gives explicit instructions on how to do the initial creation of the encrypted "volume" [note: it's not really a volume with EncFs. Each file is encrypted and stored separately, so you don't need to worry about figure out how big your "volume" will eventually be and waste space in the meantime -- the encrypted space grows as you need it]
  • Provides an interface in the unRAID web GUI to enter your encryption password and mount/unmount the encrypted "volume"
  • Unmounts the EncFs volume when unRAID is triggered to shut down

 

Thanks to boof and alphazo for doing a lot of the legwork required for getting encfs working on unraid in their respective threads (which you may want to read):

http://lime-technology.com/forum/index.php?topic=4804.0

http://lime-technology.com/forum/index.php?topic=7759.0

 

Installation

SSH to your server, then:

cd /boot/config/plugins
wget http://www.nickmerryman.com/unraid/encfs/encfs-0.1-beta.plg
installplg encfs-0.1-beta.plg

 

Release notes

2012-10-03 0.1-beta Put together an initial version.

 

DISCLAIMER: This seems to work for me on unraid 5.0-rc6-r8168-test2. I offer this plugin as a convenience and offer no warranty or guarantee.  If you lose all your encrypted data, or if through some failing the data is actually not encrypted after all, or your password gets leaked, I'm sorry, but I claim no responsibility.

 

To Do / Planned:

* If someone wants to make a prettier icon for this, that would be swell! Just send it to me and I'll include it :)

* Add configurable 'unmount after X idle minutes' option (pass as an option to the encfs mount command)

* Add support for multiple encrypted mounts

- Maybe. Don't hold your breath. I only use one encfs volume right now and so far this works fine for me as-is. Feel free to add it and re-contribute the source :)

 

Unlikely:

* Support for creating the encrypted mountpoint from within the plugin.  Too much to go wrong.  Just follow the instructions on the plugin's page to do it on the command line.

 

Notes:

* I haven't tested moving the "install directory"

* It seems like a good idea to set up your backing store to be on a specific disk, rather than on a user share. 

  encFs has one magic metadata file stored in the backing store that's required to decode any and all data (in addition to your password)

  If you set it up on a user share, make sure you back this up separately if you want a chance of recovering your data in a multi-disk failure.

* If you ever access the decrypted mountpoint via a user share rather than from a disk# share, it seems like a good idea set up unraid to ONLY write that usershare's data to that specific disk.  Otherwise, it could potentially write outside the encrypted container when copying stuff over.

 

Known issues:

* Sometimes the mounting is flaky through the 'expect' script used to mount from the web interface.  I've never had this happen when calling encfs directly from the command line, even after it happens through the web interface.

      It appears to be executing the same command, but doing an "ls $mountpoint" will come back with "Transport endpoint is not connected"

  Unmounting, deleting the mountpoint directory, and letting encfs re-create it fixed the problem for me sometimes.

  I can't figure out what causes this, and it's really annoying.  If it happens to you and you can figure it out (and even better, how to reliably fix/work around it), that would be swell.

 

* Sometimes after mounting through the webpage interface, the page remains loading indefinitely after displaying that it's trying to mount. Maybe the expect script isn't returning?

  It's got a 10 second timeout though, so that shouldn't be the issue...

  Mounting seems to always work fine when this happens, just click another link in the UI and come back if you want.

 

encfs-0.1-beta.zip

0

Share this post


Link to post
Share on other sites

On a related note, I did a small bit of benchmarking of encfs with various encryption options and using both disk and user shares before embarking on this.

 

My CPU is an AMD Phenom II X2 555, and the disk I tested on is a WD20EARX.  Parity disk is a Seagate Barracuda Green ST2000DL003.

 

I made a ramdisk to avoid interference when reading from a separate disk in the array and copied the ~475MB source file to the ram disk.

 

Read/write units are seconds required to transfer the 475MB file.

                                                    Pure write                Pure read
cypher                            block size        ramdisk->disk5(enc)       disk5(enc)->/dev/null
Non-encrypted (disk shares)        -                   39.4                    7.8
Non-encrypted (user shares)        -                   38.0                    7.8
"Standard" (disk share)            1024                36                      7.3    
"Standard" (user share)            1024                37.8                    6.5    
"Paranoid" (disk share)            1024                42                      7.6
"Paranoid" (user share)            1024                39.4                    7.8
AES-256    (disk)                  2048                39.9                    7.5
AES-256    (user)                  2048                41.1                    7.2
AES-256    (disk)                  4096                29.8                    5.5
AES-256    (user)                  4096                29.5                    5.8
blowfish (disk)                    4096                43.4                    8.5
blowfish (user)                    4096                43.6                    8.4

 

Interestingly (for this specific case) EncFs with AES with a 4k block size on top of unRAID is faster than unRAID alone for both reads and writes.  I repeated both the plain-unRAID and AES-256/4k test several times and had consistent results.  Odd.

 

Unfortunately, that doesn't hold true with a 'real world' copy though. 

This was 523GB of widely mixed sized files copied to an AES-256 4k block EncFs. I don't believe any substantial portion of the data in the source directory was stored on disk5.

time cp -R /mnt/user/test/* /mnt/disk5/decStore/test/
real    990m42.637s
user    0m9.860s
sys     37m19.040s

~9MB/sec

 

Non-encrypted test:

time cp -R /mnt/user/test/* /mnt/disk5/nonencryptedtest--safetodelete/
real    689m18.409s
user    0m9.050s
sys     66m55.070s

~12MB/sec

 

Slower, sure, but not cripplingly slow.

 

 

Aside: I have to say, I'm somewhat shocked at how slow the non-encrypted copy was.  On this same hardware with unRAID 4.7, I routinely got 20-30MB/sec writing a large file over the network (rather than copying locally).  With the 5.0rc I'm running, I only get ~15-17MB/sec doing a large network write. There's clearly a large performance regression somewhere in there.

0

Share this post


Link to post
Share on other sites

If anyone still using this?  I am still running 5.0-rc5 on one system and 5.0.5 on another.  If my servers get stolen, I do *not* want the data on my disks to be readable.

 

Is this plug-in still the best option?

0

Share this post


Link to post
Share on other sites

This is what I currently still use for the sensitive data on my own v5.x servers.

 

If/when I upgrade to v6, my intent is to investigate having a small VM (or whatever terminology we're using for these v6 "applets") running something a little more 'managed' for encryption ... but I'm not running v6 yet, and I don't have any particular solution in mind for that :)

 

edit: I have no idea if anyone at all other than me is using this.  If you are and have found it useful -- or if you think think it's absolute crap -- please let me know either way!

0

Share this post


Link to post
Share on other sites

edit: I have no idea if anyone at all other than me is using this.  If you are and have found it useful -- or if you think think it's absolute crap -- please let me know either way!

 

I would have used it if there was a v6 version *hint* ;D

0

Share this post


Link to post
Share on other sites

I would have used it if there was a v6 version *hint* ;D

 

I'm upgrading to v6 this week as soon as burn-in testing/preclear on a few new drives finish, so I'd say there's a very good chance I'll get something tested and working for v6 soon :)

0

Share this post


Link to post
Share on other sites

Where you ever able to get something working with v6?

 

With the ACD $5 sale recently, I'm looking for a way to encrypt my stuff before uploading to Amazon. When looking for different options, I stumbled across encfs, and then found your plugin when looking for an unraid variant. Thanks!

0

Share this post


Link to post
Share on other sites

Someone asked about source and contributing, so I tossed this up on github

 

https://github.com/nick5429/unraid-encfs

 

I haven't personally had time to figure out the new v6 plugin scheme in order to make this update.

 

As a starting point, the plugin could be updated with the following binaries, which work for me (running encfs manually on the command line) on 6.1: boost-1.54.0-x86_64-3.txz  encfs-1.7.4-x86_64-4gv.txz  rlog-1.4-x86_64-1pw.txz  slocate-3.1-x86_64-4.txz  tcl-8.6.1-x86_64-1.txz

 

Contributions welcome :)

 

[FWIW, I had vague notions of looking into using eCryptFS instead, which is likely a better fit for the unraid disk/usage model -- but this would require LimeTech either building in eCryptFS support to the kernel, or someone maintaining per-kernel-version module builds]

0

Share this post


Link to post
Share on other sites

Can I use this as virtual drive for Amazon Cloud Drive? Like netdrive but with encrypted files in Amazon Cloud Drive. Im on Unraid 5.

0

Share this post


Link to post
Share on other sites

Can I use this as virtual drive for Amazon Cloud Drive? Like netdrive but with encrypted files in Amazon Cloud Drive. Im on Unraid 5.

 

I would think so, if you have a way to mount the Amazon Drive, like acd-cli. Sounds like we are trying to do the same thing.

0

Share this post


Link to post
Share on other sites

Tried installing encfs using the link.

 

Got this error

 

file /boot/packages/encfs-1.7.4-i486-1sl.txz: downloading from http://darkstar.ist.utl.pt/slackware/addon/slacky/slackware-13.1/system/encfs/1.7.4/encfs-1.7.4-i486-1sl.txz ... bad download, deleting

 

Full installation log.

 

https://gist.github.com/syaifulnizamyahya/9e2587c354472b0ab9b8e97eb5c02b16#file-encfs-plg-install-failed

 

Advice welcome.

 

Edit.

 

Updated to correct links. Check github pull request.

 

Thanks.

0

Share this post


Link to post
Share on other sites

Did you try downloading it manually to see if it is a bad download or a problem with the plugin?

0

Share this post


Link to post
Share on other sites

Merged, thanks publicENEMY.  Note, I'm still pretty sure this plugin won't work on 6.1+. 

 

I still use EncFS on 6.2 from the command line using the packages I referenced a few posts back and it works fine

0

Share this post


Link to post
Share on other sites

There's an updated encfs plugin available which works with the new unraid 6.1+ plugin system.  For the moment, it's only available on the 6.2 'branch'. 

 

https://github.com/nick5429/unraid-encfs/tree/6.2

 

It still does that annoying thing where the webUI sits there "loading" forever after executing the mount command, but everything functionally works fine.  Just hit cancel and then browse away.

 

There's zero documentation on the unraid plugin system, so I'm not sure where to turn for answers.

 

I tested it briefly on 6.2.1 and it seems to work as well as it did before back on unraid v5.  Try it out and let me know what you think!

0

Share this post


Link to post
Share on other sites

There's an updated encfs plugin available which works with the new unraid 6.1+ plugin system.  For the moment, it's only available on the 6.2 'branch'. 

 

https://github.com/nick5429/unraid-encfs/tree/6.2

 

It still does that annoying thing where the webUI sits there "loading" forever after executing the mount command, but everything functionally works fine.  Just hit cancel and then browse away.

Thank you for your work on this!

 

There's zero documentation on the unraid plugin system, so I'm not sure where to turn for answers.

Perhaps it's better to say you were not able to find the docs on plugin development!  ;)  Admittedly, it appears to be scattered around, and no one person is maintaining it.  Here's what I found:

- How does the plugin system work? Documentation Added  (old, and some parts are obsolete, but provides a basic foundation)

- Plugin Authors: Changes for 6.1 release

- Important unRAID 6.1 Plugin System Changes

- Programming board  (there may a little more info scattered throughout this board for devs)

- I believe most new plugin authors start by taking apart a few selected plugins, and copying the basic structures.

 

Certainly it would be nice to have the docs in one place, and maintained by someone.  Perhaps the primary post of the first link could be updated, then stickied.

 

I tested it briefly on 6.2.1 and it seems to work as well as it did before back on unraid v5.  Try it out and let me know what you think!

This particular board is for v5, has been long deprecated, and is rarely visited by most users.  Please create a new topic for the v6 version of the plugin, in the Plugin Support board.

0

Share this post


Link to post
Share on other sites

 

Perhaps it's better to say you were not able to find the docs on plugin development!  ;)

I realized that was a bit harshly worded after posting it, and I had found all the resources you listed -- but nevertheless, 'halfway reverse engineered outdated and deprecated details' and a list of things you can't do anymore with workarounds don't add up to anywhere near 'documentation' in my book :)

 

This particular board is for v5, has been long deprecated, and is rarely visited by most users.  Please create a new topic for the v6 version of the plugin, in the Plugin Support board.

That's the plan, I just wanted anyone who may have used the old version to get an eye on it first and make sure nothing was majorly wrong before a wider release for v6.1+

 

PS: If you could get me added to "Community Developer" so I can post about this issue in the Programming board, that would be swell

0

Share this post


Link to post
Share on other sites

PS: If you could get me added to "Community Developer" so I can post about this issue in the Programming board, that would be swell

I think you need to pm jonp about that
0

Share this post


Link to post
Share on other sites

Hello, I'm just curious if there's any news about the 6.2+ version coming up and possibly being supported now?

Haven't yet seen a new thread popup anywhere, unless I missed it?

0

Share this post


Link to post
Share on other sites

from encfs vv1.9-rc2 changelog

 

allow writes in reverse mode when no header is used

 

basically, it allows read write to reverse mount. Usefull when you want to upload to acd and automatically remove uploaded files.

 

might be worth the effort to update encfs to version 1.9.1

0

Share this post


Link to post
Share on other sites
On 10/31/2016 at 8:42 PM, nick5429 said:

I just pushed a change to github that enables this plugin (working correctly without webUI hangs) for 6.2

 

I'm going to polish things up a bit before creating a new thread for it on the 6.2 plugin forum, but you can have a sneak preview :)

 

For 6.2: https://raw.githubusercontent.com/nick5429/unraid-encfs/6.2/encfs.plg

 

Hi there

 

Did you have chance to add a new thread at all? Also, tried with 6.3 and it doesn't appear to work, giving a wrong url error when downloading dependencies.

 

Thanks in advance

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.