steini84

ZFS plugin for unRAID

Recommended Posts

I decided to use your instructions to build ZFS for unraid 6.3.2 and all went well until:

 

Hunk #4 succeeded at 2515 (offset 2 lines).

Hunk #5 succeeded at 2610 (offset 2 lines).

patching file drivers/pci/quirks.c

  HOSTCC  scripts/basic/fixdep

scripts/basic/fixdep.c:105:23: fatal error: sys/types.h: No such file or directory

#include <sys/types.h>

                      ^

compilation terminated.

make[1]: *** [scripts/basic/fixdep] Error 1

make: *** [scripts_basic] Error 2

 

3.1) Do you want to run Menu Config ? [y/N] y

 

Are there other dependencies I should have installed first?

 

Thanks for your work.  I'm a long time ZFS user and would really like Lime-Tech to integrate ZFS support.  The snapshot/rollback capabilities are unparalleled.

 

Share this post


Link to post
Share on other sites

I can build it for 6.3.2 tomorrow and will let you know if I run into the same problem. 

 

 

Sent from my iPhone using Tapatalk

Share this post


Link to post
Share on other sites
On 2/19/2017 at 6:23 PM, laterdaze said:

I decided to use your instructions to build ZFS for unraid 6.3.2 and all went well until:

 

Hunk #4 succeeded at 2515 (offset 2 lines).

Hunk #5 succeeded at 2610 (offset 2 lines).

patching file drivers/pci/quirks.c

  HOSTCC  scripts/basic/fixdep

scripts/basic/fixdep.c:105:23: fatal error: sys/types.h: No such file or directory

#include <sys/types.h>

                      ^

compilation terminated.

make[1]: *** [scripts/basic/fixdep] Error 1

make: *** [scripts_basic] Error 2

 

3.1) Do you want to run Menu Config ? [y/N] y

 

Are there other dependencies I should have installed first?

 

Thanks for your work.  I'm a long time ZFS user and would really like Lime-Tech to integrate ZFS support.  The snapshot/rollback capabilities are unparalleled.

 

The plugin is updated for 6.3.2

There were some links broken in the build file, but you can try my Build script

Share this post


Link to post
Share on other sites

No, I don't have seen what improvements a GUI would bring, what were you thinking about? zpool status, pool creation or something else?



Sent from my iPhone using Tapatalk

Share this post


Link to post
Share on other sites

At the moment I am learning about NAS basics and the different server software solutions that exist. I am trying to understand what this ZFS plugin adds to unRAID. Does it allow one to run an array of drives with ZFS in a system which otherwise runs unRAID? So you could run 4-6 identical drives with ZFS in RAID-Z2 and the rest of the drives in the system with unRAID?

Share this post


Link to post
Share on other sites

Yeah you could do that. Basically it allows you to have one or more zfs pools on your server independent from your unraid protected drives.

 

My use case is three ssd drives in a raidz-1 for my vms and docker with automatic snapshots. I also replicate my pool to a 3tb external drive that I also rsync my most important data to daily (and snapshot on the USB drive)

 

It's so independent that it does not integrate at all into unraid shares, dashboard etc, so if you want to export your pools to nfs or samba you need to do it manually.

Share this post


Link to post
Share on other sites
On 2/28/2017 at 4:42 PM, steini84 said:

No, I don't have seen what improvements a GUI would bring, what were you thinking about? zpool status, pool creation or something else?



Sent from my iPhone using Tapatalk

 

hi, yes, basically all the zfs management, so it really ties into the main unraid gui - showing which drives are part of which pool, etc.

 

i've never seen a unraid screenshot with zfs enabled though...

Share this post


Link to post
Share on other sites

There is no GUI tie in at all so screen shots won't show anything different. It would be a really ambitious project to add all that to the GUI and I can't see myself do it.

If anyone want to tackle that they are welcome to use my builds and or build scripts. But maybe limetech will add zfs natively.. doubt it though :)


Sent from my iPhone using Tapatalk

Share this post


Link to post
Share on other sites
Posted (edited)
21 hours ago, steini84 said:

Yeah you could do that. Basically it allows you to have one or more zfs pools on your server independent from your unraid protected drives.

 

My use case is three ssd drives in a raidz-1 for my vms and docker with automatic snapshots. I also replicate my pool to a 3tb external drive that I also rsync my most important data to daily (and snapshot on the USB drive)

 

It's so independent that it does not integrate at all into unraid shares, dashboard etc, so if you want to export your pools to nfs or samba you need to do it manually.

 

Thanks for your reply!:) When you write: " so if you want to export your pools to nfs or samba you need to do it manually." I suppose you mean by command line interface? I've got so much to learn O.o

Edited by RiverElf

Share this post


Link to post
Share on other sites
 
Thanks for your reply! When you write: " so if you want to export your pools to nfs or samba you need to do it manually." I suppose you mean by command line interface?

Yeah was thinking via /boot/config/smb-extra.conf for samba. Don't remember what the way for samba is.

But this is all pretty advanced if people are only used to GUIs. Pleas be careful when playing and don't run with scissors ;)


Sent from my iPhone using Tapatalk

Share this post


Link to post
Share on other sites
3 minutes ago, steini84 said:


Yeah was thinking via /boot/config/smb-extra.conf for samba. Don't remember what the way for samba is.

But this is all pretty advanced if people are only used to GUIs. Pleas be careful when playing and don't run with scissors ;)


Sent from my iPhone using Tapatalk

 

Ha! :) yeah, it sounds quite advanced to someone like me. I'll try not to get cut if I ever dare go there. Nice to know that with this plugin there appears to be a way to run both ZFS and unRAID on the same system - if one has the knowledge and experience to do it safely ;-)

Share this post


Link to post
Share on other sites

Unraid a incredibly safe as an appliance if you only use the GUI.

However I have been messing with it since 2010 (4.7 if I remember correctly) and never had any catastrophes (like data loss). Still using the original setup, only with additions over the years.. and some dead drives that all rebuilt correctly :)




Sent from my iPhone using Tapatalk

Share this post


Link to post
Share on other sites

I actually used the smb-extra box within the unRAID GUI under samba settings and it worked perfectly

Share this post


Link to post
Share on other sites
I actually used the smb-extra box within the unRAID GUI under samba settings and it worked perfectly

Sorry about that. Did not know it had been integrated in the GUI.


Sent from my iPhone using Tapatalk

Share this post


Link to post
Share on other sites
11 hours ago, steini84 said:

Unraid a incredibly safe as an appliance if you only use the GUI.

However I have been messing with it since 2010 (4.7 if I remember correctly) and never had any catastrophes (like data loss). Still using the original setup, only with additions over the years.. and some dead drives that all rebuilt correctly :)




Sent from my iPhone using Tapatalk

 

Sounds good! :) unRAID looks very appealing to me. Being new to NAS one of the things I'm trying to learn more about, is how data integrity and data safety in unRAID compares to other solutions like ZFS and OpenMediaVault with the SnapRAID plugin - both of which appear to run a checksum before data is written to parity which should help protect against silent errors / bitrot. I'll probably start a thread about that in the relevant forum. It sounds like you haven't had any problems with bitrot, which is reassuring :)

Share this post


Link to post
Share on other sites
Posted (edited)

For anyone trying to set up zfs-auto-snapshot to use Previous Versions on Windows clients:

 

I placed the zfs-auto-snapshot.sh file from https://github.com/zfsonlinux/zfs-auto-snapshot/blob/master/src/zfs-auto-snapshot.sh in /boot/scripts/zfs-auto-snapshot.sh and made executable with chmod +x zfs-auto-snapshot.sh

 

I found that no matter which way I set the 'localtime' setting in smb.conf, the snapshots were not adjusting to local time and were shown in UTC time. To fix this, I removed the --utc parameter on line 537 of zfs-auto-snapshot.sh to read: DATE=$(date +%F-%H%M)

 

I then created cron entries by creating /boot/config/plugins/custom_cron/zfs-auto-snapshot.cron with the following contents:

# zfs-auto-snapshot.sh quarter hourly
*/15 * * * * /boot/scripts/zfs-auto-snapshot.sh -q -g --label=04 --keep=4 //
# zfs-auto-snapshot.sh hourly
@hourly ID=zfs-auto-snapshot-hourly /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=00 --keep=24 //
# zfs-auto-snapshot.sh daily
@daily ID=zfs-auto-snapshot-daily /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=01 --keep=31 //
# zfs-auto-snapshot.sh weekly
@weekly ID=zfs-auto-snapshot-weekly /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=02 --keep=8 //
# zfs-auto-snapshot.sh monthly
@monthly ID=zfs-auto-snapshot-monthly /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=03 --keep=12 //

Edit: I switched the cron entries to use specific times of day, days of the week, etc. primarily due to the effect of reboots on unRAID's cron handling. I would get inconsistently spaced apart snapshots with the above cron configuration.

# zfs-auto-snapshot.sh quarter hourly
*/15 * * * * /boot/config/scripts/zfs-auto-snapshot.sh -q -g --label=04 --keep=4 //
# zfs-auto-snapshot.sh hourly
0 * * * * ID=zfs-auto-snapshot-hourly /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=00 --keep=24 //
# zfs-auto-snapshot.sh daily
0 0 * * * ID=zfs-auto-snapshot-daily /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=01 --keep=31 //
# zfs-auto-snapshot.sh weekly
0 0 * * 0 ID=zfs-auto-snapshot-weekly /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=02 --keep=8 //
# zfs-auto-snapshot.sh monthly
0 0 1 * * ID=zfs-auto-snapshot-monthly /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=03 --keep=12 //

 

Run 'update_cron' to immediately enable the custom cron entries.

 

The labels differ from the zfs-auto-snapshot default labels for better compatibility with Samba. For the Samba shares, I placed the below in /boot/config/smb-extra.conf:

 

[data]
path = /mnt/zfs/data
browseable = yes
guest ok = yes
writeable = yes
read only = no
create mask = 0775
directory mask = 0775
vfs objects = shadow_copy2
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
shadow: format = zfs-auto-snap_%S-%Y-%m-%d-%H%M
shadow: localtime =  yes

 

Run 'samba reload' to refresh your Samba config. After the first scheduled snapshot is taken, you should now be able to see the snapshots in the Previous Versions dialog on a connected Windows client.

 

You'll need to modify this with your desired snapshot intervals, retention, and path names. This configuration is working well for me. I hope this helps anyone else out there trying to get ZFS snapshots working with shadow copy for Windows clients.

Edited by rinseaid
Update cron entries.
  • Upvote 1

Share this post


Link to post
Share on other sites
Posted (edited)

I also wanted to get the ZFS Event Daemon (zed) working on my unRAID setup. Most of the files needed are already built into steini84's plugin (thanks!) but zed.rc needs to be copied into the file system at each boot.

 

I created a folder /boot/config/zfs-zed/ and placed zed.rc in there - you can get the default from /usr/etc/zfs/zed.d/zed.rc.

 

Add the following lines to your go file:

 

#Start ZFS Event Daemon
cp /boot/config/zfs-zed/zed.rc /usr/etc/zfs/zed.d/
/usr/sbin/zed &

 

To use built in notifications in unRAID, and to avoid having to set up a mail server or relay,  set your zed.rc with the following options:

ZED_EMAIL_PROG="/usr/local/emhttp/webGui/scripts/notify"
ZED_EMAIL_OPTS="-i warning -s '@SUBJECT@' -d '@SUBJECT@' -m \"\`cat $pathname\`\""

 

$pathname contains the verbose output from ZED, which will be sent in the body of an email alert from unRAID. I have this set to alert level of 'warning' as I have unRAID configured to always email me for warnings. You'll also want to adjust your email address, verbosity level, and set up a debug log if desired.

 

Either place the files and manually start zed, or reboot the system for this to take effect.

 

Pro tip, if you want to test the notifications, zed will alert on a scrub finish event. If you're like me and only have a large pools that takes hours/days to scrub, you can set up a quick test pool like this:

 

truncate -s 64M /root/test.img
zpool create test /root/test.img
zpool scrub test

When you've finished testing, just destroy the pool.

 

Edited by rinseaid
spell check...

Share this post


Link to post
Share on other sites

You say you use it for KVM docker, do the pools get properly mounted and unmounted at startup/shutdown?

Share this post


Link to post
Share on other sites
You say you use it for KVM docker, do the pools get properly mounted and unmounted at startup/shutdown?

Yes


Sent from my iPhone using Tapatalk
  • Upvote 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.