ZFS plugin for unRAID


steini84

Recommended Posts

  • 2 weeks later...

I decided to use your instructions to build ZFS for unraid 6.3.2 and all went well until:

 

Hunk #4 succeeded at 2515 (offset 2 lines).

Hunk #5 succeeded at 2610 (offset 2 lines).

patching file drivers/pci/quirks.c

  HOSTCC  scripts/basic/fixdep

scripts/basic/fixdep.c:105:23: fatal error: sys/types.h: No such file or directory

#include <sys/types.h>

                      ^

compilation terminated.

make[1]: *** [scripts/basic/fixdep] Error 1

make: *** [scripts_basic] Error 2

 

3.1) Do you want to run Menu Config ? [y/N] y

 

Are there other dependencies I should have installed first?

 

Thanks for your work.  I'm a long time ZFS user and would really like Lime-Tech to integrate ZFS support.  The snapshot/rollback capabilities are unparalleled.

 

Link to comment
On 2/19/2017 at 6:23 PM, laterdaze said:

I decided to use your instructions to build ZFS for unraid 6.3.2 and all went well until:

 

Hunk #4 succeeded at 2515 (offset 2 lines).

Hunk #5 succeeded at 2610 (offset 2 lines).

patching file drivers/pci/quirks.c

  HOSTCC  scripts/basic/fixdep

scripts/basic/fixdep.c:105:23: fatal error: sys/types.h: No such file or directory

#include <sys/types.h>

                      ^

compilation terminated.

make[1]: *** [scripts/basic/fixdep] Error 1

make: *** [scripts_basic] Error 2

 

3.1) Do you want to run Menu Config ? [y/N] y

 

Are there other dependencies I should have installed first?

 

Thanks for your work.  I'm a long time ZFS user and would really like Lime-Tech to integrate ZFS support.  The snapshot/rollback capabilities are unparalleled.

 

The plugin is updated for 6.3.2

There were some links broken in the build file, but you can try my Build script

Link to comment

At the moment I am learning about NAS basics and the different server software solutions that exist. I am trying to understand what this ZFS plugin adds to unRAID. Does it allow one to run an array of drives with ZFS in a system which otherwise runs unRAID? So you could run 4-6 identical drives with ZFS in RAID-Z2 and the rest of the drives in the system with unRAID?

Link to comment

Yeah you could do that. Basically it allows you to have one or more zfs pools on your server independent from your unraid protected drives.

 

My use case is three ssd drives in a raidz-1 for my vms and docker with automatic snapshots. I also replicate my pool to a 3tb external drive that I also rsync my most important data to daily (and snapshot on the USB drive)

 

It's so independent that it does not integrate at all into unraid shares, dashboard etc, so if you want to export your pools to nfs or samba you need to do it manually.

Link to comment
On 2/28/2017 at 4:42 PM, steini84 said:

No, I don't have seen what improvements a GUI would bring, what were you thinking about? zpool status, pool creation or something else?



Sent from my iPhone using Tapatalk

 

hi, yes, basically all the zfs management, so it really ties into the main unraid gui - showing which drives are part of which pool, etc.

 

i've never seen a unraid screenshot with zfs enabled though...

Link to comment

There is no GUI tie in at all so screen shots won't show anything different. It would be a really ambitious project to add all that to the GUI and I can't see myself do it.

If anyone want to tackle that they are welcome to use my builds and or build scripts. But maybe limetech will add zfs natively.. doubt it though :)


Sent from my iPhone using Tapatalk

Link to comment
21 hours ago, steini84 said:

Yeah you could do that. Basically it allows you to have one or more zfs pools on your server independent from your unraid protected drives.

 

My use case is three ssd drives in a raidz-1 for my vms and docker with automatic snapshots. I also replicate my pool to a 3tb external drive that I also rsync my most important data to daily (and snapshot on the USB drive)

 

It's so independent that it does not integrate at all into unraid shares, dashboard etc, so if you want to export your pools to nfs or samba you need to do it manually.

 

Thanks for your reply!:) When you write: " so if you want to export your pools to nfs or samba you need to do it manually." I suppose you mean by command line interface? I've got so much to learn O.o

Edited by RiverElf
Link to comment
 
Thanks for your reply! When you write: " so if you want to export your pools to nfs or samba you need to do it manually." I suppose you mean by command line interface?

Yeah was thinking via /boot/config/smb-extra.conf for samba. Don't remember what the way for samba is.

But this is all pretty advanced if people are only used to GUIs. Pleas be careful when playing and don't run with scissors ;)


Sent from my iPhone using Tapatalk
Link to comment
3 minutes ago, steini84 said:


Yeah was thinking via /boot/config/smb-extra.conf for samba. Don't remember what the way for samba is.

But this is all pretty advanced if people are only used to GUIs. Pleas be careful when playing and don't run with scissors ;)


Sent from my iPhone using Tapatalk

 

Ha! :) yeah, it sounds quite advanced to someone like me. I'll try not to get cut if I ever dare go there. Nice to know that with this plugin there appears to be a way to run both ZFS and unRAID on the same system - if one has the knowledge and experience to do it safely ;-)

Link to comment

Unraid a incredibly safe as an appliance if you only use the GUI.

However I have been messing with it since 2010 (4.7 if I remember correctly) and never had any catastrophes (like data loss). Still using the original setup, only with additions over the years.. and some dead drives that all rebuilt correctly :)




Sent from my iPhone using Tapatalk

Link to comment
11 hours ago, steini84 said:

Unraid a incredibly safe as an appliance if you only use the GUI.

However I have been messing with it since 2010 (4.7 if I remember correctly) and never had any catastrophes (like data loss). Still using the original setup, only with additions over the years.. and some dead drives that all rebuilt correctly :)




Sent from my iPhone using Tapatalk

 

Sounds good! :) unRAID looks very appealing to me. Being new to NAS one of the things I'm trying to learn more about, is how data integrity and data safety in unRAID compares to other solutions like ZFS and OpenMediaVault with the SnapRAID plugin - both of which appear to run a checksum before data is written to parity which should help protect against silent errors / bitrot. I'll probably start a thread about that in the relevant forum. It sounds like you haven't had any problems with bitrot, which is reassuring :)

Link to comment
  • 2 weeks later...

For anyone trying to set up zfs-auto-snapshot to use Previous Versions on Windows clients:

 

I placed the zfs-auto-snapshot.sh file from https://github.com/zfsonlinux/zfs-auto-snapshot/blob/master/src/zfs-auto-snapshot.sh in /boot/scripts/zfs-auto-snapshot.sh and made executable with chmod +x zfs-auto-snapshot.sh

 

I found that no matter which way I set the 'localtime' setting in smb.conf, the snapshots were not adjusting to local time and were shown in UTC time. To fix this, I removed the --utc parameter on line 537 of zfs-auto-snapshot.sh to read: DATE=$(date +%F-%H%M)

 

I then created cron entries by creating /boot/config/plugins/custom_cron/zfs-auto-snapshot.cron with the following contents:

# zfs-auto-snapshot.sh quarter hourly
*/15 * * * * /boot/scripts/zfs-auto-snapshot.sh -q -g --label=04 --keep=4 //
# zfs-auto-snapshot.sh hourly
@hourly ID=zfs-auto-snapshot-hourly /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=00 --keep=24 //
# zfs-auto-snapshot.sh daily
@daily ID=zfs-auto-snapshot-daily /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=01 --keep=31 //
# zfs-auto-snapshot.sh weekly
@weekly ID=zfs-auto-snapshot-weekly /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=02 --keep=8 //
# zfs-auto-snapshot.sh monthly
@monthly ID=zfs-auto-snapshot-monthly /boot/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=03 --keep=12 //

Edit: I switched the cron entries to use specific times of day, days of the week, etc. primarily due to the effect of reboots on unRAID's cron handling. I would get inconsistently spaced apart snapshots with the above cron configuration.

# zfs-auto-snapshot.sh quarter hourly
*/15 * * * * /boot/config/scripts/zfs-auto-snapshot.sh -q -g --label=04 --keep=4 //
# zfs-auto-snapshot.sh hourly
0 * * * * ID=zfs-auto-snapshot-hourly /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=00 --keep=24 //
# zfs-auto-snapshot.sh daily
0 0 * * * ID=zfs-auto-snapshot-daily /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=01 --keep=31 //
# zfs-auto-snapshot.sh weekly
0 0 * * 0 ID=zfs-auto-snapshot-weekly /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=02 --keep=8 //
# zfs-auto-snapshot.sh monthly
0 0 1 * * ID=zfs-auto-snapshot-monthly /boot/config/scripts/zfs-auto-snapshot.sh --quiet --syslog --label=03 --keep=12 //

 

Run 'update_cron' to immediately enable the custom cron entries.

 

The labels differ from the zfs-auto-snapshot default labels for better compatibility with Samba. For the Samba shares, I placed the below in /boot/config/smb-extra.conf:

 

[data]
path = /mnt/zfs/data
browseable = yes
guest ok = yes
writeable = yes
read only = no
create mask = 0775
directory mask = 0775
vfs objects = shadow_copy2
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
shadow: format = zfs-auto-snap_%S-%Y-%m-%d-%H%M
shadow: localtime =  yes

 

Run 'samba reload' to refresh your Samba config. After the first scheduled snapshot is taken, you should now be able to see the snapshots in the Previous Versions dialog on a connected Windows client.

 

You'll need to modify this with your desired snapshot intervals, retention, and path names. This configuration is working well for me. I hope this helps anyone else out there trying to get ZFS snapshots working with shadow copy for Windows clients.

Edited by rinseaid
Update cron entries.
  • Like 3
  • Upvote 1
Link to comment

I also wanted to get the ZFS Event Daemon (zed) working on my unRAID setup. Most of the files needed are already built into steini84's plugin (thanks!) but zed.rc needs to be copied into the file system at each boot.

 

I created a folder /boot/config/zfs-zed/ and placed zed.rc in there - you can get the default from /usr/etc/zfs/zed.d/zed.rc.

 

Add the following lines to your go file:

 

#Start ZFS Event Daemon
cp /boot/config/zfs-zed/zed.rc /usr/etc/zfs/zed.d/
/usr/sbin/zed &

 

To use built in notifications in unRAID, and to avoid having to set up a mail server or relay,  set your zed.rc with the following options:

ZED_EMAIL_PROG="/usr/local/emhttp/webGui/scripts/notify"
ZED_EMAIL_OPTS="-i warning -s '@SUBJECT@' -d '@SUBJECT@' -m \"\`cat $pathname\`\""

 

$pathname contains the verbose output from ZED, which will be sent in the body of an email alert from unRAID. I have this set to alert level of 'warning' as I have unRAID configured to always email me for warnings. You'll also want to adjust your email address, verbosity level, and set up a debug log if desired.

 

Either place the files and manually start zed, or reboot the system for this to take effect.

 

Pro tip, if you want to test the notifications, zed will alert on a scrub finish event. If you're like me and only have a large pools that takes hours/days to scrub, you can set up a quick test pool like this:

 

truncate -s 64M /root/test.img
zpool create test /root/test.img
zpool scrub test

When you've finished testing, just destroy the pool.

 

Edited by rinseaid
spell check...
  • Like 4
Link to comment
  • 2 weeks later...
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.