Newbie Setup Help and Parity Pros/Cons

[crackerjak]

I'm am new to Unraid and I am currently thinking through how I want to configure my new system.  I have tried out Freenas and Proxmox ZFS, but I like the simplicity and flexibility that Unraid offers, but I have a couple of questions regarding configuration that I hope someone can help me with...particularly around using parity.

 

Here is what I'm working with:

Hardware

• CPU: Dual Xeon x5675 (3.07ghz)
• RAM: 48gb ECC
• HDD: 4 x 2TB Seagate NAS  (main array)
• SSD: 3 x 250GB Samsung EVOs (planned on using as cache)

Usage/Intent
This server will act as a home server (and homelab) for a 6 people. While the primary purpose will be a file server, another big function is to store and serve my media collection (HD Movies, pictures, and music) via Plex to a number of TVs and PCs throughout the house. Additionally, I will be using it as a game hosting server for my kids (i.e. Minecraft, Ark, Teamspeak 3, and any else I want to play around with). While write speed would be nice, I am primarily focused on read performance.  I keep multiple backups on/off site, and the server is on a ups, so I am willing to accept that I will have SOME level of risk.

 

With my specs and use case above, is a parity drive really needed?  If I have a drive failure, wouldn't it be faster just to restore the lost data from backup rather than wait for a rebuild and possibly have another drive die during the process?  Other than redundancy, are there any other benefits that Unraid's version of parity provides (i.e. better read/write speeds)?  Also, is the prevailing wisdom still to use XFS as the fs on the main array or is btrfs mature enough now?

 

Any help or insights are greatly appreciated!

 

Edited June 5, 2017 by crackerjak

[trurl]

Each data disk is an independent filesystem. This means that a file read can only happen at the speed of the single drive the file is being read from. And writes must update parity, so writes are actually somewhat slower than single drive writes. So, parity doesn't improve read/write performance.

 

I still recommend parity since it will allow you to take advantage of drive spanning. You don't need to worry about which files were on a drive if it fails since you can simply rebuild that one drive. And rebuilds are going to be faster than restoring from backup anyway.

 

[SSD]

On 6/5/2017 at 2:16 PM, crackerjak said:

Other than redundancy, are there any other benefits that Unraid's version of parity provides (i.e. better read/write speeds)? 

 

Also, is the prevailing wisdom still to use XFS as the fs on the main array or is btrfs mature enough now?

 

Any help or insights are greatly appreciated!

 

 

Parity exists for redundancy. If that is not a benefit, there is no need to have a parity.

 

You can still take advantage of the user shares in a non-parity protected array.

 

As trurl has already stated, parity does not affect read speed, which is only dependent on the read speed of single disk it is reading from.

 

It does substantially slow down writes.

 

Parity does have a bit of a side benefit. We typically run a monthly integrity check (parity check) that has the effect of exercising every disk and making sure parity is integral. With no parity disk there is no way to do a parity check, and this might hinder detecting problems early. But I'm sure there are other tools that could be called to do something similar.

 

The conventional wisdom here is that everyone has parity, so it is hard for us to think about use cases that don't need one. But if you have all of you data safely stored elsewhere with backups / protections in place, and want to use unRAID only as the source for media playing, there is no reason you can't configure it that way. And you will enjoy fast writes.

 

XFS is the recommended file system. A few brave souls use btrfs. Do not use RFS.

[crackerjak]

Thanks for the reply!

 

Ah I see.  Since that's the case, it seems that parity would be faster if the drive is full (since rebuilding 2TB of data could take 12-24 hrs). However; if the drive is only partially used, it might be faster just to run rsync to restore only the missing files.  (I'm completely new to Unraid and still learning how it works, so forgive me if this is a stupid conclusion).

 

Another question.  Since I am reloading the server, what is the best/fastest way to get my data into the array.  I am restoring from a USB 3.0 backup w/ approx. 1.5TB of data.  Should I just add 3 of the 2TB drives, transfer the data, and then add the last drive as a parity when done?  I would think this would be faster since it wouldn't have to do the calculations during the initial transfer.  Also, I would think that holding off on adding the 3 SSDs as a cache would be good as well since the drives couldn't hold that amount of data anyway.

 

Thoughts?

 

[UPDATE] I updated my posts to reflect 2TB drives instead of 4TB drives (I was in a raid 10 w/  4TB  usable...so I had 4TB on the brain)

Edited June 5, 2017 by crackerjak

[trurl]

24 minutes ago, crackerjak said:

Since I am reloading the server, what is the best/fastest way to get my data into the array.  I am restoring from a USB 3.0 backup w/ approx. 1.5TB of data.  Should I just add 3 of the 4TB drives, transfer the data, and then add the last drive as a parity when done?  I would think this would be faster since it wouldn't have to do the calculations during the initial transfer.  Also, I would think that holding off on adding the 3 SSDs as a cache would be good as well since the drives couldn't hold that amount of data anyway.

Yes, doing the initial data load without parity is often recommended since it allows faster writing.

 

Each user share has settings which control whether and how it uses cache. The default setting does not use cache, so even if you add cache before the initial transfer, it won't be used unless you change that setting. And if you write directly to a data disk instead of writing to a user share, cache is never used.

[crackerjak]

Perfect...thanks!

 

One last thing; Does Unraid automatically distribute files across drives, or does it fill a disk one at a time?  Just trying to figure out if I copy all data to a single drive, will it balance the data across the drives, or if there is a singular location that I copy to oo have it do that.

 

Since I haven't started the copy yet, I wanted to make sure I understand how it handles balancing before I spend many hours doing it the "wrong" way.

[trurl]

If you write data to a user share, unRAID will choose which disk to write based on the settings for that user share. The main setting you are interested in is Allocation method. The default setting of High-water is a good compromise for distributing your data without the need to constantly switch disks. Other settings that come into play are included/excluded disks, split level, and minimum free. Each page of the webUI has help, turn it on with the Help button for an explanation of these settings.

[crackerjak]

2 minutes ago, trurl said:

If you write data to a user share, unRAID will choose which disk to write based on the settings for that user share. The main setting you are interested in is Allocation method. The default setting of High-water is a good compromise for distributing your data without the need to constantly switch disks. Other settings that come into play are included/excluded disks, split level, and minimum free. Each page of the webUI has help, turn it on with the Help button for an explanation of these settings.

 

I'll give it a go.  Thanks for the help.

[SSD]

On 6/5/2017 at 2:41 PM, bjp999 said:

 

You can still take advantage of the user shares in a non-parity protected array.

 

(I was mistaken about this. User shares appear to only be active with parity in place. However, you can use custom Samba shares to set up convenient access to folders on a large RAID volume.)

 

 

I edited my post above to make a correction. Appears no user shares on non-parity protected arrays. But you can create custom Samba shares rather easily to provide convenient share access to folders on a large RAID volume.

[itimpi]

49 minutes ago, bjp999 said:

 

I edited my post above to make a correction. Appears no user shares on non-parity protected arrays. But you can create custom Samba shares rather easily to provide convenient share access to folders on a large RAID volume.

User shares do not require parity to be present.    Not sure what makes you think this is the case.

[trurl]

1 hour ago, itimpi said:

User shares do not require parity to be present.    Not sure what makes you think this is the case.

I have always run with parity so have never tried user shares without it. I think bjp999 has said he doesn't use user shares. I wonder if something in his Global Share settings might be to blame.

 

The only reason I said

On 6/5/2017 at 2:28 PM, trurl said:

I still recommend parity since it will allow you to take advantage of drive spanning.

is because you can't span drives without user shares, and if you let user shares determine where to put files, you may not know what files are on a disk if you lose it.

 

Restoring from backup instead of rebuilding from parity could still work if the backup was a backup of the actual disk.

[SSD]

You are right. I has having trouble enabling user shares on a new server that was non-parity protected. Appears array needs to be down to turn that feature on. I thought the feature was on be default, and when I couldn't turn it on led me to believe it was an issue with no parity. But it was because the array was started. Bringing down the array allowed me to enable it. I have unrevised my post above.

 

USER SHARES DO NOT REQUIRE A PARITY.

[trurl]

9 minutes ago, bjp999 said:

You are right. I has having trouble enabling user shares on a new server that was non-parity protected. Appears array needs to be down to turn that feature on. I thought the feature was on be default, and when I couldn't turn it on led me to believe 

I can see why the array would need to be stopped to turn on user shares, since starting the array creates the user shares from the top level folders of all disks.

 

But I would assume that a fresh install would have user shares enabled by default. My setups were so long ago that I can't remember, but I would expect a lot more forum questions about turning on user shares if this weren't the case.

[SSD]

45 minutes ago, trurl said:

I can see why the array would need to be stopped to turn on user shares, since starting the array creates the user shares from the top level folders of all disks.

 

But I would assume that a fresh install would have user shares enabled by default. My setups were so long ago that I can't remember, but I would expect a lot more forum questions about turning on user shares if this weren't the case.

 

I used my second key for the new server, and it had a prior configuration for something I was doing testing on. I just did a new config and updated the unRAID version. It is possible that user shares were disabled from whatever I was using that key for before.

[bonienl]

1 hour ago, trurl said:

But I would assume that a fresh install would have user shares enabled by default. My setups were so long ago that I can't remember, but I would expect a lot more forum questions about turning on user shares if this weren't the case.

 

On a new install user shares are enabled by default and disk shares are disabled by default.