Drive Performance - What drives/configurations to use?


Recommended Posts

So first of all, I'm new here - so "Hi". Secondly, thanks to the unRAID team for an amazing OS!

 

I did try to search for this but couldn't find anything immediately on the forums so thought I'd start this thread hoping to get some advice from the community.

 

Currently, I have a mixed array of disks on my primary unRAID server made up of various rotational speeds, caches, manufacturers etc. What I wanted to know is, I am starting to approach a point where I will soon need to start investing in newer, higher capacity drives to expand my storage.

 

So, my question/what I'm looking to find out is this:

 

Is there a recommended method of provisioning drives in an unRAID server to optimize the performance of the system?

 

In other words, I'm trying to work out things like:

 

* Should one try to provision the same model drives in the array?

* How much does the onboard drive cache matter in an array?

* Is performance impacted if drives vary in their cache amounts?

* Does the rotational speed of the drives matter?

* Should the parity disk be a higher performance drive compared to the storage drives in the array?

 

Reason I'm asking is because I have noticed that in some cases, a copy operation to my server often will hit speeds of just over 100 MB/s from a networked machine, but then if it's a particularly large copy, at some point it starts slowing down to a crawl at times even appearing to pause before resuming.

 

Monitoring disk activity it would seem that the parity drive appears to be grinding hard to write parity, while the storage drives in the array often appear to be waiting for the parity disk to catch up. It got me wondering about the relationship between the disk performance of the parity drive vs. the storage drives, and as the number of storage drives increase, how that has an impact on overall array performance if there is only 1 parity drive.

 

Any advice is welcome!

 

Thanks!

Link to comment

The way unRaid writes work, is that unRaid pre-reads the sector on both the data disk and parity it is about to overwrite, and using that information and the block it is about to write, is able to compute the updated parity without reading all the other disks in the array. But it does mean slow writes because unRaid has to wait a full rotation of the disk between the reads and writes on both parity and data. So no, system is not lagging waiting on parity, it is waiting on both data and parity about equally. But if parity is slower, it will be the bottleneck, albeit probably not by much. If you have two or more write streams, though, parity will be the laggert! 

 

UnRaid has another write mode called turbo write. In this mode, all of the data disks except the one being written are read, and unRaid computes the updated parity for a sector from them plus the data it is writing. In this mode, there is no wait for the full rotation and it goes considerably faster. But all disks in the array have to be spinning.

 

UnRaid does utilize caching that gives a boost to writes as they are kicking off, which on my system helps allot with files about a gig in size. But the effect is short lived and speed drops to a more modest level for bigger files.

 

The cache in the drives is tiny (64 - 128MB) compared to files being written. I do not think that it contributes in any significant way to performance. More important is the cache in the computer, which enables the burst in write speed I described.

  • Upvote 2
Link to comment

I'll add a subjective comment about unRAID performance. You really need to think about what is important to you in terms of performance. Most of my writing takes place on the server and I kick it off and don't really care if it finishes in 45 minutes or 90. I just want it to finish and be protected. But if I am working on my array, I want to be able to access my files on different disks at fast speed. If my array is in turbo write mode, all of my disks are busily supporting the background write, and my reads and usage of other drives on the array are going to be affected. I'd much rather unRAID do its traditional writes, which means parity and one data disk are basally working on that, while all of the other data disks are idle. I can then use my array to do multiple different things on multiple different disks, and none of them are in disk contention with the others.

 

In general anything I really care about performance wise I want to use my SSD drive for. Docker images, docker application data, VM images, large video image processing, and other things, if these are important you can configure your SSD drive to handle. The SSD is largely unaffected by parallel use (no seeks), so if I am doing something that I expect to be in contention with something else, I want it to happen on the SSD. For data oriented things, at the end it can copy to the array and release the space on the SSD.

  • Upvote 2
Link to comment

@bjp999 - Thanks for that! Really great information.

 

So I think what you were speaking to about the full rotation of a disk before it can continue writing is what I was getting at. In other words, if I were to start retiring older drives in my array with newer ones - would it be better for me to try and stick to the same model of drive across my array (I know it doesn't really matter since unRAID was designed to use all makes/models of drives).

 

Obviously factoring in the requirement that your data drives cannot exceed the capacity of your parity drive... 

 

I just want to make sure my array is optimised as best as possible. I'm not running an overly hardcore/big-scale system, it's more for home backup, media streaming and the like. Here are the specs of what I am currently running (rather modest).

 

It is a repurposed old desktop machine that is now in service as a server since I replaced my daily driver... it is:

 

AMD Phenom II X4 965 BE - Cooled with a Corsair Hydro h80i

16 GB DDR3 RAM (Dual Channel Mode) - non-ECC

MSI 990FX-GD80 Motherboard (about 7 years old already but still running reliably)

Parity Disk - Seagate 3TB ST3000VN000-1HJ166_W7304JLT

Parity Disk - Seagate 3TB ST3000VN000-1HJ166_W7304JQ9 / xfs filesystem

Parity Disk - Seagate 3TB ST3000DM001-1CH166_Z1F4C81K / xfs filesystem

Parity Disk - WD Red 3TB WDC_WD30EFRX-68EUZN0_WD-WCC4N6NP6CYP / xfs filesystem

Cache Disk 1 - Transcend 128GB TS128GSSD370S_D092260493 / btrfs

Cache Disk 1 - Transcend 128GB TS128GSSD370S_D092260469 / btrfs

Link to comment
2 hours ago, Alcor said:

@bjp999 - Thanks for that! Really great information.

 

So I think what you were speaking to about the full rotation of a disk before it can continue writing is what I was getting at. In other words, if I were to start retiring older drives in my array with newer ones - would it be better for me to try and stick to the same model of drive across my array (I know it doesn't really matter since unRAID was designed to use all makes/models of drives).

 

Obviously factoring in the requirement that your data drives cannot exceed the capacity of your parity drive... 

 

I just want to make sure my array is optimised as best as possible. I'm not running an overly hardcore/big-scale system, it's more for home backup, media streaming and the like. Here are the specs of what I am currently running (rather modest).

 

It is a repurposed old desktop machine that is now in service as a server since I replaced my daily driver... it is:

 

AMD Phenom II X4 965 BE - Cooled with a Corsair Hydro h80i

16 GB DDR3 RAM (Dual Channel Mode) - non-ECC

MSI 990FX-GD80 Motherboard (about 7 years old already but still running reliably)

Parity Disk - Seagate 3TB ST3000VN000-1HJ166_W7304JLT

Parity Disk - Seagate 3TB ST3000VN000-1HJ166_W7304JQ9 / xfs filesystem

Parity Disk - Seagate 3TB ST3000DM001-1CH166_Z1F4C81K / xfs filesystem

Parity Disk - WD Red 3TB WDC_WD30EFRX-68EUZN0_WD-WCC4N6NP6CYP / xfs filesystem

Cache Disk 1 - Transcend 128GB TS128GSSD370S_D092260493 / btrfs

Cache Disk 1 - Transcend 128GB TS128GSSD370S_D092260469 / btrfs

 

WOW. 4 Parity drives ?

Myself, I have moved away from the desktop drives and have gone to all NAS drives. I have both WD & HGST

  • Upvote 1
Link to comment

Whoops! LOL...

 

My bad, I was copy/pasting... Correction... The next 3 disks after the first Parity are all Data drives :-)

 

Basically, all of those with xfs filesystem.

 

Thanks for the advice on what drives - The only non-NAS drive in there left is the ST3000DM001 drive - I read they apparently suffered from failures (that particular model). I actually replaced one of them - with the WD Red drive there.

 

Does having a 2nd Parity drive make any difference? I've tried to get my head around how the 2nd parity drive actually works/what role it plays. My basic/high-level understanding of how it works is that you have enough redundancy to have 2 data drives fail on you, if you have/use a 2nd parity drive. 

 

Is there any performance benefit to the overall RW performance of the array or would you say it doesn't make much difference in a small array like mine?

Link to comment
On 6/10/2017 at 7:23 AM, bjp999 said:

You really need to think about what is important to you in terms of performance.

 

What use cases are you trying to optimize?

 

General performance feedback - larger drives are faster. I am seeing much better performance with 8T drives than I did with smaller drives. They pack the data more densely, so even when their rotation is slower, they still are faster overall.

 

1 hour ago, graywolf said:

Myself, I have moved away from the desktop drives and have gone to all NAS drives. I have both WD & HGST

 

RAID arrays have a very strict timing requirement on drives. If they don't respond in a very short period of time, the RAID is going to kick them. NAS drives are specially designed to ALWAYS respond in that time period. I think that it could means potentially less effective error recovery, but never seen that written anywhere. I am not sure that the mechanical parts are necessarily of higher quality or longer lasting, but that may also be true. That would be more a factor of mean time to failure. The NAS feature is sometimes abbreviated TLER.

 

See THS POST for a pretty good explanation. Note that unRAID is not RAID, and the TLER feature is not needed. 

  • Upvote 1
Link to comment

Awesome! I'll give that a read!

 

My use-case is primarily media streaming, and as a repository for backups. I work from home part of the time so I like to keep a backup on my server - internet bandwidth here is at a premium, so I can't use cloud storage solutions (due to bandwidth constraints). As a result, an on-prem solution is better for me.

 

Out of curiosity, it deviates slightly from this thread, but is somewhat related in a way. I noticed Lime Tech specifically tell you not to use SSD's as primary storage/parity. Why is this?

 

I mean common sense would tell me that it probably has to do with the fact that parity is probably very disk IO intensive, and you're likely to wear out the NAND on the SSD way more quickly than (for example) what a mechanical drive like a NAS drive is likely to die from. Is this it?

 

Apart from that, it simply wouldn't be cost-effective to use SSDs purely from a capacity POV. They are still too expensive per GB compared to spinning disks.

Link to comment
3 minutes ago, Alcor said:

Out of curiosity, it deviates slightly from this thread, but is somewhat related in a way. I noticed Lime Tech specifically tell you not to use SSD's as primary storage/parity. Why is this?

This is because of uncertainty with how some SSDs may handle unused areas of storage. Parity calculation assumes nothing going on behind the scenes that might change bits on a disk outside of the control of unRAID. I know some people have SSD arrays with no problems.

  • Upvote 1
Link to comment

@trurl - I see! Okay, that makes sense... you're talking about the drive wear leveling tech in the firmware. So the physical cell the parity data gets written to could be reassigned internally by the drive based on wear-levelling patterns whereas, on a spinning mechanical disk, it knows exactly where on the platter it will be written to since the drive doesn't do any "wear leveling".

 

Interesting! Okay - but then again, the sheer cost involved for some of us mere mortals would make it prohibitively expensive anyway. It'd be awesome from a performance standpoint though!

Link to comment

My understanding is that generally speaking - NAS drives are built with additional vibration dampening, sturdier construction and additional magnets for the RW armature to make it more stable/reliable.

 

They are also designed for 24/7 workloads whereas typical desktop drives are expected (on average) to operate 8 hours a day.

 

Perhaps that has something to do with it? I never power my gear down - it's always on. I can't really speak to the reliability (at least my experiences thereof) WRT NAS drives... the WD Red 3 TB I have is the first one I've used in a home-server environment. The rest have always been desktop-class drives.

Link to comment
On 6/14/2017 at 9:14 AM, Alcor said:

Awesome! I'll give that a read!

 

My use-case is primarily media streaming, and as a repository for backups. I work from home part of the time so I like to keep a backup on my server - internet bandwidth here is at a premium, so I can't use cloud storage solutions (due to bandwidth constraints). As a result, an on-prem solution is better for me.

 

Out of curiosity, it deviates slightly from this thread, but is somewhat related in a way. I noticed Lime Tech specifically tell you not to use SSD's as primary storage/parity. Why is this?

 

I mean common sense would tell me that it probably has to do with the fact that parity is probably very disk IO intensive, and you're likely to wear out the NAND on the SSD way more quickly than (for example) what a mechanical drive like a NAS drive is likely to die from. Is this it?

 

Apart from that, it simply wouldn't be cost-effective to use SSDs purely from a capacity POV. They are still too expensive per GB compared to spinning disks.

 

I think that your use cases need to be more specific.

 

Here are some use cases and how to achieve: 

 

- I want single write streams to my array to go as fast as possible. - use turbo write

- I want write streams to go to the server as quickly as possible. I am ok if they go to the array later. - cache writes on SSD

- I plan on doing a lot of parallel writes - cache writes on SSD

- I want to optimize DLing of media content - Use SSD to download and repair, and final media extracted to array disk (configure in DL program)

- I want my drives to avoid drive spinning up unless they are being directly read or written.- default unRaid writes or cached writes

- I want my read performance to be optimized when writes are occurring in the background - default unRaid writes or cached writes

 

List any others and i can maybe give some ideas to optimize ...

  • Upvote 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.