4 TB Red Drive & formatting question


Recommended Posts

Any thoughts good or bad about WD Red series?

 

https://www.newegg.com/Product/Product.aspx?Item=N82E16822236599&utm_medium=Email&utm_source=IGNEFL042817&cm_mmc=EMC-IGNEFL042817-_-EMC-042817-Index-_-DesktopInternalHardDrives-_-22236599-S0C&ignorebbr=1

 

I have 3 drives left in my array that are 8 and 9 years old.  They are slow and noisy but still dont show danger signs of sector re-allocation (probably time to run a smart test again).  I'm thinking 8/9 years is long enough though.  

 

At the price for those 4TB drives, even though I dont need the extra space right now I'm thinking of replacing my parity drives (both 3 TB) and rolling those down into the array.  Thus any future purchases can be 4 TB.  Of course the rub here is, once I swap those three out, everything is a 3TB (save one 2TB remaining) .. and swapping out 3's for 4's in the future is a very inefficient way to increase space.   Meaning maybe I should buy 5's or 6's for parity?

 

Is there any way to format a 4TB (or 5 / 6) drive as a 3TB so for now parity checks don't take the extra hours to run through dead space?

 

 

Link to comment

those drives are a yes.

 

33 minutes ago, TODDLT said:

Is there any way to format a 4TB (or 5 / 6) drive as a 3TB so for now parity checks don't take the extra hours to run through dead space?

 

This is a no. (I think.)

 

 

You should assess how many disk slots you have and how much space you anticipate needing in the future. If you think you're going to double your array size in 2 years or so, then maybe go bigger. Worse case is you get 4TB drives, and in 2 years decide you need bigger, and sell them for half what you paid.

Link to comment

WD Reds are great for unRAID. I was in the same situation as you not long ago, trying to decide if I should get the 4TB's. After finding all these deals on the 8TB models for ~$180-$200 I couldn't help but go for these. It will really help hold me over for a long time as well since I'm only working with 12 bays.

Link to comment

YES there is! Create an HPA. Look a the post below. I did this in the old days before unRAID supported drives larger than 2.2T. I bought some 3T drives and downsized them by creating a 0.8T HPA. See the thread below for the details of what I did. Recommend using a motherboard port to create the HPA. Once created, any controller will respect it. Post if questions.

 

You'll need to so some calculations to determine how large to make the HPA for your needs.

 

 

Link to comment

The whole point of unraid is being able to have different sized disks in the array.
Short stroking your parity seems like a crazy idea to me, as it will only limit your ability to grow the array in future.

If only your parity drives are 4tb and the data drives are 3tb or less, the 4th terabyte will be incredibly quick to check, as there will be no checksums there needing testing

Backblaze testing can't really be taken as church either, as they way they run the disks is far from their optimal environmentals. I only agree with them with regards to the 1.5tb seagates which were horrible


Sent from my iPhone using Tapatalk

Link to comment
19 minutes ago, miniwalks said:

If only your parity drives are 4tb and the data drives are 3tb or less, the 4th terabyte will be incredibly quick to check, as there will be no checksums there needing testing

 

I only wish this was true.  If you look at my Test Bed server spec's, I can explain that statement.  Recently, I did an experiment to answer a question for a user who want to remove his parity 2 drive.  What I did was to remove the parity 2 drive to verify what I told him.  I then wanted to reinstall the Parity 2 drive.  It took 2 hours and 44 minutes to rebuilt the parity 2 drive2 as it only had to write parity to that first TB.  The last non-correcting parity check on the entire array took 7 hours and 49 minutes as it reads the last 2TB of the Parity 1 to verify that all of those sectors can be read and that they contain zeros.  The parity calculations don't take a lot of CPU time with modern hardware.  The speed of the disks (and, sometimes, add-in SATA controllers) are the bottleneck. 

Link to comment

Now that's interesting, as I've tested this as well with 8tb parity disk and a mix of 3 and 4tb data drives, and the parity check only takes as long as it used to with a 4tb parity drive.

We're the drives precleared before putting into Parity?
If so, does a subsequent parity check run quicker?

I can't see a signature with any test server specs :-(


Sent from my iPhone using Tapatalk

Link to comment
3 hours ago, TODDLT said:

Thus any future purchases can be 4 TB.  Of course the rub here is, once I swap those three out, everything is a 3TB (save one 2TB remaining) .. and swapping out 3's for 4's in the future is a very inefficient way to increase space.   Meaning maybe I should buy 5's or 6's for parity?

 

Stop and do a money calculation at this point right now before you buy anything.  If you go to a 4TB parity drive, it will cost you approximately $135 for each additional TB of storage.  IF you go to a 6TB drive, each additional TB will drop to about $63 per TB and with a 8TB drive to about $55 per TB.  (I am ignoring the saving presently available by shucking external 8TB drives.)

Link to comment
1 minute ago, miniwalks said:

Either way I find it odd that it was checking for 0's if the disk was precleared before being added back to the array

 

The 3TB disk was being NOT rebuilt, it was the time for the last monthly non-correcting parity check that I quoted.  Only the 1TB parity 2 disk was rebuilt when it was added back in and the time quoted is for that portion.  (Parity 1 was not involved (didn't spin up) in that rebuilt!  I did not remove both parity disks for the experiment!)

Link to comment

I have an Antec 1200 case with 12 bays.  MY SSD's are in expansion card slots so they don't take up full drive bay space.  Thanks to Frank for pointing out that I could turn my signature back on :).  The drive listing is current.

 

I have 2 parity drives and 8 spinning data drives.  Leaves me 2 empty bays (one of which currently holds a spare).  

I have 13.3 TB used of 20.5 TB total space.. 65% full, so not in any danger of "needing" space having 7.2 TB free

There are 4 - 2 TB drives in the mix.  3 of them are 8-9 years old.

 

Scenario 1:

7.2 TB currently free expands to 11.2 TB if i just replace the remaining 2 TB drives with 3's.  

11.2 TB goes to 17.2 TB by filling in those last two spaces with more 3 - TB drives.

(this is at LEAST 5 years of storage growth)

That scenario buys me 10 TB of free space over what I currently have and would mean buying 6 - 3 TB drives at a cost of roughly $100 / 3 TB drive...  $600 for roughly 10 TB of added space.  $60 / TB

 

Scenario 2:

I swap the two parities out now with 4 TB drives, and used those 3 TB drives plus my warm spare to replace the 3 older 2 TB drives.  Then buy 4 TB to replace the last one plus filling in the two empty bays.  That nets me 13 TB of added space above my current capacity and costs me 5 drives at $135 each.  Then for the sake of keeping these scenarios equal i have to replace the spare...  another $135.  $810 or $62 per TB

 

I guess my thinking is trying to predict what the cost effective storage technology is going to be in 5 years is a little gray and hazy :), so not sure over investing in anything makes sense.  If Scenario 1 will last me 5 years or more (conservatively)...   wouldn't it make sense to ride it out and see what changes?  

 

I may have just talked myself into staying with 3 TB drives for now.. .but listening to opinions or thoughts.

 

**side note**.. I think I paid about $70 for 2 TB drive 8/9 years ago... so the cost per TB hasn't changed much.

 

Thanks.

Link to comment
9 hours ago, miniwalks said:

The whole point of unraid is being able to have different sized disks in the array.
Short stroking your parity seems like a crazy idea to me, as it will only limit your ability to grow the array in future.

If only your parity drives are 4tb and the data drives are 3tb or less, the 4th terabyte will be incredibly quick to check, as there will be no checksums there needing testing

Backblaze testing can't really be taken as church either, as they way they run the disks is far from their optimal environmentals. I only agree with them with regards to the 1.5tb seagates which were horrible


Sent from my iPhone using Tapatalk

 

An HPA can be created and it can be removed. There it's no permanentcy to the procedure. UnRaid does continue parity checks beyond the size of data disks, and although it reads at full speed, a terabyte takes time. If one has an 8T parity and largest data is 4, this makes a ton of sense. If the parity is 4T, with largest data of 3, not as much.

 

It does have the negative of requiring parity to be rebuilt to add a larger disk. That is the best reason not to IMO.

 

But the experience of doing it is satisfying, and one might learn something in the process! 

 

Cheers! 

Link to comment
 
An HPA can be created and it can be removed. There it's no permanentcy to the procedure.
 
But the experience of doing it is satisfying, and one might learn something in the process! 
 
Cheers! 


Oh I agree whole heartedly, I've done storage engineering as part of my background,

In the days before flash/ssd it was incredibly common to short stroke disks for those DBA types, such a waste of disk and real estate, but prior to flash, couldn't do much about it



Sent from my iPhone using Tapatalk
Link to comment
8 hours ago, miniwalks said:

Now that's interesting, as I've tested this as well with 8tb parity disk and a mix of 3 and 4tb data drives, and the parity check only takes as long as it used to with a 4tb parity drive.

 

This is just not possible, or your old parity was very sick.

Link to comment

In my opinion, hard disk technology is now a a mature technology.  There aren't going to be a lot of great leaps forward that result in lower costs in future years.  Furthermore, the demand for hard disks is either stable or declining.  (Result of smart phones and tablets)  So the cost per TB will not decline significantly and, in fact, may even increase.  You seem to be in the same situation that I am in.  You current setup has enough capacity (current installed and potentially installable) for the foreseeable future.  

 

One thing I will point out is that it appears that larger hard disks appear to have no higher failure rates than smaller hard disks.  (Seems counter intuitive but that seems to be one thing that the BackBlaze data does show.)  I personally would be waiting until I got a bit closer to being full in your current parity size situation.  If a small disk looks flakey (Remember those older smaller disks are in the very bottom of that bathtub curve and aren't likely to fail at this point in their lives), and if,at that point, you are looking at running out of storage space, then buy a 6-10TB parity drive and do a parity swap.  At that point, you can then replace any future data drives with the larger drives.   (In fact, you could actually copy the data off any future failing small drive, storing it on one of your existing larger drives and pull that drive from the array.)  My opinion is that you want to make the great leap forward-- not a baby step-- in increasing the size of parity and then only because your other options are limited. 

Link to comment
1 hour ago, Frank1940 said:

In my opinion, hard disk technology is now a a mature technology.  There aren't going to be a lot of great leaps forward that result in lower costs in future years.  Furthermore, the demand for hard disks is either stable or declining.  (Result of smart phones and tablets)  So the cost per TB will not decline significantly and, in fact, may even increase.  You seem to be in the same situation that I am in.  You current setup has enough capacity (current installed and potentially installable) for the foreseeable future.  

 

One thing I will point out is that it appears that larger hard disks appear to have no higher failure rates than smaller hard disks.  (Seems counter intuitive but that seems to be one thing that the BackBlaze data does show.)  I personally would be waiting until I got a bit closer to being full in your current parity size situation.  If a small disk looks flakey (Remember those older smaller disks are in the very bottom of that bathtub curve and aren't likely to fail at this point in their lives), and if,at that point, you are looking at running out of storage space, then buy a 6-10TB parity drive and do a parity swap.  At that point, you can then replace any future data drives with the larger drives.   (In fact, you could actually copy the data off any future failing small drive, storing it on one of your existing larger drives and pull that drive from the array.)  My opinion is that you want to make the great leap forward-- not a baby step-- in increasing the size of parity and then only because your other options are limited. 

Remember "tape is dead", and yet the amount of tape grows each year.

 

The roadmap for incoming disk drive technology continues for many years into the future. Helium, SMR, TDMR, HAMR, BPMR, HDMR are technologies to increase density. Each costs time and money, and that needs to be recouped. Having a demand is key to these technologies fruition.

 

The demand for client HDD (sub 2TB) is fading so fast, best to think of it as gone. Client HDD still ship more than capacity HDD, but no new technology is needed. The capacity HDD is now the only source of funding for future HDD technologies/implementations. While capacity SSDs do exist, they are far from price competitive. This lifts the price pressure on capacity HDD. The loss of unit count from client HDD is rippled into cost per capacity.

 

Quote

“Because we want to make sure that we have got sufficient dollars to reinvest back into our business to continue to innovate and provide compelling products for our customers, we are making selective price increases in certain enterprise markets,” said Stephen Milligan, chief executive officer of Western Digital.

 

If/When the roadmap technologies become available in the market, pressure from competing tape and SSD will determine pricing.

 

While SSD and HDD density advances have fluctuated between 10% and 40% per year, tape has been remarkably stable at 30% per year.

Link to comment
3 hours ago, Frank1940 said:

There aren't going to be a lot of great leaps forward that result in lower costs in future years. 

 

I actually think that we saw pricing stagnation for a long period, and that its ending. 3T drives were under $100 and 4T in the ~$130 range 3-4 years ago, and we had not seen much reductions beyond those levels.

 

See this post from early 2015 ...

 

6T never really managed to have the price reductions associated with prior drive sizes (1T, 2T, 3T, 4T,) which each hit the market at a premium and rather quickly because the cheapest per T in 3-6 months after release, many reaching the $100 price point or very close.

 

It is only with the 8T drives that we've seen recent record setting lows in $/TB. You can buy 8T for as low as $180. That is $22.50/T. Previously prices just under $30/T were considered excellent. We have never seen prices this low! 

 

Although we have seen a loss of demand for desktop sized drives, replaced by SSDs, there is no doubt that the absolute demand for storage continues to increase. High capacity hard disks will continue to be in demand until SSD technology becomes much cheaper than its current levels. And SSD, remarkably, is more subject to wearing out with use than magnetic, and less able to implement parity protection. I see a healthy market for hard disks for some time to come. SSDs struggle to compete as the capacity increases, due to a rather linear costs to produce high capacities.

 

And with recent HD price declines, I'm thinking we're going off of 3-4 years of relative flatness / increases and hopefully prices will continue to reduce.

 

My experience from prior years is that drives gravitate to a floor price of about $100 / drive. The power supply, controller card, housing, shipping, etc. are pretty similar regardless of the capacity. It is only in the platters, heads, and associated parts that the technology improves. And better doesn't necessarily mean more expensive to produce, mostly R&D to get there. So I remain hopeful that current 8T drives will continue to gravitate towards the $100 floor, while we see new drive sizes of 10T, 12T, and higher becoming more commonplace as drive companies continue to defy the laws of physics and produce higher capacity drives.

 

I am more more optimistic today than I was in that earlier post from 2015 about lower costs for HDs in the short to medium term.

Link to comment
5 hours ago, bjp999 said:

 

I actually think that we saw pricing stagnation for a long period, and that its ending. 3T drives were under $100 and 4T in the ~$130 range 3-4 years ago, and we had not seen much reductions beyond those levels.

 

See this post from early 2015 ...

 

6T never really managed to have the price reductions associated with prior drive sizes (1T, 2T, 3T, 4T,) which each hit the market at a premium and rather quickly because the cheapest per T in 3-6 months after release, many reaching the $100 price point or very close.

 

It is only with the 8T drives that we've seen recent record setting lows in $/TB. You can buy 8T for as low as $180. That is $22.50/T. Previously prices just under $30/T were considered excellent. We have never seen prices this low! 

 

Although we have seen a loss of demand for desktop sized drives, replaced by SSDs, there is no doubt that the absolute demand for storage continues to increase. High capacity hard disks will continue to be in demand until SSD technology becomes much cheaper than its current levels. And SSD, remarkably, is more subject to wearing out with use than magnetic, and less able to implement parity protection. I see a healthy market for hard disks for some time to come. SSDs struggle to compete as the capacity increases, due to a rather linear costs to produce high capacities.

 

And with recent HD price declines, I'm thinking we're going off of 3-4 years of relative flatness / increases and hopefully prices will continue to reduce.

 

My experience from prior years is that drives gravitate to a floor price of about $100 / drive. The power supply, controller card, housing, shipping, etc. are pretty similar regardless of the capacity. It is only in the platters, heads, and associated parts that the technology improves. And better doesn't necessarily mean more expensive to produce, mostly R&D to get there. So I remain hopeful that current 8T drives will continue to gravitate towards the $100 floor, while we see new drive sizes of 10T, 12T, and higher becoming more commonplace as drive companies continue to defy the laws of physics and produce higher capacity drives.

 

I am more more optimistic today than I was in that earlier post from 2015 about lower costs for HDs in the short to medium term.

I agree with all of this. After picking up a bunch of WD Red 8TB drives for $180 I'm already ready for those new 14TB drives that are shipping to certain partners!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.