Possible to recover from 2 drive failure?


Recommended Posts

I relocated my old Unraid server that had several disks as old as 2011 with over 20,000 hours on them. The new location was warmer then the old location and I had two drives fail. One completely dead. The other shows life on smart but red ball on main page. I added a new drive to replace completely dead one but cannot start array. Any way to get the second drive with some life back online? 

 

 

fail unraid.jpg

Edited by kciaccio
Link to comment

You are running a very old version and you have actually posted up in the thread from version 6+.  But, folks will probably try to help you out.  Couple of Questions that are intended to get a bit more information.  Did the server ever function properly after the move?  (Reason for question is that the move may have disturbed the connectors causing the problem.)  What do you mean by "some life"?

Link to comment

 

Thanks for responding...

 

It did start fine after the move but I believe a day or two of the higher temps finished off the older drives. The dead drive had no temp readings on the UNRAID GUI but running smart tests on disk two shows temps and no real failures other than pre failures due to age limits.. 

Link to comment

Read this thread.

It contains a procedure to reset the list of drives to allow rebuilding a specific drive.

 

DO NOT CARRY OUT ANYTHING IN THAT THREAD WITHOUT POSTING YOUR GAMEPLAN HERE FOR REVIEW FIRST!

 

The procedure in that thread was laid out for that specific user and situation, with a slightly different version of unraid, and different circumstances, but the general principles should apply here.

 

After you read the thread if you feel comfortable proceeding, then post your planned list of actions and commands you think you should use. It may be wise to get @limetech involved to verify that what you are planning to do is the best possible plan for your specific issue.

 

If you don't understand what the commands do and why they are laid out the way they are, don't try to muddle through. Ask for more help. Your data is at risk.

  • Upvote 1
Link to comment

If one of the disks is dead and not being recognized, he will not be able to get his configuration re-established. Hence my question about having a backup of the config directory.

 

If not, I have a couple ideas how we might be able to reestablish it.

 

The key is that the second failed disk has to be "good". If that disk is failed also, there is no hope to restore the failed disk.

Link to comment
11 minutes ago, bjp999 said:

If one of the disks is dead and not being recognized, he will not be able to get his configuration re-established. Hence my question about having a backup of the config directory.

 

If not, I have a couple ideas how we might be able to reestablish it.

 

The key is that the second failed disk has to be "good". If that disk is failed also, there is no hope to restore the failed disk.

I think the second drive has life. I just don't know how to get it to re-establish into raid alive again at least to rebuild the totally dead drive. 

Link to comment

The system is showing Drive 1 as the "wrong" drive and is showing Drive 2 as red-balled/failed.   The "before failure" picture shows a lot of errors with Disk 3.

 

So ... WHICH drive(s) had red-balled BEFORE you decided to replace a drive?    If you had 2 red-balled disks, then it's very unlikely you can rebuild either of them unless one is actually good and was just a cabling issue ... but from what you've shown above it's not at all clear which disks had failed and which one you're referring to as potentially still good.    In addition, unless you have a backup of the exact flash drive contents from before the failure, it's unlikely you can reconstitute the array to a position that will allow you to attempt a rebuild.

 

A few thoughts/comments ...

 

(a)  Do you have backups of your data?    If so, then I'd simply build a new array using your good disks and the newest version of UnRAID; and then restore the data that was on the failed disks from your backups.

 

(b)  Do you KNOW for certain which disks were marked as failed (red-balled) ??    If so, which of those is the one you THINK is good?

 

(c)  Can you read the data from either of the failed disks if you attach them to a SATA port on a Windows PC and try to read them using the free LinuxReader utility?

https://www.diskinternals.com/linux-reader/      If you don't have another PC to attach them to, there's another way to check this, but it's a bit more involved.   [Basically you just use a DIFFERENT flash drive to install the trial version of UnRAID (or the free version from an older UnRAID version); and then create a 2-drive, no parity UnRAID system with just the two disks you're checking => then see if you can read the files across your network from either of those disks.]

 

The most important thing is to absolute not do ANYTHING that WRITES to your array (or either of the disks you're testing, if you connect them to another PC).    As long as you don't do any writes, then there is a (small) chance that you could rebuild ONE of them if the other one is readable ... IF you are able to reconstitute the array as it was before the failure (but without a backup of the flash drive this is unlikely).

 

Link to comment
11 hours ago, garycase said:

The system is showing Drive 1 as the "wrong" drive and is showing Drive 2 as red-balled/failed.   The "before failure" picture shows a lot of errors with Disk 3.

 

So ... WHICH drive(s) had red-balled BEFORE you decided to replace a drive?    If you had 2 red-balled disks, then it's very unlikely you can rebuild either of them unless one is actually good and was just a cabling issue ... but from what you've shown above it's not at all clear which disks had failed and which one you're referring to as potentially still good.    In addition, unless you have a backup of the exact flash drive contents from before the failure, it's unlikely you can reconstitute the array to a position that will allow you to attempt a rebuild.

 

A few thoughts/comments ...

 

(a)  Do you have backups of your data?    If so, then I'd simply build a new array using your good disks and the newest version of UnRAID; and then restore the data that was on the failed disks from your backups.

 

(b)  Do you KNOW for certain which disks were marked as failed (red-balled) ??    If so, which of those is the one you THINK is good?

 

(c)  Can you read the data from either of the failed disks if you attach them to a SATA port on a Windows PC and try to read them using the free LinuxReader utility?

https://www.diskinternals.com/linux-reader/      If you don't have another PC to attach them to, there's another way to check this, but it's a bit more involved.   [Basically you just use a DIFFERENT flash drive to install the trial version of UnRAID (or the free version from an older UnRAID version); and then create a 2-drive, no parity UnRAID system with just the two disks you're checking => then see if you can read the files across your network from either of those disks.]

 

The most important thing is to absolute not do ANYTHING that WRITES to your array (or either of the disks you're testing, if you connect them to another PC).    As long as you don't do any writes, then there is a (small) chance that you could rebuild ONE of them if the other one is readable ... IF you are able to reconstitute the array as it was before the failure (but without a backup of the flash drive this is unlikely).

 

Drive 2 failed first. I have a second server that I built a year ago that I did windows 8 storage because it just flowed better for a plex server. It has about 80% of what I had on my UNRAID server.  
I will see if I can read from either of the drives..Thanks for the program!

 

Just used your program an I can see and play the files on the one disk. 

Edited by kciaccio
Link to comment

The rebuild is going a bit slow, but as long as it stays consistent I'd just let it finish and hope all works out okay.   One thing you COULD have done [Actually still could, but I'd be inclined not to use the array during the rebuild], was to "look" at the contents of disk2 and see if they looked okay.   You wouldn't REALLY be looking at disk2 ... you'd see the emulated contents reconstructed from all of the other disks ... in other words exactly what's now being written to the new disk as the system rebuilds disk2.

 

But since it's already underway, I'd just wait a day or two until it finishes - then look at the data and see if it seems okay.   If you have backups of SOME of the files, I'd do a binary comparison of those to confirm they're okay -- if so, that's a fairly good indication that you have a successful rebuild.

 

Link to comment

When you start an array with one disk down, that disk is emulated (by parity and all of the other disks). But from user perspective, that disk is present. You can see all of the files as though the disk were present. In theory you can ever write to the array (although I do not recommend it). At the very least, I always recommend at least checking this before doing a rebuild. Because what is getting emulated is what will be rebuilt. And if it is garbage, there is no need to proceed.

 

But for older arrays especially, instead of doing a rebuild, I recommend the user copy the files that are most important to a safe location, and gradually move to the lesser important files. The benefit of this is that, as files are copied, you know that those files are completely restored. With a disk rebuild, once it completes successfully, the entire volume would be present and all files intact. But if one of the other disks in the array were to fail in the middle, you'd have a half reconstructed disk. Since unRAID rebuilds sector by sector, with no concept of a "file", you'd have no way to know which files were completely restored, which were partially restored, and which were fully restored. It would be take forever to sort out with a bunch of content.

 

Fingers crossed for you! With a little luck all will restore and you'll be in good shape. I posted the above mostly for future readers to understand (what I think is a better) alternative.

 

Good luck!

Link to comment

I would not only have copied the files from the emulated disk; but also from the "good" disk that could be "seen" okay with LinuxReader ... since, based on the earlier "failure" of that drive; that's probably the most likely drive to fail during the rebuild.

 

But as I noted above, since the rebuild is already in progress, I'd simply let it proceed -- no need to add any additional "stress" on any of these drives.

 

  • Upvote 1
Link to comment

Thanks guys. Just put in an order to replace the drives that I have had since 2011 in my UNRAID..They have done very well. Time to retire them. There will be lots of rebuilding over the next couple weeks. 

 

 

Edited by kciaccio
Link to comment
22 minutes ago, kciaccio said:

There will be lots of rebuilding over the next couple weeks. 

And upgrade unRAID!

 

Most of us have never used the version you are on, and those that have barely remember it. Even if you don't think you need the new features for adding applications to your server, V6 is a much better NAS.

 

And we can better support it.

Link to comment
3 hours ago, trurl said:

And upgrade unRAID!

 

Most of us have never used the version you are on, and those that have barely remember it. Even if you don't think you need the new features for adding applications to your server, V6 is a much better NAS.

 

And we can better support it.

 

I am one of the few that did use this version, but memory is fuzzy!

 

Realize that until version 5, drives over 2T were not recognized by unRAID. So do not try to use larger drives until you upgrade.

Link to comment

I only upgraded my last remaining v4 server last year -- quite frankly as a NAS it was every bit as good as the newer version (although you did need to install Joe-L's excellent UnMenu to add a few features that are now standard).   The compelling reason to move beyond it was support for drives < 2TB ... and the compelling reason to move from v5 to v6 was dual parity.   In addition, v6 adds Docker and VM support, which are also excellent features IF you are using your server as more than a NAS.

 

But I absolutely agree you should move to v6 once you get the rebuild completed and before you start replacing your drives.   If you haven't already placed the order for the replacements, you should consider moving to larger drives -- although as already noted this will require upgrading UnRAID first; so that's the FIRST thing I'd do after the rebuild is completed and you've done a confirming parity check to ensure it was error-free.

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.