Upgrade disk but other disk failed during rebuild. How to recover?

[fuzzby]

I have 10 disks in my array, 8 of them data disks. 

I upgraded Disk10 from 5TB to 8TB last night but during the rebuild another disk, Disk7 started spitting out 400Million errors. The rebuild finished but Disk7 was kicked out of the array and I am running in emulated mode. Furthermore, the data on the new Disk10 is corrupted.

 

Can I reinsert my old Disk10 back into my array to affect a recovery of Disk7? Would I use NEWCONFIG after putting in my original Disk10? How would I indicate in the newconfig that Disk7 is bad? Or can I set a newconfig with a missing disk7?

 

Thanks for taking the time to read this.

Edited November 21, 2017 by fuzzby

[JorgeB]

Furthermore, the data on the new Disk10 is corrupted.

That's expected

 

Can I reinsert my old Disk10 back into my array to affect a recovery of Disk7?

Yes if the array data was unchanged since the upgrade, this includes VMs and/or docker being off if they use the array, and for best results you'd need a same size spare disk to replace disk7.

 

 

[fuzzby]

Thanks for your answer. Is there anything special I have to do after newconfig? Or will the rebuild happen automatically?

 

Also I do have changes since the upgrade but not many. This should be okay with some corruption of newer files?

[bonienl]

16 minutes ago, fuzzby said:

Is there anything special I have to do after newconfig?

 

DO NOT USE NEWCONFIG, it will make it impossible to rebuild your disk. Place your original disk 10 back.

 

johnnie.black is way smarter than me

 

Edited November 21, 2017 by bonienl

[JorgeB]

1 minute ago, fuzzby said:

Also I do have changes since the upgrade but not many.

 

This will result on on some or a lot of files being rebuilt corrupt, and not necessarily newer files, impossible to know which ones unless you have checksums.

 

If you still want to proceed follow this, if not on latest unRAID update to v6.3.5 as you don't mention what release you're on and some older ones have issues with the trust parity procedure.

 

-Tools -> New Config -> Retain current configuration: All -> Apply
-assign any missing disk(s) including the new disk7
-check both "parity is already valid" and "maintenance mode" before starting the array
-start the array
-stop array, unassign disk7
-start array, check emulated disk7 mounts and contents look correct
-stop array, reassign disk
-start array to begin rebuild

[JorgeB]

28 minutes ago, fuzzby said:

I have 10 disks in my array, 8 of them data disks. 

Before doing the above, by this do you mean you have dual parity?

[fuzzby]

20 minutes ago, johnnie.black said:

Before doing the above, by this do you mean you have dual parity?

Sorry, to be clear I have two cache disks in mirror mode. I only have one parity

[fuzzby]

23 minutes ago, johnnie.black said:

 

This will result on on some or a lot of files being rebuilt corrupt, and not necessarily newer files, impossible to know which ones unless you have checksums.

 

If you still want to proceed follow this, if not on latest unRAID update to v6.3.5 as you don't mention what release you're on and some older ones have issues with the trust parity procedure.

 

-Tools -> New Config -> Retain current configuration: All -> Apply
-assign any missing disk(s) including the new disk7
-check both "parity is already valid" and "maintenance mode" before starting the array
-start the array
-stop array, unassign disk7
-start array, check emulated disk7 mounts and contents look correct
-stop array, reassign disk
-start array to begin rebuild

Thank you for these instructions! 

[JorgeB]

Sorry, to be clear I have two cache disks in mirror mode. I only have one parity

 

OK, so, and if disk7 is really bad, i.e., it didin't drop out because of a dodgy cable or similar, the above is your best option, still if the array data changed there will be some corruption on the the rebuilt disk, maybe you can also use the old one to recover some data.

 

 

[fuzzby]

19 minutes ago, johnnie.black said:

 

OK, so, and if disk7 is really bad, i.e., it didin't drop out because of a dodgy cable or similar, the above is your best option, still if the array data changed there will be some corruption on the the rebuilt disk, maybe you can also use the old one to recover some data.

 

 

Thank you thank you thank you! It's always reassuring to discuss a battle plan with a combat vet!