Allow drive reassignments without requiring New Config

[RobJ]

The new Retain feature, part of New Config, is a great thing, makes it much easier to rearrange drive assignments.  But it's still a fairly heavy task, that causes risk and confusion to certain new or non-technical users.  It used to be much easier in earlier versions, as in 6.0 and 6.1, you could stop the array, then swap drive assignments or move drive assignments to different drive numbers/slots, without having to do New Config or being warned that parity would be erased, just start the array up with valid parity, so long as exactly the same set of drives were assigned.  It would really make life easier, and less confusing for some users, if we could return to that mode, when safe to do so.

 

An implementation suggestion to accomplish the above:  At start or when super.dat is loaded or at the stop of the array, collect all of the drive serial numbers (and model numbers too if desired), separated by line feeds, and save it as an unsorted blob (for P2).  Sort the blob (ordinary alphabetic sort) and save that as the sorted blob (for P1).  At any point thereafter, if there has been a New Config or there have been any drive assignment changes at all, before posting the warning about parity being erased, collect new blobs and compare with the saved ones.  If the sorted ones are different, then parity is not valid, erased warning should be displayed.  If the unsorted ones are different, then parity2 is no longer valid.  But if they are the same, then parity is still valid, and the messages and behavior can be adapted accordingly.  Sorting the blob puts them in a consistent order that removes the drive numbering and movement.  So long as it's exactly the same set of drives, the sorted blob will match, no matter how they have been moved around.

 

The really nice effect is that users won't unnecessarily see the "parity will be erased" warning, or have to click the "parity is valid" checkbox.  The array will just start like normal, even if a New Config had been performed (so long as the blobs matched).  You *almost* don't need the "Parity is already valid" checkbox any more.

 

The one complication to work around is that if they add a drive and unRAID clears it, or a Preclear signature is recognized on it, then the blobs won't match but parity is valid any way.  I *think* it's just a matter of where you place the test.  Or collect the blob without the new cleared disk, for comparison.

 

Edit: need to add that ANY drive assignment changes invalidates Parity2.

Edited March 30, 2017 by RobJ
add note

[itimpi]

You have left out the complications caused by dual parity as if using that any re-arrangements of the drives invalidates parity2.

[RobJ]

4 hours ago, itimpi said:

You have left out the complications caused by dual parity as if using that any re-arrangements of the drives invalidates parity2.

 

Ah, good point!  Can't have everything.  But where I'm running into this most is the wiki procedure for converting drives to XFS, and Parity2 is invalid any way, because of all the drive swapping and reassignment.

[Squid]

 
Ah, good point!  Can't have everything.  But where I'm running into this most is the wiki procedure for converting drives to XFS, and Parity2 is invalid any way, because of all the drive swapping and reassignment.

Forgive me for being daft here, but why the need to rearrange drives when converting to xfs.

Clear out space. Copy, format, copy, format. Parity remains valid at all times and nothing needs to ever be reassigned. I understand that you yourself don't utilize user shares so I know that can influence your decisions on the wiki but the wiki should be geared to simplicity. Half the problems with conversions would be solved by users doing it over the network instead of rsync (at the expense of speed)

NOTE: admittedly I have never read the wiki on conversion or followed the thread so I may be way out to lunch here and just over reaching based upon one comment of yours

Sent from my LG-D852 using Tapatalk

[trurl]

1 hour ago, Squid said:

Forgive me for being daft here, but why the need to rearrange drives when converting to xfs.

Clear out space. Copy, format, copy, format. Parity remains valid at all times and nothing needs to ever be reassigned. I understand that you yourself don't utilize user shares so I know that can influence your decisions on the wiki but the wiki should be geared to simplicity. Half the problems with conversions would be solved by users doing it over the network instead of rsync (at the expense of speed)

NOTE: admittedly I have never read the wiki on conversion or followed the thread so I may be way out to lunch here and just over reaching based upon one comment of yours

Sent from my LG-D852 using Tapatalk
 

There are plenty of opportunities for users to configure things in such a way that specific drive numbers are important, such as includes/excludes, docker volume mapping, etc. So if you move things around to empty a drive for reformatting, then files are no longer on the expected disks, and those configurations will no longer work as expected.

 

Of course, it is relatively easy to just adjust all the places where you may have specified a particular disk. and maybe that would be easier than swapping disks. But you can't have a general purpose wiki process that depends on different configuration details for different people. And people seem to need a general purpose process because they really don't understand how anything works, and that has caused some mistakes and anxiety. It has gotten to be a pretty long thread.

 

I've said more than once that if you understand parity, understand formatting, and understand user shares, then you can figure out how to approach the conversion for your specific situation.

[Squid]

38 minutes ago, trurl said:

There are plenty of opportunities for users to configure things in such a way that specific drive numbers are important, such as includes/excludes, docker volume mapping, etc. So if you move things around to empty a drive for reformatting, then files are no longer on the expected disks, and those configurations will no longer work as expected.

 

Of course, it is relatively easy to just adjust all the places where you may have specified a particular disk. and maybe that would be easier than swapping disks. But you can't have a general purpose wiki process that depends on different configuration details for different people. And people seem to need a general purpose process because they really don't understand how anything works, and that has caused some mistakes and anxiety. It has gotten to be a pretty long thread.

 

I've said more than once that if you understand parity, understand formatting, and understand user shares, then you can figure out how to approach the conversion for your specific situation.

  Rather than try and write a procedure that's suitable for all circumstances and necessitates a very complex procedure go for something that's simple that is very hard to mess up.  Vast majority of users use the share system and includes and excludes and not disk shares.

 

Give the users something that works that can't be messed up.  

 

Enable disk shares

Move disk1 to disk 2 over the network

Format disk 1

Continue.

Adjust include and excludes

 

How many users have messed up the Rsync? How many users don't know what screen is?  And now you're getting them to start reassigning  disk positions simply to avoid changing an include or exclude.   Sorry.  I don't buy the argument.   The point of that is to help users who aren't utilizing user shares which are very much in the minority now.

 

People are asking for instructions on how to use unbalance to accomplish the conversion because the entire process has been over thought and is needlessly complex..

 

We're here to help people accomplish the goal.  And that doesn't mean completely confusing the hell out of them in the process.   I'm very late to the debate but it seems to me that everyone has lost sight of what the goal is and is instead concentrating on accomplishing it in the fastest way possible which is also the most prone to mistakes.  

 

Once 6.4 is released odds on the tool that Tom has alluded to to help with this is going to be far more akin to what I'm talking about rather than the nightmare that it currently is.

 

I'm sorry  it just floored me seeing rob making a comment about reassigning disks.   Just throwing more fuel on the fire IMHO 

Edited March 31, 2017 by Squid

[Squid]

Finally got around to reading the wiki aND it does offer up different methods which don't require disk reassignment but very conspicuously missing is the simplest method that just about anyone can understand.

 

And now I'll let this topic get back on topic