Proper way to move data to new drives.

[heffneil]

Hey guys I need to clear up some space on my unraid servers now that I added some 3tb drives.  Is there an official way of moving data from a smaller drive to a new one without screwing everything up and not rebuilding the array?

 

Thanks,

 

Neil

 

[BobPhoenix]

I use the built in MC command (Midnight Commander) to move files around on my unRAID server.  You can telnet in or use the console then just type MC to get it to come up. 

 

Navigate the left side to the directory you want to move to/from then tab and setup the right to the opposite (to or from) then highlight the file or directory you want to move and press F6 answer a few questions (I just tab through) and start move operation.  You can also select a group of files with '+' before pressing F6 to move them.  NOTE you might find my memory of the correct key presses not to match the actual ones needed but the screen display enough info that you should be able to figure it out if I'm wrong.

 

[dgaschk]

You may need to run the New Permissions script after moving the files.

[chickensoup]

Is MC any faster than just moving in Windows via disk shares? E.g. Move data from \\tower\disk1\TV to \\tower\disk7\TV (assuming both disk1 & disk7 belong to the "TV" share).

[dgaschk]

Is MC any faster than just moving in Windows via disk shares? E.g. Move data from \\tower\disk1\TV to \\tower\disk7\TV (assuming both disk1 & disk7 belong to the "TV" share).

 

It should be faster. Using a disk shares requires the data be transferred via the network, i.e., server->client->server. MC performs a local copy.

[heffneil]

Here is the question that I don't understand: if I moved data from one drive to another how does parity know what is going on? I wasn't sure if that is how files are to be moved...

 

Thanks,

 

Neil

 

[dgaschk]

Parity is updated when any content of the partition changes.

[Helmonder]

Parity is updated when any content of the partition changes.

 

This is a perfect good way to transfer files, even when moving and copying around on the server parity will be updated.

 

Copying and moving between disks thru windows is possible though this means that files are transferred firstly from the unraid system to your pc and then transferred back to unraid. It is enormously slower but it does work.

 

One thing to remember when using MC is that if you move between USER SHARES the system does not quite work. If you move FROM user share TO user share the settings you have made for included and excluded disks in the shares are not taken into account (resulting in files ending up on disks where you do not want them, typically they end up on the same disk you are copying from even if this disk should not be used for that specific user share.

[heffneil]

I am using MC and moving from Disk to Disk - not using the shares.

 

I think it is very slow. I have copied like 1 TB of data on each drive and it has been a little less than 24 hours.  I Would imagine speeds should be much greater.  MC says 15 MB/s I believe...

 

Any thoughts on these speeds? 

 

Question number 2. How do I make sure that these drives aren't used in the array when I write to the shares?  I would think with the way I have it configured the data would go there first?

 

Thanks!

 

Neil

 

[Helmonder]

Looks like the same speeds I am getting.. Remember that unraid is still calculating parity while copying, this adds to the overhead.. It does not bother me much, whenever I have to move large quantities of data I just give the command to MC using SCREEN and it will just run in the background..

 

What you could do I guess is utilize the cache drive in the process..

 

On the other hand; how much does it happen you have to transfer data of these quantities, for me that happens not more then once a month; whenever I decide I want to rebalance the data over the drives..

 

I would say that if you very regularly have to transfer +1TB amounts of data then I would reconsider using unraid for that part of your storage..

[chickensoup]

15MB/Sec sounds painfully slow.. I get 30-35MB/Sec copying through Windows via disk shares.

[Helmonder]

Using a cache drive ?

[heffneil]

I use cache drive but when copying directly from one drive to another cache isn't used. My parity drive is new and 7200 rpm so I have higher hopes for performance.  Still how do I no longer utilize the old drIves?

[chickensoup]

Using a cache drive ?

 

I have a cache drive but as heffneil noted, when copying between disk shares the cache drive isn't used. My regular copy speeds are more like 90MB/Sec. If i copy data from my PC directly to a disk share I usually see around 30-35MB/Sec, which is data being written directly to the array.

[dgaschk]

I use cache drive but when copying directly from one drive to another cache isn't used. My parity drive is new and 7200 rpm so I have higher hopes for performance.  Still how do I no longer utilize the old drIves?

 

Write speed to the array is limited by the slower of the data drive and parity. Do you want to remove the old drives?

[rutherford]

This operation should have a script or plugin.

[trurl]

This operation should have a script or plugin.

This thread is more than 4 years old and is for an obsolete version of unRAID. There are plugins and dockers that help with this now, but mc (Midnight Commander) is still the preferred tool of many since it is very lightweight and built-in.

[Mlatx]

Hi All,

 

I’m in a situation where I want to replace a drive with another exact drive that is slightly newer and use the other for usb purposes.  What is a current process for copying the drive after a preclear? Then once the data is copied, how do I make the new drive active and remove the old drive? Thanks.

 

 

[itimpi]

Co-ting is a manual process.

 

there is, however another way (assuming you have a parity drive) and you want the same files on the new drive.

• Stop the array
• Unassigned the drive you want to replace
• start the Array with no device assigned.    UnRAID will now be emulating the missing drive (as if the drive has failed).   UnRAID will at this point have ‘forgotten’ the serial number of the drive you removed.
• stop the array and assign the new drive you want to use.
• start the array to rebuild the ‘emulated’ drive onto the new drive.

keep the drive you have just replaced intact until the above has finished just in case any hiccups happen during the rebuild.

[trurl]

The unassign-start steps are only needed to rebuild to the same disk, no need to forget when replacing. To rebuild to a new disk just stop-replace-assign new disk to same slot. It is not even necessary to clear the replacement, though many do just for testing.

[itimpi]

3 hours ago, trurl said:

The unassign-start steps are only needed to rebuild to the same disk, no need to forget when replacing. To rebuild to a new disk just stop-replace-assign new disk to same slot. It is not even necessary to clear the replacement, though many do just for testing.

Not true in my experience.    You can on;ly omit those steps if a disk has failed and you are replacing it.   If you omit them when unRAID thinks the disk is healthy it will simply tell you that a disk is missing and not offer the rebuild option.  The whole purpose of those steps is to simulate a disk failing.

Edited November 19, 2017 by itimpi

[JorgeB]

1 hour ago, itimpi said:

5 hours ago, trurl said:

The unassign-start steps are only needed to rebuild to the same disk, no need to forget when replacing. To rebuild to a new disk just stop-replace-assign new disk to same slot. It is not even necessary to clear the replacement, though many do just for testing.

Not true in my experience.    You can on;ly omit those steps if a disk has failed and you are replacing it.   If you omit them when unRAID thinks the disk is healthy it will simply tell you that a disk is missing and not offer the rebuild option.  The whole purpose of those steps is to simulate a disk failing.

 

Trurl is correct.

[Mlatx]

trurl was correct.  It worked like a charm.  Thanks for your help.