Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)


Recommended Posts

Hey guys, I got my LSI SAS card to replace my SUPERMICRO AOC-SASLP-MV8 card. Is it just as simple as swapping the card out and booting the server ? 

 

I also have 2x 4TB drives I want to replace some smaller drives with and move over TO XFS, I am worried I am going to mess this up (the disks are precleared) I need to merge 2 smaller drives to (1X) 4TB drive

I am really worried about this process because I don't want to loose data 

Edited by bombz
Link to comment
5 hours ago, bombz said:

I also have 2x 4TB drives I want to replace some smaller drives with and move over TO XFS

 

Note these are two different processes.    To replace a smaller drive with a 4TB drive is straightforward -- but the resulting 4TB drive will have the SAME format as the drive it replaced (e.g. RFS).    I'd replace ONE of your smaller drives with a 4TB drive;  then move as much data as possible to that drive from other smaller drives (emptying some of those drives).

 

Then I'd do a parity check to be sure all is good; and then I'd do a New Config removing all of the empty drives and adding the other 4TB drive.    After the parity sync for the new config completes, do another parity check to confirm all went well.   Then change the format of the new (and EMPTY) 4TB drive to XFS;  then copy all of the data from the 1st 4TB drive to the XFS drive (verifying that it is a good copy); and then you can format the 1st drive to XFS (this will delete all the data on it -- but you just copied it to your other drive, so it's okay).

 

Now you'll have an empty 4TB drive in XFS format, so you can move all the data from a smaller RFS drive to it; then reformat the RFS drive to XFS; and then repeat that process until you've got all XFS drives.

 

VERY IMPORTANT NOTE:  Be CERTAIN you understand the "user share copy bug" and do NOT copy in a way that will cause that issue -- if you do, you'll lose ALL of the data associated with that copy.

 

Link to comment

If you have enough ports, and if you are planning to New Config/Parity Sync like garycase suggests, you could just:

 

ADD the new precleared disks to the array to new slots as XFS drives.

Copy the data from the drives you will be removing to the new drives.

New Config/Parity Sync without those old disks.

 

That would save a lot of the complication that is discussed throughout this thread, since you wouldn't need to reformat anything.

Link to comment
6 hours ago, trurl said:

That would save a lot of the complication that is discussed throughout this thread, since you wouldn't need to reformat anything.

 

It would indeed save SOME of the reformatting, but assuming you're not replacing ALL of your drives with the two new 4TB drives, you'd still have to reformat any drives that you were planning to leave in the array after you moved the data off of them.    But if you have the ports to do it, it would save the initial rebuild of one drive that would later have to be reformatted.

Link to comment
On 11/28/2017 at 9:30 PM, Frank1940 said:

 

Yes, IF the firmware has been upgrade to the IT mode.  

 

Guess I will have to learn how to do that. It is still in the box. I have been so busy I have been putting off this upgrade and my drives are super full :-( 
I will try my best to find info on how to flash the "IT MODE" firmware

then make 2 drives I want to remove 1 drive (without losing data) 

 

Link to comment
2 hours ago, bombz said:

Guess I will have to learn how to do that. It is still in the box. I have been so busy I have been putting off this upgrade and my drives are super full :-( 
I will try my best to find info on how to flash the "IT MODE" firmware

 

Post up the model number of the card.   If it is genuine LSI card, there is a possibility that it already has the IT mode firmware installed.  Most of the used cards are not in the IT mode but rather have the RAID firmware.  

Link to comment
23 hours ago, Frank1940 said:

 

Post up the model number of the card.   If it is genuine LSI card, there is a possibility that it already has the IT mode firmware installed.  Most of the used cards are not in the IT mode but rather have the RAID firmware.  

 

Thanks man!
LSI SAS9211-8i
Still fresh in the box sealed 

Link to comment
3 hours ago, bombz said:


Damit, that's the one the community told me to buy ! 
I'll have to try to figure out how to do this, if i do it on the server NAS or toss it in a spare workstation I have here 

 

It doesn't need crossflashing, that is only when flashing LSI firmware on other brand controllers, like Dell and IBM, and although it's not very complicated it can be in some hardware, it may need flashing if it comes with IR firmware, from IR to IT mode, but that's very easy, like a bios update.

Link to comment
10 hours ago, bombz said:

I'll have to try to figure out how to do this, if i do it on the server NAS or toss it in a spare workstation I have here 

 

You can start by looking at the two threads below or googling Flashing LSI SAS9211-8i to IT mode.

 

          https://forums.lime-technology.com/topic/11753-sata-controller-cards/

 

         https://forums.lime-technology.com/topic/12114-lsi-controller-fw-updates-irit-modes/

 

Thanks @johnnie.black  I didn't realize that the  procedure was easier for the genuine LSI  boards than for the OEM sourced ones.  

 

 

 

Link to comment
13 hours ago, bombz said:


Damit, that's the one the community told me to buy ! 
I'll have to try to figure out how to do this, if i do it on the server NAS or toss it in a spare workstation I have here 

4

I assume this is the process to V20 ?

 

https://nguvu.org/freenas/Convert-LSI-HBA-card-to-IT-mode/

Looks pretty straightforward :-) I am going to try to flash this (with this method) this weekend, then I can pull out the SuperMicro and add this in. 
Next weekend I will have to try merging 2 disks down to 1 disk, and learn that process (reconfiguring) 

Edited by bombz
Link to comment
5 hours ago, bombz said:

I assume this is the process to V20 ?

 

I didn't have to flash mine as I purchased it already flashed but the procedure that you linked to looks correct.  I would print it out and read  (and reread) it until I was really familiar with it.  Then I was was doing it, I would check off each step and double verify that the screen responses were actually as described before moving on to the next step.  

 

I do know that my card was flashed to the P20 software and it has functioned flawlessly.  

Link to comment
On 12/2/2017 at 5:28 PM, Frank1940 said:

 

I didn't have to flash mine as I purchased it already flashed but the procedure that you linked to looks correct.  I would print it out and read  (and reread) it until I was really familiar with it.  Then I was was doing it, I would check off each step and double verify that the screen responses were actually as described before moving on to the next step.  

 

I do know that my card was flashed to the P20 software and it has functioned flawlessly.  

 

OK, Rock on guys!
I was so busy this weekend I got none of my tech stuff out of the way! I REALLY appreciate the clarification and assistance. I am looking forward to flashing this and getting it installed. I probably will NOT see any performance difference until I (sigh) get over to XFS (I am not looking forward to the process) I don't even know where to begin.

I hope by the end of everything my UnRAID server is top notch and working excellent ! I have a long way to go

Link to comment
On 12/4/2017 at 9:50 AM, publicENEMY said:

Just wanna say that I have successfully converted all my drives from reiserfs to xfs. I have 15+1+1 drives. All except parity have been converted to xfs.

 

Thanks for a very good guide.

Congrats man!
I have a bit to go still before I get even STARTED this process. I hope I am as successful! I hope I see a performance difference too!

Link to comment

Just flashed LSI SAS921108i 
to
20.00.07.00-IT

So just to reconfirm, I should be able to TAKE NOTE of my current array settings (the disks) and swap the SuperMicro AOC-SASLP-MV8 for the LSI card, and be good to go?

 

Basically, these are the steps I have written down from my readings that may or may not be the correct method for my situation: (questionable aspects) 
 

01. Add LSI CARD (IT MODE) -- replacing supermicro 
02. Test LSI card for week(s)
03. Copy data off (2x) 500 GB HDD in array, to External Source HDD (via USB HDD Dock || USB-direct connection to UNRAID || Unassigned Devices Add-on ) 

 

04. this is where I need direction
a) do I, copy data from the array of the drives I am removing (2x) 500GB?
b) remove the 2 drives (at once), Run 'new config' Essentially telling unraid those (2x) 500GB are 'lost drives'?
c) then add new 4TB disk (pre-cleared) to the array, formatting it XFS?
d) Once 4TB is added and parity check is completed, copy external source data (2x) 500gb back to the array (new XFS 4TB disk)
e) run parity against new data written

04-1.
Could I remove (2x) 500GB
add the 4TB to the array

run 'new config'

use the USB dock attached to the Unraid server

copy data (2x) 500 GB back to the new 4TB (XFS)?

 

05. If above is correct:
I would then have physical space in case to add another parity disk (dual parity) as (2x) 500GB were removed (Step 04) 
a) would I want to complete conversion of ALL disks to XFS, and THEN add dual parity? 
b) would adding dual parity allow the array to become more 'fault tolerant' during the conversion process of all disks in the array?

 

06. Should I use the SuperMicro and the LSI together, which would eliminate all use of the onboard SATA bus and be pushing over PCI-E? 

 

07. Considering buying another LSI card, on IT mode, and have the 2 SAS controllers running the array

I apologize for needing clarification. What worries me is how I go about pulling (2x) 500GB out at the beginning, to start the process over to XFS file system as well as adding dual parity on the fly. A lot going on. I am trying to be 100% respectively.

Thank you 

Edited by bombz
Link to comment
On 12/5/2017 at 5:57 PM, bombz said:

Just flashed LSI SAS921108i 
to
20.00.07.00-IT

So just to reconfirm, I should be able to TAKE NOTE of my current array settings (the disks) and swap the SuperMicro AOC-SASLP-MV8 for the LSI card, and be good to go?

 

Basically, these are the steps I have written down from my readings that may or may not be the correct method for my situation: (questionable aspects) 
 

01. Add LSI CARD (IT MODE) -- replacing supermicro 
02. Test LSI card for week(s)
03. Copy data off (2x) 500 GB HDD in array, to External Source HDD (via USB HDD Dock || USB-direct connection to UNRAID || Unassigned Devices Add-on ) 

 

04. this is where I need direction
a) do I, copy data from the array of the drives I am removing (2x) 500GB?
b) remove the 2 drives (at once), Run 'new config' Essentially telling unraid those (2x) 500GB are 'lost drives'?
c) then add new 4TB disk (pre-cleared) to the array, formatting it XFS?
d) Once 4TB is added and parity check is completed, copy external source data (2x) 500gb back to the array (new XFS 4TB disk)
e) run parity against new data written

04-1.
Could I remove (2x) 500GB
add the 4TB to the array

run 'new config'

use the USB dock attached to the Unraid server

copy data (2x) 500 GB back to the new 4TB (XFS)?

 

05. If above is correct:
I would then have physical space in case to add another parity disk (dual parity) as (2x) 500GB were removed (Step 04) 
a) would I want to complete conversion of ALL disks to XFS, and THEN add dual parity? 
b) would adding dual parity allow the array to become more 'fault tolerant' during the conversion process of all disks in the array?

 

06. Should I use the SuperMicro and the LSI together, which would eliminate all use of the onboard SATA bus and be pushing over PCI-E? 

 

07. Considering buying another LSI card, on IT mode, and have the 2 SAS controllers running the array

I apologize for needing clarification. What worries me is how I go about pulling (2x) 500GB out at the beginning, to start the process over to XFS file system as well as adding dual parity on the fly. A lot going on. I am trying to be 100% respectively.

Thank you 


If anyone can fill me in, I think I have my process correct, it's mainly the removal of 2 drives to be replaced by 1 drive ... then moving over to XFS (full array) then adding dual parity.

Thanks :-) 

Link to comment

@bombz -

 

So it sounds like you are completely full in terms of drive slots. So you want to copy data from 2 500G drives to an external drives, and then add a 4T disk that you'll format as xfs. This sounds like a reasonable approach. It will require rebuilding parity to remove the 500G disks and incorporate the new 4T disk containing data.

 

Swapping add-on controllers is seamless. UnRaid recognizes drives by serial number. So if you remove the old, attach the drives to the new, and boot, all the drives will be assigned to their proper slots as though nothing had changed. (Obviously connecting everything securely and not disturbing the drive cabling of other drives is critical to not causing drives to drop offline).

 

Related to ports to avoid I am confused. The SuperMicro add on controllers that use the Marvell chipset are problematic, but motherboard ports that are provided from the Intel chipset or non-Marvell chipsets are fine to use. Your post made it sound like you were trying to avoid using regular motherboard ports but maybe I misunderstood.

 

I do suggest planning for an empty slot as it is useful in some recovery scenarios. And comes in handy for preclears as well. It is also needed to convert from rfs to xfs.

  • Like 1
Link to comment
10 hours ago, SSD said:

@bombz -

 

So it sounds like you are completely full in terms of drive slots. So you want to copy data from 2 500G drives to an external drives, and then add a 4T disk that you'll format as xfs. This sounds like a reasonable approach. It will require rebuilding parity to remove the 500G disks and incorporate the new 4T disk containing data.

 

Swapping add-on controllers is seamless. UnRaid recognizes drives by serial number. So if you remove the old, attach the drives to the new, and boot, all the drives will be assigned to their proper slots as though nothing had changed. (Obviously connecting everything securely and not disturbing the drive cabling of other drives is critical to not causing drives to drop offline).

 

Related to ports to avoid I am confused. The SuperMicro add on controllers that use the Marvell chipset are problematic, but motherboard ports that are provided from the Intel chipset or non-Marvell chipsets are fine to use. Your post made it sound like you were trying to avoid using regular motherboard ports but maybe I misunderstood.

 

I do suggest planning for an empty slot as it is useful in some recovery scenarios. And comes in handy for preclears as well. It is also needed to convert from rfs to xfs.


Awesome, thanks for clarifying some things there.
So basically I can take 2x 500 out (at once) as I will have that data on an external source. When I toss the 1x 4TB in, unraid will detect a missing disk. 
Disk 5 & 6 (removed)
Disk 5 replaced (no disk 6) 

I would format the new disk as XFS, and run a parity rebuild, but would the array allow me to do that, as disk 6 is not connected, or don't I have to run a 'new config' or something
first ?

As for the supermicro, I will remove it, and just have a backup SAS card I guess :-/ I will then purchase another LSI card and run the full array out (2x) SAS cards, with hopes my performance increases, it is terrible so to speak right now

 

Link to comment

@bombz - You will need to do a new config to remove the old drives and add the new one. Don't delete the data from the 2 500G drives, and you'll have a backup just in case you have any issues with the external.

 

I do recommend a preclear to test out any new drive, but technically you would not need to preclear it. When you did a new config, parity is rebuilt.

 

What you could do is this. It's what I would do. You don't need the external.

 

Preclear the new 4T drive and second parity disk (if you have). (Recommended but not mandatory)

 

Shutdown server. Remove one of your 2 500G drives from the server, and add your 4T drive.

 

Boot server. Do a new config but keep configuration. Add the 4T add as a data drive and omit the 2 500G drives from the array configuration. (I'd also omit parity, but that's up to you. Without parity you'll be able to copy faster).

 

Change filesystem to xfs on the new 4T drive before starting array.

 

Start the array. (Let parity build if you added parity.) Format the 4T drive.

 

Use unassigned devices and mount the 500G drive. Copy all the data from the 500G drive to the 4T.

 

After the copy, shutdown the server. Swap the 500G with the other one.

 

Boot the server, start a array, use UD to mount the other 500G, and copy other 500G drive to the 4T.

 

Shutdown the server again. Remove the second 500G. I think you said you wanted to add second parity drive. This would be the time. Otherwise just leave slot empty.

 

Boot the server. Before starting the array, add your parity disk to the config (if didn't do before), and your second parity if you are adding it now. Start the array and parity(ies) will build if you added.

  • Like 1
  • Upvote 1
Link to comment
On 12/8/2017 at 9:47 PM, SSD said:

@bombz - You will need to do a new config to remove the old drives and add the new one. Don't delete the data from the 2 500G drives, and you'll have a backup just in case you have any issues with the external.

 

I do recommend a preclear to test out any new drive, but technically you would not need to preclear it. When you did a new config, parity is rebuilt.

 

What you could do is this. It's what I would do. You don't need the external.

 

Preclear the new 4T drive and second parity disk (if you have). (Recommended but not mandatory)

 

Shutdown server. Remove one of your 2 500G drives from the server, and add your 4T drive.

 

Boot server. Do a new config but keep configuration. Add the 4T add as a data drive and omit the 2 500G drives from the array configuration. (I'd also omit parity, but that's up to you. Without parity you'll be able to copy faster).

 

Change filesystem to xfs on the new 4T drive before starting array.

 

Start the array. (Let parity build if you added parity.) Format the 4T drive.

 

Use unassigned devices and mount the 500G drive. Copy all the data from the 500G drive to the 4T.

 

After the copy, shutdown the server. Swap the 500G with the other one.

 

Boot the server, start a array, use UD to mount the other 500G, and copy other 500G drive to the 4T.

 

Shutdown the server again. Remove the second 500G. I think you said you wanted to add second parity drive. This would be the time. Otherwise just leave slot empty.

 

Boot the server. Before starting the array, add your parity disk to the config (if didn't do before), and your second parity if you are adding it now. Start the array and parity(ies) will build if you added.

8

You sir ... ARE AWESOME!

Sounds like a plan. I am pumped to get this going, I was confused on some of the process taking 2 drives out and the new config process. I have only added or replaced dead drives in the past.
Once I get these steps done, I am slowing going to move the whole array over to XFS, buy another LSI card, and I should be DONE for a while (until I can get a bigger case).

 

There is always something you can spend money on with UnRAID :-) 

I love the community here, I sincerely appreciate this. 

I will keep posted how it goes (I just need to find the time this holiday season with work and life etc. to get this done now)  

Thank you... EVERYONE ! 

Edited by bombz
Link to comment
  • 2 weeks later...
1 hour ago, pcgirl said:

Can anybody advise how to convert the Cache drive?  I have converted my other drives but not sure how to do the Cache drive.

Same as any other drive. Copy content to another location, stop array, change disk format, start array, select the checkbox beside the option to format the drive, make sure that specific drive is the only one listed as needing formatted, then select format. When that's done, copy the content back in place from wherever you put it.

 

Since this is the ReiserFS to XFS thread, I'll assume you have a single cache disk that you want to convert to XFS. If you select multiple cache slots to be visible, you will lose the ability to format the cache drive in anything but BTRFS, so don't do that.

 

It's also a good idea to be sure that your data copy method and destination support sparse files and symlinks, as there are likely to be a bunch of both things on a well used app / vm / cache drive.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.