Upgrade to unRAID 6 - XFS and Cache Disk Questions


Vibe

Recommended Posts

Hi everyone!

 

I'm a long-time unRAID user with a very stable v5.06 system who would like to upgrade to v6.3.5.  I am slow to adopt new technology, especially when my current system performs flawlessly for my needs (a testament to unRAID + Dynamix webGui).  I only use my system for streaming media files, and a single disk for a storage of work related files.  I have read a lot from the forum about the upgrade process (a huge "thank you" to all that share their wealth of knowledge) as well as through the Lime Tech website and Google searches.  However, I have a few questions/concerns that I am hoping to learn more about.

 

1.  I am using an APC battery backup for automated power down during electrical outages - it works great.  When I wasn't using this, my system would begin a parity check after an outage and the wait time was the only burden I experienced.  I have never had any issues with data corruption that I am aware of.  I presume that the ReiserFS was the main factor here.  I understand that the XFS file sytem is the default format for unRAID 6 and it is designed for large files and scalability.  After a bit of online searching I have read about some users (outside of unRAID) experiencing data corruption using XFS after power outages.  The unRAID Wiki also mentions that the recovery tools for XFS are not as mature as those for ReiserFS.  One site I read where a user experienced issues where files would appear to be available, yet files sizes were "0".  However, a lot of this information appeared to be several years old.  Has anyone experienced a power failure without any major issues and/or losses using XFS?  Are there any concerns or precautions regarding data corruption to be aware of?  Have there been any issues with reliability? I hope I don't appear to be paranoid - I've been blessed with an exceptionally reliable unRAID system for 8 years, and change can be "spooky". :)

 

2.  I use one of my disks to store work files (1,000's of small documents, HTML, PHP, etc.).  I have read that XFS doesn't perform as well with small files.  This may be outdated information as well so I am uncertain if this is still valid.  Would it be better (performance wise) to continue using ReiserFS for this particular disk?  Is the ReiserFS safe to use with unRAID 6? I read about the ReiserFS regression with earlier versions of Linux, and presume that this is no longer an issue - is this correct?

 

3.  I have not had a need to use a Cache disk/pool as the slower write speeds have not been a problem.  I also write files to individual disks rather than shares.  However I am very interested in trying out Dockers as it clearly makes application installation a breeze.  I have read where users suggest keeping Docker files on the Cache disk or Cache pool, e.g. outside the array.  Now for my "dumb" question.  Is it possible to utilize a Cache disk/pool only for Docker storage - and still maintain the ability to write directly to individual disks (e.g. /server/diskX) and bypass writing to disk shares?

 

My apology for my long post - and thank you to anyone who can shed some light on my questions.  I hope to return the favor!

Link to comment

There are some issues that have cropped with ReiserFS.  The first one is that the developer ( and driving force behind it) was convicted of murdering his wife in 2008 and is currently serving a 15 years-to-life term.  As a result, development has virtually ceased!  (In fact,a change that a volunteer (of a group who decided to maintain it)  made a year or so ago, could actually caused data corruption.  (It was quickly rolled back...)

 

The second problem is that it has a tendency to become very sluggish when the disks become full.  (This may be a result that when development stopped the biggest hard disk size was about 2TB.) 

 

There are others here who can better address some of the other issue that you have and I would encourage then to do so. 

 

Point of disclosure:  I have converted both of my servers from ReiserFS to XFS.  It took two to three days for each server and was not what I would call a trivial task.  I had a set of printed instructions for the method which I used and a table to help me keep track of where I was and what was the next step in the process.  (If you decide to do this, look up the command line editing features of the BASH shell.  It can really save you a lot of time and eliminate potential mistakes.)

Link to comment

One more thing.  Every unRAID server should have a UPS.  And, after installation of that UPS, it should be tested to make sure that it will do a clean shutdown.  After adding a Docker or plugin that can access the array, it should be tested again.  If a VM is installed, it should be tested.  Periodically (say every three months after the first year), it should be tested to make sure that the ever aging battery still has the capacity to handle a shutdown. 

Edited by Frank1940
Link to comment

Hi Frank1940 - I greatly appreciate the information.  I have read about the slowness with writes on full disks (ReiserFS) and have noticed this myself to a degree.  After reading about the developers murder conviction I can see why there would be limited interest to continue.  The bug that cropped up in the past was also very concerning - I would have been extremely frustrated had I made the upgrade at that time.

 

I too will probably make the full jump to XFS as I have the time and free drives.  I will definitely follow your example and keep track during each stage.  I am currently pre-clearing two 4TB hard drives on a "trial" system with that goal in mind.  Sadly one drive may have some "issues" as it is making a repetitive noise every 5 seconds or so, and is pre-reading at 40MB/sec slower than the other (both same HGST 4TB NAS).  I may be using Amazon's return policy.

 

I completely agree with your sentiment about a UPS.  After losing power to a storm a few years ago, the next 8 hours were a "nail-biter" when power was restored and my server was doing a parity check.  Luckily I was not writing to the disks when this happened - I'm not certain if that would have had a different outcome.  However, for the price and security - it is definitely a smart thing to do.  When I make any changes I will definitely follow your recommendations regarding testing the procedure.

 

Thanks again for guidance!

Link to comment
2 hours ago, Vibe said:

experiencing data corruption using XFS after power outages.

Because of how I do things / what I play with, I'm probably very harder on the array than most users (with regards to unclean shutdowns), and I have never once had file corruption / unmountable drives, etc using XFS.

29 minutes ago, Vibe said:

the next 8 hours were a "nail-biter" when power was restored and my server was doing a parity check

If the drive(s) come up and are mountable, then you're not going to experience any major problems.  Worst case scenario pretty much is a file or two that *may* be corrupted (if a parity check indicates errors).  All of your "If I lose this my wife will kill me" files should be backed up onto physical media / cloud / whatever.

1 hour ago, Frank1940 said:

Every unRAID server should have a UPS.

Every computer should have one.  Corruptions caused by power loss on not limited to unRaid, but affect every OS equally.

Link to comment

Thank you Squid for your reply - I greatly appreciate it.  Of all unRAID users, if you're fully confident with XFS, my concerns are no longer an issue!  I do have multiple back-ups of critical data, which is definitely good advice to follow. 

 

Interestingly enough I have never had any issues after power failures with unRAID 5.  However, after moving to a new home and experiencing the first one, I quickly recognized the necessity of having a UPS.  I did have a power supply fail on me - but after a typical parity check all was just fine.

 

As far as my last question regarding using a Cache disk/pool for Docker storage - this would not negate the ability to continue writing directly to mounted disks would it?

 

Thanks again for everyone's help!

Link to comment
2 minutes ago, Vibe said:

As far as my last question regarding using a Cache disk/pool for Docker storage - this would not negate the ability to continue writing directly to mounted disks would it?

 

Not at all.  

 

As far as I'm concerned, a cache drive (whether SSD or regular HDD) is a necessity once dealing with docker apps or VMs.  Both of them (and the apps that run under them) will continually write various metadata, etc which if not stored on the cache drive will result in significant slowdowns.

 

Whether you do a pool or not is entirely up to you.  I personally can't be bothered.

Link to comment
11 minutes ago, Vibe said:

Perfect - Thanks so much Squid.  I have a couple SSD drives that I haven't been using so now they will have a new home.  I greatly appreciate your taking the time to answer my questions!

If you go the cache pool route, definitely grab a UPS.  BTRFS (required for a cache pool) is less forgiving in power failure situations than XFS (which you can use if you don't utilize a cache pool)

Link to comment

This is very good to know - I can just imagine the issues with a power failure on a cache pool.  After a power failure several years ago I did buy an APC Back-UPS 1300 (battery backup) which has been working with the APCUPSD pkg.  However, since then I have read that there is a difference between a UPS and a battery backup.  Would this be sufficient with BTRFS?

Link to comment
4 hours ago, Vibe said:

However, since then I have read that there is a difference between a UPS and a battery backup.  Would this be sufficient with BTRFS?

 

Yes.  The drive could care less.  It is the PS that you should be concerned about.  But most recent PS's will tolerate a weird AC waveform for a few minutes a few times a year without any serious consequences.  So about any UPS is satisfactory (in my opinion) if these conditions are typical.  Now if you have an outages three times a day, that is another case.  (There is an unRAID user in the PI who is in this situation!)  In that case, you want a UPS that can deliver AC power with a good sine waveform with a minimum of third harmonics.

 

One thing to think about a shutting the server as quickly as possible after you are sure that the outage is a long term outage and not a quick 'controlled' outage initiated by the power company.    (In my case, I have determined that if the power is out for thirty seconds, it will be out for hours.)  The reason is that if you run the server on battery until it only has (say) 20% of remaining capacity and then have it shut the server down.  Great, right, that is exactly what you got it for.  An hour later, the power comes back on.  You wait twenty minutes and start the server back up.  It runs for a half-hour and the power fails for a second time.   The battery hasn't had to time to fully recharge.  Now it is a race to see if the battery will last long enough to complete a clean shutdown before the battery is exhausted.

Link to comment

Feel compelled to give a dissenting opinion. I have no UPS. Had one for ~2 years - it saved me exactly one hard shutdown before the (expensive) batteries failed and wanted to be replaced. I have had a hard shutdown averaging 1 a year. It has been very rare that I see a parity error. I have never had corruption (confirmed by md5 checks). 

 

Although a parity check is an annoyance, it is not a huge concern for me. Once I had something important to do and canceled the parity check, restarting it at a more convenient time later in the evening.

 

Although I will not dispute that a UPS is beneficial for someone having frequent power outages. But for me, it just seemed an expensive luxury.

 

One reason BTRFS is not something I use is that is seems more susceptible to corruption after a hard shutdown. Although my Docker volume is BTRFS and not had any issues with it.

Link to comment

Thank you Frank1940 for the information.  I just did some further reading after your post and learned that my battery backup doesn't provide the sine waveform that you mentioned.  Very interesting stuff.  I don't have many power outages anymore, but may plan on upgrading to something that does provide this feature.  A friend of mine also mentioned issues he has experienced due to power supplies dying on servers he manages (Linux but not unRAID).  For this reason he has implemented using dual power supplies.  

 

Hi bjp999 - I certainly appreciate your perspective.  The same friend I mentioned above jokes about the batteries on his UPS having an "invisible timer" that determines when it's ready to be replaced.  He's suggested that he can add the day to his calendar for when he needs to order a new one - and they are not cheap.

 

I began using unRAID years ago when storage on my Linux home server I built (software raid1) grew beyond 250GB.  I never had any issues with data corruption when the power went out - very reliable storage indeed (despite the operator).  Once I moved to unRAID and experienced the same reliability and the ability to easily increase capacity I was hooked.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.