Hardware upgrade planned, questions around VMs, cache drive, UD, etc.


-Daedalus

Recommended Posts

Hi all,

 

I'll be doing a hardware upgrade on my server soon enough, and with it I'm planning some updates to my VM storage.

 

Right now, I have everything on 2x500GB SSDs. VMs, Docker, and most shares using it.

 

The problem is that I'm running out of space. Now, I could:

Buy bigger SSDs - Certainly possible, but I'd like to avoid this if possible given how expensive SSDs have got over the last while

Move VMs to an Unassigned SSD - I'd like to have all of BTRFS's error checking, and the RAID1 redundancy

Make some of my shares not use cache - Possible, but for the following reason it's a bit tricky.

 

When something is finished downloading, it gets moved from mnt/user/downloads to mnt/user/movies or tv as the case may be. So even if downloads is set to not use cache (which is what's currently set) stuff still gets copied to SSD when it gets moved to its final home. Can I use User0 here, or will that cause more problems?

 

 

Ideally, I'd like to have a second cache pool, only for use with VMs. I could use downloads with an unassigned device, but I don't know how well that's supported with the mover, if at all.

 

Does anyone have any options or ideas about what could be done here?

 

Thanks!

Link to comment

None of my user shares use cache, I use the cache strictly as an application drive.  Ask yourself, why am I caching writes to the array?   There are some good reasons, but some folks just defaulted into that design.  If that line of thinking doesn't give you a new option, you might look at a hardware RAID controller like Areca and implement that for your second "pool".

Link to comment

The cache drive (for write-caching) has been around a long time and was implemented when writes to the array were really slow.  So, is it a critical feature of unRAID or is it a vestigial feature like your appendix?  Depends on what you are doing with it.

 

If you're copying data from your PC to unRAID over a 10Gb connection, and you're sitting waiting for it to finish - it's a critical feature.  If you're doing the same thing over a 1Gb connection, frankly the much newer Turbo Write feature is almost as effective and doesn't involve the cache drive.  And if new content typically gets written to your array by a downloader/renamer (in other words, you aren't sitting there watching) then interactive speed doesn't matter at all.  If your main reason for using the cache drive to cache writes to the array is "because it's there" then I actually think you are missing out on some of the newer features of unRAID.  It's definitely worth thinking through how you use unRAID, and what you are trying to accomplish with the cache drive.  It's entirely possible you need to cache writes to the array - but as you pointed out in your original post things start to get complicated when you use the cache drive for caching, Dockers, and VMs...

Link to comment

I understand your reasoning for looking at my use of the cache drive, but there are reasons I'm doing it this way.

 

1) Power consumption. Turbo write would help yes, but would mean my power consumption is closer to 80W 24/7, rather than it's usual ~35. Not a huge deal, but electricity isn't cheap where I live.

2) CPU resources. My CPU isn't very powerful. A bunch of stuff getting written to the array can use 40%. I'd like to keep this down during the day when people are using other CPU intensive things.

 

So as to my original question: I'll be looking at my usage of the cache drives overall, but can you - or anyone else - suggest anything I might be able to do to have VMs and Docker humming along nicely, as well as keep most writes using cache?

Link to comment

Unfortunately you already have a good grasp of the options.  Buy bigger SSDs.  Or add more SSDs to the pool.  Or switch cache to be a large HD.  Or move VMs to an Unassigned device.  If you want redundancy in an Unassigned device then you're looking at a hardware RAID controller like Areca.

 

FWIW, turbo write does allow drives to sleep - they're just spun up as needed.

Link to comment

The cache disk was implemented when array I/O was 10-12 MB/sec. You can now get 40 with no sweat, and some get 65+ with RAID0 parity.

 

IMO, SSDs should be used to make things faster when I am sitting there waiting for them to finish. Speeding up things that are happening in the background - I don't care to speed that up. Downloads can go to an unassigned device spinner and be decompressed directly to a user share. My SSDs are used for Plex metadata, VM disk space, Docker images, and other things that make it faster for things that would otherwise make me wait.

Link to comment

I've mounted a spinner to a share folder name on the cache drive.  Then I get the larger space under that folder on the cache drive without needing more space actually on the SSD.  I only do this when writing directly to the array is a problem for me.  Like I need to free up space on another PC and my preclear isn't done yet on the new drive I'm adding to the FULL array.  That gets me access to the transferred files in the user share before I actually have added the drive to the array.  Once the array space is expanded I move the files to their new home on the array.

Link to comment

Thanks for all the replies guys.

 

It would be nice to just add a third SSD into a three-drive pool, but I've heard BTRFS RAID 5/6 equivalents are a bit unstable. Haven't looked into it in a while.

 

For the moment I've stopped downloads and media shares from using the cache, and have taken the hit on performance. We'll see if this improves VM performance anyway from not having as much stuff hitting the same SSDs.

 

Mounting a spinner to a directory on the cache sounds interesting. Does this then still work with the mover correctly? I assume it would. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.