Not enough free space transferring data


mikeyunk

Recommended Posts

I'm new to unraid.  I am currently doing the free trial to see if I like unraid and want to purchase it.  I built my server with six 3TB WB HDD's.  Three red and three green.  I used one of those as my parity drive.  I also had a old 80GB SSD sitting around so I used it for the cache drive.  But I think that SSD might be my issue.  It might just be to small to even do much of anything.

 

I created some shares and also installed Plex.

 

I copied a large amount of data to my array from a external USB HDD using Krusader.  No issues there.  Then I tried to copy some files from my Windows 10 PC.  When I copy the files (400GB worth) to the server I get an error that I don't have enough space to copy the files.  But I do have a ton of free space on the array.  3TB used of 15TB.  I have notifications setup to my gmail account.  I get a notification right around the time I get the copy error that says my cache drive is almost full, then another that it is now ull.

 

I wonder if the small SSD is causing my issue.  I know it uses the SSD to "cache" the data files before it copies them to the array.  Should I just remove the 80GB SSD from being a cache drive.  Should I replace it with a larger SSD?  SSD's are expensive though.  I can get a used 256GB for $50 from a local sale on Craiglist.  Will a 256GB SSD even be that good for a cache drive?

 

I do also have a spare 5TB & 2TB HDD that I was going to use in my array.  But I could use one of them as a cache drive instead.

 

What do you think I should do?  What do you think it causing my copy issue?

Link to comment

Hello and welcome.  I'd recommend starting by posting the share settings for the share you are copying data to, and a standard request is always Tools->Diagnostics so we can offer some help.  The issue you are facing is more likely related to setup than the size of the SSD.

 

That said, 80GB is small for a cache drive.  I'd recommend 120GB as a minimum and 250GB is common.  The cache drive in unRAID has become the defacto application drive for Dockers, Docker data, etc.  Personally, that's all I use it for - I don't cache writes to the array.  If you want to cache writes, though, make sure your cache drive is large enough for all your Docker data and the files you write to the array each day.

Link to comment
1 hour ago, mikeyunk said:

What is the advantage of using an SSD as the cache drive?  Just a little better performance?  Or is the performance difference noticeable?

You are unlikely to notice much in the way of performance gains from using a SSD for the cache drive purely for caching files that are to be later written to the main data array.   Files on the cache are moved to the main array when the ‘mover’ application runs (by default this is scheduled to run overnight).

 

Note that if using a small SSD for the cache make sure you set the Min free Space (under Sett9ngs->Global Share Settings) to be more than the largest file you intend to copy to the main array.    The idea is that if the free space on the cache falls below this value subsequent writes of files to a User Share bypass the cache (until mover runs freeing up space on the cache drive) and are written directly to the main array.   If you do not do this you can get errors trying to copy large files to unRAID as they will be created on the cache drive and then error out when free space on the cache runs out.

 

You WILL notice a big performance gain if you are using the cache drive to host docker applications or VMs.   

Edited by itimpi
Link to comment

I'm not really sure what the point of the cache drive is.  Why do I want to have files that I copy to the server go to this cache drive vs directly to the array?  I'm not going to know how much data I will need to copy over at any given time.  I'd like to just be able to copy whatever I want to the server anytime I want and not worry about it.  I'm not sure I like the idea of those files sitting on the cache drive until that evening.  I would change that setting to move any files over to the array a few times a day to address that.

 

Would it make more sense for me to use a 2TB HDD for the cache drive or a 256GB SSD for the cache drive?  I have both.  Or should I use both of these drives as cache drives?   I could then use the SSD for Docker apps like Plex as you stated earlier.

Link to comment

There's nothing that says you have to use a cache drive to cache writes to the array.  I only use my cache drive for Dockers, Docker data, and the occasional VM test.  I find that writes directly to the array are fast enough for me 90% of the time, and if that's not enough I turn on Turbo Write.  Unless you copy large amounts of data to your server while sitting there watching and waiting, or have some other specific use case, you might want to try turning write-caching off and seeing if you can live without it.  It's simpler and has the benefit that your data is immediately protected by parity.

 

To determine the size of your cache drive you need to decide whether you want cache writes to the array or not.  If you want to be able write-cache large amounts of data, use the 2TB spinner for your cache drive.  If you only want to use the cache drive as the "application drive" for Dockers, then the 250GB SSD is perfect.  For most folks a 250GB SSD would also support write-caching for a few dozen GB of data each day as well.

 

I would not  mix the 2TB HD and the 256GB SSD in the cache pool.  The cache pool works best with like sized or carefully planned devices with similar performance, try this space calculator to see what I mean http://carfax.org.uk/btrfs-usage/ .

Link to comment

OK, I am going to use the 256GB Samsung SSD as my Cache drive but only for Dockers.  I have to look at how to do that.  I already have Krusader and Plex installed, so I hope I can just change that in the settings to run them from the Cache drive.  I will add the 2TB drive to my array.  So I will have a 5TB parity drive, (6) 3TB HDD's and (1) 2TB HDD in the array.  This should server me well.

 

I THINK, I go into Shares, and set the setting for use cache disk to "only" for the App Data, ISOS, Domains shares.

 

I THINK I set my Download, Backup and Media Shares to use cache disk to "no".

 

What should i set "system" to?

Edited by mikeyunk
Link to comment

Instead of setting cache to "only" set it to "prefer" and run mover.  Mover will then move the files from the array back to the cache disk.  Set System to prefer as well.  You can then set cache to "only" if you want, but you don't need to.  Setting your other shares to "No" is correct, but run mover first to clear off your cache drive.

Link to comment

Ok, I have set app data, isos, domains & system shares to "prefer" for the use cache disk option.

I set the shares that I created (downloads, backup & media) to "no" for the use cache disk option.

 

I am noticing that my SSD is getting hot and triggering alerts.  It was at 45 degrees C, and now it is at 52 degrees C.  It is also almost full.  I am not transferring any data files at all.  Why would it be full?  That is not really that hot either but it is hotter than the alert trigger is set to which is 40 degrees C.

Edited by mikeyunk
Link to comment

There are two MKV movie files that I copied over at 2GB each and a lot of files that I copied over days ago to my download share totaling 226GB.  I was able to see these files using Krusader.  Is it safe to just move them to the array via Krusader?

 

I ran "mover" in the unraid UI but it has been running for hours and it does not look to me moving anything.  I also went back to all the shares and set them all to "prefer" on user the cache disk option.  This can be changed back later.

Link to comment

I'd do things one step at a time.  First, stop mover.  Then, for the shares where there's data on the cache drive that you want moved to the array set cache to "Yes".  For everything else set it to No.  Then run mover and make sure that your cache drive is cleaned off.  Then set Movies, etc to "No" and set the shares that you want on the cache drive to "Prefer" and run mover again - it should move things off the array back to the cache drive for Docker, etc.  Don't make any changes to share settings while Mover is running - let it run to completion.

  • Like 1
Link to comment

OK, thanks itimpi & tdallen.  Your help has been great.  I will have to wait until Sunday to do anything with my server.  For some reason one of my Xeon CPU HSF fans stopped working and the system was running way too hot and the motherboard alarm started.  The heatsink where the fan had failed was super hot.  I had to order a new HSF.  I actually ordered two since this is a dual CPU system.  I will replace the other HSF too so that they are both a matching set and since i am afraid it will also fail.  The HSF that failed is supposed to be a good one too.

 

This is what failed

http://www.ebay.com/itm/NEW-Intel-Heatsink-and-Fan-for-i7-990X-i7-980X-i7-980-i7-970-Socket-LGA1366/332322327289?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2057872.m2749.l2649

 

I purchased these tonight on Amazon

https://www.amazon.com/gp/product/B003NVPLNE/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1

 

Link to comment

My cache SSD disk got up to 100C! I didn't 'understand' the prefer option well, and had selected it on a few big shares, and I guess, my bad -- better go to comprehension classes again. :) Other than the alterts sent via email, my Splunk docker works well. If anyone's interested in it, let me know, I can share the docker-compose file.

Edited by daze
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.