Allocation of Data Oddity?

[grandprix]

I have pretty much all shares on automatic split handling, besides music which is assigned to use only disk 4 and 8 (I had planned to reduce that to only one drive entirely, but I have many CD's from my brothers DJ'ing days that I owned and ripped to FLAC and they used to be only 2TB drives so, I never did change it).

And a minimum space of either 30GB (in case of movies) and 80GB (in case of TV series) required for each share (as shown in the picture).  I'm curious, how it filled drive 8 entirely, before spreading out to the other drives with space (each drive is set to use defaults and only one spin up host exists and that too is drives 4 and 8 so that they're both spun up whenever someone is listening to music to avoid long pauses).  "HDMovies" share is showing as having 3.98TB of space available, yet, it was a recent addition of several movies (thus the "HDMovies" share) that managed to fill the drive to 0 space.

Any and all ideas/clues is greatly appreciated.
 

[grandprix]

Not an oddity I suppose and a known situation?

I moved data from that disk to two others.  Upon adding some data tonight, now disk 1, which was at 130GB free (or somewhere near that), is now at 59GB.  Each file has only been 7-8 GB, so, it again ignored the 80GB minimum free space (what it is now, as 30GB wasn't cutting it obviously).

Never had this issue, but then again, never ran out of physical hard drives this close to empty before (would normally add more, but, am at my max hard drive per controller ports).
 

[JorgeB]

On 24/05/2017 at 11:10 PM, grandprix said:

I have pretty much all shares on automatic split handling

 

According to the screenshot that's not true for the movies share, split level will take precedence over allocation method.

[grandprix]

9 hours ago, johnnie.black said:

 

According to the screenshot that's not true for the movies share, split level will take precedence over allocation method.

 

Could you reiterate for me please.  "Automatically split only top level..." is how it's set (keep in mind, I haven't changed the split levels since 4.7 and back then numbers were used, I had the Movies set to level 1, as all movie files were directly on the share (ex.  /Movies/Star Trek (1970).mkv  )  I -am- aware that "release convention" is /media/Movies/Star Trek (1970)/Star Trek (1970).mkv   -- but I had already been in a thousand or two movies in deep before I had been instilled such knowledge.

It -appeared- to be working as intended, until recently.  When you have a moment johnnie, could you elaborate where my error is/how to resolve it please.  Perhaps it was because I would just add a new drive (essentially indirectly forcing unRAID to use that new drive for new media) when I had gotten below 7TB free space in the past that I never experienced this issue?
 

Edited May 26, 2017 by grandprix

[JorgeB]

If you're coping something to the "movies" share (or any other share using the same split level) and the top folder already exists on disk4 it won't split to another disk, remove that folder or set share to "split any directories as required"

[grandprix]

Yeah that didn't do the trick.  Still, if there is only 92GB of physical space left on the drive, and there are three 8GB files that are soon to be written to the array, regardless if after the second 8GB file will drop the space on a physical drive to less than the high-water, the third or any other subsequent file will be written to that drive until it is empty.

unRAID seemed to dither, or spread out the data fine to around 100-140GB free on each drive (200-230GB before that), but once they're down to 100GB, it seems all bets are off and unRAID says "frick it", and will just hammer a single physical drive until it is empty, completely ignoring the high-water.

 

 

[JorgeB]

13 minutes ago, grandprix said:

it seems all bets are off and unRAID says "frick it", and will just hammer a single physical drive until it is empty, completely ignoring the high-water.

 

Not possible, only way to override the allocation method are the share split level and/or included/excluded disks or the included/excluded disks in the global share settings, check your config, something is not right.

[grandprix]

Oh, its very possible, because it is happening.

 

The last 2TB or so of data written to this unRAID has only been that of the same share.  So just look up top to see the configuration for that share, besides for the change you recommended of "Split any directories as needed" which, seems to have nothing to do with high-water, I set it just the same.  And instead of just 30GB high-water it is now 80GB (and was upon making the change you recommended).  Which seemed moot because it's not even following the "old" high-water of 30GB.

Pictures provided of Global Share Settings, Share Settings and of course the menu/disk/whatever proper name is given to that display screen.  I know I don't have much room left among my disks (that is soon to change on this unRAID, but not for another two months "soon"), but even still..
 

 

Edited May 27, 2017 by grandprix

[grandprix]

Sorry, I'm a hoarder and that includes evidently not closing old paint windows.  This is the share settings (the other one was an old paint window I left open).
 

[JorgeB]

Post your diagnostics, maybe I can spot something else wrong.

[grandprix]

Yeah.  I'd just as this point rather ask, what is the proper 6.3.x -> 6.2.4 reversion instructions?  From my understanding I'll just unzip 6.2.4 onto the thumb drive, keeping the "config" (which file or files that is precisely I don't know at the moment off hand) and I'll have to scrap my dockers and redo them from scratch.

I never intended to be one of those "everything worked fine before I upgraded" but the facts are simply, they did.  This box could stream 7 transcodes and 3 direct streams (Plex) while sab was unzipping and moving a file to the array.  Now, 4 streams max it seems and, forget about it if I'm moving files to the array (or sab is).  Samba issues (initially), now this issue.  Even if the errors were on my end with misconfigurations, that just goes to show 6.2.4 was more forgiving (assuming it was any settings on my end).  Just nothing but small issues with 6.3.x and I'm at my limit now.  Maybe its my hardware it's not playing nice with,  I dunno.

But yeah, if I could respectfully ask what I'll have to do to revert (since it doesn't seem to be found in a clear and concise manner in any post that I've been able to search for), I'd like to do that.

[JorgeB]

Just overwrite the 3 bz* files, docker and VMs may need to be redone.

[JorgeB]

Also, I'd expect performance with reiserfs and all disks almost full to be terrible, you may want to look how to convert them to xfs.

[grandprix]

1 hour ago, johnnie.black said:

Also, I'd expect performance with reiserfs and all disks almost full to be terrible, you may want to look how to convert them to xfs.

 

I have and it sounds like an awfully long process, one that once things are running straight again, I had planned to do once the bigger drives come in.  I'm weary about doing it, as reiserfs has served me well for 6 years now, never a hitch.  xfs though I swear I see threads on it here and there about having to do some sort of maintenance on them.. perhaps that's just a side effect of more people using xfs (aka statistics), but, does concern me just the same.  Once the 8TB for this unRAID come in (just 4 atm which unfortunately two have to go to the parities of course) I'm definitely going to give it a whirl though.  I'll try nearly anything once (read nearly lol)

Thanks for all your help johnnie.  I'm not much concerned about rebuilding the docker/image, that's such a breeze to redo since docker authors make things so simple (thanks to the authors).

[BRiT]

XFS has required less maintenance than RFS for me and has never had any performance hiccups or corruptions. I can't say the same for BTRFS.

 

If you need to move data, definitely have a look at unBALANCE. It's a real gem.

[grandprix]

5 minutes ago, BRiT said:

XFS has required less maintenance than RFS for me and has never had any performance hiccups or corruptions. I can't say the same for BTRFS.

 

If you need to move data, definitely have a look at unBALANCE. It's a real gem.

Oh you know, you're right, I think it may have been BTRFS that I recall seeing maintenance threads on.  Either way, I'll definitely give it a go.  And thank you, I was just wracking my brain last night trying to remember the name of the plugin/app for moving files around so I wouldn't have to putty in and use MC.

[JonathanM]

2 hours ago, grandprix said:

This box could stream 7 transcodes and 3 direct streams (Plex) while sab was unzipping and moving a file to the array.  Now, 4 streams max it seems and, forget about it if I'm moving files to the array (or sab is).

From personal experience, I'd say this has much more to do with topping off ReiserFS disks than it does which version of unraid you are running.

 

I suspect you are going to see much of the same behaviour with your current disk situation under 6.2.4

[grandprix]

Yeah, usually I would have added a disk, but, that isn't an option, only upgrading existing ones.  Amazon sent them SurePost, sooo, now instead of receving them yesterday (as would've been the case using UPS Ground or heck even USPS Priority), I won't be getting them until Tuesday.  I hate packages sitting at the post office, this is what I see happening the longer it sits there: