User shares


Recommended Posts

6 hours ago, SSD said:

es it is. :P 

 

If an Windows or Linux had this "issue", it would absolutely be a bug.

 

Understand it is hard to fix, and benefits far outweigh the risks, but I still call it a bug.

It IS actually a Linux bug and is easy to recreate on any Linux system.     It is a side-erect of having multiple views of the same data in such a way that Linux does not realise the different views are referring to the same data.

Link to comment
43 minutes ago, itimpi said:

It IS actually a Linux bug and is easy to recreate on any Linux system.     It is a side-erect of having multiple views of the same data in such a way that Linux does not realise the different views are referring to the same data.

I always thought its more of having two different file systems (fuse & xfs/btrfs) both referencing the same data.

 

eg: if in linux you do a

mount -o bind /tmp/test /tmp/test2

 

and then copy a file from test to test2 it will return an error that the file is in use.

Edited by Squid
Link to comment
2 hours ago, itimpi said:

It IS actually a Linux bug and is easy to recreate on any Linux system.     It is a side-erect of having multiple views of the same data in such a way that Linux does not realise the different views are referring to the same data.

 

Not doubting there are ways to confuse LInux enough to make this happen. But it would be called a bug and something they would fix!

Link to comment

Thanks. I have it working. Still a bit worried that I turn into corrupting everything, but let me keep fingers crossed that I remember what not to do. @SSD actually nicely laid out nicely the biggest risk scenario: removing a disk from user share and the copying things into the user share. If not done on system level, this will wipe it all.

Link to comment

Of course, it is very easy to avoid and @Squid's is correct that odds for it to happen are very low.

 

This assumes though that you set things up and rarely change things. I am using Unraid for the fun of changing things and trying new things out. The tinkering around it is at least as much of the fun as using the actual system.

 

Having the user shares up and running now, I realize how great of a feature this is. Really cool!

 

One thought so, why would Unraid still write to a disk within a user share that has been excluded to be part of the user share. I understand the logic of it still reading it, but don't understand the logic why it should still read to it if no other disk has free.

Link to comment

Your last sentence wasn't so clear. I can't speak to the "why", only to the behavior.

 

Split level is supreme. If a file is being sent to a disk because that a folder already exists beyond the split level on a disk - there is not stopping it. Excluded or not. unRAID has "no choice" where to put the file. Even if a disk is completely full and a completely empty disk is present and assigned to the user share - unRAID will try to write it to the full disk and you'll get an out of space error.

 

Include and exclude affect media where unRAID "has a choice" of where to put the file.

 

If you give unRAID ultimate control to split everything, as apparently Squid does, unRAID would never write to an excluded disk, because split level would never come into play. You might say unRAID "always has a choice" in that configuration.

 

I know it sounds a bit nonsensical, but you get used to it and it actually works pretty well. I keep full TV shows (all seasons) each on a single disk (which I don't recommend, but its how I am set up and not ready to change it). So I monitor my free space on the disks in the share and if it drops below 30G or so, I'll move a show to another disk with lots of free space so I don't have risk of running out of space. I have allocation method set to fill up, but min free set very high - like 400G. So any new shows try to go to a disk with a lot of empty space. leaving other disks with generous free space to accept new seasons and episodes of shows on those disks. When a disk has all the shows it can handle, I typically remove it from the included disks. That way unRAID will never put a new TV show on it, but will continue to fill in episodes for TV shows already on that disk. Works well for me. Once my free space drops below about 500G on the most empty disk, I'll look to add a new disk to the TV share (or delete some old TV series I don't want to keep any more).  A new disk will become the instant recipient of some shows from disks that are getting anywhere near full. Looking to have between 200G and 300G on the other disks so it will be a good long time before I need to do anything.

 

Might sound a little high maintenance, but I rarely have to do much with it. Maybe once every couple of months I'll move a TV show to a different disk. Occasionally I actually run out of space on a disk and am alerted, which is not a big deal actually.

 

Hope this helps! Enjoy your array!!!

Link to comment

Thanks for taking the time to reply to my messages. This is very helpful.


Everything is working very well for me now. I have set things set as "manual", which very well serves my purpose.

 

I am an intellectually curious person, so have a few more questions how this really works.

 

1) How does Unraid handle if I happen to have two files in the same folder on two disks that are part of the same user share? For example "movies/myvideo.mkv" on disk 1 and "movies/myvideo.mkv" on disk 2. Would this also lead to corruption? Or over-write? Or somehoe miraculously, the user share would be able to display both movies. In my example, I assume that the two movies are different, but happen to have the same name.

 

2) I still have not fully understood how the split level works. Let me give an example to illustrate:

 

Disk1/TV/Show1/

(...)

Disk2/TV/Show1/

Disk2/TV/Show2/

(...)

Disk3/TV/Show3/

(...)

 

I have things set to manual. See my questions below:

 

A) Assuming things are set to manual, adding a movie to user share "/TV/Show1" will never lead to things written to disk3. Even if both disk1 and disk2 are both full. Correct?

 

B) I now go ahead and exclude disk2 from the user share. From what I understand, all of the files in "disk2/TV/Show1" would remain part of the user share and all files are being displayed. When writing to the user share (folder show1), Unraid will always write to disk1/TV/Show1 first independent of all split settings. However, Unraid will over-rule the setting and still write to the excluded disk2 once disk1 is full. I don't understand why it was implemented this way, but doesn't bother me. Just curious whether this is indeed the design principle?

Link to comment
1 minute ago, steve1977 said:

"movies/myvideo.mkv" on disk 1 and "movies/myvideo.mkv" on disk 2.

The file on the lowest numbered disk takes precedence on reading from the user shares (you would also only see one file when browsing the user share), and you would get very confused as to what's going on.  That's one of the problems with disk shares.  When only using user shares, that situation can't happen.

Link to comment
1 minute ago, Squid said:

The file on the lowest numbered disk takes precedence on reading from the user shares (you would also only see one file when browsing the user share), and you would get very confused as to what's going on.  That's one of the problems with disk shares.  When only using user shares, that situation can't happen.

 

Got it, very clear. This is about "display/access", right? So even in this example, the second file would not be "destroyed"? It is "just" not accessible via the user share.


I am planning to fully migrate to user shares, so this issue will eventually go away. I now just have the "legacy" situation. Is there some form of command line that will allow me to spot whether I have duplicates across disks?

Link to comment

Thanks again guys. it’s working very well. two more questions:

1) would i benefit significantly from adding a dedicated cache disk for caching?

2) i still have permission issues caused by radarr. i can fix them, but need to find a way that radarr doesn’t cause them again and again. any thoughts? i’ve tried to set to 0777 for file permission in radarr settings, but this didn’t fix it.


Sent from my iPhone using Tapatalk

Link to comment

I believe I have solved the permission. Had to set both folder and file permission to 0777. Only did for file, which wasn't sufficient.

 

Any thoughts about the use of a cache disk appreciated. I could change my current cache disk (where my VMs and dockers reside) to an unassigned disk. And then have a dedicated cache SSD disk for caching. Thoughts?

Link to comment

The cache drive serves two purposes these days.  First it’s the default application drive for Dockers and VMs.  Second it still has the traditional role of caching writes to the array.  Based on the wording of your question I assume we are talking about the second part.  Writes to the cache drive can be faster than writes to the parity protected array, but unless you implement a BTRFS cache pool your data will not be protected while on the cache.

 

Will you benefit?  Dunno, that’s really up to you.  It’s entirely an optional feature.  If you are spending time watching your PC copy files to your server then it may help you.  If you download files to the cache drive and then watch them immediately before mover runs, it may help you by not having to spin up array drives.  Personally I don’t use write-caching.  I like my files to be immediately protected by parity and I don’t sit around watching file operations.  If I want things to go faster I turn on turbo write.

 

I’d use the cache drive for Dockers and move VMs to an Unassigned device if you feel the need.

Link to comment
On 11/25/2017 at 10:08 PM, Squid said:

Extremely unlikely you would ever do this, even accidentally.  You would have to be going out of your way to try and force the user share copy issue

 

Nothing happened, but I realized an easy easy way for this issue to arise with Radarr.

 

All my movies are on disk shares /movies. I have now created a user share /movies. When bulk importing from a disk share into Radarr, it would copy the file to the user share and thus kill the file? Same for Sonarr.


Way to avoid is to import from the user share or change the folder name from /movies to /movies2.

 

Just reporting this back as others may trip into this.

Link to comment

But then why reference disk shares with Radarr and the equivalent user share.  Wouldn't it all be simpler to just only reference the user share.  Any rename by bulk rename would simply rename it in place.

 

Like I said, you're going out of your way to try and mess it up by including both user and disk shares in the template, and both are ultimately referring to the same thing.

Link to comment

On a positive note, I didn't do it, so no harm happened.

 

I do think though it would be a common use case.

 

Many of us will have /movies as user share. Many of us will set this user share as target directory for radarr.

 

When bulk-importing, it feels natural to first try things out with one disk (share) rather than putting all movies on all disks in the user share through the import process. Of course, I can rename the folder of one share to /movies2 to achieve the same. Just saying that it could naturally happen.

 

The root cause is actually that I find the radarr bulk-import process particularly complicated: renames the movie folders and files upon bulk-import and copying movies is not exactly straight forward and quite cumbersome.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.