Please audit my server lockdown method (ransomware)


jumperalex

Recommended Posts

Hey everyone. With the new spat of social media ransomware attackes, and an SO that loves her social media, I though I'd check in to see if the illuminati had any thoughts on my system lockdown.

 

So here is my setup.

 

Media Shares

- All media shares are SECURE and all users set to READ-ONLY

  -- These are written to, typically by a torrent docker which of course can access them without needing smb permission.

  -- In the occasional scenario when I need to add something over SMB I can set my PC's user (jumperalex) to R/W for the few minutes it takes to copy over and then revert.

  -- I've considered creating a new user, "jumperalex_rw" (with R/W access to all media shares) that I can enter into Windows Network Credentials dialog pop-up when I try to access a media share. So long as I don't select "Remember my credentials" I feel like this should prevent ransomware on any external machine from writing to the SMB shares.

 

Data Backup Shares

- I created backup shares for each computer; "Alex_PC_Backup" and "Lisa_PC_Backup" and set them to PRIVATE, but not yet hidden

- I created two new users "alex_backup_user" and "lisa_backup_user" and I granted them each R/W access to only their respective backup shares; all other users have NO ACCESS to these shares

- I setup the backup process on each machine to send the backups to their respective shares, and when I did that I entered the credentials for the share's respective user when prompted ... this was all in an admin elevated session.

  -- For reference I did this using Windows 10's "Backup and Restore (Windows 7)" on my machine and Acronis True Image 2013 on Lisa's

- At this point I set the backup shares to HIDDEN

- I tested if both the standard user and admin could access the backup shares and they could not; not their own and not the "other" machine's. Heck they couldn't even see them once set to HIDDEN.

- I then of course tested the backup creation and all worked as hoped on both machines.

 

So as best I can tell if any machine gets hit by ransomware it will itself of course get got. But it can't touch any media shares. Similarly as best I can tell my backup shares will all be protected except if the malware is able to grep the machine's backup_user credentials from where ever they are stored when I typed them in to allow the backup program to access the share. But that seems to be segregated from normal, or even admin user, access as evidenced by the fact that I couldn't access the shares that way. Also the stricken machine has never seen the credentials for the other machine's backup share, which isn't even visible.

 

For the record my REALLY important, non-recreatable, data is also regularly synced to a thumbdrive and stored in a firebox. My next goal is something like peer-to-peer crashplan but last i checked it was still a fight everytime they updated something.

 

In the mean time, what are anyone's thoughts? What have I missed / assumed incorrectly? What else should I test?

Link to comment

You are taking a very strong response, but locking things up too tightly can result in you leaving the back door unlocked for your own convenience. Only you can determine if you can live this tightly tied up in knots.

 

Did you check up the Ransomware plugin?  Was it not enough?

 

Offsite backups of critical files would be important too.  Looks like your thumb drive covers that.  Just having a portable USB hard drive at work would also qualify.  Remember the firebox may not survive a fire.

 

Link to comment

On your backup:  you can still type a hidden share for access.    On my second machine no shares are exported to SMB and all are private.    I have a VM (ubuntu)  that turns on with the local Al shares passed through and the main machines shares mounted as read only users.  I use rsync and only copy if new/no delete for media shares.    All other shares I use rsync to copy all.    Crashplan to cloud,  also o  machine 2, takes care of historical non-media share off-site backup.

Link to comment

You are taking a very strong response, but locking things up too tightly can result in you leaving the back door unlocked for your own convenience. Only you can determine if you can live this tightly tied up in knots.

 

An excellent point. For sure I will be self-auditing my own behavior over time to see if I find myself constantly flipping any shares to/from R/W. If I do I'll certainly look into other options such as more robust backups to mitigate the risk of an open sharing being hit. One thing I did do was put in a feature request http://lime-technology.com/forum/index.php?topic=54196.0 looking for better visibility, auditing, and bulk control of share permissions (technically RobJ asked for that last bit but I was trying to take baby steps :) ) That would make it easier to avoid mistakes.

 

FWIW, over time I've been unmapping and hidding many shares as a test and I've found myself not missing them. So that was sort of my first test to see if I could go this far.

 

Did you check up the Ransomware plugin?  Was it not enough?

 

Honestly no. I applaud the author for his efforts and I might even add the tool myself. But I consider it a line of defense AFTER appropriate application of permission policies and good backups. I also think there are more robust solutions that are possible but require Limetech's buy-in to make them happen. They've been discussed in various ransom-ware threads

 

Offsite backups of critical files would be important too.  Looks like your thumb drive covers that.  Just having a portable USB hard drive at work would also qualify.  Remember the firebox may not survive a fire.

 

Yeah I wish I could have a portable anything at work, but that isn't an option.

 

You're right about the firebox. Worse it sits 3 ft from the server :o I'd move it but its literally in the safest place in the house (basement, surrounded by concrete, least amount of timber overhead). I should probably buy another and put it in the shed that is 50 ft from the house. Of course the better option is to settle on a cloud solution for my really important files (not that numerous) and consider an external spinner for my media collection (which really isn't critical but would be annoying to lose). A single 8TB would do me well for a while especially since I don't horde media, I delete 90% of it once watched. I'd consider a second unraid with rsync if I had a geographically separated place to put it, but I don't, so an HDD placed outside the house is my best bet without paying for large cloud storage. Then again, honestly, cost isn't the factor other than hating to waste money and wanting control over my data, so the right solution is still something I'm looking for.

 

 

Link to comment

But I consider it a line of defense AFTER appropriate application of permission policies and good backups. I also think there are more robust solutions that are possible but require Limetech's buy-in to make them happen. They've been discussed in various ransom-ware threads

And I agree with you.  There are better ways to handle this that all require large changes in how the share system works in unRaid.  But even there the plugin does have its place, but one of its major short comings is that the more options you enable in it, the more likely an inadvertent innocent trip of the plugin becomes more likely.

 

But ultimately, since at some point you still require write access to the server over SMB / AFP, the plugin will still have its place.  The plugin is the last desperate line of defense against an attack.

 

But no matter how you cut it, the problem with ransomware isn't securing your server's.  The problem is securing the devices that connect to it, as that is where the attack is going to come from. 

Link to comment

On your backup:  you can still type a hidden share for access.    On my second machine no shares are exported to SMB and all are private.    I have a VM (ubuntu)  that turns on with the local Al shares passed through and the main machines shares mounted as read only users.  I use rsync and only copy if new/no delete for media shares.    All other shares I use rsync to copy all.    Crashplan to cloud,  also o  machine 2, takes care of historical non-media share off-site backup.

 

Yeah that was one thing I wasn't sure about. Are you saying that, for the most part, a hidden, unmapped SMB share is ransomeware safe? My gut says it should be since how would any ransomeware know to even find it. I mean, if they're REALLY good they'd write something that sits and waits for a a few days to collect info on hidden shares, but is any ransomwhere doing that?

Link to comment

But I consider it a line of defense AFTER appropriate application of permission policies and good backups. I also think there are more robust solutions that are possible but require Limetech's buy-in to make them happen. They've been discussed in various ransom-ware threads

And I agree with you.  There are better ways to handle this that all require large changes in how the share system works in unRaid.  But even there the plugin does have its place, but one of its major short comings is that the more options you enable in it, the more likely an inadvertent innocent trip of the plugin becomes more likely.

 

yeah honestly that was some of, having to deal with inadvertant triggers and fine tuning it. Since my system was set to all PUBLIC before this weekend I figured that was the bigger bang for the buck in my case.

 

But ultimately, since at some point you still require write access to the server over SMB / AFP, the plugin will still have its place.  The plugin is the last desperate line of defense against an attack.

for sure. And that's why I might eventually add it if I can't make one of my other ideas work. Basically a R/W "drop box" for every share that is emptied into its paired share periodically / on inotify trigger so that that the final destination never needs to be R/W except when I want to delete over SMB ... and right now I do that even less than I currently write over SMB.

 

But no matter how you cut it, the problem with ransomware isn't securing your server's.  The problem is securing the devices that connect to it, as that is where the attack is going to come from.

for sure. I'm about as locked down as I think I can get with FF running NoScript and uBlock and my SO is running FF with AdBlock+ and both of us running Windows Defender (not always the best in tests, but not always the worst). I'll probably convert her to uBlock soon but her tolerance for breakage is much lower than mine ... well MY tolerance for dealing with her when there is web breakage is low  :-X but that is also why I made an effort to isolate our backups from eachother that way no single machine can take down the whole system. Also neither of our machines is actually sharing with each other in anyway but I need to research more about how to prevent malware from crawling my network.

 

But as with everything networked, there is no ONE most important thing, it is all defense in depth and creating firebreaks where possible; because it is impossible to Secure anything 100%

Link to comment

Sounds like you're very well protected except for the "firebox", which I suspect is not very fire proof (most inexpensive ones are not).  I use a waterproof, fireproof, data-rated safe to store my backups in, but it was NOT inexpensive.  You can get larger (and more expensive) ones, but this is what I have:  https://www.amazon.com/Wilson-Safe-Media-Ds070e-Electronic/dp/B01DZ07WPM

 

I do the same thing r.e. backups -- I have a set of SyncBack profiles that backup to the server using a R/W account that has credentials stored for that logon; but all other access to the servers is via R/O accounts except when I'm personally updating the server (e.g. adding media).

 

In addition, I keep my servers OFF these days -- as yet-another Ransomware protection measure.  When my backup utility runs, it sends WOL commands to my servers (3 of them); waits 3 minutes; then runs the backup profiles; and then sends commands to turn off the servers.

 

Link to comment

:o ooff for that kinda money I'll just pay for enough cloud storage to also protect myself from theft / catastrophe. Then again my data isn't THAT important hahaha

 

My firebox is one of the better ones, but of course still not up to your standard. I've also considered upgrading my thumbdrive to something a bit more robust to tolerate higher temps inside any enclosure. Like, all metal and waterproofed.

 

Any worries about malware triggering the WOL? I don't know enough to know if that is a silly question  :-[ Either way, to do that would require a second backup only server, which is an option, but I just haven't committed yet.

Link to comment

I'm not concerned about malware running a WOL command.

 

To do that, it would have to (a) know which command to run; (b) know to wait long enough for the remote system to boot up; and then © would still have to know the account details for the R/W account on the server => and in fact there's only ONE of my 3 servers that's accessed with a R/W account during the backups -- the other two are R/O accounts, since those servers are being backed up ... NOT written to.

 

But just-in-case, I DO have a complete set of backups that I update quarterly that are stored in that trusty safe  :)

 

I am, by most standards, VERY well backed up.  The flaw in my strategy is I don't have off-site backups for all of my media (I DO for all of our personal stuff -- finances, pictures, documents, spreadsheets, etc.)  => but the media IS fully backed up on both a backup server that's at the "other end" of the house AND on bare disks stored in the safe.

 

 

Link to comment

On your backup:  you can still type a hidden share for access.    On my second machine no shares are exported to SMB and all are private.    I have a VM (ubuntu)  that turns on with the local Al shares passed through and the main machines shares mounted as read only users.  I use rsync and only copy if new/no delete for media shares.    All other shares I use rsync to copy all.    Crashplan to cloud,  also o  machine 2, takes care of historical non-media share off-site backup.

 

Yeah that was one thing I wasn't sure about. Are you saying that, for the most part, a hidden, unmapped SMB share is ransomeware safe? My gut says it should be since how would any ransomeware know to even find it. I mean, if they're REALLY good they'd write something that sits and waits for a a few days to collect info on hidden shares, but is any ransomwhere doing that?

 

That I do not know.  I am assuming your backup software has the mapping info saved as a potentual source of the info?    What about SMB use sniffing?  No answers,  just questions

Link to comment

I suppose one possible vulnerability would be an SMB use "sniffer" that ran in the background and kept monitoring network accesses; and when the backup utility became active might "see" that activity and then attack that path.

 

There's simply no "absolute" way to guarantee NO possible intrusions => but personally I feel VERY safe with the way mine's set up ... and I think jumperalex is safe as well.

 

But nothing is "absolute".

 

 

Link to comment

  -- These are written to, typically by a torrent docker which of course can access them without needing smb permission.

 

I'm trying to get a better handle on how Dockers, Mover, and user scripts (say, rsync) play in this topic and my *nix knowledge (just enough to be dangerous) is failing me.  If I shut down a user share as secured and read-only, are Dockers or system processes like Mover impacted?  I'm guessing Mover isn't since it can sudo to whatever ID it needs with appropriate file system access, but how about Dockers?  Container authors can specify that their container runs under any ID they want (commonly running as nobody...). And how about cron type scripts launched from CA or similar?

 

Also, assuming all those can run without issue given locked down user shares - does that create any additional exposure?

 

Thanks for any insights, or feel free to point me in the right direction if there's a good reference I should know about on how this all hangs together.

Link to comment

You bring up a good point -- i.e. ransomware that attacks Linux.    My setup -- and jumperalex's -- and I suspect most folks who are concerned with this -- is focused on Ransomware on a client PC that might propagate to the server.    I'm very comfortable with my setup in this regard (and alex's as well) ... but if a ransomware Trojan became active on the server itself that's an entirely different issue.

 

In my case, my servers are all pure NAS's => NO Dockers, NO VM's, etc. ... and I never boot them in GUI mode, so there's never an active browser.  I THINK this means they're very safe, although v6 DOES do some transactions via the internet (time sync, checks for updates, etc.).    I don't think those represent any real risk, but I don't know that for a fact.

 

However, in the case where you have active torrents, I'm far less confident.  This certainly seems like a potential "window of vulnerability" -- but I just don't know enough about Nix versions of Ransomware to know just how much risk this might entail.  Perhaps installing a Linux antivirus utility might be a good idea if you're using Dockers that do internet access.

 

 

 

Link to comment

So the three places that come immediately to mind for my server are .torrents, rar files,  media files (that might have been rar'ed or not in the first place).

 

Looking at each file type and their ability to cause arbitrary code execution / privileges escalation if maliciously crafted:

 

.torrents would need to trigger rutorrent 

 

rar files get auto extracted after download. So the unrar application (i think bundled with ruTorrent or part of the Docker system) would need to be exploited.

 

media files get processed by Plex (specifically their instance of ffmpeg) sometimes but are more often than not direct streamed to my Roku, so the malicious file would need trigger Plex's ffmpeg or the Roku to execute arbitrary code or elevate privileges.

 

Now, I have no idea how to audit that, so I can only look into how do we do our best to isolate those dockers so they can do no harm.

 

First and foremost is the rule of Least Privilege; don't give the docker access to more than it needs and don't give it R/W access if it doesn't need it. Right now, as best I can tell, the Docker App can only access the /mnts we give it access to.

 

Plex of course has access to all media, but it really only NEEDS read access unless we want to allow it to delete from the GUI (I actually like that for deleting things after watching them).

I'm not at my machine, can we limit Dockers to R/O of mounts?

 

rutorrent would require a safe space to write torrents to that is regularly swept into an R/O area. In my case that is easy enough though it means double storage. In the case of rar'ed files I have that already anyway (keeping the rar files for seeding) so all I'd need to do is sweep the extracted file. for non-rar'ed files I'd be making a second copy. That seems fair enough.

 

So anything that can compromise the application w/in the container should be limited to damaging anything the container has R/W access.

 

Then of course there is worst case, breaking out of the container. In my book, that is what backups are for because there is a limit to what I can do to prevent exploiting Docker beyond keeping my server up to date with Limetech.

 

Just my thoughts as I avoid work ;-)

Link to comment

Yeah, I guess there's really two questions buried in there.

 

First, if I lock down the user shares are Mover, cron scripts, and Dockers still going to be able to write to those shares?  And second, if they can still write to those shares, doesn't that mean there's still an exposure (something on Windows that could take advantage of that or potentially Linux malware)?

Link to comment

Well remember Move and Docker do not access shares via SMB. They access them directly through /mnt. So rework your entire thinking with that in mind.

 

so yes Mover can access ALL file shares, not the least of which because it is run as root (99% sure of that at least hehe). But windows has no way of making mover do anything because its only toehold into the server is via SMB. That means 1) dropping a file into a share that is malicious and 2) an application with root (or elevated to root through chicanery) executing that file (everything I said about rutorrent, rar, and plex above).

 

But the mover doesn't execute files persay. I suppose a really malformed filename might make the mover script or MV crash and execute arbitrary code. But I mean, I haven't heard about that happening and what would that filename even look like? I have the same thoughts about cron scripts.

 

Docker is, as I said above, a whole other animal. The docker service runs with deep hooks into the system so anything that can break out of that is cause of severe concern, but little we can do about it other than "physical" isolation of backups.

 

Again, just my musings as I avoid work long enough to just leave. I hope someone comes in and corrects me because learning will occur :)

Link to comment

Hmm ...

 

First, we had Windows PC's with Solitaire, MineSweeper, and Tetris that could occupy lots of time while "avoiding work" in the 90's.  That resulted in "Boss Key" utilities that would pop a spreadsheet on your screen over the offending game  :)

 

Now it seems that work-avoidance is taken up by forums, which can look pretty "official" just by themselves -- at least you're at your PC and typing "stuff" rather than just moving cards or figures around on it.

 

In my working days we actually did work that was related to our employment  :)

Link to comment

... Just my thoughts as I avoid work ;-)

 

... Again, just my musings as I avoid work long enough to just leave.

 

... I'm working  :P

 

Seems to be a bit of a consistency challenge in these comments  :) :)

Hey discussing, researching, and evaluating various security best practices are well within my job description  ;D

Link to comment

So just to bring my own thread back OT I feel like we're got a few open questions concerning the real threat surface of:

  • The mover script / MV at risk of malformed file names; I can't imagine this is easy or lucrative enough for a black-hat to even try
  • Docker apps being granted R/O access to shares they don't need R/W access; this can at least reduce the impact of an attack
  • Docker itself and how to mitigate its compromise; I think the answer is backups ... anyone else?
  • Compromise of any media player, like Plex's custom ffmpeg, via malformed media file posted to torrent sites; could be lucrative if its possible. Let's say, a poisoned GoT torrent hmmmm
  • Ditto for a malicious RAR and the tool being used to extract it

I mean these are the server side attacks that I can think of for a box like mine, but the way I see it, all except a Docker exploit can only hit my media/torrent which is less of a concern than my PC backups which hold actual important data.

 

Still it would be nice to discuss ways to further prevent an attack and limit the extent of the damage should it happen.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.