ken-ji

[6.3.0+] How to setup Dockers without sharing unRAID IP address

Recommended Posts

ken-ji    34

How to setup Dockers to have own IP address without sharing the host IP address:

This is only valid in unRAID 6.3 series going forward.

 

Single NIC only:

  • Some assumptions:
    We'll be using a shared interface br0 (This allows us to use the same nic with virtual machines, otherwise its alright to use eth0)
    The IP address details are:
    unRAID = 192.168.1.2
    Gateway/router = 192.168.1.1
    Subnet = 192.168.1.0/24
    Docker IP pool = 192.168.1.128/25 (192.168.1.128-254)
    A new docker network will be established called homenet
  • Login via SSH and execute this:
# docker network create \
-o parent=br0 \
--driver macvlan \
--subnet 192.168.1.0/24 \
--ip-range 192.168.1.128/25 \
--gateway 192.168.1.1 \
homenet
 
  • Modify any Docker via the WebUI in Advanced mode
  • Set Network to None
  • Remove any port mappings
  • Fill in the Extra Parameters with: --network homenet
  • Apply and start the docker
  • The docker is assigned an IP from the pool 192.168.1.128 - 192.168.1.254; typically the first docker gets the first IP address
# docker inspect container | grep IPAddress
            "SecondaryIPAddresses": null,
            "IPAddress": "",
                    "IPAddress": "192.168.1.128",
# docker exec container ping www.google.com
PING www.google.com (122.2.129.167): 56 data bytes
64 bytes from 122.2.129.167: seq=0 ttl=57 time=36.842 ms
64 bytes from 122.2.129.167: seq=1 ttl=57 time=36.496 ms
^C
# docker exec container ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2): 56 data bytes
^C
#
  • At this point, your gateway/router will have a first class network citizen with the specified IP address
  • An additional Extra Parameter can be specified to fix the IP address: --ip 192.168.1.128
  • The container will not be allowed to talk to unRAID host due to the underlying security implementation with the macvlan driver used by Docker. This is by design
  • That's it.

 

Secondary NIC is available:

  • Some assumptions:
    We'll be using a dedicated interface br1 (the native eth1 interface can used here too)
    There is no IP address assigned to the interface
    The IP address details are:
    Gateway/router = 10.0.3.1
    Subnet = 10.0.3.0/24
    Docker IP pool = 10.0.3.128/25 (10.0.3.128-254)
    A new docker network will be established called docker1
    unRAID has an ip of 10.0.3.2
  • Login via SSH and execute this:
# docker network create \
-o parent=br1 \
--driver macvlan \
--subnet 10.0.3.0/24 \
--ip-range 10.0.3.128/25 \
--gateway 10.0.3.1 \
docker1
 
  • Modify any Docker via the WebUI in Advanced mode
  • Set Network to None
  • Remove any port mappings
  • Fill in the Extra Parameters with: --network docker1
  • Apply and start the docker
  • The docker is assigned an IP from the pool 10.0.3.128 - 10.0.3.254; typically the first docker gets the first IP address
# docker inspect container | grep IPAddress
            "SecondaryIPAddresses": null,
            "IPAddress": "",
                    "IPAddress": "10.0.3.128",
# docker exec container ping www.google.com
PING www.google.com (122.2.129.167): 56 data bytes
64 bytes from 122.2.129.167: seq=0 ttl=57 time=36.842 ms
64 bytes from 122.2.129.167: seq=1 ttl=57 time=36.496 ms
^C
# docker exec container ping 10.0.3.2
PING 10.0.3.2 (10.0.3.2): 56 data bytes
64 bytes from 10.0.3.2: seq=0 ttl=64 time=0.102 ms
64 bytes from 10.0.3.2: seq=1 ttl=64 time=0.075 ms
64 bytes from 10.0.3.2: seq=2 ttl=64 time=0.065 ms
64 bytes from 10.0.3.2: seq=3 ttl=64 time=0.069 ms
^C
--- 10.0.3.2 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.065/0.077/0.102 ms

  • At this point, your gateway/router will have a first class network citizen with the specified IP address
  • An additional Extra Parameter can be specified to fix the IP address: --ip 10.0.3.128
  • The container can happily talk to unRAID as the packets go out via br1 and talk to the host on br0
  • That's it.

 

Some caveats:

  • With only a single NIC, and no VLAN support on your network, it is impossible for the host unRAID to talk to the containers and vice versa; the macvlan driver specifically prohibits this. This situation prevents a reverse proxy docker from proxying unRAID, but will work with all other containers on the new docker network.
  • I cannot confirm yet what happens in the case of two or more NICs bridged/bonded together (but it should be the same as a single NIC)

 

Capture.PNG.eca20b4719650a5a4ff273cb26189830.PNG

 

Edited by ken-ji
Clarified caveat of inability to talk to unRAID in case of single NIC
  • Upvote 7

Share this post


Link to post
Share on other sites
testdasi    0

Does this need to be done every time unRAID is restarted?

And why 6.3.0+ only? Was it a new undocumented feature or something?

Share this post


Link to post
Share on other sites
bonienl    163

This is quite interesting. I've been looking into network segregation for Dockers in the past but couldn't make it to work properly.

 

I'll have a look at translating your approach to GUI support.

 

Do you have a good reference (URL?) which provides more background information?

 

  • Upvote 2

Share this post


Link to post
Share on other sites
ken-ji    34

Does this need to be done every time unRAID is restarted?

And why 6.3.0+ only? Was it a new undocumented feature or something?

 

I think its permanent (as long as the docker.img is intact) since that's where all the docker related meta data is persisted.

6.3.0 used docker 1.12 which is when the macvlan plugin was released as stable

 

This is quite interesting. I've been looking into network segregation for Dockers in the past but couldn't make it to work properly.

 

I'll have a look at translating your approach to GUI support.

 

Do you have a good reference (URL?) which provides more background information?

 

 

Probably these:

https://github.com/docker/libnetwork/blob/master/docs/macvlan.md

https://docs.docker.com/engine/userguide/networking/get-started-macvlan/

 

  • Upvote 2

Share this post


Link to post
Share on other sites
CHBMB    171

Damn this is interesting.....

Share this post


Link to post
Share on other sites
BRiT    45

I'll have to give this a shot once I have some spare time.

Share this post


Link to post
Share on other sites
DestroidUK    1

Brilliant ken-ji!

 

Works a treat and solved some Plex and Crashplan docker issues I was having, I nearly set up new VM's to solve those issues until I found your post.

 

Router is happy and so am I!

 

It seems to help bridge the gap between containers and VM's allowing them to live on the physical network rather than port mapping or NAT'ing.

  • Upvote 1

Share this post


Link to post
Share on other sites
CHBMB    171
Posted (edited)

Just starting to play around with this.  Got this working and an IP address allocated from the DHCP pool, using a single NIC.

 

docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn linuxserver/plex

But if I try to use

 --IP=192.168.0.128

for fix the IP address I get

docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn --IP 192.168.0.150 linuxserver/plex
unknown flag: --IP

 

EDIT:  Schoolboy error, @Malykai kindly pointed out that it needs to be --ip in lowercase.  I feel kinda stupid now for copy pasta without thinking. 

Edited by CHBMB

Share this post


Link to post
Share on other sites
CHBMB    171

@ken-ji This works brilliantly. All my WAN traffic goes over a VPN using pfsense, that however breaks Plex remote access.  Managed to fix it with this macvlan setup.  Thanks man.  I'm going to write a guide on how to do this in pfsense at some point.  Credit to you of course.

Share this post


Link to post
Share on other sites
ken-ji    34
9 hours ago, CHBMB said:

Just starting to play around with this.  Got this working and an IP address allocated from the DHCP pool, using a single NIC.

 


docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn linuxserver/plex

But if I try to use


 --IP=192.168.0.128

for fix the IP address I get


docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn --IP 192.168.0.150 linuxserver/plex
unknown flag: --IP

 

EDIT:  Schoolboy error, @Malykai kindly pointed out that it needs to be --ip in lowercase.  I feel kinda stupid now for copy pasta without thinking. 

Oops. just noticed now the wrong capitalization in the post. Corrected.

Share this post


Link to post
Share on other sites
Spritzup    1

So I'm considering doing this to get PiHole working properly.  However, I'm concerned with conflicts on with my network DHCP server... is it possible to just assign a static IP and not use a range?

 

Thanks.

 

~Spritz

  • Upvote 1

Share this post


Link to post
Share on other sites
CHBMB    171

Just start the container up, let it get an address from DHCP, then allocate it a fixed IP address. Easy.

Sent from my LG-H815 using Tapatalk

  • Upvote 1

Share this post


Link to post
Share on other sites
Spritzup    1

So it doesn't matter if I use the same DHCP range that my router is using, since I will be manually forcing an IP?  That makes sense... and it won't impact my other containers, unless I use the appropriate switch (--network).

 

Thanks!


~Spritz

Share this post


Link to post
Share on other sites

I'm using this and it works really well. Everything survives restarts and what not. I'm going to end up using this to point to another port on my nic and have all of that traffic route to a router that vpn's all the traffic. This seems to be useful enough to deserve to be pinned no? or added to the "FAQ for unRAID v6" topic maybe?

Share this post


Link to post
Share on other sites
Helmonder    8
[mention=62359]ken-ji[/mention] This works brilliantly. All my WAN traffic goes over a VPN using pfsense, that however breaks Plex remote access.  Managed to fix it with this macvlan setup.  Thanks man.  I'm going to write a guide on how to do this in pfsense at some point.  Credit to you of course.



I must try this out... what you are describing is the exact reason i have plex running in a dedicated vm.. would love to have a docker with a seperate ip address..

I am somewhat reluctant in doing it this way though.. since it is not formally supporter it could break with an update ? Soinds like something that wpuld be great to fit in the gui itself..


Verzonden vanaf mijn iPhone met Tapatalk

Share this post


Link to post
Share on other sites
CHBMB    171
11 minutes ago, Helmonder said:

 

 


I must try this out... what you are describing is the exact reason i have plex running in a dedicated vm.. would love to have a docker with a seperate ip address..

I am somewhat reluctant in doing it this way though.. since it is not formally supporter it could break with an update ? Soinds like something that wpuld be great to fit in the gui itself..


Verzonden vanaf mijn iPhone met Tapatalk

 

 

It's supported by docker so unless they deprecate the feature then it shouldn't be a problem.

 

I actually managed to figure out how to do this via pfsense, so I'm not currently using this method, but it worked flawlessly.

Share this post


Link to post
Share on other sites
jrdnlc    6
On 4/20/2017 at 2:55 AM, CHBMB said:

It's supported by docker so unless they deprecate the feature then it shouldn't be a problem.

 

I actually managed to figure out how to do this via pfsense, so I'm not currently using this method, but it worked flawlessly.

 

I'll be patiently waiting for this guide B|

Share this post


Link to post
Share on other sites
CHBMB    171

What guide?

Sent from my LG-H815 using Tapatalk

Share this post


Link to post
Share on other sites
BRiT    45
7 hours ago, CHBMB said:

What guide?
 

 

Vowels and How to Use Them.

 

 

B|

Share this post


Link to post
Share on other sites
jrdnlc    6
On 4/21/2017 at 11:32 PM, CHBMB said:

What guide?

Sent from my LG-H815 using Tapatalk
 

 

On 4/3/2017 at 8:09 AM, CHBMB said:

@ken-ji This works brilliantly. All my WAN traffic goes over a VPN using pfsense, that however breaks Plex remote access.  Managed to fix it with this macvlan setup.  Thanks man.  I'm going to write a guide on how to do this in pfsense at some point.  Credit to you of course.

 

Share this post


Link to post
Share on other sites
jrdnlc    6

@ken-ji How would I set this up if i'm using 802.3ad? I have a total of 4 NIC's

Share this post


Link to post
Share on other sites
CHBMB    171

Oh yeah, I forgot about that. I'd even started it as well.....

Sent from my LG-H815 using Tapatalk

Share this post


Link to post
Share on other sites
CHBMB    171
On 2017-4-22 at 7:00 AM, jrdnlc said:

 

I'll be patiently waiting for this guide B|

Here you go.

  • Upvote 1

Share this post


Link to post
Share on other sites
bonienl    163

Perhaps you would be interested to know that macvlan support is added in the upcoming version of unRAID, it allows you to select additional 'custom' networks from the GUI.

 

  • Upvote 3

Share this post


Link to post
Share on other sites
CHBMB    171
3 minutes ago, bonienl said:

Perhaps you would be interested to know that macvlan support is added in the upcoming version of unRAID, it allows you to select additional 'custom' networks from the GUI.

 

Errr, I've kinda changed my approach now, doing it all at the firewall level.  But I did enjoy messing around with the macvlan stuff and I can definitely see how it would be very useful for others.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Copyright © 2005-2017 Lime Technology, Inc. unRAID® is a registered trademark of Lime Technology, Inc.