harshl Posted March 25, 2017 Share Posted March 25, 2017 I have recently created several VLAN interfaces in the GUI in order to deploy a new virtual firewall. I have been fighting with it for hours and have not been able to get anything to talk to or from the firewall. I decided to simplify my troubleshooting and install a simple CentOS host to test with. Any VM using br0, works just fine. If I assign any of the VLAN interfaces to a VM, in my case, br0.10, br0.20, br0.30 or br0.5, no traffic will pass anywhere. Attached are screenshots of the config from the GUI, network-1.png and network-2.png. The LACP bond has been in play for a long time and works fine. I can see from the CLI that br0.10 has MAC Addresses for the unRAID interface and the VM interface that is assigned in the VLAN: root@serv:~# brctl showmacs br0.10 port no mac addr is local? ageing timer 1 d0:50:99:6c:7d:b6 yes 0.00 1 d0:50:99:6c:7d:b6 yes 0.00 2 fe:54:00:71:39:f0 yes 0.00 2 fe:54:00:71:39:f0 yes 0.00 Also attached is the output of ifconfig -a showing all of the deployed interfaces and the status of each, ifconfig -a.txt. My switch is configured properly to run both tagged VLANs and one specific VLAN untagged up into the unRAID host. Trouble is, no MAC Addresses or ARPs get learned on the switch from any of the vnet interfaces assigned on a br0.x VLAN interface on the unRAID host. You can also see from the output above, that the switch MAC doesn't get learned on the unRAID host either. There is a complete L2 breakdown happening here somewhere, and it seems to be on the unRAID host. You will find the MAC address learned on the switch in the attached file show mac.txt. You will not find the 39:f0 MAC listed, however, you will find the 7d:b6 MAC which is used on all of the bond and br0 interfaces. From the test CentOS machine I have setup, I have tested with both untagged and tagged settings on the interface, neither allow it to communicate, not even with the unRAID host address. That bit is what tells me that the switch doesn't even matter at this point, since I can't even talk from the VM to the host tagged or untagged. If I alter the VM config to use br0 instead of br0.10, no issue at all, I can ping the host and everything else out in my network as expected. I have read a fair bit about various setups in other distributions, but nothing has pointed me in the right direction so far. Looking for any advice anyone might have to trouble shoot this. Haven't altered anything on the CLI, config is as it was set from the GUI. Any tips? Thanks all! -Landon ifconfig -a.txt show mac.txt 1 Quote Link to comment
amberbycode Posted March 26, 2017 Share Posted March 26, 2017 Hi there, I ran into this same problem and my fix was to just resetup your network adapters as they seem to have changed in an update to unRAID's VM manager.This SuperUser thread helped with me however since you are using CentOS it may be a bit different. - amberbycode Quote Link to comment
harshl Posted March 26, 2017 Author Share Posted March 26, 2017 9 minutes ago, amberbycode said: Hi there, I ran into this same problem and my fix was to just resetup your network adapters as they seem to have changed in an update to unRAID's VM manager.This SuperUser thread helped with me however since you are using CentOS it may be a bit different. - amberbycode Thanks for the response, but this looks like it is just dealing with the settings in the guest from the quick look I did. My issue resides on the unRAID side. The CentOS machine I am testing with is brand new and created just to validate this issue wasn't specific to the other VM I ultimately want to use. Any other ideas out there? Thanks! Quote Link to comment
ken-ji Posted March 27, 2017 Share Posted March 27, 2017 I'll take a stab at this. Please elaborate, you have the VLANs in use accdg to you unRAID config: untagged VLAN (unRAID uses this too) tagged VLAN 10 tagged VLAN 20 tagged VLAN 30 tagged VLAN 100 Which ones are used by your switch? What bridge (br0, br0.5, br0,10, br0.20, br0.30, br0.100, vibr0) is being used by the CentOS VM? what else is tied to the bridges? (# brctl show) Posting the diagnostics should help a lot too. Quote Link to comment
harshl Posted March 27, 2017 Author Share Posted March 27, 2017 8 minutes ago, ken-ji said: I'll take a stab at this. Please elaborate, you have the VLANs in use accdg to you unRAID config: untagged VLAN (unRAID uses this too) tagged VLAN 10 tagged VLAN 20 tagged VLAN 30 tagged VLAN 100 Which ones are used by your switch? What bridge (br0, br0.5, br0,10, br0.20, br0.30, br0.100, vibr0) is being used by the CentOS VM? what else is tied to the bridges? (# brctl show) Posting the diagnostics should help a lot too. Thanks ken-ji! VLANs are as follows: untagged VLAN - same as tagged VLAN 10 and is my primary private network. My switch is able to be configured this way. What happens is all untagged packets get dropped onto VLAN 10, all tagged packets are handled according to the tag. tagged VLAN 20 - extended family access network, same SSID and key at everyones homes provides easy access. tagged VLAN 30 - guest network tagged VLAN 100 - going to be removed, was intended to be the internet facing VLAN, but due to my configuration requirements, limitations within unRAID and my firewall of choice, I ended up passing through a couple of physical NICs instead. tagged VLAN 5 is my management network The switch is setup with LACP to unRAID, that has been working for a long time. The LACP interfaces are participating as follows: VLAN 5 - tagged VLAN 10 - both tagged and untagged traffic VLAN 20 - tagged VLAN 30 - tagged VLAN 5 - tagged The CentOS VM is just for testing and is setup with a single interface on br0.10 and the VM interface within the guest is set for tagging as well. If I strip the tagging from the VM interface config and flip the VM XML config over to br0, it works fine. Here is the output of 'brctl show', vnet1 is the adapter of the CentOS machine being used for testing: root@serv:~# brctl show bridge name bridge id STP enabled interfaces br0 8000.d050996c7db6 no bond0 vnet0 br0.10 8000.d050996c7db6 no bond0.10 vnet1 br0.100 8000.d050996c7db6 no bond0.100 br0.20 8000.d050996c7db6 no bond0.20 br0.30 8000.d050996c7db6 no bond0.30 br0.5 8000.d050996c7db6 no bond0.5 docker0 8000.024253fcfc21 no veth475d643 veth5a423d9 veth6f12e66 vethe997a46 vethf5e5009 virbr0 8000.5254003d3dbe yes virbr0-nic The anonymized diagnostics are attached here as well. Thanks for the response and any tips you can offer! serv-diagnostics-20170326-2217.zip Quote Link to comment
ken-ji Posted March 27, 2017 Share Posted March 27, 2017 I think your problem is somewhat due to the fact that the native(default) VLAN is also present as a subinterface (br0.10) VLAN10. This is a config that never made sense to me. And the Cisco Network Engineers I've spoken with says the native vlan for a link should never be part of the tagged VLANs on that link (unless you are preparing to change the native VLAN between switches with minimal disruption) Also, your CentOS VM can do tagged packets - but it should be on br0 not br0.10 The subinterfaces .X assume that anything going through it from the host (unRAID) will be tagged with VLAN id X. So if you turn on tagging, and are on br0.10, your packets get double tagged. And its anybody's guess where the packet goes from there. 1 Quote Link to comment
harshl Posted March 28, 2017 Author Share Posted March 28, 2017 9 minutes ago, ken-ji said: I think your problem is somewhat due to the fact that the native(default) VLAN is also present as a subinterface (br0.10) VLAN10. This is a config that never made sense to me. And the Cisco Network Engineers I've spoken with says the native vlan for a link should never be part of the tagged VLANs on that link (unless you are preparing to change the native VLAN between switches with minimal disruption) Also, your CentOS VM can do tagged packets - but it should be on br0 not br0.10 The subinterfaces .X assume that anything going through it from the host (unRAID) will be tagged with VLAN id X. So if you turn on tagging, and are on br0.10, your packets get double tagged. And its anybody's guess where the packet goes from there. Hi ken-ji, Thanks for the response. I tested the CentOS host on br0.5 without tagging at the guest and that does in fact work. I guess my problem was, I was trying to treat the br0.x interfaces as if they were a virtual switch, but they aren't, they are a simple bridge that applies the tags automatically. This does point to the issues of br0 and br0.10 carrying the same network. It seems unRAIDs current implementation can't handle that, which is fine. I can work around that now that I know the limitation exists. I need to stop trying to drive this unRAID host as if it was ESXi. As for the config of having both tagged and untagged traffic on the same interface, there are actually many scenarios where you would do this. Take a VoIP phone in an enterprise for instance. The phone is designed to be able to tag it's packets so they can be differentiated from regular traffic, but if a PC is plugged into the VoIP phone, the PC will not tag it's traffic. The phone knows this an passes traffic from the PC port on the phone untagged and the traffic it originates itself tagged on the voice VLAN per the admins configuration. This means the switch must be capable of understanding both the tagged traffic from the phone and the untagged traffic from the PC being sent through the phone in order to drop each traffic type onto the proper VLAN. There are several other scenarios where you would want this type of functionality as well. I think in my situation, I will be able to do everything I need on VLAN 10 with untagged traffic on br0. I will just have to create some additional virtual interfaces which is no big deal. All of the other VLAN bridges seem to be functioning as they should. Now onto implementing the actual firewall device. Thanks for pointing me in the right direction! -Landon Quote Link to comment
ken-ji Posted March 28, 2017 Share Posted March 28, 2017 2 minutes ago, harshl said: As for the config of having both tagged and untagged traffic on the same interface Just to be clear as I seem to have been misunderstood - you normally don't place a single VLAN as both tagged and untagged at the same time on an interface. That's just weird... Quote Link to comment
harshl Posted March 28, 2017 Author Share Posted March 28, 2017 15 minutes ago, ken-ji said: Just to be clear as I seem to have been misunderstood - you normally don't place a single VLAN as both tagged and untagged at the same time on an interface. That's just weird... My explanation below of a phone is exactly what would warrant such a configuration on a switch. I understand why it seems weird on the unRAID host, but on a switch it is relatively common. Thanks again, -Landon Quote Link to comment
bonienl Posted March 28, 2017 Share Posted March 28, 2017 To give some background information on the current VLAN implementation on unRAID. It is purposely done to keep the untagged interface as is, while adding additional VLANs, this prevents that unknowing people shoot themshelves in the foot and cut-off their GUI communication to unraid. As a site-effect you can not mix tagged and untagged traffic. There is no support to set a native VLAN. Your phone example is valid - we use that too, but it is not intended way for unRAID to work. The simple solution, as you have already found is to create a separate VLAN tag for each VM you have and wish to isolate. Quote Link to comment
harshl Posted March 28, 2017 Author Share Posted March 28, 2017 2 hours ago, bonienl said: To give some background information on the current VLAN implementation on unRAID. It is purposely done to keep the untagged interface as is, while adding additional VLANs, this prevents that unknowing people shoot themshelves in the foot and cut-off their GUI communication to unraid. As a site-effect you can not mix tagged and untagged traffic. There is no support to set a native VLAN. Your phone example is valid - we use that too, but it is not intended way for unRAID to work. The simple solution, as you have already found is to create a separate VLAN tag for each VM you have and wish to isolate. Makes perfect sense and I agree entirely. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.