nxos and inter-vlan ACLs
-
Hello ITPro. I am pulling my hair out trying to figure out how ACLs work on the nxos platform with inter-vlan routing. Hoping someone with experience can explain to me why with the following configuration, packets from source 192.168.120.0/24 are not blocked when destined for the 192.168.119.0/24 network.
Note, this is an HSRP setup, so two switches sharing virtual IP for gateway on both vlans.
WILNXLAB-01: interface Vlan1 no shutdown ip address 192.168.119.3/24 hsrp version 2 hsrp 119 preempt priority 90 ip 192.168.119.2 track 119 track 120 track 121 track 122 interface Vlan120 no shutdown ip access-group test in <-----------enable ACL ip address 192.168.120.3/24 hsrp version 2 hsrp 120 preempt priority 90 ip 192.168.120.2 track 119 track 120 track 121 track 122 WILNXLAB-02: WILNXLAB-02: interface Vlan1 no shutdown ip address 192.168.119.4/24 hsrp version 2 hsrp 119 preempt priority 80 ip 192.168.119.2 track 119 track 120 track 121 track 122 interface Vlan120 no shutdown ip access-group test in <-----------enable ACL ip address 192.168.120.4/24 hsrp version 2 hsrp 120 preempt priority 80 ip 192.168.120.2 track 119 track 120 track 121 track 122 ---------------------Access List on each switch IP access list test 10 deny ip 192.168.120.0 0.0.0.255 any 20 deny icmp 192.168.120.0 0.0.0.255
-
@Adam-Tyler said in nxos and inter-vlan ACLs:
IP access list test
10 deny ip 192.168.120.0 0.0.0.255 any
20 deny icmp 192.168.120.0 0.0.0.255try the following to see what shows up:
show running-config aclmgr
show vlan filter
show vlan access-map
You may have to consider using VLAN ACLs (VACL) instead of traditional ACLs.
switch(config)# show running-config aclmgr !Command: show running-config aclmgr !Time: Tue Apr 9 20:23:04 2019 ip access-list DENY_ACL_MAP 10 deny ip 192.168.120.0/24 any 20 deny icmp 192.168.120.0/24 any 30 permit ip any any vlan access-map DENY_TEST 10 match ip address DENY_ACL_MAP action drop vlan filter DENY_TEST vlan-list 120
Now run the same verification commands now
show running-config aclmgr
show vlan filter
show vlan access-map
Let me know if I've not missed your point completely.
-
Thanks Ronnie, you are tracking right with me. I thought of using VACL's too, but I can't get it work for some reason. Here is the output from "show run aclmgr" after entering your commands. Both workstations can still connect via ping and SMB with this configuratoin, just tested.. The heck?
WILNXLAB-01(config)# show run aclmgr !Command: show running-config aclmgr !Running configuration last done at: Tue Apr 9 13:49:37 2019 !Time: Tue Apr 9 13:49:40 2019 version 7.0(3)I7(6) Bios:version ip access-list DENY_ACL_MAP 10 deny ip 192.168.120.0/24 any 20 deny icmp 192.168.120.0/24 any 30 permit ip any any vlan access-map DENY_TEST 10 match ip address DENY_ACL_MAP action drop vlan filter DENY_TEST vlan-list 120
WILNXLAB-02(config)# show run aclmgr !Command: show running-config aclmgr !Running configuration last done at: Tue Apr 9 13:49:33 2019 !Time: Tue Apr 9 13:49:52 2019 version 7.0(3)I7(6) Bios:version ip access-list DENY_ACL_MAP 10 deny ip 192.168.120.0/24 any 20 deny icmp 192.168.120.0/24 any 30 permit ip any any vlan access-map DENY_TEST 10 match ip address DENY_ACL_MAP action drop vlan filter DENY_TEST vlan-list 120
-
try this...I think I messed up and forgot that when using match statements, the extended ACL to use must permit, then match can drop it (yeah, intuitive right?). Try this. I don't have a complete lab setup for testing only the NX-OS demo.
SW1(config)# show running-config aclmgr !Command: show running-config aclmgr !Time: Wed Apr 10 10:20:24 2019 version 7.3(0)D1(1) ip access-list DENY2VLAN1 10 permit ip 192.168.120.0/24 any 20 permit icmp 192.168.120.0/24 any vlan access-map DENY2VLAN1MAP 10 match ip address DENY2VLAN1 action drop vlan filter DENY2VLAN1MAP vlan-list 120
hopefully that works...
or try ..and configure the filter on vlan 1 instead?
this is me is relatively confusing since these things don't have ingress or egress attached to them. sigh -
Thanks Ronnie. I did actually try the permit statements on the VACL approach too. I've tried a bunch of combinations now and the only thing I can seem to get working reliably is as follows..
This allows anything on the 192.168.119.0/24 network to ping anything on the 192.168.120.0/24 network as well as the reverse. It ONLY allows hosts on the 192.168.120.0/24 network to access SMB shares hosted on the 192.168.119.0/24 network (VLAN1). Additionally all other traffic is blocked. I am able to enable "statistics per-entry" and see the hit counter of the "deny ip any any" entry increment. Again, it only ever seems to block traffic "egressing/out" to its final destination.
ip access-list 120_out statistics per-entry 10 remark ------Allow file sharing return traffic to VLAN 120 20 permit udp 192.168.119.0/24 eq netbios-ss 192.168.120.0/24 30 permit udp 192.168.119.0/24 eq netbios-ns 192.168.120.0/24 40 permit tcp 192.168.119.0/24 eq 137 192.168.120.0/24 50 permit tcp 192.168.119.0/24 eq 135 192.168.120.0/24 60 permit tcp 192.168.119.0/24 eq 139 192.168.120.0/24 70 permit tcp 192.168.119.0/24 eq 445 192.168.120.0/24 80 remark ------Allow echo to VLAN 120 90 permit icmp 192.168.119.0/24 any echo 100 remark ------Allow echo reply traffic to VLAN 120 110 permit icmp any any echo-reply 120 remark ------Allow ospf 130 permit ospf any any 140 remark ------Deny everthing else 150 deny ip any any ip access-list 1_out statistics per-entry 10 remark ------Allow file sharing to VLAN 1 20 permit udp 192.168.120.0/24 192.168.119.0/24 eq netbios-ss 30 permit udp 192.168.120.0/24 192.168.119.0/24 eq netbios-ns 40 permit tcp 192.168.120.0/24 192.168.119.0/24 eq 445 50 permit tcp 192.168.120.0/24 192.168.119.0/24 eq 139 60 permit tcp 192.168.120.0/24 192.168.119.0/24 eq 135 70 permit tcp 192.168.120.0/24 192.168.119.0/24 eq 137 80 remark ------Allow echo to VLAN 1 90 permit icmp 192.168.120.0/24 any echo 100 remark ------Allow echo reply traffic to VLAN1 110 permit icmp any any echo-reply 120 remark ------Allow ospf 130 permit ospf any any 140 remark ------Deny everything else 150 deny ip any any interface Vlan1 ip access-group 1_out out interface Vlan120 ip access-group 120_out out
So it's a mystery as to why ACLs only seem to work when applied to each interface "out" instead of "in". Any ideas on that one?
Additionally I did a ton of testing with the "established" ACL option trying to get the switch to respect return flow at least for TCP traffic. For example: 40 permit tcp 192.168.120.0/24 192.168.119.0/24 eq 445 established. This unfortunately never worked. I am surprised the Nexus doesn't seem to support session aware ACLs?
Why didn't the VACL work? I was reading an article recently about VACLs and there was some confusion around the VACL applying to traffic within the same VLAN only vs routed/inter-vlan traffic..
Thanks for your help and thoughts.
Regards,
Adam Tyler -
@Adam-Tyler said in nxos and inter-vlan ACLs:
Why does this work?
You must view this from the standpoint of the vlan interface: So don't look at it independently but something similar to the following e.g. :
interface Vlan1
ip access-group 1_out inshould be viewed as more as
inside to outside
interface Vlan1
ip access-group 1_out outshould be viewed as more as
outside to inside
according to https://community.cisco.com/t5/switching/acl-between-vlans-on-3560g/td-p/1539959
This is why this works.
-
This post is deleted!
-
Thanks Ronnie, I am following the logic. Based on the article and your comments, it sounds like perhaps it's suggested that the ACL is simply built incorrectly to successfully apply inbound? Just to 100% confirm, I went ahead and created a new ACL that shows as follows:
ip access-list 120_in statistics per-entry 10 deny ip any any log 20 deny icmp any any log
After applying this to interface vlan 120 "in", ICMP still happily flows between hosts on VLAN 120 and VLAN 1. Did I miss something, because this is still not making any sense to me... This rule should block inbound echo from VLAN 120...? It should not apply to echo-reply coming from 119.
I also am getting no statistic counters or log entries indicating the rule is being actively applied to packets.
Regards,
Adam Tyler -
@Adam-Tyler said in nxos and inter-vlan ACLs:
I also am getting no statistic counters or log entries indicating the rule is being actively applied to packets.
You need at least one permit statement.
SW1# show access-list 120_in IP access list 120_in statistics per-entry 10 deny ip any any log [match=0] 20 deny icmp any any log [match=0] 30 permit ip any any [match=0]
I'm still working the answer for the other part... let me get some more information
are you doing this between vlans on the same switch? or between vlans on different switches? -
Hmm... Still no dice.. Starting to wonder if I have buggy firmware or something. These are 9000v VMware virtual machines running in the lab. It was a real trick figuring out how to get a VM on the same VMware host to connect to a switchport on the "virtual" nexus switch. I ended up making a separate VMware vSwitch for each host that needed to connect.
IP access list 120_in<------------Still not counters.. statistics per-entry 8 deny icmp any any log 9 deny ip any any log 10 permit ip any any log 20 permit icmp any any log WILNXLAB-01(config)# show run int vlan 120 !Command: show running-config interface Vlan120 !Running configuration last done at: Fri Apr 12 14:51:50 2019 !Time: Fri Apr 12 14:52:07 2019 version 7.0(3)I7(6) Bios:version interface Vlan120 no shutdown ip access-group 120_in in ip access-group 120_out out
Outbound seems to work exactly as expected.... WEIRD!
WILNXLAB-01(config)# show ip access-lists 120_out IP access list 120_out statistics per-entry 10 remark ------Allow file sharing return traffic to VLAN 120 20 permit udp 192.168.119.0/24 eq netbios-ss 192.168.120.0/24 [match=0] 30 permit udp 192.168.119.0/24 eq netbios-ns 192.168.120.0/24 [match=0] 40 permit tcp 192.168.119.0/24 eq 137 192.168.120.0/24 [match=0] 50 permit tcp 192.168.119.0/24 eq 135 192.168.120.0/24 [match=0] 60 permit tcp 192.168.119.0/24 eq 139 192.168.120.0/24 [match=0] 70 permit tcp 192.168.119.0/24 eq 445 192.168.120.0/24 [match=9326] 80 remark ------Allow echo to VLAN 120 90 permit icmp 192.168.119.0/24 any echo [match=0] 100 remark ------Allow echo reply traffic to VLAN 120 110 permit icmp any any echo-reply [match=83322] 120 remark ------Allow ospf 130 permit ospf any any [match=0] 140 remark ------Deny everthing else 150 deny ip any any [match=18]
-
remember on traffic between vlans. The inbound and outbound terminology are weird. We have to think of directionality for the processing.
ip access-group 120_in in
when applied to the vlan interface is (from within the VLAN 120 outbound to another) . Andip access-group 120_in out
when applied to the vlan interface means (from outside the vlan 120 heading in to the vlan).so it's a bit twisted in our syntax to the meaning of the ACL.
-
Hey Ronnie, sorry for the delay on this. I checked in with a few consultants we work with regularly and the consensus is that this Cisco device is not working as it should. It's a 9000v Nexus virtual appliance and apparently ACLs aren't officially supported. Super interested in testing inbound ACLs on a physical Nexus device, but I don't have a spare handy.
Regards,
Adam Tyler -
@Adam-Tyler,
Yes...I only have access to the simulator -
Just to close the loop here, I ended up buying a couple of nexus devices for the lab and could not re-create this problem. It seems the nexus virtual appliance simply doesn't support inbound ACLs. That was fun. You just can't get the same level of design experience from virtual labs in some cases.
Regards,
Adam Tyler -
Yes. Sometimes there are limitations to what they can do. sigh