Clarification on sk110096 / Trunk connections with Cisco devices

[Perry_McGrew]

We have CP3200's at some remote sites that have P2P VPN to corporate Datacenter.  The current equipment is Public ISP Router > CP3200 > Cisco 3560CX switch > user devices PCs (including Cisco wireless Controller)

All the end user devices are VLAN 1 on Cisco switch that is defined as Access Port connected to CP3200.  I have read sk110096 and see that Gaia does not support VLAN 1 and only accepts untagged traffic.  The Local Wireless Controller's Management interface IP is on the same VLAN 1.  

The Cisco wireless controller is unsupported, and I want to retire it.  There are several VLANs in the Wireless controller that the Access Points use for specific SSIDs.   To remove the local Wireless Controller, the Cisco Access Points would need to be converted to FLEXWAN to connect back to the Wireless Controller in our Datacenter.  But these APs need trunk ports to carry all the subnets / SSIDs

Can this be setup on 3200?  

Thx

[Duane_Toler · Accepted solution]

Yep, I know what you mean.  The article is referring to tagging a native VLAN on a trunk port.  When you're doing VLAN configurations on a host device (not a switch), the untagged VLAN on the host ("eth0" interface) implies the 802.1q "native VLAN" on the switch port.

int gi0/1
sw mode trunk
sw trunk allow vlan 1-50
sw trunk native vlan 20

Here, on a trunk port, VLAN 20 will be untagged as the frame exits the port.  When the end host receives it, untagged, this is the "eth0" interface on the host.  The Linux VLAN driver won't tag VLAN 1, however.  In this particular switch port configuration, if you DID want VLAN 1 to work across the trunk port, it would then have to be tagged; Gaia (the VLAN driver) won't do this.

Likely not an issue for you.  For you, your native (untagged) VLAN on your network is likely already VLAN 1 anyway.

Check it on your switch port (here's mine on one of my switches):

#show int gi0/8 sw
Name: Gi0/8
Switchport: Enabled
Administrative Mode: trunk
Operational Mode: trunk
Administrative Trunking Encapsulation: dot1q
Operational Trunking Encapsulation: dot1q
Negotiation of Trunking: Off
Access Mode VLAN: 20 (External_WAN)
Trunking Native Mode VLAN: 255 (Native)
...
Trunking VLANs Enabled: 10-12,20,254,255,901-903

On your 3200 gateway, your "eth0" (or "eth1", whatever) native interface is the IP for your network's "VLAN 1" subnet.

Configure all the other VLANs like you want (VLAN 10, VLAN 20) for your FlexConnect VLANs, configure the gateway's VLAN sub-interfaces (eth1.10, eth1.20), add those VLANs to the switch port VLAN trunk, and your FlexConnect AP trunk port.

"show span vlan X" to see if you got it right.

Caution:  *PLEASE* use "switchport trunk allowed vlan x,y,z".  Allow only the VLANs you require.  Why?  Just because your host doesn't have a VLAN for that network, doesn't mean the switch won't try to emit tagged frames for it!  If you don't want VLAN 250 on that port, then don't allow it.  Multicast frames gets the same treatment as Broadcast frames, and you can end up flooding your interfaces for traffic you didn't expect (#HasHappened; I got the t-shirt, and the emotional scar).

If you use "sw trunk allow vlan add X" then you can remove a VLAN with "sw trunk allow vlan remove X".  Yes, "sw trunk allow vlan all" is a command but you're not gonna use that, right? 🙂

[Perry_McGrew]

I am hoping this would be valid

For example, on the CP3200

Eth2 (no IP Address) > connected to the Cisco Switch that is defined as Trunk.  The wireless Access Points would be connected to Cisco switch ports)

Eth2.340 (VLAN 340, Native / untagged VLAN on Cisco Switch Port, DHCP Scope on CP3200 with Vendor Option 53)

Eth2.341 (VLAN 341 subnet & DHCP Scope on CP3200)

Eth2.342 (VLAN 342 subnet & DHCP scope on CP3200)

Eth2.343 (VLAN 343 subnet & DHCP scope on CP3200)

Eth2.344 (VLAN 344 subnet & DHCP scope on CP3200)

[Duane_Toler]

Ok so VLAN 340 is your untagged native.  On the switch port, set "sw trunk native vlan 340".

On the Check Point gateway, don't add VLAN 340.  Just use eth2 natively.  That's the "untagged" interface on the host.  Yes, it looks a little odd, but that's correct.  You can't add eth2.340, because "eth2.340" implies you are tagging VLAN 340, which you aren't, because it's.... untagged.  The IP address of the untagged interface eth2 should be the IP of your VLAN 340 subnet. (10.3.40.1, or whatever you are using).

Everything else is ok.

[emmap]

I know it works but it's not supported to use the untagged part of a trunk interface. If there's VLANs configured on an interface, there should be no IP address configured or used on the base interface.

[the_rock]

As @Duane_Toler said, just make sure you are not tagging vlan thats supposed to be untagged. All else looks good to me as well.

Best,

Andy

[Perry_McGrew]

Sorry to come back to this but I want to run this by you for another set of eyes.  These sites are remote and really not looking to have to completely redo the entire setup.  CP3200 is the P2P VPN.  The CP3200 Eth3 is connected to a Cisco Managed Switch using VLAN 1 subnet (192.168.x.y/24) and the CP3200 Eth3 subnet is defined as Ethernet with IP as it is the default GW for end user devices.  The site's devices (PCs, printers etc) are on other Cisco switch ports on the same VLAN 1 subnet.

So I need to add a Cisco Access Point to the mix.   It needs a trunk port to carry the various SSID subnets.  It needs to connect to the wireless controller located back in the corporate datacenter.  

I connected the AP to the Cisco Switch port Gi0/4

interface GigabitEthernet0/4
description Test_AP_1 to CP3200 ETH4
switchport access vlan 340
switchport trunk native vlan 340
switchport mode trunk

Based on what I've read I think I have only 2 options.  One is to move the current 192.168.x.y subnet and all the local devices to another VLAN, make the Cisco Interfaces "switchport access vlan z".  Does the CP3200 Eth3 need to change from Ethernet with IP to Ethernet with VLAN sub-if with the 192.168.x.y /24 IP?   Really not looking to have to do this!

The other thought was to make a second connection from the Cisco Switch to one of the CP3200 unused ports - Eth4 and define CP3200 Eth4 as follows:

Eth4 Ethernet (no IP)

Eth4.340 (VLAN 340 10.20.a.b /24) Native / untagged VLAN on Cisco Switch Port, DHCP Scope on CP3200 with Vendor Option 53)

Eth4.341 (VLAN 341 10.20.c.d /24 subnet & DHCP Scope on CP3200)

Eth4.342 (VLAN 342 10.20.e.f /24 subnet & DHCP scope on CP3200)

Eth4.343 (VLAN 343 10.20.g.h /24 subnet & DHCP scope on CP3200)

Eth4.344 (VLAN 344 10.20.i.j /24 subnet & DHCP scope on CP3200)

I think this should work.  Only question here is from comment @Duane_Toler  made.   Should I define the VLAN 340 IP address directly on Eth4 and NOT as a VLAN sub-if?  

[Duane_Toler]

If VLAN 340 is the untagged native, then you won't have a VLAN 340 subinterface on the host interface (the firewall eth4 interface).  Here's an example:

This is from a customer's firewall.  They use port-channels, but the config is the same for "eth#".

add interface bond0 vlan 3 
add interface bond0 vlan 5 
add interface bond0 vlan 12 
set interface bond0 state on 

set interface bond0 comments "Net_Guest" 
set interface bond0 state on 
set interface bond0 mtu 1500 
set interface bond0 ipv4-address 172.31.31.1 mask-length 24
 
set interface bond0.3 comments "Net_10_10" 
set interface bond0.3 state on 
set interface bond0.3 mtu 1500 
set interface bond0.3 ipv4-address 10.10.10.1 mask-length 24 

set interface bond0.5 comments "Net_10_20" 
set interface bond0.5 state on 
set interface bond0.5 mtu 1500 
set interface bond0.5 ipv4-address 10.10.20.1 mask-length 24 

set interface bond0.12 comments "Net_30_30" 
set interface bond0.12 state on 
set interface bond0.12 mtu 1500 
set interface bond0.12 ipv4-address 172.30.30.1 mask-length 24 

Three VLANs and each are tagged on the interface.  The untagged VLAN (whatever it may be on the switch), is the native interface IP.  If the switch has "switchport trunk native vlan 340", then that's the subnet to use for the untagged host interface; not necessarily "VLAN 1".  That's a switch-side configuration.

You need a VLAN 1 native and untagged interface for your PCs and printers (your current eth3).

For all the other VLANs and the native AP mgmt VLAN, that needs to be a separate interface (eth4).  Eth4 is untagged and will be your native VLAN 340 on the switch and have IP 10.20.a.b/24. 

Like so:

set interface eth3 comments "PCs and printers"
set interface eth3 ipv4-address 192.168.1.254 mask-length 24

set interface eth4 state on
add interface eth4 vlan 341
add interface eth4 vlan 342
add interface eth4 vlan 343
add interface eth4 vlan 344

set interface eth4 comments "native vlan, _sw tr native vlan 340_ on switch"
set interface eth4 ipv4-address 10.20.a.b mask-length 24

set interface eth4.341 comments "VLAN 341, _sw tr allow vlan add 341_ on switch"
set interface eth4.341 ipv4-address 10.20.c.d mask-length 24

set interface eth4.342 comments "VLAN 342, _sw tr allow vlan add 342_ on switch"
set interface eth4.342 ipv4-address 10.20.e.f mask-length 24

set interface eth4.343 comments "VLAN 342, _sw tr allow vlan add 343_ on switch"
set interface eth4.343 ipv4-address 10.20.g.h mask-length 24

set interface eth4.344 comments "VLAN 344, _sw tr allow vlan add 344_ on switch"
set interface eth4.344 ipv4-address 10.20.i.j mask-length 24

For your AP port, add "spanning-tree portfast trunk"  since you can be relatively certain this port is connected to a trunk-capable device.  This will bring up the port a few seconds faster.

One extra tip for all your switch ports:  Add "switchport nonegotiate" as well; access ports and trunk ports.  Especially on Cisco switches, there's a small window when the switch port is coming up where it could be negotiating DTP (for 2 seconds), and with a rogue switch you could force an access port to become a VLAN trunk port.  Once a port is a VLAN trunk, then it will run VTP, and if you don't have VTP disabled (or in transparent mode), then the rogue switch can either destroy your VLAN topology or it can become the root bridge ("all frames flow towards the root"). Best case is this rogue just destabilizes the network; worst case is the rogue runs a mirror port to capture all your packets as it forwards them to an outgoing port.  Extra bonus, "sw nonego" also results in switch ports coming up another 2 seconds faster.

[Perry_McGrew]

@Duane_Toler ,

Thanks for the details.  I am familiar with the Cisco side.  VTP is disabled.   I have a Cisco Access Point connected to the Cisco Switch Gi/0/4

interface GigabitEthernet0/4
description Test_AP_1
switchport trunk native vlan 340
switchport mode trunk
srr-queue bandwidth share 1 30 35 5
priority-queue out
mls qos trust cos
auto qos trust
storm-control broadcast level 15.00
storm-control multicast level 30.00
storm-control action trap
spanning-tree portfast edge trunk.

CISCO-SW#sh int gi0/4 sw
Name: Gi0/4
Switchport: Enabled
Administrative Mode: trunk
Operational Mode: trunk
Administrative Trunking Encapsulation: dot1q
Operational Trunking Encapsulation: dot1q
Negotiation of Trunking: On
Access Mode VLAN: 1 (default)
Trunking Native Mode VLAN: 340 (AP MGT FLEXWAN)
Administrative Native VLAN tagging: disabled
Voice VLAN: none
Administrative private-vlan host-association: none
Administrative private-vlan mapping: none
Administrative private-vlan trunk native VLAN: none
Administrative private-vlan trunk Native VLAN tagging: enabled
Administrative private-vlan trunk encapsulation: dot1q
Administrative private-vlan trunk normal VLANs: none
Administrative private-vlan trunk associations: none
Administrative private-vlan trunk mappings: none
Operational private-vlan: none
Trunking VLANs Enabled: ALL
Pruning VLANs Enabled: 2-1001
Capture Mode Disabled
Capture VLANs Allowed: ALL

Protected: false
Unknown unicast blocked: disabled
Unknown multicast blocked: disabled
Appliance trust: none

The Cisco Switch has a connection from Gi0/12 to the CP3200 Eth3 that enables the VLAN 1 devices.  I added another connection from the Cisco Switch Gi0/11 to CP3200's Eth4 to support the Access Point subnets

interface GigabitEthernet0/11
description CP3200 Port Eth4
switchport trunk allowed vlan 340-344
switchport trunk native vlan 340
switchport mode trunk
spanning-tree portfast edge trunk

The Access Point is preconfigured to use DHCP for local IPv4 and register to the Wireless Controller (using VLAN 340) located in our Corp Datacenter. The AP has the Wireless Controller's IP hard coded.  I can Ping the Wireless Controller from the CP3200's site and I can Ping all the VLAN IPs I created on the CP3200 Eth4 from the Corp HQ.  So that tells me the traffic can traverse the VPN tunnel. 

Yesterday morning, I created the eth4 interface and the VLAN Sub-Ifs.  They look like they match what your CLI commands would have created.   I have attached screen shot from the CP3200 WebUI of eth3 (the "VLAN 1" local PCs etc, connected to the switch) and the Eth4 I am trying to get to work with the Wi-Fi subnets / VLANs.   

It still is not working.   I am not familiar with the bond0 on the CP3200.   Are these required? 

I had also created and enabled DHCP on the CP3200 for all the VLAN 340 - 344 subnets.  The Access Point is NOT getting an IP from the CP3200.  I have not figured this out yet -- and may need to find a debug to troubleshoot.  The local devices are getting IPs from the CP3200 "VLAN 1" connection on Eth3.  Unless those Bond0 ports are needed, I wonder if DHCP issue is the reason.

CISCO-SW#sh cdp nei gi0/4 detail
-------------------------
Device ID: TEST_AP_1
Entry address(es):
IPv6 address: FE80::3FD0:3295:F19C:EC7E (link-local)
Platform: cisco CW9164I-B, Capabilities: Router Trans-Bridge
Interface: GigabitEthernet0/4, Port ID (outgoing port): GigabitEthernet0
Holdtime : 178 sec

Version :
Cisco AP Software, ap1g6a-k9w8 Version: 17.12.2.35
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 2014-2015 by Cisco Systems, Inc.

advertisement version: 2
Power drawn: 30.000 Watts
Power request id: 4013, Power management id: 2
Power request levels are:30000 15400 0 0 0
Management address(es):

CISCO-SW#sh mac address-table interf gi0/4
Mac Address Table
-------------------------------------------

Vlan Mac Address Type Ports
---- ----------- -------- -----
340 c4d6.6622.5b20 DYNAMIC Gi0/4

Thoughts?  

[Duane_Toler]

Looks good so far.  No, you don't need the bond0 interfaces; that's just an example from my other customer. 

On the switch, "show mac addr vlan 340" should should you MAC addresses on Gig0/4 and Gig/0/11.  Gig0/4 will be that AP MAC, and Gig0/11 will be the MAC addr of your 3200 appliance eth4 interface.

"show mac addr int gi/0/11" will show you VLANs 340-344 and the same MAC addr for the 3200 appliance eth4 interface.

If you don't see that, then did you define the VLANs? Spanning tree instances have to be created for them.

conf t
vlan 340
name Net_10.20.0
vlan 341
name Net_10.20.1
vlan 342
name Net_10.20.2
vlan 343
name Net_10.20.3
vlan 344
name Net_10.20.4
end
!
show vlan brief
show span vlan 340
show span vlan 341
show span vlan 342
show span vlan 343
show span vlan 344

Also prune all the VLANs from your AP port (or just set "sw tr allow vlan X-Y").  Otherwise, BUMs on VLAN 1 are forwarding out that port, too.

If you have VLANs, and spanning-tree, and you see MACs on the expected ports, then check your DHCP configuration, wherever your DHCP server is.  If you're using DHCP-relay (bootp relay) on the firewall, then the bootp relay commands need to specify the interface (or VLAN sub interface) to listen for those packets and forward them. 

[Perry_McGrew]

@Duane_Toler ,

I will upload text file with more of the configs and verifying outputs.   I really can't see where / why it's not working.  At least the AP should be getting an IP address assigned by the CP3200 DHCP I set up.  I did edit /etc/dhcpd.conf to add the Option 43 to the VLAN 340 IP pool.  It provides the IP address of the Wireless Controller the AP needs to register too.   I pre-setup the AP in my office and inputted the Wireless Controller's IP address.  The FW logs show nothing as well as the Wireless Controller AP Join Logs.  dhcpd is running...

[Perry_McGrew]

I just set up the Cisco Switch as DHCP server and the AP acquired IPv4 address and registered to the Wireless Controller thru the VPN tunnel.

[Duane_Toler]

Yeah that's probably gonna be your better bet.  That'll be easier to manage with the extra options.

The output you uploaded had a curious item missing, tho.  "sh span vlan 340" didn't show the AP MAC address, but the AP could have been rebooting when you ran that command.  "show mac addr vlan 340" did show it later, tho, which is good.

All of your other outputs look good and as expected.  Since you moved the DHCP services and tested, now you know the DHCP issue was in the firewall policy.  Of course, now you have to get a wireless client on the VLAN 34x WLANs. 🙂

Be sure you have the FlexConnect VLANs added to that FlexConnect template and you should be good to go!  If you have any issues, "show mac addr vlan X" will be your best troubleshooting command on the switch, and "arp -n -i eth4.X" on the 3200 appliance.

Happy networking!

[Perry_McGrew]

Yes.   The AP's will timeout and reboot to go thru the discovery process.  Our small sites with CP3200s have Cisco 2504 WLCs with 2702i APs.  The 2504's are unsupported.   I replaced all our 5508's (also unsupported) with single Cisco 9800L-F WLC in the Datacenter and the unsupported APs with 91xx series models.  Going from 5508/2504 AireOS to 9800L IOS-XE has been a chore.  This was my 1st attempt to install APs at a CP3200 site and have it connect (FlexWan) to the datacenter's 9800L across the P2P VPN.  

I am going to move all the DHCP function off CP3200 to the local Cisco Switch.  It's easy to set up and seems to be more ways to debug it.  I'm probably going to convert these sites to SDWAN through our service provider which would eliminate the CP3200's which go EoL in 2025.  

Thanks for the 2nd set of eyes on the config issues.   Helped me validate the setup.  Would not hurt if CP could create an SK that clearly showed how to connect a CP device to a Trunk port.  Sometimes a simple diagram is worth 1,000 words! 

Perry

[Duane_Toler]

Excellent to hear, except for tossing the out the Check Point gear. 😞  Glad it's working for you.  I would agree about moving DHCP to the L3 switch.  Gaia DHCP is meant to "get you online", not "provide complex detailed specific needs" (at least not yet).  You could also use DNS records for CISCO-CAPWAP-CONTROLLER.domain instead of Option 43 (for DHCP servers that are hard to configure with options.... like Gaia ).

Doing VLANs in CLISH makes more sense than WebUI; there's a mismatch in WebUI between some of the options in CLISH such as what you were running into.  Plus, WebUI can get you in a stuck position when trying to undo some configs.  Portchannels (bonds) have a similar mismatch.  802.1q VLANs are "interesting" (versus Ye Olde ISL VLANs) because of the provision for untagged/native, so you have to be mindful of that.  ISL VLANs tagged everything.  Then the "management VLAN" concept dials it up one notch higher, depending on your switch.  Management VLAN != Native VLAN != VLAN 1.  (but by default, they're all VLAN 1) 

Additionally don't fall into the trap thinking that every L2 VLAN needs a matching "interface VLANx". Doing so creates a switch-local VLAN routing option that can be a backdoor on your local network.  The SVI "interface VLANx" is for switch-local routing and/or management VLAN purposes.  You only need 1 management VLAN.

Now you have a working template for any of your other sites!  Copy, paste, paste, paste, paste. 🙂  Enjoy!