Cisco NX-OS: VXLAN and External Connectivity

By default:

  • underlay – default VRF
  • overlay – “tenant” VRF, hosts in VXLAN are isolated

Border Leafs are used to connect the internal fabric to external networks. Not necessary a box, just configuration on the Leaf. It maintains the following routing control planes:

  1. MP-BGP L2VPN EVPN – inside VXLAN fabric
  2. “tenant” VRF BGP or IGP to external routes
  3. MP-BGP to BGP/IGP redistribution

Main consideration: Border Leaf maintains all /32 host routes for all VRFs, but we need

  • aggregate (summarization during the redistribution) them
  • use traffic engineering based on the longest match (in case there are multiple external routers)

VRF on External router:

  • if there is overlapping addressing inside Tenant networks => VRF => VRF=Lite => some sort of nat is needed
  • if all addresses are unique, it can mix them in the default routing table

Configuration

R1 (External router) Configuration

interface Loopback 0
    ip address 77.77.77.77 255.255.255.255

interface Ethernet0/0
    ip address 77.77.1.1 255.255.255.0

router bgp 777
    bgp log-neighbor-changes
    neighbor 77.77.1.8 remote-as 65001
    address-family ipv4
        network 77.77.77.77 mask 255.255.255.255
        neighbor 77.77.1.8 activate

NXOS8 (Border Leaf) Configuration

interface Ethernet1/7
    description R1 External Router
    no switchport
    vrf member CUSTOMER1
    ip address 77.77.1.8/24
    no shutdown

!

router bgp 1
    log-neighbor-changes
    address-family ipv4 unicast
    neighbor 1.1.1.51
        remote-as 1
        update-source loopback0
        address-family l2vpn evpn
            send-community extended
    neighbor 1.1.1.52
        remote-as 1
        update-source loopback0
        address-family l2vpn evpn
            send-community extended
    vrf CUSTOMER1
        log-neighbor-changes
        address-family ipv4 unicast
            redistribute direct route-map DIRECT
    neighbor 77.77.1.1
        remote-as 777
        local-as 65001
        address-family ipv4 unicast

 

 

R1 receives all routes (/24 and all /32)

  • redistribute direct route-map DIRECT – /24 subnets
  • evpn routes – /32 routes

Note: Cisco made import of the VPNv4 prefixes from L2VPN EVPN into unicast implicit, so there is no “advertise l2vpn evpn command under BGP vrf configuration, which was in NXOS v7 – it happens automatically.

The goal is to advertise the less specific route by using route-map, prefix-lists etc

NXOS8

conf t
ip prefix-list NO_HOST_SUBNETS seq 5 deny 0.0.0.0/0 ge 32
ip prefix-list NO_HOST_SUBNETS seq 10 permit 0.0.0.0/0 le 31
!
router bgp 1
    vrf CUSTOMER1
        neighbor 77.77.1.1
            address-family ipv4 unicast
                prefix-list NO_HOST_SUBNETS out

 
/32 routes have been filtered

We can use any IGP, but BGP is more typical for this design because:

  • redistribution happens automatically
  • easy to apply policy on BGP
  • easier to filter routes and do aggregation

Multiple Exit Points

If there are multiple exit points and configure NXOS7 the same way as NXOS8:

NXOS7

interface Ethernet1/7
    description R1 External Router
    no switchport
    vrf member CUSTOMER1
    ip address 77.77.1.7/24
    no shutdown
!
router bgp 1
    log-neighbor-changes
    neighbor 1.1.1.51
        remote-as 1
        update-source loopback0
        address-family l2vpn evpn
            send-community extended
    neighbor 1.1.1.52
        remote-as 1
        update-source loopback0
        address-family l2vpn evpn
            send-community extended
    vrf CUSTOMER1
        log-neighbor-changes
        address-family ipv4 unicast
            redistribute direct route-map DIRECT
      neighbor 77.77.1.1
          remote-as 777
          local-as 65001
          address-family ipv4 unicast

Traffic engineering: /32 more specific from NXOS7 and /24 from NXOS8, Will fail over if NXOS7 goes down

Here is a way to manage how traffic is leaving – using Local Preference.

LP200 on NXOS8 (more preferred) and default LP100 on NXOS7. It will be advertised to the RR, RR will prefer NXOS8 and reflects to other Leafs (they will see only one possible exit point)

NXOS8

route-map LOCAL_PREF permit 10
    set local-preference 200
!
router bgp 1
    vrf CUSTOMER1
        neighbor 77.77.1.1
             remote-as 777
             local-as 65001
             address-family ipv4 unicast
             prefix-list NO_HOST_SUBNETS out
            route-map LOCAL_PREF in

Remote Leaf now using NXOS8 for 77.77.77.77
Share

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *