Two control planes for the VXLAN technology:
- Multicast control plane (flood and learn)
- MP-BGP EVPN control plane
Virtual Extensible Local Area Networks (VXLANs) allows to extend reachability of a VLAN within a data center over Layer 3.
Every VTEP with specific VXLAN and certain VNI will join the same multicast group. To learn remote MAC addresses, the VTEP will use conversational MAC address learning technique: learn only actively speaking MAC addresses.
note: Always Read Guidelines and Limitation for specific hardware
Cisco Nexus 5600 Series NX-OS Layer 2 Switching Configuration Guide, Release 7.x
- Set switching mode to store-and-forwarding
- IP unicast reachability between VTEPs
- PIM BIDIR reachability between VTEPs
- Enable features “vn-segment-vlan-based” and “nv overlay”
conf t hardware ethernet store-and-fwd-switching copy run start reboot
IP unicast reachability between VTEPs
EIGRP as an example. The goal is just connectivity between Loopback interfaces:
feature eigrp ! router eigrp 100 address-family ipv4 unicast autonomous-system 100 router-id 126.96.36.199 interface Ethernet1/1 no switchport ip address 172.16.12.1/30 ip router eigrp 100 no shutdown interface Ethernet1/2 no switchport ip address 172.16.13.1/24 ip router eigrp 100 no shutdown interface loopback0 ip address 188.8.131.52/32 ip router eigrp 100
PIM BIDIR reachability between VTEPs
Configure PIM first, check that is working and than enable BIDIR. Configure PIM with rp-address 184.108.40.206 (NXOS3) on all Nexus devices:
conf t feature pim ip pim rp-address 220.127.116.11 group-list 18.104.22.168/4 interface loopback0 ip pim sparse-mode interface Ethernet1/1 ip pim sparse-mode interface Ethernet1/2 ip pim sparse-mode
How to check multicast receiving:
On one Leaf Nexus Loopback interface configure join-group:
NXOS1: interface loopback0 ip igmp join-group 22.214.171.124
Check (*, 126.96.36.199 ) in the RP pim routes => receiver is ok:
How to check multicast sending:
On the second Leaf Nexus run “ping multicast 188.8.131.52 interface e1/1“
NXOS2# ping multicast 184.108.40.206 interface e1/1 PING 220.127.116.11 (18.104.22.168): 56 data bytes 64 bytes from 172.16.12.1: icmp_seq=0 ttl=254 time=6.742 ms 64 bytes from 172.16.12.1: icmp_seq=1 ttl=254 time=10.17 ms 64 bytes from 172.16.12.1: icmp_seq=2 ttl=254 time=13.882 ms 64 bytes from 172.16.12.1: icmp_seq=3 ttl=254 time=11.793 ms 64 bytes from 172.16.12.1: icmp_seq=4 ttl=254 time=8.43 ms --- 22.214.171.124 ping multicast statistics --- 5 packets transmitted, From member 172.16.12.1: 5 packets received, 0.00% packet loss --- in total, 1 group member responded ---
Check on the RP pim routes. There should be 2 entries: (*, 126.96.36.199 ) and for Leaf NXOS2 (172.16.12.2/32, 188.8.131.52/32) – (S,G) – sender is ok:
To enable bidirectional (PIM BIDIR) – configure switches with the following command (just add bidir):
ip pim rp-address 184.108.40.206 group-list 220.127.116.11/4 bidir
Do the same test with join-group 18.104.22.168 and on the rp there should be just one route with (*,G) => bidirectional is ok:
Why BIDIR? It will be used for BUM traffic (Broadcast, Unknown-Unicast and Multicast traffic) – like ARP
conf t feature vn-segment-vlan-based feature nv overlay
Mapping VLAN to VXLAN
conf t vlan 10 vn-segment 101010
Create Network Virtualization Edge (VNE) interface
conf t interface nve1 source-interface loopback0 member vni 101010 mcast-group 22.214.171.124
There is communication between SW5 and SW6 over VXLAN:
SW5#ping 10.10.10.6 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.10.10.6, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 14/15/16 ms
A Classic Remote VTEP Discovery from here:
- End station sends Address Resolution Protocol (ARP) packet for remote end station.
- Packet reaches to VTEP-A and since VTEP-A does not know about the VTEP-B, it encapsulates the packet inside VXLAN header. It puts the multicast IP address as the destination IP address. Since the same multicast address is used by all VTEPs, all joins the same multicast group.
- This packet reaches to all VTEP and is decapsulated, in this way remote VTEP learns about the other VTEP. Since the decapsulated VTEP has the VNID, it is forwarded in the VLAN that has the same VNID configured.
- Now, remote end sends the ARP reply packet and it reaches to VTEP-B, since now VTEP-B knows about VTEP-A it again encapsulates the orginal frame but now the destination IP address is of VTEP-B and it is the unicast IP address.
- ARP reply reaches to VTEP-A and now VTEP-A gets to know about VTEP-B it forms the neighbor relationship with VTEP-B.
show mac address-table show nve peer show nve vni