Two control planes for the VXLAN technology:
- Multicast control plane (flood and learn)
- MP-BGP EVPN control plane
Virtual Extensible Local Area Networks (VXLANs) allows to extend reachability of a VLAN within a data center over Layer 3.
Every VTEP with specific VXLAN and certain VNI will join the same multicast group. To learn remote MAC addresses, the VTEP will use conversational MAC address learning technique: learn only actively speaking MAC addresses.
note: Always Read Guidelines and Limitation for specific hardware
Cisco Nexus 5600 Series NX-OS Layer 2 Switching Configuration Guide, Release 7.x
- Set switching mode to store-and-forwarding
- IP unicast reachability between VTEPs
- PIM BIDIR reachability between VTEPs
- Enable features “vn-segment-vlan-based” and “nv overlay”
conf t hardware ethernet store-and-fwd-switching copy run start reboot
IP unicast reachability between VTEPs
EIGRP as an example. The goal is just connectivity between Loopback interfaces:
feature eigrp ! router eigrp 100 address-family ipv4 unicast autonomous-system 100 router-id 18.104.22.168 interface Ethernet1/1 no switchport ip address 172.16.12.1/30 ip router eigrp 100 no shutdown interface Ethernet1/2 no switchport ip address 172.16.13.1/24 ip router eigrp 100 no shutdown interface loopback0 ip address 22.214.171.124/32 ip router eigrp 100
PIM BIDIR reachability between VTEPs
Configure PIM first, check that is working and than enable BIDIR. Configure PIM with rp-address 126.96.36.199 (NXOS3) on all Nexus devices:
conf t feature pim ip pim rp-address 188.8.131.52 group-list 184.108.40.206/4 interface loopback0 ip pim sparse-mode interface Ethernet1/1 ip pim sparse-mode interface Ethernet1/2 ip pim sparse-mode
How to check multicast receiving:
On one Leaf Nexus Loopback interface configure join-group:
NXOS1: interface loopback0 ip igmp join-group 220.127.116.11
Check (*, 18.104.22.168 ) in the RP pim routes => receiver is ok:
How to check multicast sending:
On the second Leaf Nexus run “ping multicast 22.214.171.124 interface e1/1“
NXOS2# ping multicast 126.96.36.199 interface e1/1 PING 188.8.131.52 (184.108.40.206): 56 data bytes 64 bytes from 172.16.12.1: icmp_seq=0 ttl=254 time=6.742 ms 64 bytes from 172.16.12.1: icmp_seq=1 ttl=254 time=10.17 ms 64 bytes from 172.16.12.1: icmp_seq=2 ttl=254 time=13.882 ms 64 bytes from 172.16.12.1: icmp_seq=3 ttl=254 time=11.793 ms 64 bytes from 172.16.12.1: icmp_seq=4 ttl=254 time=8.43 ms --- 220.127.116.11 ping multicast statistics --- 5 packets transmitted, From member 172.16.12.1: 5 packets received, 0.00% packet loss --- in total, 1 group member responded ---
Check on the RP pim routes. There should be 2 entries: (*, 18.104.22.168 ) and for Leaf NXOS2 (172.16.12.2/32, 22.214.171.124/32) – (S,G) – sender is ok:
To enable bidirectional (PIM BIDIR) – configure switches with the following command (just add bidir):
ip pim rp-address 126.96.36.199 group-list 188.8.131.52/4 bidir
Do the same test with join-group 184.108.40.206 and on the rp there should be just one route with (*,G) => bidirectional is ok:
Why BIDIR? It will be used for BUM traffic (Broadcast, Unknown-Unicast and Multicast traffic) – like ARP
conf t feature vn-segment-vlan-based feature nv overlay
Mapping VLAN to VXLAN
conf t vlan 10 vn-segment 101010
Create Network Virtualization Edge (VNE) interface
conf t interface nve1 source-interface loopback0 member vni 101010 mcast-group 220.127.116.11
There is communication between SW5 and SW6 over VXLAN:
SW5#ping 10.10.10.6 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.10.10.6, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 14/15/16 ms
A Classic Remote VTEP Discovery from here:
- End station sends Address Resolution Protocol (ARP) packet for remote end station.
- Packet reaches to VTEP-A and since VTEP-A does not know about the VTEP-B, it encapsulates the packet inside VXLAN header. It puts the multicast IP address as the destination IP address. Since the same multicast address is used by all VTEPs, all joins the same multicast group.
- This packet reaches to all VTEP and is decapsulated, in this way remote VTEP learns about the other VTEP. Since the decapsulated VTEP has the VNID, it is forwarded in the VLAN that has the same VNID configured.
- Now, remote end sends the ARP reply packet and it reaches to VTEP-B, since now VTEP-B knows about VTEP-A it again encapsulates the orginal frame but now the destination IP address is of VTEP-B and it is the unicast IP address.
- ARP reply reaches to VTEP-A and now VTEP-A gets to know about VTEP-B it forms the neighbor relationship with VTEP-B.
show mac address-table show nve peer show nve vni