Cisco NX-OS: vPC & Failures

vPC Orphan Ports Traffic from remote Orphan is allowed over Peer Link and exit via local Member Traffic from remote Member is allowed over Peer Link and exit via local Orphan -Orphans ports should be avoided at all costs because PL is a bottleneck of the system Ideal: vPC Peers only have vPC Member Ports and all downstream devices are dual attached vPC Consistency Checks Type 1 Global and Interface Consistency Check if global mismatch - vPC failing to form if interface mismatch - VLANs being suspended Type 2 Consistency Check if mismatch - log messaged but not vPC failure, but could be data plane failures Failure: vPC peer-link failure (link loss) Secondary waits for hold-timeout and keepalive timeouts trying to reach out to the Primary over Keep-alive link After timers expire if vPC Primary is alive: disable Member port on Secondary disable SVI on Secondary => Secondary is disabled => force all traffic to go over Primary if vPC Primary is dead: promote vPC Secondary to Operational Primary traffic over new vPC Primary if vPC Primary is alive: NXOS1(config)# int po50 NXOS1(config-if)# shutdown 2019 Oct 22 05:15:26 NXOS1 %$ VDC-1 %$ %VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on secondary....

October 29, 2019 · 3 min · Dmitry Golovach

Cisco NX-OS: vPC & FHRP

FHRP acts as active/active forwarding over vPC: traffic received in vPC Member Port of FHRP Standby to FHRP Virtual MAC is not forwarded over Peer Link to Active FHRP - essentially HSRP Standby acts as HSRP Active peer-gateway allows to proxy not only virtual active MAC address but also to proxy physical primary MAC address (in case destination MAC address is an Active device, but not a control/management plane of the box itself) the goal - avoid using Peer Link for data plane and it should forward traffic to the upstream L3 router etc in general use GLBP for this behavior, but for vPC it is a default Nexus SVI configuration:...

October 24, 2019 · 2 min · Dmitry Golovach

Cisco NX-OS: From vPC to Back-to-Back vPC

The vPC Peer Link should never be blocking because this link carries important traffic such as the Cisco Fabric Services over Ethernet (CFSoE) Protocol. The peer link is always forwarding. STP from SW8 and SW9: STP from NXOS1 and NXOS2: In the correct design, the vPC Peer Link should be used only in case of failure. All links are up and active: The link between SW8 and NXOS2 is down:...

October 22, 2019 · 2 min · Dmitry Golovach

Cisco NX-OS: vPC Configuration

vPC Order of Operations IP connectivity for Peer Keepalive Enable vPC & LACP globally Create vPC domain define Peer Keepalive address configure vPC role priority (Optional) - lower priority => vPC primary switch. (default 32667) Establish Port Channel for vPC Peer link Verify vPC Consistency Parameters Disable vPC Member Port (optional but recommended) Configure vPC Member Ports Enable vPC Member Ports Make sure keepalive links is up and check IP reachability (mgmt0 could be used)...

October 17, 2019 · 2 min · Dmitry Golovach

Cisco NX-OS: Virtual Port Channel (vPC)

Three Main Types of MCEC (Multi Chassis EtherChannel) C3750 Cross Stack Port Channels (StackWise) single control plane C6500 Virtual Switching System (VSS) single control plane via Virtual Switch Link (VSL) Nexus Virtaul Port Channel (vPC) separate control planes separate control plane protocol instances (STP/IGPs/BGP/FHRP) via a Peer Link (like VSS’s VSL) Each vPC peer has Peer Link to sync control plane between vPC peers (CAM/ARP/IGMP) uses CFSoE (Cisco Fabric Service over Ethernet) used to elect a vPC Primary and vPC Secondary Role normally not used for the data plane => much lower BW Peer Keepalive Link L3 link used as heartbeat in the control plane used to prevent active/active or “Split Brain” vPC roles not used in the vPC data plane could back to back or over routed infrastructure (vrf) vPC Member ports from downstream neighbor the vPC peers is one switch Note: VLANS on vPC Member ports must be added on the Peer link too...

October 15, 2019 · 2 min · Dmitry Golovach

Cisco NX-OS: FEX Designs

Basic topology: Host Port-Channel: vPC Implementation problem - configuration must be synced between different control planes: config sync command Dual vPC or EvPC - Enhanced vPC - only N5K N7K

October 10, 2019 · 1 min · Dmitry Golovach

Cisco NX-OS: Fabric Extender (FEX)

Nexus 2000 Series Fabric Extenders acts as a remote line card of N7K or N5K chassis. N2K FEX - ToR - Top of the Rack N5K/N7K - EoR - End of the Row Why? Solve the problem of wiring cables in the data center. Keep all cables inside the rack as much as possible Simplify the management and reduce number of management devices. All management performed on Parent Switch (management/upgrade etc) Limitations and Parent Switches No local switching inside the N2K FEX 5K as FEX’s Parent static pinning (ports on N2K pinned to the uplink port) vPC topologies FEX ports are L2 switchports 7K as FEX’s Parent not all line cards support FEX static pinning only FEX link must be Port-Channel FEX ports are L2 switchports or native L3 routed interfaces L2 FEX ports are STP “edge” ports run BPDUGuard -> not switch could be connected to the FEX port FEX configuration Enable FEX feature: on N5K:...

October 8, 2019 · 2 min · Dmitry Golovach

Cisco NX-OS: Virtual Device Contexts (VDCs)

Nexus Virtual Device Contexts (VDCs) vitualize physical hardware (like contexts in ASA) also vitualize control plane protocols. Separate control plane per VDC (vlan 10 in VDC 1 is not vlan 10 in VDC 2) Each VDC has its own: Management plane Control plane Data plane Why use VDC: multiple logical roles (Core & Distribution on the same box) VDCs as a managed service to customers lab enviroment for later production use some features can not co-exist in the same VDC (OTV and SVIs) VDC limitations:...

October 3, 2019 · 2 min · Dmitry Golovach

Cisco NX-OS: CLI notes

NX-OS supports aliases “cli alias name <name of alias> <command>” conf t cli alias name wr copy run star** => can use “wr” to save config NX-OS also supports multiple commands aliases. Use “;” as separator: conf t cli alias name commit end ; copy run start ; copy running-config bootflash:///$(SWITCHNAME).$(TIMESTAMP.cfg IOS range command CTRL+0 => clear the screen Tab to complete the command Admins of efault VDC can verify or save config in all VDCs: show run vdc-all copy run start vdc-all NX-OS has checkpoints - save well known working config before the maintenance window start....

September 30, 2019 · 2 min · Dmitry Golovach

Cisco ISE: Update HotSpot access-code Daily

Several times, I run into the question if there is an option to “automatically” change the guest HotSpot access code at a given interval (lets say daily) and I came up with the following solution: ISE API + Python + Task Scheduler Steps: Enable API on ISE Create Python Script Configure Task Scheduler Enable the ERS APIs The ERS APIs are disabled by default. Login to your ISE PAN Navigate to Administration > System > Settings and select ERS Settings from the left panel....

September 26, 2019 · 3 min · Dmitry Golovach