Dell EMC's portfolio covers these key areas to provide the best in service and customer ..... automated settings and tar
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network VLT and Spanning Tree configuration examples for Dell MXL and M I/O Aggregator modular switches connected to Cisco Nexus 5000 series switches Dell Networking Solutions Engineering May 2016
A Dell EMC Deployment Guide
Revisions Date
Revision
Description
Authors
May 2016
2.0
Revised document throughout. Added VLT feature and M IOA platform.
Jim Slaughter, Dennis Dadey, Davis Smith
Dec 2012
1.1
Original document titled "Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment"
Jason Pearce, Manjesh Siddamurthy, Kevin Horton
Copyright © [2012-2016] Dell Inc. or its subsidiaries. All Rights Reserved. Except as stated below, no part of this document may be reproduced, distributed or transmitted in any form or by any means, without express permission of Dell. You may distribute this document within your company or organization only, without alteration of its contents. THIS DOCUMENT IS PROVIDED “AS-IS”, AND WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED. IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE SPECIFICALLY DISCLAIMED. PRODUCT WARRANTIES APPLICABLE TO THE DELL PRODUCTS DESCRIBED IN THIS DOCUMENT MAY BE FOUND AT: http://www.dell.com/learn/us/en/vn/terms-of-sale-commercial-and-public-sector-warranties Performance of network reference architectures discussed in this document may vary with differing deployment conditions, network loads, and the like. Third party products may be included in reference architectures for the convenience of the reader. Inclusion of such third party products does not necessarily constitute Dell’s recommendation of those products. Please consult your Dell representative for additional information. Trademarks used in this text: Dell™, the Dell logo, Dell Boomi™, PowerEdge™, PowerVault™, PowerConnect™, OpenManage™, EqualLogic™, Compellent™, KACE™, FlexAddress™, Force10™ and Vostro™ are trademarks of Dell Inc. EMC VNX®, and EMC Unisphere® are registered trademarks of Dell. Other Dell trademarks may be used in this document. Cisco Nexus®, Cisco MDS®, Cisco NX-0S®, and other Cisco Catalyst® are registered trademarks of Cisco System Inc. Intel®, Pentium®, Xeon®, Core® and Celeron® are registered trademarks of Intel Corporation in the U.S. and other countries. AMD® is a registered trademark and AMD Opteron™, AMD Phenom™ and AMD Sempron™ are trademarks of Advanced Micro Devices, Inc. Microsoft®, Windows®, Windows Server®, Internet Explorer®, MS-DOS®, Windows Vista® and Active Directory® are either trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Red Hat® and Red Hat® Enterprise Linux® are registered trademarks of Red Hat, Inc. in the United States and/or other countries. Novell® and SUSE® are registered trademarks of Novell Inc. in the United States and other countries. Oracle® is a registered trademark of Oracle Corporation and/or its affiliates. VMware®, Virtual SMP®, vMotion®, vCenter® and vSphere® are registered trademarks or trademarks of VMware, Inc. in the United States or other countries. IBM® is a registered trademark of International Business Machines Corporation. Broadcom® and NetXtreme® are registered trademarks of QLogic is a registered trademark of QLogic Corporation. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and/or names or their products and are the property of their respective owners. Dell disclaims proprietary interest in the marks and names of others.
2
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Table of contents Revisions.............................................................................................................................................................................2 1
Introduction ...................................................................................................................................................................5
2
Hardware ......................................................................................................................................................................7 2.1
Dell PowerEdge M1000e ....................................................................................................................................7
2.2
M1000e MXL and M I/O Aggregator blade switches ..........................................................................................7
2.2.1 Dell Networking MXL 10/40Gb Ethernet Switch .................................................................................................8 2.2.2 Dell PowerEdge M I/O Aggregator .....................................................................................................................8 2.2.3 FlexIO Expansion Modules .................................................................................................................................9 3
Scenario One – VLT Environment..............................................................................................................................10 3.1
VLT Overview ...................................................................................................................................................10
3.2
Spanning Tree with VLT ...................................................................................................................................11
3.3
VLT Interoperability with Cisco Nexus vPC ......................................................................................................12
3.3.1 Dell Networking VLT Components ...................................................................................................................12 3.3.2 Cisco Nexus vPC Components ........................................................................................................................13 3.4
VLT and vPC Configuration ..............................................................................................................................13
3.4.1 MXL VLT Configuration ....................................................................................................................................15 3.4.2 M IOA VLT Configuration .................................................................................................................................18 3.4.3 Nexus vPC Configuration .................................................................................................................................20 3.4.4 Server Network Adapter Configuration .............................................................................................................23 3.5
Commands to verify VLT and vPC ...................................................................................................................23
3.5.1 MXL / M IOA commands and output ................................................................................................................23 3.5.2 Nexus commands and output ...........................................................................................................................28 4
Scenario Two – Non-VLT Environment ......................................................................................................................35 4.1
Overview ...........................................................................................................................................................35
4.2
Configuration ....................................................................................................................................................36
4.3
PVST+ Configuration ........................................................................................................................................36
4.3.1 MXL Configuration ............................................................................................................................................38 4.3.2 Nexus Configuration .........................................................................................................................................41 4.3.3 Server Network Adapter Configuration .............................................................................................................43 4.3.4 Commands to verify PVST+ .............................................................................................................................43 4.4
3
MSTP Configuration .........................................................................................................................................50
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.4.1 MXL Configuration ............................................................................................................................................52 4.4.2 Nexus Configuration .........................................................................................................................................55 4.4.3 Server Network Adapter Configuration .............................................................................................................57 4.4.4 Commands to verify MSTP ...............................................................................................................................57 A
References .................................................................................................................................................................64
B
Components used in this guide ..................................................................................................................................65
C
Terminology ................................................................................................................................................................66
D
Resetting the MXL or M IOA to factory defaults .........................................................................................................68
E
MXL and M IOA Port Mapping ...................................................................................................................................69
F
Modifying Spanning Tree Port Cost and Priority ........................................................................................................70
G
Support and Feedback ...............................................................................................................................................71
About Dell EMC ................................................................................................................................................................71
4
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
1
Introduction Dell Networking provides customers with the most efficient use of current networking equipment at the lowest cost, while still providing great new technologies focused around the explosive data growth in the industry. Application demands have driven the need for increased bandwidth, lower latency and converged infrastructure in today’s networks. Dell EMC’s portfolio covers these key areas to provide the best in service and customer experience. Today’s businesses find it difficult to keep pace with the changing networking and enterprise landscape. With limited resources, they must support a variety of devices that provide key business functions, deliver reliable and flexible IT services, and provide discernible cost savings.
VRTX WAN
Remote/Branch Office
Internet
Dell Campus Networking Dell Data Center Networking
Campus
Storage Network
Dell Servers Dell Network Controllers and Security
Remote Data Centers
Public Cloud
Dell Storage
Data Center
Networking architecture overview The Dell PowerEdge M1000e chassis provides a revolutionary approach to shared infrastructure for enterprise computing. It combines the density and innovative agility of hyper-scale computing with the easeof-use and efficiency of advanced management technologies. The M1000e chassis provides a straightforward, modular approach to infrastructure and introduces a more practical way to manage the complex IT needs of businesses. The M1000e chassis has the capacity for up to six modular switches including the Dell Networking MXL and M I/O Aggregator (M IOA). This deployment guide provides examples of connecting MXL or M IOA switches to upstream Cisco Nexus 5000 series switches in a fault-tolerant network.
5
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
1.1
Typographical conventions This document uses the following typographical conventions:
6
Monospace text
Command Line Interface (CLI) examples
Bold monospace text
Commands entered at the CLI prompt
Underlined monospace text
CLI examples that word wrap (text is entered as a single command)
Italic monospace text
Variables in CLI examples
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
2
Hardware This section describes the hardware used to validate the topologies covered in this deployment guide. Refer to Appendix B for software versions used.
2.1
Dell PowerEdge M1000e The Dell PowerEdge M1000e modular blade enclosure, see Figure 2, provides an extremely reliable and efficient platform for building any IT infrastructure. The design of the M1000e enclosure reduces data center sprawl and IT complexity. It delivers one of the most energy efficient, flexible and manageable blade server implementations on the market. The PowerEdge M1000e supports server modules, network, storage, and cluster interconnect modules (switches and pass-through modules), power supplies, fans, and integrated keyboard-video-mouse (KVM) and Chassis Management Controller (CMC). The PowerEdge M1000e uses redundant and hot-pluggable components throughout to provide maximum uptime. Virtually unlimited in scalability, the PowerEdge M1000e chassis provides ultimate flexibility in server processor and chipset architectures. The M1000e infrastructure supports Intel and AMD server architectures simultaneously, while cutting-edge mechanical, electrical and software interface definitions enable multigenerational server support and expansion. For more information about the Dell PowerEdge M1000e, visit http://www.dell.com/us/business/p/poweredge-m1000e/pd.
Dell PowerEdge M1000e modular blade enclosure
2.2
M1000e MXL and M I/O Aggregator blade switches M1000e blade environments expand quickly and easily. Dell EMC’s complete, scale-on-demand MXL and M I/O Aggregator (M IOA) switches with FlexIO expansion modules support three fully redundant fabrics.
7
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
2.2.1
Dell Networking MXL 10/40Gb Ethernet Switch Figure 3 shows the Dell Networking MXL, a multilayer blade switch. It has two fixed 40GbE ports on the base module and support for two optional FlexIO Expansion Modules. It provides thirty-two 10GbE internal ports to server blades in the M1000e chassis. The MXL runs version 9 of the Dell Networking Operating System, providing switching, bridging and routing functionality for transmitting data, storage and server traffic.
Expansion Slot 1
Expansion Slot 0
Fixed 40GbE QSFP+ Ports
Dell Networking MXL
2.2.2
Dell PowerEdge M I/O Aggregator Figure 4 shows the Dell PowerEdge M IOA, a preconfigured, low-touch, Layer 2 blade switch. It has two fixed 40GbE ports on the base module and support for two optional FlexIO Expansion Modules. It provides thirtytwo 10GbE internal ports to server blades in the M1000e chassis. The M IOA runs version 9 of the Dell Networking Operating System, providing switching and bridging functionality for transmitting data, storage and server traffic.
I/O Bay 1 Expansion Slot 1
Expansion Slot 0
Fixed 40GbE QSFP+ Ports
Dell PowerEdge M I/0 Aggregator with one FlexIO Expansion Module installed
8
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
2.2.3
FlexIO Expansion Modules The Dell MXL and M IOA support a combination of four types of FlexIO Expansion Modules as shown in Figure 5. Supported modules include the following:
Four-port Fibre Channel 8Gb module Four-port 10Gb Base-T FlexIO module (only one 10Gbase-T module can be used per switch) Four-port 10Gb SFP+ FlexIO module Two-port 40Gb QSFP+ FlexIO module
FlexIO expansion modules Environments this document describes can use any supported combination of Ethernet expansion modules (10 GB Base-T, 10Gb SFP+, or 40 GB QSFP+) and the two fixed, QSFP+ ports.
9
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
3
Scenario One – VLT Environment
3.1
VLT Overview Virtual Link Trunking (VLT) allows physical links between two switching devices to appear as a single virtual link to other networking devices. VLT reduces the role of spanning tree protocols by allowing link aggregation group (LAG) terminations on two separate switches, and by supporting a loop-free topology. VLT provides Layer 2 multipathing, creating redundancy through increased bandwidth, enabling multiple parallel paths between nodes and load-balancing traffic where alternative paths exist. In a non-VLT environment utilizing the spanning tree protocol, redundancy involves blocked ports as shown in Figure 6.
Dell Switch
Nexus Switch
Dell Switch
Nexus Switch
Server
Non-VLT environment with spanning tree blocked ports
10
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
In a VLT environment, all paths are active, utilizing all available links and switches to their fullest potential as shown in Figure 7. This increases available bandwidth and performance.
Dell Switch
Nexus Switch
Dell Switch
Nexus Switch
Server
VLT Environment with all paths active
3.2
Spanning Tree with VLT As a precautionary measure, Dell EMC recommends running the Rapid Spanning Tree Protocol (RSTP) on MXLs in VLT environments. (See the note below for M IOA.) Because VLT environments are loop-free, simultaneously running spanning tree is optional though considered a best practice in case of switch misconfiguration or improperly connected cables. In a properly configured VLT environment, there will be no ports blocked by spanning tree. Cisco enables Rapid per Vlan Spanning Tree Plus (RPVST+), its implementation of RSTP, on Nexus by default. Notes: M IOA does not support spanning tree protocols. The M IOA configuration described later in this guide places the system in a preconfigured state called "VLT mode." On the M IOA, VLT mode places all external uplink ports in the same port channel, which cannot be modified. This eliminates the possibility of loops in VLT mode on the M IOA. You can use the Per-VLAN Spanning Tree Plus (PVST+) protocol instead of RSTP on the MXL. Either protocol will interoperate with Nexus RPVST+ in a VLT environment.
11
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
3.3
VLT Interoperability with Cisco Nexus vPC In this environment, Dell Networking VLT runs on a pair of MXL or M IOA switches. It interoperates with a pair of Nexus 5000 series switches running Cisco virtual Port Channel (vPC), as shown in Figure 8. Since all paths are active, the result is a highly fault-tolerant environment that makes full use of the throughput capabilities of all switches and links.
VLT Domain
vPC Domain
MXL or MIOA
Nexus 5000
VLTi
vPC Peer Link
MXL or MIOA
Nexus 5000
PowerEdge Blade Server
Dell Networking VLT interoperability with Cisco Nexus vPC Note: The Dell Networking Switch pair in the VLT domain consists of either two MXLs or two M IOAs. VLT does not support mixing one MXL and one M IOA in the same domain.
3.3.1
Dell Networking VLT Components The following are key components of a Dell VLT network: Virtual link trunk (VLT): The combined port channel between an attached device and the VLT peer switches. In Figure 9, Port Channel (Po) 1, Po 2, and Po 128 are VLTs. VLT peer device: One of a pair of devices that are connected with the special port channel known as the VLT interconnect. In Figure 9, the two MXLs or M IOAs are VLT peer devices. VLT interconnect (VLTi): The link used to synchronize states between the VLT peer switches. Both ends must be on 10G or 40G interfaces. In Figure 9, Po 127 is the VLTi. VLT backup link: The backup link monitors the vitality of VLT peer switches. The backup link sends periodic keep alive messages between the VLT peer switches. The backup link is normally on the management network. VLT domain: This domain includes both VLT peer devices, the VLT interconnect and all of the port channels in the VLT connected to attached devices.
12
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
3.3.2
Cisco Nexus vPC Components The following are key components of a Cisco vPC network: Virtual Port Channel (vPC): The combined port channel between the vPC peer devices and the downstream device. In Figure 9, Po 200 and Po 129 are vPCs. vPC peer device: One of a pair of devices connected with the special port channel known as the vPC peer link. In Figure 9, the two Nexus 5000 switches are vPC peer devices. vPC peer link: The link used to synchronize states between the vPC peer devices. In Figure 9, Po 55 is the vPC peer link. vPC member port: Interfaces that belong to the vPCs. vPC peer-keepalive link: The peer-keepalive link monitors the vitality of a vPC peer Cisco Nexus device. The peer-keepalive link sends configurable, periodic keepalive messages between vPC peer devices. The management network normally provides this connection. vPC domain: The domain that includes both vPC peer devices, the vPC peer-keepalive link, and all of the port channels in the vPC connected to the downstream devices.
3.4
VLT and vPC Configuration This section provides detailed configuration steps to build the VLT/vPC network shown in Figure 9. The cloud shown in the diagram represents the upstream network.
VLT Domain 1 LACP NIC Team Po 1
Blade Server 1 VLAN 10
Te0/1
vPC Domain 1
Po 128 Po 129
Te0/41
E1/1
MXL or MIOA Te0/43
E1/3
Po 200
Nexus 5000 E1/9
Te0/2 Fo0/33
Blade Server 2 VLAN 20
E1/7 E1/8
Fo0/37
Po 127
Po 55
Fo0/33 Fo0/37
E1/7 E1/8
Te0/1 Te0/2 LACP NIC Team Po 2
MXL or MIOA
Te0/43 Te0/41
E1/3 E1/1
E1/9
Nexus 5000
VLT-vPC network diagram
13
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Blade Server 1 is in VLAN 10 and Blade Server 2 is in VLAN 20. Blade servers 1 and 2 reside in server slots 1 and 2 of the M1000e chassis respectively. The Dell switches reside in slots A1 and A2 in the back of the M1000e chassis. The configuration sections that follow use the same port numbers and port channel numbers shown in Figure 9. For more information on port mapping of MXL and M IOA ports, see Appendix E. The out-of-band management network is not shown in Figure 9 to minimize clutter. The Dell switches connect to the management network via the CMC network port on the M1000e chassis. Each Nexus switch has a dedicated management Ethernet port connected to the management network. In addition to switch administration, the management network is used by the switches to send VLT/vPC peer-keepalive messages to the management IP address of the peer. Proper configuration of the management interfaces is important and is included in the configuration sections that follow. Two sections, MXL VLT Configuration and M IOA VLT Configuration, provide the Dell Networking switch configuration. The Nexus vPC Configuration section is the same in this environment regardless of whether it is connected to a pair of MXLs or a pair of M IOAs. The configuration text is formatted so it can be cut and pasted into a plain text editor (Microsoft Notepad, Notepad++, for example) or directly into a switch console session. Notes: The configuration steps were developed with the switches starting at their default configurations. This may be helpful if there is an existing configuration on a switch that conflicts with this environment. To reset Dell MXL or M IOA switches to factory defaults, step-by-step instructions are provided in Appendix D. The Cisco Nexus 5000 series switches were reset by using the #write erase command followed by #reload. When prompted after reload, "Power On Auto Provisioning" was not used, the admin password was configured, and the Nexus basic configuration dialog was not used. Refer to the Nexus system documentation for more information. When an MXL or M IOA switch is connected to a Cisco Nexus switch, a number of DCBX (Data Center Bridging Exchange) error messages may be displayed on the serial console. To suppress these messages, either configure DCBX on the switches (beyond the scope of this guide) or simply disable DCB on the Dell Networking switches. To do this, use the command: (conf)#no dcb enable
14
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
3.4.1
MXL VLT Configuration MXL-A1
15
MXL-A2
Set the hostname, enable RSTP, and configure the out-of-band management address. This example uses a static management IP address. DHCP may be used instead.
Set the hostname, enable RSTP, and configure the out-of-band management address. This example uses a static management IP address. DHCP may be used instead.
enable configure hostname MXL-A1 protocol spanning-tree rstp no disable exit
enable configure hostname MXL-A2 protocol spanning-tree rstp no disable exit
interface ManagementEthernet 0/0 ip address 172.25.112.91/24
interface ManagementEthernet 0/0 ip address 172.25.112.92/24
If prompted - Proceed with Static IP [confirm yes/no]: Y Configure the management route.
If prompted - Proceed with Static IP [confirm yes/no]: Y Configure the management route.
exit management route 0.0.0.0/0 172.25.112.254
exit management route 0.0.0.0/0 172.25.112.254
Configure the VLTi LAG (without LACP) and the VLTi LAG interfaces.
Configure the VLTi LAG (without LACP) and the VLTi LAG interfaces.
interface Port-channel 127 description "VLTi LAG to MXL-A2" channel-member fortyGigE 0/33,37 no shutdown exit
interface Port-channel 127 description "VLTi LAG to MXL-A1" channel-member fortyGigE 0/33,37 no shutdown exit
interface range fortyGigE 0/33, fortyGigE 0/37 description "VLTi link to MXL-A2" no shutdown exit
interface range fortyGigE 0/33, fortyGigE 0/37 description "VLTi link to MXL-A1" no shutdown exit
Configure the same VLT domain # on both peers. Set the backup (peer-keepalive) address to the management IP of MXL-A2. Set this switch as the VLT primary peer, configure the same VLT domain MAC address on both peers, and configure the unique unit id.
Configure the same VLT domain # on both peers. Set the backup (peer-keepalive) address to the management IP of MXL-A1. Set this switch as the VLT secondary peer, configure the same VLT domain MAC address
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MXL-A1
MXL-A2 on both peers, and configure the unique unit id.
16
vlt domain 1 peer-link port-channel 127 back-up destination 172.25.112.92 primary-priority 1 system-mac mac-address 00:aa:bb:cc:dd:ee unit-id 0 exit
vlt domain 1 peer-link port-channel 127 back-up destination 172.25.112.91 primary-priority 2 system-mac mac-address 00:aa:bb:cc:dd:ee unit-id 1 exit
Create the downstream (server-facing) VLT portchannel interfaces and configure ports. Repeat these commands as needed for each server in the M1000e chassis.
Create the downstream (server-facing) VLT portchannel interfaces and configure ports. Repeat these commands as needed for each server in the M1000e chassis.
interface Port-channel 1 description "VLT LAG to Blade Server Slot 1" switchport vlt-peer-lag port-channel 1 spanning-tree rstp edge-port no shutdown exit
interface Port-channel 1 description "VLT LAG to Blade Server Slot 1" switchport vlt-peer-lag port-channel 1 spanning-tree rstp edge-port no shutdown exit
interface Port-channel 2 description "VLT LAG to Blade Server Slot 2" switchport vlt-peer-lag port-channel 2 spanning-tree rstp edge-port no shutdown exit
interface Port-channel 2 description "VLT LAG to Blade Server Slot 2" switchport vlt-peer-lag port-channel 2 spanning-tree rstp edge-port no shutdown e exit
interface TenGigabitEthernet 0/1 description "Link to Blade Server Slot 1" no switchport no portmode hybrid port-channel-protocol LACP port-channel 1 mode active exit no shutdown exit
interface TenGigabitEthernet 0/1 description "Link to Blade Server Slot 1" no switchport no portmode hybrid port-channel-protocol LACP port-channel 1 mode active exit no shutdown exit
interface TenGigabitEthernet 0/2
interface TenGigabitEthernet 0/2
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
17
MXL-A1
MXL-A2
description "Link to Blade Server Slot 2" no switchport no portmode hybrid port-channel-protocol LACP port-channel 2 mode active exit no shutdown exit
description "Link to Blade Server Slot 2" no switchport no portmode hybrid port-channel-protocol LACP port-channel 2 mode active exit no shutdown exit
Create the upstream VLT portchannel interface and configure ports.
Create the upstream VLT portchannel interface and configure ports.
interface Port-channel 128 description "VLT LAG to N5K" portmode hybrid switchport vlt-peer-lag port-channel 128 no shutdown exit
interface Port-channel 128 description "VLT LAG to N5K" portmode hybrid switchport vlt-peer-lag port-channel 128 no shutdown exit
interface range TenGigabitEthernet 0/41, TenGigabitEthernet 0/43 description "Link to N5K" no switchport port-channel-protocol LACP port-channel 128 mode active exit no shutdown
interface range TenGigabitEthernet 0/41, TenGigabitEthernet 0/43 description "Link to N5K" no switchport port-channel-protocol LACP port-channel 128 mode active exit no shutdown
Create VLANs 10 and 20 and tag the port channels in the VLANs.
Create VLANs 10 and 20 and tag the port channels in the VLANs.
interface vlan 10 tagged port-channel 1,127-128 no shutdown exit
interface vlan 10 tagged port-channel 1,127-128 no shutdown exit
interface vlan 20 tagged port-channel 2,127-128 no shutdown end
interface vlan 20 tagged port-channel 2,127-128 no shutdown end
Save the configuration.
Save the configuration.
write
write
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MXL configuration notes: Specification of a VLT system-mac mac-address is optional but Dell EMC recommends this to minimize traffic disruption if a VLT failover occurs. This address is used as the system ID in LACP PDUs, and must be the same on both VLT peers. If a VLT system-mac mac-address is not configured, the MAC address of the VLT primary peer is used. The system automatically assigns a unique unit ID (0 or 1) to each peer switch upon creation of the VLT domain. Optionally, the unit ID can be manually configured as shown above to minimize the time required for the VLT system to converge when one peer switch reboots. It is a best practice to configure spanning tree first when a spanning tree protocol is used with VLT, Use a static LAG on the VLT interconnect (port channel 127 in this example). All other VLT LAGs should use LACP. The portmode hybrid command allows an Ethernet interface to carry multiple tagged VLANs and a single untagged (also called a native) VLAN simultaneously. If a given port is only expected to carry tagged VLANs or a single untagged VLAN, the portmode hybrid command is not required. In this environment, RTSP communicates with Nexus RPVST+ using the untagged default VLAN (VLAN 1). Since port channel 128 also carries tagged VLANs 10 and 20, port channel 128 is put in hybrid mode. See the Nexus vPC Configuration section to configure the upstream switches.
3.4.2
M IOA VLT Configuration The benefit of the M IOA is ease-of-configuration with VLT mode. VLT mode is a low-touch mode with automated settings and targeted at server administrators. The only items that require configuration in VLT mode are VLANs and the management IP address (if DHCP is not used). VLT mode preconfigures the two fixed 40 Gb ports (Fo 0/33 and Fo 0/37) as the VLT interconnects. They must be used for this purpose. All other external ports are configured for 10 Gb and are added to uplink port channel 128. Note: Before putting the M IOA in VLT mode, restore it to factory defaults as detailed in Appendix D. MIOA-A1
18
MIOA-A2
Set the switch to VLT mode.
Set the switch to VLT mode.
enable config stack-unit 0 iom-mode vlt exit reload
enable config stack-unit 0 iom-mode vlt exit reload
After reloading, set the hostname and configure the out-of-band management address. This example uses a static management IP address. DHCP may be used instead.
After reloading, set the hostname and configure the out-of-band management address. This example uses a static management IP address. DHCP may be used instead.
enable configure
enable configure
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MIOA-A1
MIOA-A2
hostname MIOA-A1 interface ManagementEthernet 0/0 ip address 172.25.112.91/24
hostname MIOA-A2 interface ManagementEthernet 0/0 ip address 172.25.112.92/24
If prompted - Proceed with Static IP [confirm yes/no]: Y Configure the management route.
If prompted - Proceed with Static IP [confirm yes/no]: Y Configure the management route.
exit management route 0.0.0.0/0 172.25.112.254
exit management route 0.0.0.0/0 172.25.112.254
Create VLANs 10 and 20 by tagging the server ports in the VLANs.
Create VLANs 10 and 20 by tagging the server ports in the VLANs.
interface TenGigabitEthernet 0/1 vlan tagged 10 interface TenGigabitEthernet 0/2 vlan tagged 20 end
interface TenGigabitEthernet 0/1 vlan tagged 10 interface TenGigabitEthernet 0/2 e vlan tagged 20 end
Save the configuration.
Save the configuration.
write
write
M IOA configuration notes: To verify the system is in VLT mode, run the command show system stack-unit 0 iom-mode. The "Boot-Mode" field must be "vlt". The system creates the server-facing port channels automatically on the M IOA after LACP configuration on the blade server network adapters is complete as described in Server Network Adapter Configuration. By default, network ports on the blade servers installed in the M1000e chassis are down until the uplink port channel (po 128) becomes operational on the M IOA. This is due to a feature called Uplink Failure Detection (UFD) whereby, when upstream connectivity fails, the M IOA disables the downstream links. See the M IOA configuration guide (link in Appendix A) for more information on UFD.
19
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
3.4.3
Nexus vPC Configuration This section describes vPC configuration on the Cisco Nexus 5000 series switches. This section is the same regardless of whether the Nexus switches are connected to a pair of MXLs or M IOAs. N5K-1
20
N5K-2
Enable LACP and vPC features. Create VLANs 10 and 20. Configure this switch as the primary spanning tree root bridge for the native VLAN (VLAN 1) and VLANs 10 and 20.
Enable LACP and vPC features. Create VLANs 10 and 20. Configure this switch as the secondary spanning tree root bridge for the native VLAN (VLAN 1) and VLANs 10 and 20.
configure feature lacp feature vpc vlan 10,20 spanning-tree vlan 1,10,20 root primary
configure feature lacp feature vpc vlan 10,20 spanning-tree vlan 1,10,20 root secondary
Configure the hostname, the management interface address and the management route.
Configure the hostname, the management interface address and the management route.
hostname N5K-1 interface mgmt 0 ip address 172.25.115.37/24 vrf member management no shutdown exit vrf context management ip route 0.0.0.0/0 172.25.115.254 exit
hostname N5K-2 interface mgmt 0 ip address 172.25.115.36/24 vrf member management no shutdown exit vrf context management ip route 0.0.0.0/0 172.25.115.254 exit
Create the VPC domain. Assign a role priority of 1 to make this switch the vPC primary. Specify the management IP of switch N5K-1 for the peer-keepalive link.
Create the VPC domain. Assign a role priority of 65535 to make this switch the vPC secondary. Specify the management IP of switch N5K-2 for the peer-keepalive link.
vpc domain 1 role priority 1 peer-keepalive dest 172.25.115.36 exit
vpc domain 1 role priority 65535 peer-keepalive dest 172.25.115.37 exit
Configure the port channel and port channel members for the vPC peer link.
Configure the port channel and port channel members for the vPC peer link.
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
21
N5K-1
N5K-2
interface port-channel 55 description "vPC Peer-Link to N5K-2" switchport mode trunk switchport trunk allowed vlan 1,10,20 no shutdown vpc peer-link exit
interface port-channel 55 description "vPC Peer-Link to N5K-1" switchport mode trunk switchport trunk allowed vlan 1,10,20 no shutdown vpc peer-link exit
interface ethernet 1/7-8 description "Link to N5K-2" switchport mode trunk switchport trunk allowed vlan 1,10,20 channel-group 55 mode active no shutdown exit
interface ethernet 1/7-8 description "Link to N5K-1" switchport mode trunk switchport trunk allowed vlan 1,10,20 channel-group 55 mode active no shutdown exit
Configure the port channel and port channel member links for downstream connectivity to the Dell Networking switches.
Configure the port channel and port channel member links for downstream connectivity to the Dell Networking switches.
interface port-channel 129 description "vPC to Dell" switchport mode trunk switchport trunk allowed vlan 1,10,20 vpc 129 no shutdown exit
interface port-channel 129 description "vPC to Dell" switchport mode trunk switchport trunk allowed vlan 1,10,20 vpc 129 no shutdown exit
interface ethernet 1/1, ethernet 1/3 description "Link to Dell" switchport mode trunk switchport trunk allowed vlan 1,10,20 channel-group 129 mode active no shutdown exit
interface ethernet 1/1, ethernet 1/3 description "Link to Dell" switchport mode trunk switchport trunk allowed vlan 1,10,20 channel-group 129 mode active no shutdown exit
Configure the port channel and port channel members for upstream connectivity to the external network.
Configure the port channel and port channel members for upstream connectivity to the external network
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
N5K-1
N5K-2
interface port-channel 200 description "vPC to External Network" switchport mode trunk switchport trunk allowed vlan 10,20 vpc 200 no shutdown exit
interface port-channel 200 description "vPC to External Network" switchport mode trunk switchport trunk allowed vlan 10,20 vpc 200 no shutdown exit
interface ethernet 1/9 description "Link to External Network" switchport mode trunk switchport trunk allowed vlan 10,20 channel-group 200 mode active no shutdown end
interface ethernet 1/9 description "Link to External Network" switchport mode trunk switchport trunk allowed vlan 10,20 channel-group 200 mode active no shutdown end
Save the configuration
Save the configuration
copy running-config startup-config
copy running-config startup-config
Nexus configuration notes: It is a best practice to make the spanning tree root bridge as close as possible to the network core. In this environment, the Nexus switches will serve as the spanning tree root primary and secondary switches. On Nexus, the “primary” command above sets the priority to 24576, and the “secondary” command above sets the priority to 28672. The MXL switches each have a default priority of 32768. The switch with the lowest priority becomes the root bridge, so N5K-1 becomes the root bridge for the network and N5K-2 becomes the alternate. This configuration uses the native VLAN (VLAN 1) for spanning-tree communication between the Nexus switches running RPVST+ and the MXL switch running RSTP. If the configuration uses M IOA instead of MXL, VLAN 1 does not need to be allowed on port channel 129 since the M IOA does not use spanning tree. The peer-keepalive address is the management address of the vPC peer switch. This setting allows keepalive heartbeat packets to flow through the management network.
22
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
3.4.4
Server Network Adapter Configuration In this VLT environment, the server adapter ports are configured as LACP NIC teams. On Blade Server 1, the team is tagged in VLAN 10. On Blade Server 2, the team is tagged in VLAN 20. Note: Refer to the network adapter or operating system documentation for NIC teaming instructions. The network adapter software may refer to LACP as 802.3ad or Dynamic Link Aggregation. For an overview of NIC teaming in Windows 2012 R2, see http://technet.microsoft.com/en-us/library/hh831648.aspx. The network adapter sees the MXL pair or M IOA pair as a single switch. For a network adapter with two 10 Gb ports, the LACP NIC team has 20 Gbps bandwidth with both ports active.
3.5
Commands to verify VLT and vPC
3.5.1
MXL / M IOA commands and output Verify the configuration on the Dell Networking switches using the commands listed in Table 1: Dell Networking show commands Command
Output
show running-config vlt
Displays the running VLT domain configuration
show vlt brief
Displays the role (Primary or Secondary), and the status of the ICL Link, Heartbeat and VLT Peer
show vlt detail
Shows the VLT LAG information for both the local and VLT peer switch (The LAG Status must be UP in both the Local and Peer columns for the LAG to be functional.)
show vlt mismatch
Shows configuration inconsistencies between the VLT peers that may result in VLT errors
show mac-address-table
Displays the MAC address table Addresses learned from the VLT peer are marked with (N)
show spanning-tree rstp brief
VLT port channels are identified and are in the forwarding state
Note: MXL and M IOA command output is similar. To avoid duplicating all output, the command output examples that follow change MXL and M IOA switch names to DELL-A1 and DELL-A2, respectively.
3.5.1.1
show running-config vlt This command displays the VLT domain configuration settings. The back-up destination is the management IP address of the peer switch. The system-mac mac-address, if configured, must be identical on both switches.
23
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
DELL-A1#show running-config vlt ! vlt domain 1 peer-link port-channel 127 back-up destination 172.25.112.92 primary-priority 1 system-mac mac-address 00:aa:bb:cc:dd:ee unit-id 0 DELL-A2#show running-config vlt ! vlt domain 1 peer-link port-channel 127 back-up destination 172.25.112.91 primary-priority 2 system-mac mac-address 00:aa:bb:cc:dd:ee unit-id 1
3.5.1.2
show vlt brief The ICL Link Status, Heat Beat Status, and VLT Peer Status must all be up. One switch is primary and the other secondary. DELL-A1#show vlt brief VLT Domain Brief -----------------Domain ID: Role: Role Priority: ICL Link Status: HeartBeat Status: VLT Peer Status: Local Unit Id: Version: Local System MAC address: Remote System MAC address: Configured System MAC address: Remote system version: Delay-Restore timer: Delay-Restore Abort Threshold: Peer-Routing : Peer-Routing-Timeout timer: Multicast peer-routing timeout:
24
1 Primary 1 Up Up Up 0 6(6) 00:1e:c9:f1:04:ba 00:1e:c9:f1:03:92 00:aa:bb:cc:dd:ee 6(6) 90 seconds 60 seconds Disabled 0 seconds 150 seconds
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
DELL-A2#show vlt brief VLT Domain Brief -----------------Domain ID: Role: Role Priority: ICL Link Status: HeartBeat Status: VLT Peer Status: Local Unit Id: Version: Local System MAC address: Remote System MAC address: Configured System MAC address: Remote system version: Delay-Restore timer: Delay-Restore Abort Threshold: Peer-Routing : Peer-Routing-Timeout timer: Multicast peer-routing timeout:
3.5.1.3
1 Secondary 2 Up Up Up 1 6(6) 00:1e:c9:f1:03:92 00:1e:c9:f1:04:ba 00:aa:bb:cc:dd:ee 6(6) 90 seconds 60 seconds Disabled 0 seconds 150 seconds
show vlt detail The Local and Peer VLT LAGs are all up when all connected devices are up. DELL-A1#show vlt detail Local LAG Id Peer LAG Id ------------ ----------1 1 2 2 128 128
Local Status -----------UP UP UP
Peer Status ----------UP UP UP
Active VLANs ------------10 20 1, 10, 20
Notes: Output is identical on switch A2.
3.5.1.4
show vlt mismatch There should be no output to this command on either switch. If there is, resolve the mismatch. Mismatch examples include incompatible VLT configuration settings, VLAN differences, different switch operating system versions and spanning-tree inconsistencies. DELL-A1#show vlt mismatch DELL-A1#
Note: Output is identical on switch A2.
25
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
3.5.1.5
show mac-address-table Displays the MAC address table. Addresses learned from the VLT peer are marked with (N). DELL-A1#show mac-address-table Codes: *N - VLT Peer Synced MAC *I - Internal MAC Address used for Inter Process Communication VlanId Mac Address Type Interface 1 00:26:0b:f1:f0:81 Dynamic Po 128 1 00:2a:6a:f7:5b:68 Dynamic Po 128 1 00:2a:6a:f7:5b:6a Dynamic (N) Po 128 1 00:2a:6a:f7:80:48 Dynamic (N) Po 128 1 00:2a:6a:f7:80:4a Dynamic Po 128 10 00:10:18:d6:b3:e0 Dynamic Po 128 10 00:26:0b:f1:f0:81 Dynamic Po 128 10 e0:db:55:11:7b:44 Dynamic (N) Po 1 20 00:10:18:d6:b3:e0 Dynamic Po 128 20 00:26:0b:f1:f0:81 Dynamic Po 128 20 bc:30:5b:ef:d8:e0 Dynamic Po 2
State Active Active Active Active Active Active Active Active Active Active Active
DELL-A2#show mac-address-table Codes: *N - VLT Peer Synced MAC *I - Internal MAC Address used for Inter Process Communication VlanId Mac Address Type Interface 1 00:26:0b:f1:f0:81 Dynamic (N) Po 128 1 00:2a:6a:f7:5b:68 Dynamic (N) Po 128 1 00:2a:6a:f7:5b:6a Dynamic Po 128 1 00:2a:6a:f7:80:48 Dynamic Po 128 1 00:2a:6a:f7:80:4a Dynamic (N) Po 128 10 00:10:18:d6:b3:e0 Dynamic (N) Po 128 10 00:26:0b:f1:f0:81 Dynamic (N) Po 128 10 e0:db:55:11:7b:44 Dynamic Po 1 20 00:10:18:d6:b3:e0 Dynamic (N) Po 128 20 00:26:0b:f1:f0:81 Dynamic (N) Po 128 20 bc:30:5b:ef:d8:e0 Dynamic (N) Po 2
26
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
State Active Active Active Active Active Active Active Active Active Active Active
3.5.1.6
show spanning-tree rstp brief Note: M IOA does not support this command. The Root ID must match the MAC address of the Nexus switch configured as the spanning tree root bridge. RSTP runs in this environment as a precautionary measure only. If any ports are blocked (“BLK” in the "Sts" column), there is likely a configuration or cable connection issue causing a loop that has been blocked by spanning tree. The VLT or VLTi port channels will be identified as VLT or VLTi and should all be in the forwarding state (“FWD” in the "Sts" column) on both Dell switches. Ports that are administratively shutdown will appear as "DIS." MXL-A1#show spanning-tree rstp brief Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24577, Address 002a.6af7.5bbc Root Bridge hello time 2, max age 20, forward delay 15 Bridge ID Priority 32768, Address 001e.c9f1.04ba Configured hello time 2, max age 20, forward delay 15 Bpdu filter disabled globally Interface Name PortID ---------- -------Po 1 128.2 Po 2 128.3 Po 127 128.128 Po 128 128.129 Te 0/2 128.131 | (output truncated)
Prio ---128 128 128 128 128
Cost ------1800 1800 600 1400 2000
Interface Name Role PortID ---------- ------ -------Po 1 Desg 128.2 Po 2 Desg 128.3 Po 127 Desg 128.128 Po 128 Root 128.129 Te 0/2 Dis 128.131 | (output truncated)
Prio ---128 128 128 128 128
Sts ----------FWD(vlt) FWD(vlt) FWD(vltI) FWD(vlt) DIS
Cost ------1800 1800 600 1400 2000
Cost ------1400 1400 1400 1400 1400
Sts ----------FWD FWD FWD FWD DIS
Designated Bridge ID -------------------32768 001e.c9f1.04ba 32768 001e.c9f1.04ba 32768 001e.c9f1.04ba 24577 0023.04ee.be01 32768 001e.c9f1.04ba
Cost ------1400 1400 1400 1400 1400
MXL-A2#sh spanning-tree rstp brief Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24577, Address 002a.6af7.5bbc Root Bridge hello time 2, max age 20, forward delay 15 Bridge ID Priority 32768, Address 001e.c9f1.0392 Configured hello time 2, max age 20, forward delay 15 Bpdu filter disabled globally
27
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Link-type --------(vlt) P2P (vlt) P2P (vltI)P2P (vlt) P2P P2P
PortID -------128.2 128.3 128.128 144.128 128.131
Edge ---Yes Yes No No No
Bpdu Filter -----No No No No No
Interface Name PortID ---------- -------Po 1 128.2 Po 2 128.3 Po 127 128.128 Po 128 128.129 Te 0/2 128.131 | (output truncated)
Prio ---128 128 128 128 128
Cost ------1800 1800 600 1400 2000
Interface Name Role PortID ---------- ------ -------Po 1 Desg 128.2 Po 2 Desg 128.3 Po 127 Root 128.128 Po 128 Root 128.129 Te 0/2 Dis 128.131 | (output truncated)
3.5.2
Prio ---128 128 128 128 128
Sts ----------FWD(vlt) FWD(vlt) FWD(vltI) FWD(vlt) DIS
Cost ------1800 1800 600 1400 2000
Cost ------2000 2000 2000 2000 2000
Sts ----------FWD FWD FWD FWD DIS
Designated Bridge ID -------------------32768 001e.c9f1.0392 32768 001e.c9f1.0392 32768 001e.c9f1.04ba 32768 001e.c9f1.0392 32768 001e.c9f1.0392
Cost ------2000 2000 2000 2000 2000
Link-type --------(vlt) P2P (vlt) P2P (vltI)P2P (vlt) P2P P2P
PortID -------128.2 128.3 128.128 128.129 128.131
Edge ---Yes Yes No No No
Bpdu Filter -----No No No No No
Nexus commands and output Verify the configuration on the Nexus switches using the commands listed in Table 2: Nexus show commands
3.5.2.1
Command
Output
show feature
Reports whether LACP and vPC are enabled
show running-config vpc
Displays vPC domain configuration
show vpc
Displays vPC peer and LAG status
show vpc consistencyparameters
Displays any configuration inconsistencies between the peers that may need to be corrected
show spanning-tree
vPC port channels are identified and are in the forwarding state
show feature LACP and vPC features must be enabled for this environment. Other features shown are optional. N5K-1# show feature | grep enabled lacp 1 enabled lldp 1 enabled sshServer 1 enabled telnetServer 1 enabled vpc 1 enabled
28
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Note: Switch 2 not shown, output is identical.
3.5.2.2
show running-config vpc This command displays the vPC domain and port channel configuration settings. The peer-keepalive destination is the management IP address of the peer switch. N5K-1# show running-config vpc !Command: show running-config vpc !Time: Tue Apr 12 17:53:10 2016 version 7.0(5)N1(1) feature vpc vpc domain 1 role priority 1 peer-keepalive destination 172.25.115.36 delay restore 150 interface port-channel55 vpc peer-link interface port-channel129 vpc 129 interface port-channel200 vpc 200 N5K-2# show running-config vpc !Command: show running-config vpc !Time: Tue Apr 12 17:54:30 2016 version 7.0(5)N1(1) feature vpc vpc domain 1 role priority 65535 peer-keepalive destination 172.25.115.37 delay restore 150 interface port-channel55 vpc peer-link interface port-channel129 vpc 129
29
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
interface port-channel200 vpc 200
3.5.2.3
show vpc This command displays the vpc status. Peer status and vPC keep-alive status must be as shown. The consistency status fields should all show “success”. If not, see the show vpc consistency-parameters command. N5K-1# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id Peer status vPC keep-alive status Configuration consistency status Per-vlan consistency status Type-2 consistency status vPC role Number of vPCs configured Peer Gateway Dual-active excluded VLANs Graceful Consistency Check Auto-recovery status
: : : : : : : : : : : :
1 peer adjacency formed ok peer is alive success success success primary 2 Disabled Enabled Enabled (timeout = 240 seconds)
vPC Peer-link status --------------------------------------------------------------------id Port Status Active vlans ---------- -------------------------------------------------1 Po55 up 1,10,20 vPC status ---------------------------------------------------------------------------id Port Status Consistency Reason Active vlans ------ ----------- ------ ----------- -------------------------- ----------129 Po129 up success success 1,10,20 200 Po200 up success success 10,20 N5K-2# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id Peer status vPC keep-alive status Configuration consistency status
30
: : : :
1 peer adjacency formed ok peer is alive success
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Per-vlan consistency status Type-2 consistency status vPC role Number of vPCs configured Peer Gateway Dual-active excluded VLANs Graceful Consistency Check Auto-recovery status
: : : : : : : :
success success secondary 2 Disabled Enabled Enabled (timeout = 240 seconds)
vPC Peer-link status --------------------------------------------------------------------id Port Status Active vlans ---------- -------------------------------------------------1 Po55 up 1,10,20 vPC status ---------------------------------------------------------------------------id Port Status Consistency Reason Active vlans ------ ----------- ------ ----------- -------------------------- ----------129 Po129 up success success 1,10,20 200 Po200 up success success 10,20
3.5.2.4
show vpc consistency-parameters This command pinpoints inconsistencies between the vPC peers. Depending on the severity of the misconfiguration, vPC may either warn the user (Type-2 misconfiguration) or suspend the Port Channel (Type-1 misconfiguration). In the specific case of a VLAN mismatch, only the VLAN that differs between the vPC member ports is suspended on the vPC Port Channels. N5K-1# sh vpc consistency-parameters ? global Global Parameters interface Specify interface vlans Vlans vpc Virtual Port Channel configuration N5K-1# sh vpc consistency-parameters global Legend: Type 1 : vPC will be suspended in case of mismatch
31
Name ------------QoS
Type ---2
Network QoS (MTU) Network Qos (Pause) Input Queuing (Bandwidth) Input Queuing (Absolute
2 2 2 2
Local Value ---------------------([], [], [], [], [], []) (1538, 0, 0, 0, 0, 0) (F, F, F, F, F, F) (100, 0, 0, 0, 0, 0) (F, F, F, F, F, F)
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Peer Value ----------------------([], [], [], [], [], []) (1538, 0, 0, 0, 0, 0) (F, F, F, F, F, F) (100, 0, 0, 0, 0, 0) (F, F, F, F, F, F)
Priority) Output Queuing (Bandwidth) Output Queuing (Absolute Priority) Vlan to Vn-segment Map STP Mode STP Disabled STP MST Region Name STP MST Region Revision STP MST Region Instance to VLAN Mapping STP Loopguard STP Bridge Assurance STP Port Type, Edge BPDUFilter, Edge BPDUGuard STP MST Simulate PVST IGMP Snooping Group-Limit Allowed VLANs Local suspended VLANs
2 2
(100, 0, 0, 0, 0, 0) (F, F, F, F, F, F)
(100, 0, 0, 0, 0, 0) (F, F, F, F, F, F)
1 1 1 1 1 1
No Relevant Maps Rapid-PVST None "" 0
No Relevant Maps Rapid-PVST None "" 0
1 1 1
Disabled Enabled Normal, Disabled, Disabled Enabled 4000 1,10,20 -
Disabled Enabled Normal, Disabled, Disabled Enabled 4000 1,10,20 -
1 2 -
Note: Output is identical on switch N5K-2.
3.5.2.5
show spanning-tree The Root ID must match the MAC address of the Nexus switch configured as the spanning tree root bridge for each VLAN. RPVST+ runs in this environment as a precautionary measure only. If any ports are blocked (“BLK” in the Sts column) in this topology, there is likely a configuration or cable connection issue. All vPC/vPC peer-link port channels (po 55, 129, 200) are identified as vPC and in an active (forwarding) state on both Nexus switches. N5K-1# show spanning-tree VLAN0001 Spanning tree enabled protocol rstp Root ID Priority 24577 Address 002a.6af7.5bbc This bridge is the root Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po129
32
Role ---Desg Desg
Forward Delay 15 sec
24577 (priority 24576 sys-id-ext 1) 002a.6af7.5bbc 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD
Cost --------1 1
Prio.Nbr -------128.4150 128.4224
Type -------------------------------(vPC peer-link) Network P2p (vPC) P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
VLAN0010 Spanning tree enabled protocol rstp Root ID Priority 24586 Address 002a.6af7.5bbc This bridge is the root Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po129 Po200
Role ---Desg Desg Desg
24586 (priority 24576 sys-id-ext 10) 002a.6af7.5bbc 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD FWD
Cost --------1 1 1
Prio.Nbr -------128.4150 128.4224 128.4295
Type -------------------------------(vPC peer-link) Network P2p (vPC) P2p (vPC) P2p
VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 24596 Address 002a.6af7.5bbc This bridge is the root Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po129 Po200
Role ---Desg Desg Desg
Forward Delay 15 sec
Forward Delay 15 sec
24596 (priority 24576 sys-id-ext 20) 002a.6af7.5bbc 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD FWD
Cost --------1 1 1
Prio.Nbr -------128.4150 128.4224 128.4295
Type -------------------------------(vPC peer-link) Network P2p (vPC) P2p (vPC) P2p
N5K-2# show spanning-tree VLAN0001 Spanning tree enabled protocol rstp Root ID Priority 24577 Address 002a.6af7.5bbc Cost 1 Port 4150 (port-channel55) Hello Time 2 sec Max Age 20 sec Bridge ID
33
Priority Address
Forward Delay 15 sec
28673 (priority 28672 sys-id-ext 1) 002a.6af7.8081
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Hello Time Interface ---------------Po55 Po129
Role ---Root Desg
2
Sts --FWD FWD
sec
Max Age 20 sec
Cost --------1 1
Prio.Nbr -------128.4150 128.4224
Type -------------------------------(vPC peer-link) Network P2p (vPC) P2p
VLAN0010 Spanning tree enabled protocol rstp Root ID Priority 24586 Address 002a.6af7.5bbc Cost 1 Port 4150 (port-channel55) Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po129 Po200
Role ---Root Desg Desg
Sts --FWD FWD FWD
Priority Address Hello Time
Interface ---------------Po55 Po129 Po200
34
Role ---Root Desg Desg
Forward Delay 15 sec
28682 (priority 28672 sys-id-ext 10) 002a.6af7.8081 2 sec Max Age 20 sec Forward Delay 15 sec Cost --------1 1 1
Prio.Nbr -------128.4150 128.4224 128.4295
Type -------------------------------(vPC peer-link) Network P2p (vPC) P2p (vPC) P2p
VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 24596 Address 002a.6af7.5bbc Cost 1 Port 4150 (port-channel55) Hello Time 2 sec Max Age 20 sec Bridge ID
Forward Delay 15 sec
Forward Delay 15 sec
28692 (priority 28672 sys-id-ext 20) 002a.6af7.8081 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD FWD
Cost --------1 1 1
Prio.Nbr -------128.4150 128.4224 128.4295
Type -------------------------------(vPC peer-link) Network P2p (vPC) P2p (vPC) P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4
Scenario Two – Non-VLT Environment This section presents an alternative to VLT for legacy Dell Networking MXL installations. It relies on the spanning tree protocol to protect against loops for redundant connections to Nexus switch pairs. Reliance on spanning tree for redundancy as presented here is not the preferred solution. Dell EMC recommends using the loop-free VLT environment presented in Scenario One – VLT Environment for MXL and M IOA switches. Note: The Dell Networking M IOA does not support spanning tree protocols. Therefore, this section does not include M IOA.
4.1
Overview Without VLT (or vPC on Nexus), redundant paths can form loops that must be blocked by a spanning tree protocol as shown in Figure 10.
MXL-A1
N5K-1
MXL-A2
N5K-2
Blade Server
Redundant links blocked by spanning tree in a fully meshed topology If an active link fails or is shut down, the spanning tree protocol enables fault tolerance by placing a blocked link in the forwarding state until the issue is resolved. Per VLAN Spanning Tree Plus (PVST+) and Multiple Spanning Tree Protocol (MSTP) allow for configuration of spanning tree on a per-VLAN basis. This allows network administrators to run different spanning tree instances and better utilize network bandwidth. Paths blocked for one VLAN can be made active for a different VLAN (or a group of VLANs in the case of MSTP).
35
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.2
Configuration The switch configuration examples are divided into two sections based on the spanning tree protocol used: PVST+ Configuration and MSTP Configuration. The configuration text is formatted so it can be cut and pasted into a plain text editor (Microsoft Notepad, Notepad++, for example) or directly into a switch console session. The configuration steps were developed with the switches starting at their default configurations. This may be helpful if there is an existing configuration on a switch that conflicts with this environment. Notes: To reset Dell MXL switches to factory defaults, step-by-step instructions are provided in Appendix D. The Cisco Nexus 5000 series switches were reset by using the #write erase command followed by #reload. When prompted after reload, "Power on Auto Provisioning" was not used, the admin password was configured, and the Nexus basic configuration dialog was not used. Refer to the Nexus system documentation for more information. When an MXL switch is connected to a Cisco Nexus switch, a number of DCBX (Data Center Bridging Exchange) error messages may be displayed on the serial console. To suppress these messages, either configure DCBX on the switches (beyond the scope of this guide) or disable DCB on the Dell Networking switches. To do this, use the command: (conf)#no dcb enable
4.3
PVST+ Configuration Per-VLAN Spanning Tree Plus (PVST+) is an implementation of the spanning-tree protocol where a separate spanning tree runs inside each VLAN. The Cisco Nexus implementation of this protocol is called Rapid Per VLAN Spanning Tree Plus (RPVST+) and is enabled on Nexus switches by default. This section provides detailed configuration steps to build the network running PVST+ shown in Figure 11 and Figure 12 . The cloud in the diagrams represents the upstream network. The network has two tagged VLANs: VLAN 30 and VLAN 40. The network adapter on Blade Server 1 is tagged in VLAN 30 and the network adapter on Blade Server 2 is tagged in VLAN 40. Blade servers 1 and 2 reside in slots 1 and 2, respectively, of the M1000e chassis. The MXLs reside in slots A1 and A2 in the back of the M1000e chassis.
36
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
PVST+ VLAN 30
VLAN 30 Root Bridge
Po101
Te0/1
Te0/41
E1/1
Te0/42
E1/2
MXL-A1
E1/3
N5K-1
Te0/43
Blade Server 1 VLAN 30
Fo0/33
Po102
Te Fo0/37 0/44
E1/9
E1/4
Po 127 Fo0/33 Fo0/37
Te0/1
MXL-A2
E1/7 E1/8
Po 55 E1/7 E1/8
Te 0/43
Po103
E1/3 E1/9 E1/4
Te0/44 Te0/41
E1/1
Te0/42
E1/2
N5K-2 Spanning Tree Blocked Interface
Po104
Interfaces blocked by PVST+ on VLAN 30
PVST+ VLAN 40 Po101
Te0/2
Te0/41
E1/1
Te0/42
E1/2
MXL-A1
E1/3
N5K-1
Te0/43
Blade Server 2 VLAN 40
Fo0/33
Te Fo0/37 0/44
Po102
E1/9 E1/4
Po 127 Fo0/33 Fo0/37
Te0/2
MXL-A2
E1/7 E1/8
Po 55 Te 0/43
E1/7 E1/8 Po103
E1/3
E1/9 E1/4
Te0/44 Te0/41
E1/1
Te0/42
E1/2
N5K-2 Spanning Tree Blocked Interface
Po104 VLAN 40 Root Bridge
Interfaces blocked by PVST+ on VLAN 40 Configuration of switch NK5-1 as VLAN 30's root bridge and switch NK5-2 as VLAN 40's root bridge blocks different port channels for each VLAN. Port channels 101 and 103 forward traffic on VLAN 30 and port channels 102 and 104 forward traffic on VLAN 40. This allows for better utilization of the available links.
37
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
The configuration sections that follow use the same port numbers and port channel numbers shown in Figure 11 and Figure 12. For more information on port mapping of MXL ports, see Appendix E.
4.3.1
MXL Configuration MXL-A1
MXL-A2
Set the hostname and configure the out-of-band management address.
Set the hostname and configure the out-of-band management address.
enable configure hostname MXL-A1
enable configure hostname MXL-A2
interface ManagementEthernet 0/0 ip address 172.25.112.91/24
interface ManagementEthernet 0/0 ip address 172.25.112.92/24
If prompted - Proceed with Static IP [confirm yes/no]: y Configure the management route.
If prompted - Proceed with Static IP [confirm yes/no]: y Configure the management route.
exit management route 0.0.0.0/0 172.25.112.254
exit management route 0.0.0.0/0 172.25.112.254
Enable PVST+ and globally disable BPDUs on edge ports.
Enable PVST+ and globally disable BPDUs on edge ports.
protocol spanning-tree pvst no disable edge-port bpdufilter default exit Configure the internal server-facing interfaces and make them edge ports.
Configure the internal server-facing interfaces and make them edge ports.
interface range tengigabitethernet 0/1, tengigabitethernet 0/2 no portmode hybrid switchport no shutdown spanning-tree pvst edge-port exit
interface range tengigabitethernet 0/1, tengigabitethernet 0/2 no portmode hybrid switchport no shutdown spanning-tree pvst edge-port exit
Configure the remaining interfaces and port channels.
Configure the remaining interfaces and port channels.
interface range tengigabitethernet 0/41-42
interface range tengigabitethernet 0/41-42
port-channel-protocol LACP port-channel 101 mode active exit
38
protocol spanning-tree pvst no disable edge-port bpdufilter default exit
port-channel-protocol LACP port-channel 104 mode active exit
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MXL-A1 no shutdown exit interface range tengigabitethernet 0/43-44
no shutdown exit interface range tengigabitethernet 0/43-44
port-channel-protocol LACP port-channel 102 mode active exit no shutdown exit
port-channel-protocol LACP port-channel 103 mode active exit no shutdown exit
interface range fortyGigE 0/33,fortyGigE 0/37 port-channel-protocol LACP port-channel 127 mode active exit no shutdown exit
interface range fortyGigE 0/33,fortyGigE 0/37 port-channel-protocol LACP port-channel 127 mode active exit no shutdown exit
interface Port-channel 101 switchport no shutdown exit
interface Port-channel 103 switchport no shutdown exit
interface Port-channel 102 switchport no shutdown exit
interface Port-channel 104 switchport no shutdown exit
interface Port-channel 127 switchport no shutdown exit
39
MXL-A2
interface Port-channel 127 switchport no shutdown exit
Tag the interfaces in VLANs 30 and 40.
Tag the interfaces in VLANs 30 and 40.
int vlan 30 tagged tengigabitethernet 0/1 tagged port-channel 101-102,127 no shutdown exit
int vlan 30 tagged tengigabitethernet 0/1 tagged port-channel 103-104,127 no shutdown exit
int vlan 40 tagged tengigabitethernet 0/2
int vlan 40 tagged tengigabitethernet 0/2
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MXL-A1
MXL-A2
tagged port-channel 101-102,127 no shutdown exit end
tagged port-channel 103-104,127 no shutdown exit end
Save the configuration.
Save the configuration.
write
write
MXL PVST+ configuration notes: The portmode hybrid command allows an Ethernet interface to carry multiple tagged VLANs and a single untagged (native) VLAN. If a given port is only expected to carry tagged VLANs or a single untagged VLAN, the portmode hybrid command may be omitted from the configuration. In this environment, both VLANs (30 and 40) are tagged so the hybrid port state is not used. Dell Networking systems do not expect a PVST+ BPDU on an untagged port. If this situation occurs, the Dell Networking OS places the port in an Error-Disable state. This behavior might result in the network not converging. To prevent the system from executing this action, use the no spanning-tree pvst err-disable cause invalid-pvst-bpdu command on each untagged port connected to another switch. Only run the spanning-tree pvst edge-port command on ports connecting to servers or other end nodes and never on ports that connect to other switches. This command designates a port as an edge of the spanning tree (only switches participate in spanning tree). This enables it to begin forwarding traffic as soon as its link is active. This happens many seconds before the spanning-tree protocol would otherwise allow it to forward traffic. MXL ports te 0/1-32 can safely be made edge ports. As fixed internal ports, they can only be connected to server blades or storage devices inside the M1000e chassis. The optional command edge-port bpdufilter default stops transmission of BPDUs on edge ports.
40
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.3.2
Nexus Configuration N5K-1
41
N5K-2
Enable the LACP feature.
Enable the LACP feature.
configure feature lacp
configure feature lacp
Configure the hostname, the management interface address and the management route.
Configure the hostname, the management interface address and the management route.
hostname N5K-1 interface mgmt 0 ip address 172.25.115.37/24 vrf member management no shutdown exit vrf context management ip route 0.0.0.0/0 172.25.115.254 exit
hostname N5K-2 interface mgmt 0 ip address 172.25.115.36/24 vrf member management no shutdown exit vrf context management ip route 0.0.0.0/0 172.25.115.254 exit
Ensure RPVST+ is enabled. Create the VLANs. Configure spanning tree settings.
Ensure RPVST+ is enabled. Create the VLANs. Configure spanning tree settings.
spanning-tree vlan 30,40 spanning-tree spanning-tree spanning-tree secondary
spanning-tree vlan 30,40 spanning-tree spanning-tree secondary spanning-tree
mode rapid-pvst pathcost method long vlan 30 root primary vlan 40 root
mode rapid-pvst pathcost method long vlan 30 root vlan 40 root primary
Configure the port channels and interfaces.
Configure the port channels and interfaces.
interface port-channel 101 switchport mode trunk switchport trunk allowed vlan 30,40 exit
interface port-channel 102 switchport mode trunk switchport trunk allowed vlan 30,40 exit
interface port-channel 103 switchport mode trunk
interface port-channel 104 switchport mode trunk
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
N5K-1
42
N5K-2
switchport trunk allowed vlan 30,40 exit
switchport trunk allowed vlan 30,40 exit
interface port-channel 55 switchport mode trunk switchport trunk allowed vlan 30,40 exit
interface port-channel 55 switchport mode trunk switchport trunk allowed vlan 30,40 exit
interface ethernet 1/1-2 switchport mode trunk switchport trunk allowed vlan 30,40 channel-group 101 mode active exit
interface ethernet 1/1-2 switchport mode trunk switchport trunk allowed vlan 30,40 channel-group 104 mode active exit
interface ethernet 1/3-4 switchport mode trunk switchport trunk allowed vlan 30,40 channel-group 103 mode active exit
interface ethernet 1/3-4 switchport mode trunk switchport trunk allowed vlan 30,40 channel-group 102 mode active exit
interface ethernet 1/7-8 switchport mode trunk switchport trunk allowed vlan 30,40 channel-group 55 mode active exit
interface ethernet 1/7-8 switchport mode trunk switchport trunk allowed vlan 30,40 channel-group 55 mode active exit
interface Ethernet1/9 switchport mode trunk switchport trunk allowed vlan 30,40 exit end
interface Ethernet1/9 switchport mode trunk switchport trunk allowed vlan 30,40 exit end
Save the configuration.
Save the configuration.
copy running-config startup-config
copy running-config startup-config
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Nexus configuration notes Nexus switches enable RPVST+ by default. The example shows the spanning-tree mode rapidpvst command in case the Nexus is not at default settings. To see current spanning tree settings on Nexus, run the command show running-config spanning-tree all. The Dell Networking OS implementation of PVST+ uses IEEE 802.1s costs as the default port costs. For a more consistent path cost calculation with MXL, run the spanning-tree pathcost method long command on the Nexus switches. It is a best practice to make the root bridge as close as possible to the network core. In this environment, the Nexus switches serve as the spanning tree root primary / alternate switches. On Nexus, the “primary” command above sets the priority to 24576, and the “secondary” command above sets the priority to 28672. The Dell switches have a default priority of 32768. The switch with the lowest priority becomes the root bridge.
4.3.3
Server Network Adapter Configuration In this topology, the server adapter ports are configured as load balancing NIC teams. Traffic is load balanced across the ports, and the team provides fault tolerance in case of a link failure. On Blade Server 1, the NIC team is tagged in VLAN 30. On Blade Server 2, the NIC team is tagged in VLAN 40. Note: Refer to the network adapter or operating system documentation for NIC teaming instructions. For an overview of NIC teaming in Windows 2012 R2, see http://technet.microsoft.com/enus/library/hh831648.aspx. Network adapters cannot be configured as LACP teams in this environment. To use LACP NIC teams, use VLT as covered in Scenario One – VLT Environment.
4.3.4
Commands to verify PVST+ For this environment, the spanning tree algorithm blocked ports appropriately based on the selection of different root bridges. If further adjustments are desired, additional interface parameters (port cost and port priority) can be adjusted to increase or decrease the probability that a port becomes a forwarding port. For more information, see Appendix F for Dell switches and your Nexus system documentation. In the command output, the Root ID must match the MAC address of the Nexus switch configured as the spanning tree root bridge for the VLAN. Blocked interfaces will be marked as “BLK” in the "Sts" column and forwarding interfaces will be marked as "FWD".
43
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.3.4.1
Spanning tree commands and output – PVST VLAN 30 PVST+ VLAN 30
VLAN 30 Root Bridge
Po101 Te0/41
E1/1
Te0/42
E1/2
MXL-A1
Te0/1
E1/3
N5K-1
Te0/43
Blade Server 1 VLAN 30
Fo0/33
Te Fo0/37 0/44
E1/9
Po102
E1/4
Po 127
Po 55
Fo0/33 Fo0/37
Te0/1
E1/7 E1/8
Te 0/43
MXL-A2
E1/7 E1/8 Po103
Te0/44
Te0/41
Port Channels 101 and 103 are Forwarding
E1/3 E1/9 E1/4
N5K-2
Spanning Tree Blocked Interface
E1/1 E1/2
VLAN 30 blocked interfaces Verify PVST+ on VLAN 30 using the following commands: MXL-A1#show spanning-tree pvst vlan 30 brief VLAN 30 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24606, Address 002a.6af7.5bbc Root Bridge hello time 2, max age 20, forward delay 15 Bridge ID Priority 32768, Address 001e.c9f1.04ba Configured hello time 2, max age 20, forward delay 15 Bpdu filter enabled globally Interface Name PortID -------- -------Po 101 128.102 Po 102 128.103 Po 127 128.128 Te 0/1 128.130
Prio ---128 128 128 128
Interface Name ---------Po 101 Po 102 Po 127 Te 0/1
PortID -------128.102 128.103 128.128 128.130
Role -----Root Altr Altr Desg
Cost -----1800 1800 600 2000
Designated Sts Cost Bridge ID PortID ---------- ------- -------------------- -------FWD 1800 24606 002a.6af7.5bbc 144.100 BLK 1800 28702 002a.6af7.8081 144.101 BLK 1800 32768 001e.c9f1.0392 128.128 FWD 1800 32768 001e.c9f1.04ba 128.130
Prio ---128 128 128 128
Cost ------1800 1800 600 2000
Sts ----------FWD BLK BLK FWD
Cost ------1800 1800 1800 1800
MXL-A2#show spanning-tree pvst vlan 30 brief
44
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Link-type --------P2P P2P P2P P2P
Edge ---No No No Yes
BpduFilter ---------No No No No
VLAN 30 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24606, Address 002a.6af7.5bbc Root Bridge hello time 2, max age 20, forward delay 15 Bridge ID Priority 32768, Address 001e.c9f1.0392 Configured hello time 2, max age 20, forward delay 15 Bpdu filter enabled globally Interface Name ---------Po 103 Po 104 Po 127 Te 0/1
PortID -------128.104 128.105 128.128 128.130
Interface Name ---------Po 103 Po 104 Po 127 Te 0/1
Role -----Root Altr Desg Desg
Prio ---128 128 128 128
Cost -----1800 1800 600 2000
PortID -------128.104 128.105 128.128 128.130
Prio ---128 128 128 128
Sts ----------FWD BLK FWD FWD
Cost ------1800 1800 600 2000
Cost ------1800 1800 1800 1800
Sts ----------FWD BLK FWD FWD
Designated Bridge ID PortID -------------------- -------24606 002a.6af7.5bbc 144.102 28702 002a.6af7.8081 144.103 32768 001e.c9f1.0392 128.128 32768 001e.c9f1.0392 128.130
Cost ------1800 1800 1800 1800
Link-type --------P2P P2P P2P P2P
Edge ---No No No Yes
BpduFilter ---------No No No No
N5K-1# show spanning-tree vlan 30 VLAN0030 Spanning tree enabled protocol rstp Root ID Priority 24606 Address 002a.6af7.5bbc This bridge is the root Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po101 Po103 Eth1/9
45
Role ---Desg Desg Desg Desg
Forward Delay 15 sec
24606 (priority 24576 sys-id-ext 30) 002a.6af7.5bbc 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
Prio.Nbr -------128.4150 128.4196 128.4198 128.137
Type -------------------------------P2p P2p P2p P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
N5K-2# show spanning-tree vlan 30 VLAN0030 Spanning tree enabled protocol rstp Root ID Priority 24606 Address 002a.6af7.5bbc Cost 1000 Port 4150 (port-channel55) Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po102 Po104 Eth1/9
46
Role ---Root Desg Desg Desg
Forward Delay 15 sec
28702 (priority 28672 sys-id-ext 30) 002a.6af7.8081 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
Prio.Nbr -------128.4150 128.4197 128.4199 128.137
Type -------------------------------P2p P2p P2p P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.3.4.2
Spanning tree commands and output – PVST VLAN 40 PVST+ VLAN 40 Po101
Te0/41
E1/1
Te0/42
E1/2
MXL-A1
Te0/2
E1/3
N5K-1
Te0/43
Blade Server 2 VLAN 40
Fo0/33
Te Fo0/37 0/44
E1/9
Po102
E1/4
Po 127
Po 55
Fo0/33 Fo0/37
MXL-A2
Te0/2
Port Channels 102 and 104 are Forwarding
E1/7 E1/8
E1/7 E1/8
Te 0/43
E1/3
Po103
E1/9 E1/4
Te0/44 Te0/41
E1/1
Te0/42
E1/2
N5K-2 Spanning Tree Blocked Interface
Po104 VLAN 40 Root Bridge
VLAN 40 blocked interfaces Verify PVST+ on VLAN 40 using the following commands: MXL-A1#show spanning-tree pvst vlan 40 brief VLAN 40 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24616, Address 002a.6af7.8081 Root Bridge hello time 2, max age 20, forward delay 15 Bridge ID Priority 32768, Address 001e.c9f1.04ba Configured hello time 2, max age 20, forward delay 15 Bpdu filter enabled globally
47
Interface Name ---------Po 101 Po 102 Po 127 Te 0/2
PortID -------128.102 128.103 128.128 128.140
Interface Name ---------Po 101 Po 102
Role -----Altr Root
Prio ---128 128 128 128
Cost -----1800 1800 600 2000
PortID -------128.102 128.103
Prio ---128 128
Sts ----------BLK FWD BLK FWD
Cost ------1800 1800
Cost ------1800 1800 1800 1800
Sts ----------BLK FWD
Designated Bridge ID PortID -------------------- -------28712 002a.6af7.5bbc 144.100 24616 002a.6af7.8081 144.101 32768 001e.c9f1.0392 128.128 32768 001e.c9f1.04ba 128.140
Cost ------1800 1800
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Link-type --------P2P P2P
Edge ---No No
BpduFilter ---------No No
Po 127 Te 0/2
Altr Desg
128.128 128.140
128 128
600 2000
BLK FWD
1800 1800
P2P P2P
No Yes
No No
MXL-A2#show spanning-tree pvst vlan 40 brief VLAN 40 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24616, Address 002a.6af7.8081 Root Bridge hello time 2, max age 20, forward delay 15 Bridge ID Priority 32768, Address 001e.c9f1.0392 Configured hello time 2, max age 20, forward delay 15 Bpdu filter enabled globally Interface Name ---------Po 103 Po 104 Po 127 Te 0/2
PortID -------128.104 128.105 128.128 128.140
Interface Name ---------Po 103 Po 104 Po 127 Te 0/2
Role -----Altr Root Desg Desg
Prio ---128 128 128 128
Cost -----1800 1800 600 2000
PortID -------128.104 128.105 128.128 128.140
Prio ---128 128 128 128
Sts ----------BLK FWD FWD FWD
Cost ------1800 1800 600 2000
Cost ------1800 1800 1800 1800
Sts ----------BLK FWD FWD FWD
Designated Bridge ID PortID -------------------- -------28712 002a.6af7.5bbc 144.102 24616 002a.6af7.8081 144.103 32768 001e.c9f1.0392 128.128 32768 001e.c9f1.0392 128.140
Cost ------1800 1800 1800 1800
Link-type --------P2P P2P P2P P2P
Edge ---No No No Yes
BpduFilter ---------No No No No
N5K-1# show spanning-tree vlan 40 VLAN0040 Spanning tree enabled protocol rstp Root ID Priority 24616 Address 002a.6af7.8081 Cost 1000 Port 4150 (port-channel55) Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po101 Po103 Eth1/9
48
Role ---Root Desg Desg Desg
Forward Delay 15 sec
28712 (priority 28672 sys-id-ext 40) 002a.6af7.5bbc 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
Prio.Nbr -------128.4150 128.4196 128.4198 128.137
Type -------------------------------P2p P2p P2p P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
N5K-2# show spanning-tree vlan 40 VLAN0040 Spanning tree enabled protocol rstp Root ID Priority 24616 Address 002a.6af7.8081 This bridge is the root Hello Time 2 sec Max Age 20 sec Bridge ID
Priority Address Hello Time
Interface ---------------Po55 Po102 Po104 Eth1/9
49
Role ---Desg Desg Desg Desg
Forward Delay 15 sec
24616 (priority 24576 sys-id-ext 40) 002a.6af7.8081 2 sec Max Age 20 sec Forward Delay 15 sec
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
Prio.Nbr -------128.4150 128.4197 128.4199 128.137
Type -------------------------------P2p P2p P2p P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.4
MSTP Configuration Multiple Spanning Tree Protocol (MSTP), specified in IEEE 802.1s and later merged into 802.1Q-2005, is a RSTP-based spanning tree variation. It can be an improvement to PVST+ in some environments. PVST+ allows one spanning tree instance for each VLAN. The 1:1 approach of PVST+ may not be suitable for a large number of VLANs, because each spanning tree instance consumes bandwidth and processing resources. Like PVST+, MSTP allows multiple spanning tree instances. However, MSTP allows the mapping of multiple VLANs to one spanning tree instance to reduce the total number of required spanning tree instances. With MSTP enabled, MST 0 includes all 4094 VLANs by default on the Dell and Nexus switches. Administrators can then group and move VLANs into new instances as needed. This section provides detailed configuration steps to build the network running MSTP shown in Figure 15 and Figure 16. The cloud in the diagrams represents the upstream network. This topology is a simplified example that uses three VLANS: 30, 40, and 50. VLANs 30 and 40 are configured in MST instance 1, and VLAN 50 is configured in MST instance 2. The network adapter on Blade Server 1 is tagged in VLANs 30 and 40 and the network adapter on Blade Server 2 is tagged in VLAN 50. Blade servers 1 and 2 reside in slots 1 and 2, respectively, of the M1000e chassis. The MXLs reside in slots A1 and A2 in the back of the M1000e chassis.
50
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MST Instance 1 VLANs 30 & 40
MST 1 Root Bridge
Po101
Te0/1
Te0/41
E1/1
Te0/42
E1/2
MXL-A1
E1/3
N5K-1
Te0/43
Blade Server 1 VLAN 30 & 40
Fo0/33
Te Fo0/37 0/44
Po102
E1/9
E1/4
Po 127 Fo0/33 Fo0/37
Te0/1
MXL-A2
E1/7 E1/8
Po 55 E1/7 E1/8
Te 0/43
Po103
E1/3 E1/9 E1/4
Te0/44 Te0/41
E1/1
Te0/42
E1/2
N5K-2 Spanning Tree Blocked Interface
Po104
Interfaces blocked in MST instance 1
MST Instance 2 VLAN 50 Po101
Te0/2
Te0/41
E1/1
Te0/42
E1/2
MXL-A1
E1/3
N5K-1
Te0/43
Blade Server 2 VLAN 50
Fo0/33
Te Fo0/37 0/44
Po102
E1/9 E1/4
Po 127 Fo0/33 Fo0/37
Te0/2
MXL-A2
E1/7 E1/8
Po 55 Te 0/43
E1/7 E1/8 Po103
E1/3
E1/9 E1/4
Te0/44 Te0/41
E1/1
Te0/42
E1/2
N5K-2 Spanning Tree Blocked Interface
Po104 MST 2 Root Bridge
Interfaces blocked in MST instance 2
51
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.4.1
MXL Configuration MXL-A1 Set the hostname and configure the out-of-band management address.
Set the hostname and configure the out-of-band management address.
enable configure hostname MXL-A1 interface ManagementEthernet 0/0 ip address 172.25.112.91/24
enable configure hostname MXL-A2 interface ManagementEthernet 0/0 ip address 172.25.112.92/24
If prompted - Proceed with Static IP
If prompted - Proceed with Static IP
[confirm yes/no]: y Configure the management route.
[confirm yes/no]: y Configure the management route.
exit management route 0.0.0.0/0 172.25.112.254
exit management route 0.0.0.0/0 172.25.112.254
Enable MSTP, disable BPDU’s on edge ports, place VLANs 30 and 40 in MST instance 1 and VLAN 50 in MST instance 2.
Enable MSTP, disable BPDU’s on edge ports, place VLANs 30 and 40 in MST instance 1 and VLAN 50 in MST instance 2.
protocol spanning-tree mstp no disable edge-port bpdufilter default msti 1 vlan 30,40 msti 2 vlan 50 exit
protocol spanning-tree mstp no disable edge-port bpdufilter default msti 1 vlan 30,40 msti 2 vlan 50 exit
Configure the internal server-facing interfaces and make them edge ports.
Configure the internal server-facing interfaces and make them edge ports.
interface range tengigabitethernet 0/1, tengigabitethernet 0/2 switchport no shutdown spanning-tree mstp edge-port exit
interface range tengigabitethernet 0/1, tengigabitethernet 0/2 switchport no shutdown spanning-tree mstp edge-port exit
Configure the remaining interfaces and port channels.
Configure the remaining interfaces and port channels.
interface range tengigabitethernet 0/41-42
interface range tengigabitethernet 0/41-42
port-channel-protocol LACP port-channel 101 mode active exit no shutdown
52
MXL-A2
port-channel-protocol LACP port-channel 104 mode active exit no shutdown
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MXL-A1 exit interface range tengigabitethernet 0/43-44
exit interface range tengigabitethernet 0/43-44
port-channel-protocol LACP port-channel 102 mode active exit no shutdown exit
port-channel-protocol LACP port-channel 103 mode active exit no shutdown exit
interface range fortyGigE 0/33,fortyGigE 0/37 port-channel-protocol LACP port-channel 127 mode active exit no shutdown exit
interface range fortyGigE 0/33,fortyGigE 0/37 port-channel-protocol LACP port-channel 127 mode active exit no shutdown exit
interface Port-channel 101 description LINK TO N5K-1 switchport no shutdown exit
interface Port-channel 103 description LINK TO N5K-1 switchport no shutdown exit
interface Port-channel 102 description LINK TO N5K-2 switchport no shutdown exit
interface Port-channel 104 description LINK TO N5K-2 switchport no shutdown exit
interface Port-channel 127 description LINK TO MXL-A2 switchport no shutdown exit
53
MXL-A2
interface Port-channel 127 description LINK TO MXL-A1 switchport no shutdown exit
Tag the interfaces in VLANs 30, 40 and 50.
Tag the interfaces in VLANs 30, 40 and 50.
int vlan 30 tagged tengigabitethernet 0/1 tagged port-channel 101-102,127 no shutdown exit
int vlan 30 tagged tengigabitethernet 0/1 tagged port-channel 103-104,127 no shutdown exit
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MXL-A1
MXL-A2
int vlan 40 tagged tengigabitethernet 0/1 tagged port-channel 101-102,127 no shutdown exit
int vlan 40 tagged tengigabitethernet 0/1 tagged port-channel 103-104,127 no shutdown exit
int vlan 50 tagged tengigabitethernet 0/2 tagged port-channel 101-102,127 no shutdown exit end
int vlan 50 tagged tengigabitethernet 0/2 tagged port-channel 103-104,127 no shutdown exit end
Save the configuration.
Save the configuration.
write
write
MXL MSTP configuration notes: The portmode hybrid command allows an Ethernet interface to carry multiple tagged VLANs and a single untagged (native) VLAN. If a given port is only expected to carry tagged VLANs or a single untagged VLAN, the portmode hybrid command may be omitted from the configuration. In this environment, all VLANs are tagged so the hybrid port state is not used. Only run the spanning-tree mstp edge-port command on ports connecting to servers or other end nodes and never on ports that connect to other switches. This command designates a port as an edge of the spanning tree (only switches participate in spanning tree). This enables it to begin forwarding traffic as soon as its link is active. This happens many seconds before the spanning-tree protocol would otherwise allow it to forward traffic. MXL ports te 0/1-32 can safely be made edge ports. As fixed internal ports, they can only be connected to server blades or storage devices inside the M1000e chassis. The optional command edge-port bpdufilter default stops transmission of BPDUs on edge ports.
54
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.4.2
Nexus Configuration N5K-1
55
N5K-2
Enable the LACP feature.
Enable the LACP feature.
configure feature lacp
configure feature lacp
Configure the hostname, the management interface address and the management route.
Configure the hostname, the management interface address and the management route.
hostname N5K-1 interface mgmt 0 ip address 172.25.115.37/24 vrf member management no shutdown exit vrf context management ip route 0.0.0.0/0 172.25.115.254 exit
hostname N5K-2 interface mgmt 0 ip address 172.25.115.36/24 vrf member management no shutdown exit vrf context management ip route 0.0.0.0/0 172.25.115.254 exit
Enable MSTP. Create the VLANs. Configure spanning tree settings.
Enable MSTP. Create the VLANs. Configure spanning tree settings.
spanning-tree mode mst vlan 30,40,50 spanning-tree pathcost method long spanning-tree mst 1 root primary spanning-tree mst 2 root secondary spanning-tree mst configuration instance 1 vlan 30,40 instance 2 vlan 50
spanning-tree mode mst vlan 30,40,50 spanning-tree pathcost method long spanning-tree mst 2 root primary spanning-tree mst 1 root secondary spanning-tree mst configuration instance 1 vlan 30,40 instance 2 vlan 50
Configure the port channels and interfaces.
Configure the port channels and interfaces.
interface port-channel 101 switchport mode trunk switchport trunk allowed vlan 30,40,50 exit
interface port-channel 102 switchport mode trunk switchport trunk allowed vlan 30,40,50 exit
interface port-channel 103 switchport mode trunk
interface port-channel 104 switchport mode trunk
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
N5K-1
56
N5K-2
switchport trunk allowed vlan 30,40,50 exit
switchport trunk allowed vlan 30,40,50 exit
interface port-channel 55 switchport mode trunk switchport trunk allowed vlan 30,40,50 exit
interface port-channel 55 switchport mode trunk switchport trunk allowed vlan 30,40,50 exit
interface ethernet 1/1-2 switchport mode trunk switchport trunk allowed vlan 30,40,50 channel-group 101 mode active exit
interface ethernet 1/1-2 switchport mode trunk switchport trunk allowed vlan 30,40,50 channel-group 104 mode active exit
interface ethernet 1/3-4 switchport mode trunk switchport trunk allowed vlan 30,40,50 channel-group 103 mode active exit
interface ethernet 1/3-4 switchport mode trunk switchport trunk allowed vlan 30,40,50 channel-group 102 mode active exit
interface ethernet 1/7-8 switchport mode trunk switchport trunk allowed vlan 30,40,50 channel-group 55 mode active exit
interface ethernet 1/7-8 switchport mode trunk switchport trunk allowed vlan 30,40,50 channel-group 55 mode active exit
interface Ethernet1/9 switchport mode trunk switchport trunk allowed vlan 30,40,50 exit
interface Ethernet1/9 switchport mode trunk switchport trunk allowed vlan 30,40,50 exit
Save the configuration.
Save the configuration.
copy running-config startup-config
copy running-config startup-config
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Nexus configuration notes: The Dell Networking OS implementation of MSTP uses IEEE 802.1s costs as the default port costs. For a more consistent path cost calculation with MXL, run the spanning-tree pathcost method long command on the Nexus switches. It is a best practice to make the root bridge as close as possible to the network core. In this environment, the nexus switches serve as the spanning tree root primary / alternate switches. On Nexus, the “primary” command above sets the priority to 24576 and the “secondary” command above sets the priority to 28672. Dell switches have a default priority of 32768. The switch with the lowest priority becomes the root bridge.
General MSTP configuration notes: In addition to the MST instance number, MSTP allows for the configuration of name and revision values. All three of these values must match on all connected switches to form an MSTP region. On Nexus and MXL, the default values for the name (null) and revision (0) match, which makes configuration of these fields unnecessary. With other 3rd party switches, these fields may require configuration in order to match. All bridges in the MSTP region must have the same VLAN-to-instance mapping.
4.4.3
Server Network Adapter Configuration In this topology, the server adapter ports are configured as load balancing NIC teams. Traffic is load balanced across the ports, and the team provides fault tolerance in case of a link failure. On Blade Server 1, the NIC team is tagged in VLANs 30 and 40. On Blade Server 2, the NIC team is tagged in VLAN 50. Note: Refer to the network adapter or operating system documentation for NIC teaming instructions. For an overview of NIC teaming in Windows 2012 R2, see http://technet.microsoft.com/enus/library/hh831648.aspx. Network adapters cannot be configured as LACP teams in this environment. To use LACP NIC teams, use VLT as covered in Scenario One – VLT Environment.
4.4.4
Commands to verify MSTP For this environment, the MSTP algorithm blocks interfaces appropriately based on the selection of different root bridges for each MST instance. If further adjustments are desired, interface parameters (port cost and port priority) can be adjusted to increase or decrease the probability that the system puts a particular interface in the forwarding state. For more information on modifying port cost and priority, see Appendix F for Dell switches and Nexus documentation for your system. Run the following commands to verify the MST instance-to-VLAN mapping is consistent across all four switches in the topology.
57
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
Run the following commands on each MXL switch: MXL-A1#show spanning-tree mst configuration Revision: 0 MSTI VID 1 30,40 2 50
Note: Output is identical on MXL-A2. The MXL does not include the default instance (MSTI 0) in the output above. Run the following command to confirm the instance-to-VLAN mapping for MSTI 0: MXL-A1#sh spanning-tree msti 0 brief MSTI 0 VLANs mapped 1-29, 31-39, 41-49, 51-4094 | (output truncated) Note: Output is identical on MXL-A2.
Run the following command on each Nexus switch: N5K-1# show spanning-tree mst configuration Name [] Revision 0 Instances configured 3 Instance Vlans mapped -------- --------------------------------------------------------------------0 1-29,31-39,41-49,51-4094 1 30,40 2 50 -------------------------------------------------------------------------------
Note: Output is identical on N5K-2.
Notes: Consistency among all switches in the topology is important. The MST instance numbers and mapped VLANs must match on all four switches. Switch configuration does not include the name value. Therefore, it appears as [] on Nexus and is not included in the output on MXL (unless a name is configured). The default revision number is 0 which appears in the command output. If a name or revision number is configured, the value must match on all four switches.
58
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
4.4.4.1
Spanning tree commands and output for MST 1 MST Instance 1 VLANs 30 & 40
MST 1 Root Bridge
Po101
Te0/1
Te0/41
E1/1
Te0/42
E1/2
MXL-A1
E1/3
N5K-1
Te0/43
Blade Server 1 VLAN 30 & 40
Fo0/33
Te Fo0/37 0/44
E1/9
Po102
E1/4
Po 127
Po 55
Fo0/33 Fo0/37
Port Channels 101 and 103 are Forwarding
Te0/1
E1/7 E1/8
MXL-A2
Te 0/43
E1/7 E1/8 Po103
E1/3 E1/9 E1/4
Te0/44 Te0/41
E1/1
Te0/42
E1/2
N5K-2
Po104
Spanning Tree Blocked Interface
MST instance 1 blocked interfaces In the command output, the Root ID must match the MAC address of the Nexus switch configured as the spanning tree root bridge for MST instance 1. Blocked interfaces will be marked as “BLK” in the "Sts" column and forwarding interfaces will be marked as "FWD". MXL-A1#show spanning-tree msti 1 brief MSTI 1 VLANs mapped 30, 40 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24576, Address 002a.6af7.5bbc Root Bridge hello time 2, max age 20, forward delay 15, max hops 19 Bridge ID Priority 32768, Address 001e.c9f1.04ba Configured hello time 2, max age 20, forward delay 15, max hops 20 Bpdu filter enabled globally Interface Name ---------Po 101 Po 102 Po 127 Te 0/1
PortID -------128.102 128.103 128.128 128.130
Prio ---128 128 128 128
Cost ------1800 1800 600 2000
Sts ----------FWD BLK BLK FWD
Cost ------1800 1800 1800 1800
Designated Bridge ID -------------------24576 002a.6af7.5bbc 28672 002a.6af7.8081 32768 001e.c9f1.0392 32768 001e.c9f1.04ba
Interface
59
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
PortID -------144.100 144.101 128.128 128.130 Bpdu
Name ---------Po 101 Po 102 Po 127 Te 0/1
Role -----Root Altr Altr Desg
PortID -------128.102 128.103 128.128 128.130
Prio ---128 128 128 128
Cost ------1800 1800 600 2000
Sts --------FWD BLK BLK FWD
Cost ------1800 1800 1800 1800
Link-type --------P2P P2P P2P P2P
Edge ---No No No Yes
Filter -----No No No No
Boundary -------No No No No
MXL-A2#show spanning-tree msti 1 brief MSTI 1 VLANs mapped 30, 40 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24576, Address 002a.6af7.5bbc Root Bridge hello time 2, max age 20, forward delay 15, max hops 19 Bridge ID Priority 32768, Address 001e.c9f1.0392 Configured hello time 2, max age 20, forward delay 15, max hops 20 Bpdu filter enabled globally Interface Name ---------Po 103 Po 104 Po 127 Te 0/1
PortID -------128.104 128.105 128.128 128.130
Interface Name ---------Po 103 Po 104 Po 127 Te 0/1
Role -----Root Altr Desg Desg
Prio ---128 128 128 128
Cost ------1800 1800 600 2000
PortID -------128.104 128.105 128.128 128.130
Prio ---128 128 128 128
Sts ----------FWD BLK FWD FWD
Cost ------1800 1800 600 2000
Cost ------1800 1800 1800 1800
Sts ---------FWD BLK FWD FWD
Designated Bridge ID -------------------24576 002a.6af7.5bbc 28672 002a.6af7.8081 32768 001e.c9f1.0392 32768 001e.c9f1.0392
Cost ------1800 1800 1800 1800
Link-type --------P2P P2P P2P P2P
Edge ---No No No Yes
PortID -------144.102 144.103 128.128 128.130 Bpdu Filter -----No No No No
Boundary -------No No No No
N5K-1# show spanning-tree mst 1 ##### MST1 Bridge Root
vlans mapped: 30,40 address 002a.6af7.5bbc this switch for MST1
Interface ---------------Po55 Po101 Po103 Eth1/9
60
Role ---Desg Desg Desg Desg
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
priority
Prio.Nbr -------128.4150 128.4196 128.4198 128.137
24577 (24576 sysid 1)
Type -------------------------------P2p P2p P2p P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
N5K-2# show spanning-tree mst 1 ##### MST1 Bridge Root
vlans mapped: 30,40 address 002a.6af7.8081 address 002a.6af7.5bbc port Po55
Interface ---------------Po55 Po102 Po104 Eth1/9
4.4.4.2
Role ---Root Desg Desg Desg
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
priority priority cost Prio.Nbr -------128.4150 128.4197 128.4199 128.137
28673 (28672 sysid 1) 24577 (24576 sysid 1) 1000 rem hops 19
Type -------------------------------P2p P2p P2p P2p
Spanning tree commands and output for MST 2 MST Instance 2 VLAN 50 Po101
Te0/2
Te0/41
E1/1
Te0/42
E1/2
MXL-A1
E1/3
N5K-1
Te0/43
Blade Server 2 VLAN 50
Fo0/33
Te Fo0/37 0/44
Po102
E1/9 E1/4
Po 127 Fo0/33 Fo0/37
Te0/2
Port Channels 102 and 104 are Forwarding
MXL-A2
E1/7 E1/8
Po 55 Te 0/43
E1/7 E1/8 Po103
E1/3
E1/9 E1/4
Te0/44 Te0/41
E1/1
Te0/42
E1/2
N5K-2 Spanning Tree Blocked Interface
Po104 MST 2 Root Bridge
MST instance 2 blocked interfaces
In the command output, the Root ID must match the MAC address of the Nexus switch configured as the spanning tree root bridge for MST instance 2. Blocked interfaces will be marked as “BLK” in the "Sts" column and forwarding interfaces will be marked as "FWD".
61
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
MXL-A1#sh spanning-tree msti 2 brief MSTI 2 VLANs mapped 50 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24576, Address 002a.6af7.8081 Root Bridge hello time 2, max age 20, forward delay 15, max hops 19 Bridge ID Priority 32768, Address 001e.c9f1.04ba Configured hello time 2, max age 20, forward delay 15, max hops 20 Bpdu filter enabled globally Interface Name ---------Po 101 Po 102 Po 127 Te 0/1
PortID -------128.102 128.103 128.128 128.130
Interface Name ---------Po 101 Po 102 Po 127 Te 0/1
Role -----Altr Root Altr Desg
Prio ---128 128 128 128
Cost ------1800 1800 600 2000
PortID -------128.102 128.103 128.128 128.130
Prio ---128 128 128 128
Sts ----------BLK FWD BLK FWD
Cost ------1800 1800 600 2000
Cost ------1800 1800 1800 1800
Designated Bridge ID -------------------28672 002a.6af7.5bbc 24576 002a.6af7.8081 32768 001e.c9f1.0392 32768 001e.c9f1.04ba
PortID -------144.100 144.101 128.128 128.130
Bpdu Sts Cost Link-type Edge Filter Boundary ---------- ------- --------- ---- ------ -------BLK 1800 P2P No No No FWD 1800 P2P No No No BLK 1800 P2P No No No FWD 1800 P2P Yes No No
MXL-A2#show spanning-tree msti 2 brief MSTI 2 VLANs mapped 50 Executing IEEE compatible Spanning Tree Protocol Root ID Priority 24576, Address 002a.6af7.8081 Root Bridge hello time 2, max age 20, forward delay 15, max hops 19 Bridge ID Priority 32768, Address 001e.c9f1.0392 Configured hello time 2, max age 20, forward delay 15, max hops 20 Bpdu filter enabled globally
62
Interface Name ---------Po 103 Po 104 Po 127 Te 0/1
PortID -------128.104 128.105 128.128 128.130
Interface Name ---------Po 103 Po 104 Po 127 Te 0/1
Role -----Altr Root Desg Desg
Prio ---128 128 128 128
Cost ------1800 1800 600 2000
PortID -------128.104 128.105 128.128 128.130
Prio ---128 128 128 128
Sts ----------BLK FWD FWD FWD
Cost ------1800 1800 600 2000
Cost ------1800 1800 1800 1800
Designated Bridge ID -------------------28672 002a.6af7.5bbc 24576 002a.6af7.8081 32768 001e.c9f1.0392 32768 001e.c9f1.0392
PortID -------144.102 144.103 128.128 128.130
Bpdu Sts Cost Link-type Edge Filter Boundary ---------- ------- --------- ---- ------ -------BLK 1800 P2P No No No FWD 1800 P2P No No No FWD 1800 P2P No No No FWD 1800 P2P Yes No No
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
N5K-1# show spanning-tree mst 2 ##### MST2 Bridge Root
vlans mapped: 50 address 002a.6af7.5bbc address 002a.6af7.8081 port Po55
Interface ---------------Po55 Po101 Po103 Eth1/9
Role ---Root Desg Desg Desg
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
priority priority cost Prio.Nbr -------128.4150 128.4196 128.4198 128.137
28674 (28672 sysid 2) 24578 (24576 sysid 2) 1000 rem hops 19
Type -------------------------------P2p P2p P2p P2p
N5K-2# show spanning-tree mst 2 ##### MST2 Bridge Root
vlans mapped: 50 address 002a.6af7.8081 this switch for MST2
Interface ---------------Po55 Po102 Po104 Eth1/9
63
Role ---Desg Desg Desg Desg
Sts --FWD FWD FWD FWD
Cost --------1000 1000 1000 2000
priority
Prio.Nbr -------128.4150 128.4197 128.4199 128.137
24578 (24576 sysid 2)
Type -------------------------------P2p P2p P2p P2p
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
A
References Dell Virtual Link Trunking Reference Architecture 2.0 Dell Networking Configuration Guide for the MXL 10/40GbE Switch I/O Module 9.10(0.0) Dell PowerEdge Configuration Guide for the M I/O Aggregator 9.10(0.0) Cisco NX-OS Software Virtual PortChannel: Fundamental Concepts 5.0 (Cisco Inc.) Cisco Nexus 5600 Series NX-OS Interfaces Configuration Guide, Release 7.x
64
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
B
Components used in this guide The software versions and licenses used while developing this guide are shown below.
65
Switch
Software Version
Licenses
Dell Networking MXL
Dell Networking OS 9.10(0.0)
Not Applicable
Dell PowerEdge M IOA
Dell Networking OS 9.10(0.0)
Not Applicable
Cisco Nexus 5672UP
Cisco NX-OS 7.0(5)N1(1)
LAN_BASE_SERVICES_PKG cisco 1.0 ENTERPRISE_PKG cisco 1.0
Cisco Nexus 5548
Cisco NX-OS 7.0(8)N1(1)
LAN_BASE_SERVICES_PKG cisco 1.0 ENTERPRISE_PKG cisco 1.0
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
C
Terminology CMC (Chassis Management Controller): The embedded management interface of the Dell PowerEdge™ M1000e blade server chassis. Among other functions, the CMC provides network and console access to installed IO modules including the Dell Networking MXL and M IOA switches. DHCP (Dynamic Host Configuration Protocol): A network management protocol used to dynamically assign network settings to devices connected to a network. Fault Tolerance: Enables a network to continue operating properly in the event of a failure of some of its components. LACP (Link Aggregation Control Protocol): LACP is used to control the bundling of several physical ports together to form a single logical channel. LACP allows a network device to negotiate an automatic bundling of links by sending LACP packets to the peer (which is a directly connected device that also implements LACP). LAG (Link Aggregation Group): Two or more network links bundled together to function as a single link. LAGs provide increased bandwidth, redundancy and load balancing. MSTP: (Multiple Spanning-Tree Protocol): A modified version of the rapid spanning tree protocol that carries multiple spanning tree instances within its rapid spanning-tree protocol packet. Spanning-tree instance 0 is assigned to the common rapid spanning-tree instance and additional instances above 0 may be configured. For each spanning-tree instance, a root switch is elected and unique active and backup links can be chosen providing potentially unique traffic paths on the network per instance. Each VLAN can then be assigned to a spanning-tree instance allowing active traffic on separate VLANs to potentially utilize separate paths across the network. In order for interconnected switches to participate together in more than the common spanning-tree instance, they must have the same MSTP configuration. MSTP was originally defined in the IEEE 802.1s standard and is included in IEEE 802.1Q standard. NIC Teaming: NIC Teaming allows multiple network interfaces on a computer to be placed into a team to combine bandwidth or provide traffic failover to prevent connectivity loss in the event of a network component failure. Out-of-Band: An out-of-band interface provides management connectivity to a device without participating in or relying on a device’s in-band (normal-use) data interfaces. Port Channel: See LAG. PVST+ (Per-VLAN Spanning-Tree Plus): A vendor-specific implementation of the spanning tree protocol that maintains separate instances for each VLAN, passing that instance’s protocol frames within the VLAN it manages. RSTP (Rapid Spanning-Tree Protocol): A standards-based modified version of the basic spanning tree protocol that allows for much faster convergence times of spanning tree instances and provides for special administratively-assigned port states that improve behavior in certain circumstances. RSTP is defined in IEEE 802.1w.
66
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
STP (Spanning Tree Protocol): A family of layer-2 management protocols used by Ethernet bridges or switches to establish a loop-free forwarding topology. Protocols include the original STP (IEEE 802.1d), RSTP, MSTP, and PVST+. Ethernet can easily forward traffic in endless loops if cabled or configured incorrectly. The spanning-tree protocols provide a standard way for an interconnected set of switches to only use links that will not cause traffic flows to loop. (Loops will rapidly saturate all available bandwidth). In a spanning tree, a single bridge is elected the root bridge—either by lowest assigned priority or by having the lowest MAC address. Once a root bridge is elected, every other bridge keeps its one link with the lowest path cost to the root bridge active. Links with redundant paths are then blocked by switches that don’t have the lowest path cost for that link. As a result, with spanning tree each non-root bridge effectively has only one active link between it and the root bridge and the topology of these unblocked links draws a tree to the root bridge. Note: VLT terminology is in section 3.3.1 and vPC terminology is in section 3.3.2.
67
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
D
Resetting the MXL or M IOA to factory defaults To restore the Dell Networking MXL or M IOA to its factory default settings, perform the following steps via a serial console connection: Dell#restore factory-defaults stack-unit 0 clear-all Proceed with factory settings? Confirm [yes/no]:yes The system reloads with factory default settings. When the system is done with the boot process, the following message is displayed: This device is in Bare Metal Provisioning (BMP) mode. BMP is attempting to download an image, configuration file or boot script using DHCP. To continue with the standard manual interactive mode, it is necessary to abort BMP. Press A to abort BMP now. Press C to continue with BMP. Press L to toggle BMP syslog and console messages. Press S to display the BMP status. [A/C/L/S]:A
(“A” is entered at the above prompt since we are not using the BMP feature.) % Warning: The bmp process will stop ... Dell>enable Dell#
The system is now ready for configuration from its default state.
68
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
E
MXL and M IOA Port Mapping Figure 19 shows the port mapping for the Dell Networking MXLs and M IOAs installed in the M1000e chassis. External port mapping, detailed in the figure below, is dependent on the FlexIO Expansion module(s) installed in each MXL or M IOA. Mapping of server-facing internal ports (numbered 1-32) is detailed in the attachment titled M1000e_Internal_Port_Mapping_1.1.pdf
Internal 10/1 Gb
Fixed QSFP Ports
Expansion Slot 0
Expansion Slot 1
Dell Networking MXL and Dell PowerEdge M I/O Aggregator – Port Mapping
MXL and M IOA port mapping for the M1000e blade enclosure
69
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
F
Modifying Spanning Tree Port Cost and Priority You can adjust two interface parameters (port cost and port priority) to increase or decrease the probability that a port becomes a forwarding port on Dell Networking switches.
Port cost — a value that is based on the interface type. The greater the port cost, the less likely the port is selected to be a forwarding port. Port priority — influences the likelihood that a port is selected to be a forwarding port in case that several ports have the same port cost.
The following table lists the default values for port cost by interface. Interface Type 1000-Mb/s Ethernet interface 10-Gigabit Ethernet interface 40-Gigabit Ethernet interface Port Channel with one 10-Gigabit Ethernet interface Port Channel with one 40-Gigabit Ethernet interface Port Channel with two 10-Gigabit Ethernet interfaces Port Channel with two 40-Gigabit Ethernet interfaces
Port Cost Default Value 20000 2000 1400 2000 1400 1800 600
To change the port cost or port priority of an interface, use the following commands. Change the port cost of an interface. In INTERFACE mode: PVST: spanning-tree pvst vlan vlan-id cost cost-value MSTP: spanning-tree msti msti-number cost cost-value The cost value range is from 1 to 2000000. Change the port priority of an interface. In INTERFACE mode:
PVST: spanning-tree pvst vlan vlan-id priority priority-value MSTP: spanning-tree msti msti-number priority priority-value
The priority value is from 0 to 240, in increments of 16. The default is 128. To view the current values for these interface parameters, use the show config command from INTERFACE mode.
70
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0
G
Support and Feedback Contacting Technical Support Support Contact Information
Web: http://Support.Dell.com/ Telephone: USA: 1-800-945-3355
Feedback for this document We encourage readers to provide feedback on the quality and usefulness of this deployment guide by sending an email to
[email protected]
About Dell EMC Dell EMC is a worldwide leader in data center and campus solutions, which includes the manufacturing and distribution of servers, network switches, storage devices, personal computers, and related hardware and software. For more information on these and other products, please visit the Dell EMC website at http://www.dell.com.
71
Deploying Dell Networking MXL and M IOA switches into a Cisco Nexus Network | version 2.0