Summary of the content on the page No. 1
Third party information brought
to you courtesy of Dell.
SwitchX 56Gb/s InfiniBand Blade Switch I/O
Module for PowerEdge M-Series User Manual
PN: M4001F, M4001Q
Rev 1.0
www.mellanox.com
Summary of the content on the page No. 2
Rev 1.0 NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE PRO- VIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUS- TOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES
Summary of the content on the page No. 3
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 Contents Contents 3 Revision History 4 About this Manual 5 Chapter 1 Overview 7 1.1 InfiniBand Connectors 8 1.2 Switch Status Lights 9 1.3 I2C Interface RJ-45 Connector 10 Chapter 2 Installation and Basic Operation 11 2.1 Installation Safety Warnings 11 2.2 Mechanical Installation 13 2.2.1 Removing an Old Switch From the Chassis 13 2.2.2 Removing an I/O Module Blank From the Chassis 14 2.2.3 Installin
Summary of the content on the page No. 4
Rev 1.0 Revision History Table 1 - Revision History Table Date Revision Description November 2011 1.0 Initial release 4 Mellanox Technologies
Summary of the content on the page No. 5
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 About this Manual This manual describes the installation and basic use of the SwitchX M4001F 56 Gb/s, and the M4001Q 40 Gb/s InfiniBand blade switch I/O modules for the PowerEdge M-Series chassis. Intended Audience This manual is intended for users and system administrators responsible for installing and setting up the switch platforms listed above. The manual assumes familiarity with the InfiniBand® Architecture Specification.
Summary of the content on the page No. 6
Rev 1.0 This symbol indicates information that is helpful to the user. This symbol indicates a situation that can potentially cause damage to hardware or software. BEWARE! This symbol indicates a situation that can potentially cause personal injury or damage to hardware or software. 6 Mellanox Technologies
Summary of the content on the page No. 7
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 1Overview The SwitchX M4001F 56 Gb/s, and the M4001Q 40 Gb/s InfiniBand blade switch I/O module for the PowerEdge M-Series chassis provides a high bandwidth, low latency fabric for Enterprise Data Centers (EDC), High-Performance Computing (HPC), and Embedded environments. Based on the th st 5 generation InfiniBand switch device (1 Generation SwitchX). This I/O module (IOM) deliv- ers up to 56Gb/s at FDR speed and 40 Gb/s at QDR
Summary of the content on the page No. 8
Rev 1.0 Overview Figure 1: Switch Front Panel and Locking Mechanism Locking Mechanism Port LEDs Switch Status Lights 1.1 InfiniBand Connectors This switch has 16 QSFP connectors through the front panel. The remaining 16 interfaces are through the Midplane Connector out of the back of the switch. Figure 1 shows the front 16 ports. Each of the InfiniBand ports has two LEDs located between the connectors and the I2C interface. The green LED, when lit, indicates that a valid physical connection t
Summary of the content on the page No. 9
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 Figure 2: Physical and Logical Link Indication LEDs LED Name Connection Status Physical Link - Green Off – No Physical Link ON – Physical Link Blinking – indicates Data Transfer Constant on – indicates Link exists Data Activity - Yellow with no Data Transfer taking place Off with green LED lit – indicates that the Subnet Manager may not be running 1.2 Switch Status Lights The switch Status lights indicate whether the switch is
Summary of the content on the page No. 10
Rev 1.0 Overview LED Switch Status Indication Green Status ON BLINKING BLUE The CMC is identifying the newly installed switch Blinking Blue ON ON BLUE Switch is on and operating Normally ON ON or BLINKING Fault in System AMBER Self-diagnosed OFF ON or BLINKING Fault in System AMBER CMC-detected 1.3 I2C Interface RJ-45 Connector There is an “I2C”interface on the front panel. Figure 4: I2C RJ-45 Interface The I2C connection provides access to Flash and EEPROMs. The I2C conn
Summary of the content on the page No. 11
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 2 Installation and Basic Operation Installation and initialization of the I/O module are straightforward processes, requiring attention to the normal mechanical, power, and thermal precautions for rack-mounted equipment. The I/O module does not require any programming or configuration to operate as a basic InfiniBand switch and includes all of the necessary functionality to operate with external standard InfiniBand Subnet Mana
Summary of the content on the page No. 12
Rev 1.0 Installation and Basic Operation 5. Copper InfiniBand Cable Connecting/Disconnecting Copper InfiniBand cables are heavy and not flexible, as such they should be carefully attached to or detached from the connectors. Support the cable’s weight using the rack and a cable management system. Refer to the cable manufacturer for special warnings and instructions. 6. Rack Mounting and Servicing When this product is mounted or serviced in a rack, special precautions must be taken to ensure t
Summary of the content on the page No. 13
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 2.2 Mechanical Installation These switches are hot pluggable. It is not necessary to power down the Dell Chassis to install a new switch or to replace an old switch with a new one. Figure 5: Rear View of the Dell PowerEdge M1000e Chassis Latch Release Buttons Locking Arms A1 B1 C1 C2 B2 A2 Channel 1 slots Channel 2 slots M4001[F, Q] blades are not allowed in the Fabric A slots. The switch can only be placed in channel slots
Summary of the content on the page No. 14
Rev 1.0 Installation and Basic Operation 2.2.2 Removing an I/O Module Blank From the Chassis 1. Unlock the I/O Module Blank by pushing the red latch release button. 2. Pull the locking arm down to a position perpendicular to the front of the chassis. 3. Pull the I/O Module Blank out of the chassis using the locking arm. 2.2.3 Installing the Mellanox M4001[F, Q] Switch Into the Dell Chassis Make sure the rack is stable on a solid floor and that the rack is filled from the bottom up. This will k
Summary of the content on the page No. 15
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 To remove, disengage the locks and slowly pull the connector away from the port receptacle. Both LED indicators will turn off when the cable is unseated. Care should be taken not to impede the air exhaust flow through the chassis. Cable lengths should be used which allow for routing horizontally around to the side of the chassis before bending upward or downward in the rack. Mellanox Technologies 15
Summary of the content on the page No. 16
Rev 1.0 Cluster Management and Firmware 3 Cluster Management and Firmware 3.1 Network Management and Clustering Software Download and install, on all nodes, the Mellanox OpenFabric software package for Linux, Windows, or other operating systems from the Mellanox software website: http://www.mellanox.com >Downloads > InfiniBand SW/Drivers. This software package provides connectivity for server and storage systems utilizing High Perfor- mance Computing (HPC) or enterprise data center (EDC) appli
Summary of the content on the page No. 17
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 4 Troubleshooting As soon as a switch is inserted in make sure that the power LED comes on. The power LED for the switch does not come on: 1. Check that the chassis has power. 2. Remove and reinstall the switch. The status LED for the switch is blinking amber: 1. Remove the switch from the chassis and re-insert it (verify that the switch is all the way in the chassis and the lever is firmly closed). 2. If the amber LED continu
Summary of the content on the page No. 18
Rev 1.0 Appendix A: Specifications Table 4 - M4001[F, Q] Specification Data Physical Power and Environmental H x W x D: 12.2 x 1.34 x 10.7 inches Typ Power: 310.7 x 34.1 x 271.8 mm FDR Weight: 2.6Kg fully configured Passive: 80.66W or 275.22BTUs/hr Mounting: Vertically mounted rack Active: 108.26W or 369.40BTUs/hr SerDes Speeds 10, 20, 40, 56 Gb/s per port QDR 60.42W or 206.16BTUs/hr Passive: Connectors: QSFP Active: 88.02W or 300.34BTUs/hr 16 external QSFP connectors 16 internal backplane
Summary of the content on the page No. 19
InfiniBand Blade Switch I/O Module for PowerEdge M-Series Rev 1.0 Physical Power and Environmental Scalability and Performance Switching Performance: Simultaneous wire-speed any port to any port 48K Unicast Addresses Max. per Subnet Addressing: 16K Multicast Addresses per Subnet 2.56 Tb/s Switching Capacity Mellanox Technologies 19
Summary of the content on the page No. 20
Rev 1.0 Appendix B: EMC Certification Statements 20 Mellanox Technologies
