Summary of the content on the page No. 1
Cisco SFS 7008 InfiniBand Server Switch
Hardware Guide (Topspin 270 Hardware Guide)
Release 2.2.0 Update 1
Part Number: OL-7725-01
Product ID: 140265
Summary of the content on the page No. 2
Copyright © 2004 - 2005 Topspin Communications, Inc. All rights reserved. The Topspin Switched Computing System, Topspin Host Channel Adapter, Topspin Element Manager, and collateral software programs and documentation are subject to and made available only pursuant to the license agreement signed by you and Topspin, Communications, Inc. (or if no signed license agreement exists, the license agreement included with your original media) and may only be used in accordance with the terms of that
Summary of the content on the page No. 3
iii Regulatory Notices........................................ vii FCC Statement.......................................................................................................................................... vii Safety Information .................................................................................................................................... vii Electrical Cautions......................................................................................................
Summary of the content on the page No. 4
iv Connect Management Devices ..................................................................................................................26 Attach a Serial Console Cable to a PC or Terminal ..................................................................... 26 Connect an Ethernet Cable to the Ethernet Management Port ..................................................... 27 Recommended Management Configuration ..............................................................................
Summary of the content on the page No. 5
v Upgrading a Topspin 270 with Element Manager........................................................................ 54 Upgrading an Individual Controller Card..................................................................................................56 Upgrading a Controller Card with the CLI................................................................................... 57 Upgrading Individual Node Cards................................................................................
Summary of the content on the page No. 6
vi 7: Specifications and Compliance Certifications 103 Chassis and Management Interface ............................................................................................ 103 Electrical Specifications ............................................................................................................. 104 EMC/Immunity........................................................................................................................... 104 Safety ...........................
Summary of the content on the page No. 7
vii Regulatory Notices FCC Statement This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference
Summary of the content on the page No. 8
viii CAUTION: Grounding is supplied by the ground-prong on the 3-prong power cable. Do not attach a separate ground cable. Do not use adapter plugs. Do not remove the ground prong from the cable. Ensure the ground connection on the power supply is correct and functioning before applying power to the Topspin system. CAUTION: Always ground yourself before touching any internal Topspin system component to avoid damage from electrostatic discharge (ESD). CAUTION: The Topspin system has two pow
Summary of the content on the page No. 9
ix CAUTION: Never place your hand inside an empty card or module bay. You should never have cause to place a hand anywhere inside the Topspin chassis. Unused card and module bays should always have a Topspin cover over the bay to ensure proper safety, ventilation, and cooling. General Cautions CAUTION: No user is authorized to remove the Topspin system enclosure cover. The internal chassis contains no user-serviceable components. Removing the enclosure cover voids your warranty. See the war
Summary of the content on the page No. 10
x
Summary of the content on the page No. 11
11 1 Features Overview The Topspin 270 Switched Computing System provides data center managers with a high-performance, low-latency interconnect. • “System Architecture” on page 11. • “About the Topspin 270 Chassis” on page 13. • “Administrative Features” on page 20. System Architecture Hot Standby On the Topspin 270, the inactive controller card remains on hot standby so that it can assume control in the event that the active controller card fails over. The hot standby feature lets the Serve
Summary of the content on the page No. 12
12 Example 1. Topspin-270# show card 2. 3. 4. ========================================================================= 5. Card Information 6. ========================================================================= 7. admin oper admin oper oper 8. slot type type status status code 9. ------------------------------------------------------------------------- 10
Summary of the content on the page No. 13
13 Fabric For redundancy, InfiniBand Host Channel Adapters (HCAs) can be dual-connected to a redundant pair of Topspin 270s. In an InfiniBand fabric that includes more than one Topspin 270: if the subnet manager on the Topspin 270 that is acting as the master fails, another subnet manager will take over within seconds. All necessary state information is kept in sync. Non-Blocking Architecture The Topspin 270 provides non-blocking switch element architecture with full bi-directional bandwidth
Summary of the content on the page No. 14
14 Figure 1-2: Back Chassis View
Summary of the content on the page No. 15
15 Chassis Slot Numbering Slot numbers are printed on the chassis for easy identification. Power Supply ID 1 Power Supply ID 2 Fan ID 1 Fan ID 2 Slot # 9 Slot # 10 Slot # 11 Slot # 12 Slot # 13 Slot # 14 Figure 1-3: Chassis Slot Numbering in Front (Non-port Side)
Summary of the content on the page No. 16
16 Slot # 15 Slot # 16 Slot # 17 Slot # 1 Slot # 2 Slot # 3 Slot # 4 Slot # 5 Slot # 6 Slot # 7 Slot # 8 Figure 1-4: Chassis Slot Numbering in Back (Service Side) Dimensions • Height: 6U unit • Width: 17.04” • Depth: 8” - 23.7” • Weight: maximum 106 pounds (full chassis configuration with 4x copper Line Interface Modules) Connections • 96 ports of 10 Gbps 4X Copper InfiniBand. • Two 10/100 Ethernet RJ-45 Management-Ethernet ports for out-of-band management. • Two RJ-45 Console Port used to conf
Summary of the content on the page No. 17
17 • The power supplies are 1U, 48V, 1200W • The supplies provide regulated +48V DC to all other modules in the system. • Each power supply has self contained fans for cooling. Fan Trays Refer to “Installing a Fan Module” on page 41 for installation instructions. The fan trays are redundant, hot-swappable cooling units. • Only one fan tray, in either of the two slots, is required to cool the system. However, a blanking panel is required for any unused bay. • The replacement of any one fan
Summary of the content on the page No. 18
18 Understanding the Core Slot to Mgmt Interface Module Connection Each Core slot in the chassis (refer to “The Core Slot” on page 18) is connected to 1 port on the Management Interface module. The controller (fabric interface module inserted in the core slot) selects the Management Interface module to which it will connect. The pairing of Management Interface module to core slot is as follows: • The Management Interface module in slot #15 is connected to the Fabric Controller Module in the
Summary of the content on the page No. 19
19 • At least one Fabric Controller module must be installed in the Core slot for the chassis to boot. • When a module is installed in a Core slot, it acts as either the master or standby controller for the chassis, with all modules in the system being slaves to the master controller. • The subnet manager runs on the fabric module installed in the Core slot that has arbitrated to be the master. • The Core slot module also acts as the switch fabric spine, interconnecting the end-nodes. Each
Summary of the content on the page No. 20
20 Administrative Features Real-Time Clock The Topspin 270 maintains correct time regardless of power conditions or connectivity. Latency The Topspin 270 has port to port latency of less than 600ns. Non-Volatile Memory The memory supports up to: • three stored system images (not including recovery image) • one week of log files at normal verbosity; and one day of log files at maximum verbosity Vital Product Data Storage Vital Product Data (VPD) is stored in non-volatile memory in the Chassis ID
