Dell HPC Omni-Path Fabric-Supported Architecture and Application Study
Summary
Technical white paper from Dell HPC Engineering (June 2016, authors Deepthi Cherlopalle and Joshua Weage) examining the Dell HPC Omni-Path Fabric architecture and application performance. Analyzes Intel Omni-Path Host Fabric Interface (HFI) with server support matrix, Dell Networking H-Series Fabric components, and performance benchmarks comparing Omni-Path to InfiniBand interconnects across HPC applications. Provides architecture design guidance for high-performance computing cluster deployment
Page 1 Text Content
Dell HPC Omni-Path Fabric: Supported Architecture and Application Study Deepthi Cherlopalle Joshua Weage Dell HPC Engineering June 2016
Page Summary Contents For Dell HPC Omni-Path Fabric-Supported Architecture and Application Study
Page 1
Dell HPC Omni-Path Fabric: Supported Architecture and Application Study Deepthi Cherlopalle Joshua Weage Dell HPC Engineering June 2016
Page 2
Revisions Date Description June 2016 Initial release – v1 THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED ...
Page 3
Table of contents Revisions ...............................................................................................................................................................................
Page 4
List of Figures Figure 1 Intel® Omni-Path Host Fabric Interface Adapter 100 series 1 port PCIe x16 ........................................................9 Figure 2 Dell H-Series Switches ..............
Page 5
Executive Summary In the world of High Performance Computing (HPC), servers with high-speed interconnects play a key role in the pursuit to achieve exascale performance. Intel’s Omni-Path Architecture...
Page 6
6 Dell HPC Omni-Path Fabric: Supported Architecture and Application Study June 2016
Page 7
1 Introduction The High Performance Computing (HPC) domain primarily deals with problems which surpass the capabilities of a standalone machine. With the advent of parallel programming, applications c...
Page 8
Sending the data Programmed I/O (PIO): This supports the on-load model. The host can be used to send small messages since these can be sent by the CPU faster than an RDMA setup time. Send DMA (SDM...
Page 9
2 Dell Networking H-Series Fabric Dell Networking H-Series Fabric is a comprehensive fabric solution that includes host adapters, edge and director class switches, cabling and complete software and ma...
Page 10
Figure 2 Dell H-Series Switches 2.2.1 Dell H-Series Edge Switches Dell H-Series Edge Switches based on the Intel® Omni-Path Architecture consist of two models supporting 100Gbps for all ports: an entr...
Page 11
2.2.2 Dell H-Series Director-Class Switches Dell H-Series Director-Class Switches based on the Intel® Omni-Path Architecture consist of two models supporting 100Gbps for all ports: a 192-port switch a...
Page 12
3 Intel® Omni-Path Fabric Software 3.1 Available Installation Packages The following packages [4] are available for an Intel® Omni-Path Fabric: Intel® Omni-Path Fabric Host Software – This is the ba...
Page 13
3.2 Intel® Omni-Path Fabric Manager GUI Fabric Manager GUI [5] provides a set of analysis tools for graphically monitoring fabric status and managing fabric health. This package is open source and can...
Page 14
3.3 Chassis Viewer Chassis viewer is a web interface which can be used to manage basic functionalities on edge switches with and without management cards. The following picture exemplifies the basic l...
Page 15
3.4 OPA Fast Fabric Fast Fabric [6] is a set of fabric management tools used for fabric deployments, switch management and host management. This can be accessed if Intel Fabric Suite is installed on t...
Page 16
Sweeping the fabric to discover topology changes, managing those changes when nodes are added or deleted, etc. Adaptive routing Congestion Control Subnet Administration Subnet Administrator (SA)...
Page 17
Figure 9 Starting Embedded Subnet Manager Controlling Subnet Manager using Switch CLI The default login credentials for the switch is admin/adminpass. From the switch CLI, use the following commands t...
Page 18
4 Test bed and configuration This section explains the server configuration, BIOS options and application versions used for the application performance study. This study utilized the Zenith cluster lo...
Page 19
Application Version MPI Benchmark osu_latency OSU 4.4.1 Open MPI-hfi-1.10 osu_bw osu_bibw Apoa1 NAMD 2.11 Intel MPI 5.1.3 F1atpase WRF 3.8 Intel MPI 5.1.3 Conus 2.5km eddy_417k pump_2m aircraft_wing_2...
Page 20
5 Performance Benchmarking Results 5.1 Latency Ti e( s) OSU Micro-benchmarks were used to determine latency. These latency tests were done in Ping-Pong fashion. HPC applications need low latency and h...
Page 21
OSU Bandwidth an id th (G /s el at iv to Message Size(bytes) Bi-directional Uni-directional Figure 11 OSU Bandwidth values based on Intel® Xeon® CPU E5-2697 v4 processor. 5.3 Weather Research Forecast...
Page 22
5.4 NAMD NAMD [9] is a parallel molecular dynamics code designed for high-performance simulation of large bio- molecular systems. Three proteins Apo A1 (92,224 atoms), F1ATpase (327,506 atoms) and STM...
Page 23
ANSYS® Fluent® Relative Performance er fo rm an ce el at iv to re (1 e) er fo rm an ce el at iv to re (1 e) Number of Cores (nodes) aircraft_wing_14m combustor_12m fluidized_bed_2m landing_gear_15m lm...
Page 24
5.6 CD-adapco® STAR-CCM+® At the time of publication of this whitepaper, Intel Omni-Path was not officially supported by CD-adapco® STAR-CCM+®. In order to obtain preliminary performance data for this...
Page 25
STAR-CCM+® Relative Performance er fo rm an ce el at iv to re (1 e) Number of Cores (nodes) Eglin Store Separation Kcs With Physics Turbo Charger Figure 17 STAR-CCM+® Relative Performance (2/2) 25 Del...
Page 26
6 Conclusion and Future Work The Intel® Omni-Path Architecture is a new option available for low-latency, high-bandwidth cluster fabrics. The micro benchmark results prove that OPA is an ideal candida...
Page 27
7 References [1] [Online]. Available: http://www.intel.com/content/dam/www/public/us/en/documents/product- briefs/transforming-economics-hpc-fabrics-opa-brief.pdf. [2] [Online]. Available: http://www....
Manual Details
| Brand | Dell |
|---|---|
| Pages | 27 |
| File Size | 1.43 MB |
| Published | June 17, 2026 |
8 views
Enter the captcha to get the download link:
Frequently Asked Questions
What tools are available for monitoring and managing the Omni-Path fabric?
Users can utilize freely available fabric monitoring and managing tools such as Fabric Manager GUI, Chassis Viewer, and opafastfabric.
How is the OPA FastFabric configured?
Setup procedures cover distinct steps for both the chassis, the switches, and the hosts.
What processors are used for performance evaluation in this study?
The evaluation uses Intel Omni-Path and Intel Broadwell Processors combined with Dell Servers.
What advanced fabric features might be evaluated later?
Future plans include evaluating dynamic lane scaling, packet integrity protection, and traffic flow optimization.