Dell HPC Lustre Storage IEEL3.0 with OPA Owner's Manual
Summary
Discover high-speed data management with Dell's HPC Lustre Storage solution, optimized for demanding High Performance Computing workloads using Intel Omni-Path technology. This comprehensive white paper is essential reading for IT architects and system engineers planning large-scale storage deployments. The guide details the advanced architecture—including metadata/object servers and scalability methods—and provides deep dives into performance tuning, evaluation parameters, and best practices required to maximize throughput in mission-critical data environments.
Page 1 Text Content
Dell HPC Lustre Storage A Dell Technical White Paper
Quy Ta
Dell HPC Engineering Innovations Lab
September 2016 | Version 1.1
Page Summary Contents For Dell HPC Lustre Storage IEEL3.0 with OPA Owner's Manual
Page 1
Dell HPC Lustre Storage A Dell Technical White Paper Quy Ta Dell HPC Engineering Innovations Lab September 2016 | Version 1.1
Page 2
Dell HPC Lustre Storage solution with Intel Omni-Path THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS I...
Page 3
Dell HPC Lustre Storage solution with Intel Omni-Path Contents Figures ...................................................................................................................... iv Tables ...
Page 4
Dell HPC Lustre Storage solution with Intel Omni-Path Figures Figure 1: Lustre based storage solution components ............................................................... 2 Figure 2: Dell HPC Lu...
Page 5
Dell HPC Lustre Storage solution with Intel Omni-Path Tables Table 1: Test Client Cluster Details .................................................................................. 12 Table 2: Dell HP...
Page 6
Dell HPC Lustre Storage solution with Intel Omni-Path 1. Introduction In high performance computing (HPC), the efficient delivery of data to and from the compute nodes is critical and often complicate...
Page 7
Dell HPC Lustre Storage solution with Intel Omni-Path pairs. Each additional OSS increases the existing networking throughput, while each additional OST increases the storage capacity. Figure 1 shows ...
Page 8
Dell HPC Lustre Storage solution with Intel Omni-Path Metadata Storage Server (MDS) – Manages the MDT, providing Lustre clients access to files. Object Storage Target (OST) – Stores the data strip...
Page 9
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 2: Dell HPC Lustre Storage Solution Components Overview There are several new architectural changes in this release compared to previous re...
Page 10
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 3: Dell Power Edge R730 i DRAC 3.1 Management Server The Intel Manager Server is a single server connected to the Metadata servers and Obje...
Page 11
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 4: Metadata Server Pair i DRAC i DRAC MD3420 #1 SERVER SAS PCI SLOT SAS PORT MD3420 MD3420 MD3420 ARRAY CONTROLLER CONTROLLER ieel3-mds1 Sl...
Page 12
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 5: Metadata Server Pair with Lustre DNE option ieel-mds1 ieel-mds2 i DRAC i DRAC MD3420 #1 MD3420 #2 SERVER SAS PCI SLOT SAS PORT MD3420 MD...
Page 13
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 6: Object Storage Server Pair ieel-oss1 ieel-oss2 i DRAC i DRAC MD3460 #1 MD3460 #2 MD3460 #3 MD3460 #4 SERVER SAS PCI SLOT SAS PORT MD3460...
Page 14
Dell HPC Lustre Storage solution with Intel Omni-Path Targets per enclosure. By using RAID 6, the solution provides higher reliability at a marginal cost on write performance (due to the extra set of ...
Page 15
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 8: OSS Scalability Management Metadata Object Storage Object Storage Target (MGT) Target (MDT) Targets (OSTs) Targets (OSTs) Management Met...
Page 16
Dell HPC Lustre Storage solution with Intel Omni-Path Path HFI interface using IPo IB (i.e. ifcfg-ib0) as well as your 10Gb E Ethernet interface (i.e. eth0) on your OSS servers to both participate in ...
Page 17
Dell HPC Lustre Storage solution with Intel Omni-Path 4. Performance Evaluation and Analysis The performance studies presented in this paper profile the capabilities of the Dell HPC Lustre Storage wit...
Page 18
Dell HPC Lustre Storage solution with Intel Omni-Path number of threads above 64 were simulated by increasing the number of threads per client across all clients. For instance, for 128 threads, each o...
Page 19
Dell HPC Lustre Storage solution with Intel Omni-Path sync echo 3 > /proc/sys/vm/drop_caches Th ro gh in /s In addition, to simulate a cold cache on the server, a “sync” was performed on all th...
Page 20
Dell HPC Lustre Storage solution with Intel Omni-Path We found that single client performance were consistent at 1GB/s to 1.3GB/s for writes and reads respectively. The write and read performance rise...
Page 21
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 12: N-to-N Random reads and writes Iozone Random - Dell HPC Lustre Storage with Intel Omni-Path Number of concurrent threads Read Write 4.3...
Page 22
Dell HPC Lustre Storage solution with Intel Omni-Path Also during the preliminary metadata testing, we concluded that the number of files per directory significantly affects the results, even while ke...
Page 23
Dell HPC Lustre Storage solution with Intel Omni-Path Figure 13: File Metadata Operations MDtest Files Metadata - Dell HPC Lustre Storage with Intel Omni-Path Threads File Create File Stat File Remove...
Page 24
Dell HPC Lustre Storage solution with Intel Omni-Path The max_pages_per_rpc parameter is a tunable that sets the maximum number of pages that will undergo I/O in a single RPC to that OST. [root@node05...
Page 25
Dell HPC Lustre Storage solution with Intel Omni-Path 5.3 Misc. tunings NOTE: Setup of the Intel Omni-Path HFI interconnect will utilize IPo IB and the ifcfg-ib0 configuration file. Consult the Dell H...
Page 26
Dell HPC Lustre Storage solution with Intel Omni-Path IOzone Sequential Reads - iozone -i 1 -c -e -w -r 1024K -I -s $Size -t $Thread -+n -+m /root/list.$Thread IOzone IOPS Random Reads / Writes – iozo...
Page 27
Dell HPC Lustre Storage solution with Intel Omni-Path Dell Power Vault MD3460 http://www.dell.com/support/home/us/en/04/product-support/product/powervault- md3460/research Lustre Home Pages http://www...
Manual Details
| Brand | Dell |
|---|---|
| Pages | 27 |
| File Size | 1.03 MB |
| Published | June 23, 2026 |
10 views
Enter the captcha to get the download link:
Frequently Asked Questions
What are the core component types in this HPC Lustre storage solution?
The architecture consists of Management Servers, Metadata Servers, and Object Storage Servers.
How can I test metadata performance using the provided tools?
Metadata tests require using the MDtest command with specific arguments like -d (directory) or -i (iterations).
Does the solution use dedicated networking for different purposes?
Yes, it utilizes separate connections designated as the Management Network and the Data Network.
What is stored on the storage arrays?
RAID6 layouts are used on component arrays, such as those shown in the MD3460 figures.