Dell HPC Lustre Storage IEEL3.0 with EDR Documentation
Summary
Dell HPC Lustre Storage is a high-performance computing storage solution leveraging the Lustre file system with Mellanox InfiniBand EDR networking. This technical white paper covers Lustre file system architecture, management server configuration, metadata servers, object storage servers, scalability options, networking infrastructure, and Intel Enterprise Edition for Lustre integration. Target users include HPC system architects, storage administrators, and technical professionals designing high-performance storage solutions for compute-intensive environments.
Page 1 Text Content
Dell HPC Lustre Storage A Dell Technical White Paper
Quy Ta
Dell HPC Engineering Innovations Lab
September 2016 | Version 1.0
Page Summary Contents For Dell HPC Lustre Storage IEEL3.0 with EDR Documentation
Page 1
Dell HPC Lustre Storage A Dell Technical White Paper Quy Ta Dell HPC Engineering Innovations Lab September 2016 | Version 1.0
Page 2
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVI...
Page 3
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Contents 1. Introduction ........................................................................................................... 1 2. ...
Page 4
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR 1. Introduction In high performance computing (HPC), the efficient delivery of data to and from the compute nodes is critical and often co...
Page 5
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR pairs. Each additional OSS increases the existing networking throughput, while each additional OST increases the storage capacity. Figure ...
Page 6
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Object Storage Target (OST) – Stores the data stripes or extents of the files on a file system. Object Storage Server (OSS) – Manages ...
Page 7
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 2: Dell HPC Lustre Storage Solution Components Overview There are several new architectural changes in this release compared to pre...
Page 8
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 3: Dell Power Edge R730 i DRAC 3.1 Management Server The Intel Manager Server is a single server connected to the Metadata servers ...
Page 9
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 4: Metadata Server Pair i DRAC i DRAC MD3420 #1 SERVER SAS PCI SLOT SAS PORT MD3420 MD3420 MD3420 ARRAY CONTROLLER CONTROLLER ieel3...
Page 10
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 5: Metadata Server Pair with Lustre DNE option ieel-mds1 ieel-mds2 i DRAC i DRAC MD3420 #1 MD3420 #2 SERVER SAS PCI SLOT SAS PORT M...
Page 11
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 6: Object Storage Server Pair ieel-oss1 ieel-oss2 i DRAC i DRAC MD3460 #1 MD3460 #2 MD3460 #3 MD3460 #4 SERVER SAS PCI SLOT SAS POR...
Page 12
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Targets per enclosure. By using RAID 6, the solution provides higher reliability at a marginal cost on write performance (due to the extra...
Page 13
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 8: OSS Scalability Management Metadata Object Storage Object Storage Target (MGT) Target (MDT) Targets (OSTs) Targets (OSTs) Manage...
Page 14
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR interface using IPo IB (i.e. ifcfg-ib0) as well as your 10Gb E Ethernet interface (i.e. eth0) on your OSS servers to both participate in t...
Page 15
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR 4. Performance Evaluation and Analysis The performance studies presented in this paper profile the capabilities of the Dell HPC Lustre Sto...
Page 16
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR The test environment for the solution has a single MDS pair and a single OSS pair with a total of 960TB of raw disk space. The OSS pair co...
Page 17
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR echo 3 > /proc/sys/vm/drop_caches In addition, to simulate a cold cache on the server, a “sync” was performed on all the active serve...
Page 18
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR respectively. The write and read performance rise sharply as we increase the number of process threads up to 24 where we see level out as ...
Page 19
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 11: N-to-N Random reads and writes Iozone Random - Dell HPC Lustre Storage with Infiniband EDR Number of concurrent threads Read Wr...
Page 20
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Also during the preliminary metadata testing, we concluded that the number of files per directory significantly affects the results, even ...
Page 21
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR Figure 12: File Metadata Operations MDtest Files Metadata - Dell HPC Lustre Storage with Infiniband EDR Threads File Create File Stat File...
Page 22
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR The max_pages_per_rpc parameter is a tunable that sets the maximum number of pages that will undergo I/O in a single RPC to that OST. [roo...
Page 23
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR The continued use of generally available, industry-standard benchmark tools like IOzone and MDtest provide an easy way to match current an...
Page 24
Dell HPC Lustre Storage solution with Mellanox Infiniband EDR -I number of items per directory in tree -y sync file after writing -u unique working directory for each task -t time unique working direc...
Manual Details
| Brand | Dell |
|---|---|
| Pages | 24 |
| File Size | 1.01 MB |
| Published | June 24, 2026 |
9 views
Enter the captcha to get the download link:
Frequently Asked Questions
What is the Dell HPC Lustre Storage solution?
It is a fully-supported, easy-to-use, high-throughput, scale-out and cost-effective parallel file system storage solution using Intel Enterprise Edition for Lustre software v.3.0.
What are the main components of the solution?
The solution consists of a Management Server, Metadata Servers, Object Storage Servers, and a Data Network utilizing Mellanox Infiniband EDR.
How does the management interface work?
The solution utilizes an intelligent, extensive, and intuitive management interface that allows users to bypass the cache on compute nodes where IO threads are running.
What are the key performance testing commands?
Key commands include N-to-N Sequential Reads/Writes, Random Reads and Writes, and Metadata testing using tools like IOzone and MDtest.