Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS - User's Guide and Manual
Summary
This Dell technical whitepaper (Version 1.0, March 2013) explains how to improve Network File System I/O performance in High Performance Computing clusters using Dell Fluid Cache for Direct Attached Storage. Author Garima Kochhar from Dell HPC Engineering discusses the architecture of Fluid Cache for DAS, integration methods with NFS exports, benchmark results demonstrating I/O performance gains, and deployment recommendations for HPC environments. The paper targets HPC system architects and adm
Page 1 Text Content
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS This Dell technical white paper explains how to improve Network File System I/O performance by using Dell Fluid Cache for Direct Attached Storage in a High Performance Computing Cluster.
Garima Kochhar
Dell HPC Engineering
March 2013, Version 1.0
Page Summary Contents For Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS - User's Guide and Manual
Page 1
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS This Dell technical white paper explains how to improve Network File System I/O performance by using Dell Fluid Cache for Direct...
Page 2
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS This document is for informational purposes only and may contain typographical errors and technical inaccuracies. The content is...
Page 3
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Contents Executive Summary ........................................................................................................
Page 4
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Tables Table 1. NFS server and storage hardware configuration ...................................................... 8 Table 2. ...
Page 5
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Executive Summary Most High Performance Computing clusters use some form of a Network File System (NFS) based storage solution f...
Page 6
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS 1. Introduction A Network File System (NFS) based storage solution is a popular choice for High Performance Computing Clusters (...
Page 7
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS sections provide details on each of these components as well as information on tuning and monitoring the solution. 2.1. NFS stor...
Page 8
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS NFS server Figure 2. NFS server and storage hardware configuration Table 1. Server configuration NFS SERVER Power Edge R720 PROC...
Page 9
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS NFS server software and firmware configuration Table 2. Software OPERATING SYSTEM Red Hat Enterprise Linux (RHEL) 6.3z KERNEL VE...
Page 10
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Hardware configuration for DFC Table 3. Server configuration NFS SERVER Power Edge R720 CACHE POOL Two 350GB Dell Power Edge Exp...
Page 11
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS I/O cluster details Table 5. I/O cluster configuration CLIENTS 64 Power Edge M420 blade servers 32 blades in each of two Power E...
Page 12
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS OFED Mellanox OFED 1.5.3-3.0.0 Test bed Figure 3. 2.4. Solution tuning The NFS server and the attached storage arrays are config...
Page 13
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS 2.4.1. Storage • 3TB NL SAS disks are selected for large capacity at a cost-effective price point. • Virtual disks are created u...
Page 14
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS • NFSv3 is recommended over NFSv4 based on the performance results of a previous study.4 It was found that metadata create opera...
Page 15
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS The warranty of the device is expressed in number of years and number of Petabytes written (PBW). For the recommended 350GB SSD ...
Page 16
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS 2.5.3. Dell Fluid Cache for DAS health and monitoring DFC provides a very simple command-line utility /opt/dell/fluidcache/bin/f...
Page 17
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS • DFC in Write-Back mode (DFC-WB) – This configuration builds on the baseline by adding DFC as described in Section 2.2, and DFC...
Page 18
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Large sequential write performance Figure 5. ro in i B /s ro in i B /s Sequential writes Number of concurrent clients baseline D...
Page 19
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS better than the baseline since the data is already in the DFC cache. As expected on read operations, WB and WT tests have simila...
Page 20
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Random read performance Figure 8. Random reads Number of concurrent clients baseline DFC-WB DFC-WT 3.3. Metadata tests This sect...
Page 21
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Metadata file create performance Figure 9. er st at er ec File create er cr ea te er ec Number of concurrent clients baseline DF...
Page 22
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Metadata file remove performance Figure 11. er re ve er ec File remove Number of concurrent clients baseline DFC-WB DFC-WT 3.4. ...
Page 23
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Figure 12 shows that on a cold-cache read for sequential tests, the throughput of the DFC configuration drops from a peak of ~3,...
Page 24
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Figure 13 shows that on a cold-cache read for the random tests, peak IOPS of the DFC configurations drop from ~123,000 IOPs to ~...
Page 25
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS 4. Conclusion This Dell technical white paper describes a method to improve NFS performance using Dell Fluid Cache for DAS in an...
Page 26
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS http://en.community.dell.com/techcenter/systems-management/w/wiki/1760.openmanage- server-administrator-omsa.aspx 8. Dell Power ...
Page 27
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Appendix A: Step-by-step configuration of Dell Fluid Cache for NFS This appendix provides detailed step-by-step instructions on ...
Page 28
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Solution cabling Figure 15. A.2. NFS server set up After the Power Edge R720 server is ready and cabled to the Power Vault MD120...
Page 29
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS 5. Install the Red Hat Scalable File System (XFS) packages that are part of RHEL 6.3 add-on. xfsprogs-3.1.1-7.el6.x86_64 and xfs...
Page 30
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS 2. Change the OS I/O scheduler to “deadline”. To the end of the kernel line in /etc/grub.conf for the .14.1 errata kernel, add e...
Page 31
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Partitions : Available Hot Spare Policy violated : Not Applicable Encrypted : No Layout : RAID-0 Size : 557.75 GB (598879502336 ...
Page 32
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Status : Ok Name : PERC H810 Adapter Slot ID : PCI Slot 7 State : Ready Firmware Version : 21.1.0-0007 Minimum Required Firmware...
Page 33
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Stripe Element Size : 512 KB Disk Cache Policy : Disabled A.5. XFS and DFC configuration In this final step of the configuration...
Page 34
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Example on the client: [root@compute-0-0 ~]# mount –o vers=3 <ib0-IP-of-NFS-server>:/home/xfs <mount-point-on-client>...
Page 35
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Appendix B: Benchmarks and tests The iozone benchmark was used to measure sequential read and write throughput (Mi B/sec) as wel...
Page 36
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS IOzone Argument Description -t Number of threads +m Location of clients to run IOzone on when in clustered -w Does not unlink (d...
Page 37
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS B.2. mdtest mdtest can be downloaded from http://sourceforge.net/projects/mdtest/. Version 1.8.3 was used in these tests. It was...
Page 38
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Metadata file and directory creation test: # mpirun -np 32 -rr --hostfile ./hosts /nfs/share/mdtest -d /nfs/share/filedir -i 6 -...
Manual Details
| Brand | Dell |
|---|---|
| Pages | 38 |
| File Size | 1.04 MB |
| Published | June 22, 2026 |
6 views
Enter the captcha to get the download link:
Frequently Asked Questions
What three areas need tuning when optimizing the NFS performance?
Optimization requires tuning Storage parameters, the NFS server configuration, and the Dell Fluid Cache for DAS settings.
How do I minimize cache effects during metadata testing?
You must unmount the NFS share on clients, stop the NFS service and umount XFS on the server, and then remount all components.
What is used to test metadata file and directory removal performance?
The `mdtest` command is used with specific arguments (e.g., -r) running concurrently from multiple NFS clients.
What file system structure is required for the Dell Fluid Cache configuration?
The recommended setup includes using XFS file systems configured in conjunction with the Dell Fluid Cache technology.