Dell Need for Speed - Comparing FDR and EDR InfiniBand - Part 1 Specifications User Guide
Summary
This Dell HPC blog post evaluates Mellanox Technologies FDR (14 Data Rate) and EDR (Enhanced Data Rate) InfiniBand performance with theoretical speeds of 56Gb/s and 100Gb/s respectively. Part 1 compares latency, bandwidth, and High-Performance Linpack results between ConnectX-3 based FDR adapters and ConnectX-4 based EDR adapters. EDR achieves 150 million messages per second messaging rate versus FDR's 90 million, with x16 Gen3 bus width. The analysis uses industry-standard micro-benchmarks on H
Page 1 Text Content
Need for Speed: Comparing FDR and EDR Infini Band (Part 1)
By Olumide Olusanya and Munira Hussain
The goal of this blog is to evaluate the performance of Mellanox Technologies’ FDR (Fourteen Data Rate) Infiniband and their latest EDR (Enhanced Data Rate) Infiniband with speeds of 56Gb/s and 100Gb/s respectively. This is the first of our two series blog and we will be showing how these interconnects perform on a cluster using industry-wide micro-level benchmarks and applications on HPC cluster configuration. In this part, we will show latency, bandwidth and HPL results for FDR vs EDR and in part 2 we will share more results with other applications which include ANSYS Fluent, WRF, and NAS Parallel Benchmarks. You should also keep in mind that while some applications would benefit from the higher bandwidth in EDR, other applications which have low communication overhead would show little performance improvement in comparison.
General Overview: Mellanox EDR adapters are based on a new generation ASIC also known as Connect X-4 while the FDR adapters are based on Connect X-3. The theoretical uni-directional bandwidth for EDR is 100 Gb/s versus FDR which is 56Gb/s. Another difference is that EDR adapters are x16 adapters while FDR adapters are available in x8 and x16. Both of these adapters operate at a bus width of 4X link. The messaging rate for EDR can reach up to 150 million messages per second compared with FDR Connect X-3 adapters which deliver more than 90 million messages per second. Table 1 below shows the difference between EDR and FDR and Table 2 describes the configuration of the cluster used in the test while Table 3 lists the applications and benchmarks used for this test.
Table 1 - Difference between EDR and FDR
FDR EDR Chipset Connect X-3 Connect X-4 Link x8 and x16 Gen3 x16 Gen3 Theoretical BW 56 Gb/s 100 Gb/s Messaging rate 90 MMS 150 MMS Port QSFP QSFP28
Table 2 - Cluster configuration
Components Details Server 16 nodes x Power Edge C6320 [ 4 chassis ] Processor Intel®Xeon®Intel Xeon E5-2660 v3 @2.6/2.2 GHz , 10 cores, 105W
Page Summary Contents For Dell Need for Speed - Comparing FDR and EDR InfiniBand - Part 1 Specifications User Guide
Manual Details
| Brand | Dell |
|---|---|
| Pages | 5 |
| File Size | 943.11 KB |
| Published | June 18, 2026 |
Enter the captcha to get the download link:
Frequently Asked Questions
What is the primary theoretical bandwidth difference between EDR and FDR interconnects?
EDR has a theoretical uni-directional bandwidth of 100 Gb/s, compared to FDR's ConnectX-3 adapters which are rated at 56 Gb/s.
How does the bidirectional bandwidth performance compare between the two technologies using OSU benchmarks?
EDR achieves 24.2 GB/s (193.6Gb/s), showing a 124% improvement over FDR's bidirectional rate of 10.8 GB/s (86.4Gb/s).
Is the performance benefit of EDR obvious in compute-intensive benchmarks like HPL?
No, for compute-intensive apps like HPL, large performance differences are not expected because they spend more than 80% of runtime on computation versus communication.
What is the difference in messaging rate between connected EDR and FDR adapters?
The messaging rate for EDR can reach up to 150 million messages per second, compared with FDR ConnectX-3 adapters which deliver over 90 million messages per second.