10GbE Outperforms IB in Range of HPC Applications

According to a new benchmark study recently conducted by IBM Corporation, 10Gb Ethernet delivered better performance than InfiniBand and Gigabit Ethernet for a range of HPC (high performance computing) applications.

IBM researchers compared the performance of several applications in HPC using 4x InfiniBand DDR, 10 Gigabit Ethernet using iWARP, and Gigabit Ethernet using TCP/IP. The results showed 10Gb Ethernet is superior in performance to InfiniBand in some standard performance benchmark test suites and comparable in others.

The test configuration was a computing cluster using a 2.3 GHz quad-core Opteron processor, with each node having 16Gb memory and two quad-core processors. Each node was configured with a 4x DDR InfiniBand connected to a 96-port Cisco DDR switch, using dual-port DDR ConnectX adapters from Mellanox; a 10Gb Ethernet network using Chelsio dual-port adapters with full offload capability, connected to a 20-port Force10 10Gb Ethernet switch; and a Gigabit Ethernet network connected through a Cisco switch. The software configuration was standards-based as well, running RedHat Enterprise Linux Server release 5.2.

IBM ran eight applications over the networks, including NETPERF – a benchmark suite that is commonly used to measure various aspects of networking performance, with a primary focus on bulk data transfer and request/response performance using either TCP/IP or UDP and the Berkeley Socket Interface (BSI). The NETPERF results showed 10Gb Ethernet greatly outperforming InfiniBand for both TCP and UDP, and the Gigabit Ethernet was a distant third.

"We are pleased that IBM has published the results of its extensive performance testing for key HPC applications and the results clearly show that 10Gb Ethernet is superior to InfiniBand for a broad base of HPC applications and very similar in others – validating the reasons behind the rapidly growing success of Ethernet as the HPC interconnect technology," said Kianoosh Naghshineh, president and CEO of Chelsio Communications, Inc.

To read the full report, IBM Benchmarks 10Gb Ethernet and InfiniBand for HPC Applications