Indiana University Using Lustre Over WAN

Indiana University and four research technology partners have succeeded in using the Lustre file system over a wide area network (WAN), saturating the world’s first commercial 100 gigabit link.

Provided by T-Systems and Alcatel-Lucent in June, the connection uses a single wavelength to bridge the 60 kilometers (37 miles) between Dresden and Freiberg, Germany. IU, the Center for Information Services and High Performance Computing, DataDirect Networks, and Whamcloud joined forces to achieve this milestone.

"Using the full duplex capability of the 100 gigabit link, we have been able to achieve an aggregate average transfer rate of 21.904 gigabytes per second. This is over 87% of the theoretical maximum allowed by the link," said Data Capacitor project lead Stephen Simms of the Pervasive Technology Institute at Indiana University. "At that rate, we’re sending enough data through to fill 285 DVDs every minute."

With equipment provided by DataDirect Networks (DDN) and Hewlett Packard, and in consultation with Whamcloud, the team deployed Lustre file systems in both Dresden and Freiberg. The preliminary goals of the project were to stress test Data Direct Network’s S2A9900 and SFA10K disk appliances and fully exercise Lustre across the 100 gigabit connection.

The longer-term goal is to explore the possibility of having one or more wide area file systems that provide Dresden and Freiberg with massive data transfer capabilities, enabling faster exchange of shared data and insights.

IU has used Lustre across the WAN since 2006 and has worked with the Center for Information Services and High Performance Computing in the area of data intensive computing since 2007, when an IU-led team won the SuperComputing Bandwidth Challenge along with Technische Universität Dresden and DDN. Using the IU Data Capacitor and a 10 gigabit network, the team achieved a peak transfer rate of 18.21 Gigabits per second, which corresponds to the content of 29 DVDs per minute.

Pervasive Technology Institute (PTI) at Indiana University is an organization dedicated to the development and delivery of information technology to advance research, education, industry, and society. Supported in part by a $15-million grant from the Lilly Endowment, Inc., PTI is built upon a spirit of collaboration and brings together researchers and technologists from a range of disciplines and organizations, including the IU School of Informatics and Computing at IU Bloomington, the IU Maurer School of Law, and University Information Technology Services at Indiana University.

About Center for Information Services
and High Performance Computing
The Center for Information Services and High Performance Computing (ZIH) is a central scientific unit of the Technische Universität Dresden offering a broad spectrum of systems, services, and research competencies supporting both the university and the research community in the state of Saxony. The Center started to focus its research on data intensive computing with the establishment of the HRSK system in 2006. The connection of the network infrastructure of the university to the German Research Network and the Internet ranges among the fastest in Germany.