Fujitsu File System for Advanced x86 HPC Cluster Systems

Fujitsu Limited FEFS (Derived from Fujitsu Exabyte File System), a scalable file system software package for building file systems for x86 HPC clusters in Japan.

FEFS is software for x86 HPC cluster systems that enables high-speed parallel distributed processing of large amounts of read/write transactions from the compute nodes. The software achieves the world’s highest throughput speed of 1 TB/s from the compute nodes to the file system.

In addition, it includes features for system scalability, high reliability for zero operational downtime, and actual operational convenience. It delivers high speeds and large-scale data processing performance increasingly demanded of file systems in accordance with performance improvements and increases in the scale of cluster systems. This contributes to improvements in overall system performance.

To meet the needs of customers, Fujitsu is offering file system solutions that combine its PRIMERGY x86 servers with its ETERNUS storage system and the new FEFS.

Background
Computer-based analysis and simulation can be used to reduce costs and shorten development times. It is currently being actively used in manufacturing and many other industries. Increasingly, x86 HPC clusters, which use multiple x86 servers for parallel processing, are becoming the dominant platform used for such analyses and simulations.

With the improved performance of x86 HPC cluster systems in recent times, file systems have emerged as a source of performance bottlenecks. There is an increasing need for file systems that can deliver higher speeds and large-scale data processing capabilities.

Fujitsu’s newly-developed FEFS software enables high-speed parallel distributed processing of large amounts of read/write transactions from the compute nodes, creating a large-scale file system with performance and reliability.

Fujitsu is committed to providing the best file system solutions for analysis and simulation applications and a variety of other fields.

FEFS was developed based on the Lustre open source software, with proprietary feature enhancements added by Fujitsu. From x86 HPC cluster systems consisting of several dozen servers to massive systems comprised of up to a million servers, FEFS enables file systems with scalability, performance, reliability, and convenience to support a range of systems.

Main feature enhancements of FEFS:

Scalability

• It enables scalability of file systems from terabyte-scale systems to a maximum of 8 exabytes (1,000 petabytes), depending on data volume requirements.
• It can be used as a file system offering superior price-performance for clusters consisting of several dozen nodes, and it can be used for large scale clusters comprised of up to a million servers.

Performance

• It enables the configuration of systems consisting of 10,000 storage systems with the throughput speed of 1 TB/s.
• It achieves metadata management performance capable of creating several tens of thousands of files per second, approximately 1-3 times the performance of Lustre.

Reliability

• Due to built-in redundancies at all levels of the file system (such as disk RAID configuration, IB network multipath configuration, and configurations of multiple servers and storage units), it enables failovers while jobs are being executed.

Convenience

• Fair share features (Refers to features that allocate server processing capacity in a way that is fair to each user) for allocating resources among users prevent a particular user from monopolizing I/O processing resources.
• Priority control settings for the operation of each node guarantees I/O processing bandwidth for each node.
• Directory level quota (A limit set on the number of files that can be created or total file data capacity for each individual file owner, group, or larger affiliation, in which usage beyond the limit is prohibited.) functions enable efficient use of disk capacity by monitoring and managing fine levels of file system activities.

System Configuration and Pricing
Smallest configuration: four PRIMERGY RX300 S6 servers (with IB connection), three ETERNUS DX80 S2 storage units, and FEFS license. System Price: starting from 21,430,000JPY.