Gitex Global: Huawei AFA OceanStor A310 AI Storage Solution

At Gitex Global 2023, Huawei Technologies Co., Ltd. has launched an AI storage model, the OceanStor A310, designed for the era of large AI models.

The solution provides storage capability for basic model training, industry model training, and inference in segmented scenario models, unleashing new AI capabilities.

The launch demonstrates the company’s alignment with the latest trends of AI development in the era of large models. The OceanStor A310 DL data lake storage caters to different industries and scenarios in large AI model applications.

Peter Zhou, president, data storage product line, Huawei said: “With the OceanStor A310, we are bringing our customers and partners a cutting-edge AI storage solution to unleash new AI capabilities and create value for various industries. Visitors to our GITEX stand can witness how Huawei is committed to accelerating the intelligent era, leveraging a trusted, secure, and reliable OceanStor platform used by some of the leading businesses in the world.”

Enterprises face 4 significant challenges in the development and implementation of large model applications.

First, data preparation takes a long time, with dispersed data sources and slow aggregation. Currently, preprocessing about a hundred terabytes of data takes around ten days.

Secondly, for multi-modal large models using massive text and images as training sets, the current loading speed for a large number of small files is less than 100MB/s, resulting in low efficiency in loading training sets.

Third, frequent tuning of large model parameters and instability in the training platform leads to training interruptions approximately every 2 days on average. The Checkpoint mechanism is needed to recover training, and the fault recovery process takes more than a day.

Finally, the high threshold for large model implementation, complex system construction, difficult resource scheduling, and GPU resource utilization is usually below 40%. The OceanStor A310’s DL data lake storage caters to different industries and scenarios in large model applications.

In the context of basic/industry large model scenarios, OceanStor A310 is a DL data lake storage solution that offers unlimited horizontal scalability and high performance for mixed workloads. It enables end-to-end massive data storage management for AI processes, including data collection, preprocessing, training, and inference. It possesses data analysis capabilities consistent with HPC and big data, allowing for the processing of data from similar sources.

The company is a Gold Sponsor of Gitex Global 2023 and the lead sponsor for the event’s cybersecurity program, Cyber Valley.