Alluxio: Hybrid Cloud Solution Fueled by 3rd Gen Xeon Scalable Processors and Optane Persistent Memory 200 Series

Alluxio, Inc. announced a go-to-market solution in collaboration with Intel Corp. to offer an in-memory acceleration layer with 3rd Gen Xeon Scalable processors and Optane persistent memory (PMem) 200 series.

Click to enlarge

The solution enables performance analytics and AI pipelines at a massive scale while minimizing the TCO by more managing data in local storage tiers close to compute. Optane PMem provides a cost-effective storage tier for Alluxio managed data, while accelerating performance with a disaggregated compute and storage architecture.

“With an explosion in the amount of data being managed by the Alluxio Data Orchestration system, effective memory and storage media provided by Intel has a huge value-add,” said Haoyuan Li, founder and CEO.

3rd gen Xeon Scalable processors with Optane persistent memory 200 series delivers up to 6TB total memory per socket and on average 32% higher memory bandwidth than the previous gen. ⁽¹⁾ Using Storage over App Direct, a feature of PMem App Direct mode, allows the company’s solution to access a high-performance block storage tier to provide the data access acceleration.

The solution builds on the company‘s collaboration with Intel aimed at improving their customers’ experience with managing and processing their data, such as optimizations for Intel Deep Learning Boost and the AI acceleration technology built into the latest Xeon Scalable processors. With its latest release, Alluxio now improves POSIX API support to further accelerate ML and AI workloads in addition to new storage connectors for cloud object storage. The firm will bring these solutions to market to help fuel next-gen data, analytics and AI applications.

“We are excited to collaborate with Alluxio to help them bring innovative solutions to market,” said Alper Ilkbahar, VP, data platforms group and GM, Optane group, Intel. “The advancement of compute technologies for both advanced analytics and AI applications introduces performance challenges at the data access layer. Alluxio’s Data Orchestration Platform coupled with Intel Optane persistent memory is uniquely positioned to accelerate and simplify management at the data access layer across multiple environments.“

Resource:
Blog: Accelerating Analytics and AI with Alluxio and NVIDIA GPUs

⁽¹⁾ Based on testing by Intel as of April 27, 2020 (Baseline) and March 23, 2021 (New). Baseline configuration: 1-node, 1 xXeon Platinum 8280L processor (28 cores at 2.7GHz) on Neon City with a single Intel Optane PMem module configuration (6x32GB DRAM; 1x{128, 256, 512GB} Optane PMem module), ucode rev: 04002F00 running Fedora 29 kernel 5.1.18-200.fc29.x86_64 and Intel Memory Latency Checker (Intel MLC) version 3.8 with App Direct Mode.
New Configuration: 1-node, 1xpre-production 3rd Gen Xeon Scalable processor (38 cores at 2.0GHz) on Wilson City with a single Optane PMem module configuration (8x 32GB DRAM; 1x{128, 256, 512GB} Optane PMem module), ucode rev: 8d000270 running RHEL 8.1 kernel 4.18.0-147.el8.x86_64 and Intel MLC version 3.9 with App Direct Mode.