SC19: InfiniteIO Platform Reduces Latency and Accelerates Performance for ML, AI and Analytics

InfiniteIO, Inc. announced the Application Accelerator, which delivers performance improvements for critical applications by processing file metadata independently from on-premises storage or cloud systems.

This platform provides organizations across industries low latency for their mission-critical applications, such as AI/ML, HPC and genomics, while minimizing disruption to IT teams.

“Bandwidth and I/O challenges have been largely overcome, yet reducing latency remains a significant barrier to improving application performance,” said Henry Baltazar, VP, research, 451 Research. “Metadata requests are a large part of file system latency, making up the vast majority of requests to a storage system or cloud. InfiniteIO’s approach to abstracting metadata from file data offers IT managers a nondisruptive way to immediately accelerate application performance.“

Dashboard Application Accelerator
Click to enlarge

As unstructured data has grown, requests for file metadata-information such as file attributes and access privileges-have also skyrocketed to become a bottleneck for application performance. The company’s latest release, built on its Metadata Engine (IME) architecture, responds to file metadata requests directly from the network instead of the NAS or cloud storage system. The metadata abstraction can reduce latency from seconds to microseconds for all files in a hybrid cloud environment. This results in faster access to data and speeds up application performance.

“Reducing latency is the last frontier in improving application performance. The tech industry has been focused on making incremental performance improvements with faster storage and file systems, when the biggest opportunity is in removing the file system latency created by processing metadata requests,” said Mark Cree, CEO, InfiniteIO. “Separating metadata processing from file I/O decreases application latency, which translates into reduced product development cycles and greater worker productivity.“

Turbocharging data-intensive applications
The Application Accelerator allows organizations to go faster and implement innovation, with minimal to no disruption to existing IT operations:

Completely offloads and responds to metadata requests at the speed of DRAM, which improves performance up to 50% for workloads such as ML training models
Delivers more than three million metadata operations per second, doubling the performance of the previous-gen product
Deploys without requiring changes to existing applications or storage systems
Supports enterprise growth by scaling to one billion files in a single node or two billion files in a clustered configuration
Enhances the performance of all NAS, and all-flash and NVMe arrays
Can be upgraded with performance data tiering capabilities

The company also released software features that continue to simplify and accelerate tiering of cold data from primary NAS systems to lower-cost cloud storage, such as the ability to scan one billion files in a day and a API for cloud usage charge-back. The IME architecture never recalls metadata back from the cloud, increasing performance and avoiding cloud egress charges. Policies automatically tier files so that even rarely accessed information is available on-demand, without disruption or performance compromises.

Resources:
InfiniteIO Blog
Application Accelerator DS
Video: How InfiniteIO Accelerates Performance