SwiftStack: Multi-Cloud Artificial Intelligence/Machine and Deep Learning Data Management Solution

SwiftStack, Inc. announced a customer-proven edge-to-core-to-cloud solution that supports large-scale Artificial Intelligence/Machine and Deep Learning (AI/ML/DL) workflows.

Click to enlarge

The company has recently deployed this solution stack in two autonomous vehicle use cases.

The firm’s AI/ML solution delivers massive storage parallelism and throughput, needed for ingest, training and inferencing; a scale-out global namespace for access to data whether on-premises or in one or more clouds; data services such as tagging, search, and metadata management to support AI/ML workflows; and Kubernetes and TensorFlow support. Additionally the solution extends to public clouds to take advantage of cloud-bursting and economies of scale, while data is secured on-premises.

“Infrastructure challenges are the primary inhibitor for broader adoption of AI/ML workflows,” said Amita Potnis, research director, infrastructure systems, platforms and technologies group, IDC. “SwiftStack’s multi-cloud data management solution is the first of its kind in the industry and effectively handles storage I/O challenges faced by edge-to-core-to-cloud, large-scale AI/ML data pipelines.“

The solution includes an integration of SwiftStack with Valohai‘s deep learning platform-as-a-service to provide machine orchestration, version control, and AI/ML pipeline management.

“With this integration users can quickly adopt an AI/ML platform, easily use multi-cloud workflows and frameworks, and scale to petabytes of storage and hundreds of gigabytes of bandwidth,” said Eero Laaksonen, CEO, Valohai. “This gives them the ability to create a deep learning infrastructure, even at an enormous scale, in a fraction of the time.“

“The SwiftStack solution accelerates data pipelines, eliminates storage silos, and enables multi-cloud workflows, thus delivering faster business outcomes,” said Jason Blum, CTO, GPL Technologies, an NVIDIA and SwiftStack elite partner. “SwiftStack provides us with the flexibility, technology leadership and breakthrough economics to build tailored solutions for our customers.”

GPL has created multiple ways to implement the solution, with NVIDIA DGX-1 GPU server(s), NVIDIA GPU Cloud, and other system hardware.

Click to enlarge

The emergence of AI/ML workloads has created a new set of challenges for organizations, while the rise of GPU computing is enabling massive parallelism and several petaflops (floating-point operations per second) of computational power. It has been difficult for these environments to build and manage a storage infrastructure that provides appropriate scale and concurrent performance.

“Traditional storage architectures are not designed for these new distributed workloads and fall short of performance, scale, and value, so storage services needed to be rethought to accommodate AI/ML pipelines,” said Shailesh Manjrekar, head, product and AI/ML solutions marketing, SwiftStack. “Successful customer deployments are proving that we have created a solution that enables them to put their GPU cycles to work, bring cloud and AI to the data and scale affordably as the workflow grows.“