WekaIO and Stability AI Partner

WekaIO, Inc. is pioneering a new solution in collaboration with its customer Stability AI Ltd, an open-source generative AI company.

The Weka data platform converged mode for cloud solution will enhance Stability AI’s ability to train multiple AI models in the cloud, including its Stable Diffusion model, and extend efficiency, cost and sustainability benefits to its customers.

AI processes such as training and inference require significant energy and computing resources. Few companies, especially those pioneering the next frontier of generative AI (GenAI) solutions, can invest the time or expense needed to acquire and deploy the necessary horsepower on-premises, and training and inference of AI models in the cloud is often prohibitively expensive.

Weka has developed a way to improve this equation for cloud-first GenAI customers. Its data platform’s converged mode solution is the a scale-out storage on deep learning capable instances available to users running workloads in the cloud. It uses ephemeral local storage and memory in cloud AI instances to yield exponential cost savings and performance improvements for large-scale generative AI resources compared to traditional data architectures.

Stability AI is seeking to use existing cloud resources more efficiently. The company began working with Weka to develop a “converged cloud” approach with the Weka data platform and recently concluded a successful trial of the solution. Its converged architecture provides a single, performing set of resources to support the application and data platform simultaneously. The result is more optimal training of Stability AI’s models and better utility for its research teams.

“As the only independent, open, and multimodal generative AI company, we understand the limitations of running converged workloads on-premises,” said Tom Mason, CTO, Stability AI. “Weka gives our customers more options by enabling training on converged mode in the public cloud. Our hope is that this partnership will increase our hardware utilization and scale to thousands of instances and essentially, do more with less, more sustainably.“

Once deployed, Stability AI hopes to be able to extend the benefit of performance gains and significant cloud cost-savings from Weka’s data platform, all while lowering carbon emissions and energy consumption. It expects to use the Weka solution to underpin its forthcoming sustainability initiatives as its deployment matures and grows.

“The Weka data platform can increase GPU-stack storage efficiency by 10-50x, driving a significant cost advantage for generative AI companies by helping to maximize resources they’ve already paid for,” said Weka co-founder and CEO Liran Zvibel. “At the same time, Weka makes it simple and affordable to run AI model training and inference in the public cloud, helping to shrink their data infrastructure and energy and carbon footprint, delivering a double-bottom line benefit for customers that want to harness the power of AI without compromising their corporate sustainability goals.“

Weka’s converged mode for cloud solution will be released as a public preview this year, initially on AWS, with other clouds to follow.