Databricks Acquires Start-Up Rubicon

The team behind Rubicon is joining Databricks, Inc.

Founded by large scale infrastructure builders, Akhil Gupta and Sergei Tsarev, Rubicon is a start-up working on building storage systems for AI.

Databricks was started as a company over 10 years ago with the goal to make data and AI applications dramatically easier to build.

It soon realized that traditional storage systems such as enterprise data warehouses were not sufficient to process data needed for AI applications. That’s why it built Delta Lake, a core storage foundation for the Lakehouse that combines the best of data warehousing and data lake storage.

With recent innovations in AI like LLMs, there is a greater need for storage solutions that enable processing diverse types of data and seamlessly integrating both the online stack (for serving) and the offline stack (for training and experimentation). This is reflected in conversations with customers who have been asking for a simple easy to use platform that enables them to leverage these new exciting technologies. 

Gupta and Tsarev are 2 known figures in the storage infrastructure space. Akhil began his career at Google almost 2 decades ago, where he was responsible for the ads infrastructure, which is one of the largest scale use cases of AI on the planet to date. Sergei co-founded Clustrix, one of the first scale-out OLTP database systems for high value, high-transaction applications. The two met at Dropbox, where they helped build, among other things, one of the world’s largest, most efficient, and secure storage systems which served trillions of files and exabytes of data. This included processing immense volumes of unstructured data, e.g. documents, images, and videos, precisely the type of data that customers rely on for AI applications. It is no surprise that given their past experiences, both of them saw the potential of AI, as well as the gaps in storage and serving infrastructure needed to support those workloads. This led them to founding a company focused on solving the problem.

“We launched Rubicon.io with the goal of enabling the next gen of storage infrastructure for the AI age. A great developer experience with world class performance at scale is at the core of our mission. We find our vision aligns perfectly with Databricks and we are thrilled that our team will be joining Databricks to deliver an extraordinary experience for developers on the Lakehouse,” said Gupta and Tsarev.

When the company first met them, it quickly realized its shared outlook and the potential for the technologies they have been developing to accelerate our AI infrastructure work.

It is thrilled to bring team Rubicon’s pioneering vision into the Databricks Lakehouse Platform. At the upcoming Data + AI Summit, it will unveil how we will integrate Rubicon’s technical approach to simplify building AI applications.