Dremio Unveils Hybrid Data Catalog for Apache Iceberg Delivers Flexibility and Governance Across All Deployment Models

Dremio Corp. announced that its Data Catalog for Apache Iceberg supports all deployment options – on-prem, cloud, and hybrid – making the company the only lakehouse provider to deliver full architecture flexibility.

Additionally, it is announcing integrations with Snowflake’s managed service of Apache Polaris (incubating) and Databricks’ Unity Catalog managed service. This allows customers to choose the best catalog for their needs, while using Dremio to deliver analytics across all data. Databricks and Snowflake customers can choose the catalog that makes the most sense for their business, reducing TCO and avoiding unnecessary infrastructure expenses. By enabling full governance and security, the solution helps organizations unify their data infrastructure while maintaining control and optimizing performance.

First hybrid data catalog for Apache Iceberg
“Enterprises face extraordinary pressure to access, prepare, and govern distributed datasets for consumption by analytics and AI applications. To meet this demand, they need to catalog diverse data and metadata across data centers, regions, and clouds,” said Kevin Petrie, VP, research, BARC US. “Dremio is taking a logical step to enable this with an open catalog that is based on Apache Iceberg, the emerging standard for flexible table formats, thereby integrating with an ecosystem of popular platforms.” 

Built on the open-source Project Nessie, Dremio’s Data Catalog for Apache Iceberg introduces key features and functionality that includes:

• Flexible and open interoperability: Supports all Iceberg engines, such as Dremio and Spark, through the Iceberg REST catalog API, providing a flexible, open and future-proof solution.
• Centralized data governance: Enables centralized data governance across all data, with role-based access control and fine-grained access privileges to ensure data compliance and security.
• Automated table optimization: Automates table optimization tasks like compaction and garbage collection, enhancing performance and lowering storage costs.
• Data branching and versioning: Git-like branching and version control supports  experimentation, virtual development environments, and time-travel without data duplication while preventing risk to production data.
• Simplified data management: Reduces complexity and increases data management efficiency by addressing common pain points such as convoluted catalog setups, lack of governance controls, and insufficient maintenance tools.

Integration with managed service catalogs
With augmented support for Snowflake and Databricks, the company is strengthening its ongoing commitment to deliver open, scalable, and flexible lakehouse architectures that streamline data integration and analytics across any environment. Now, Dremio customers no longer have to choose between vendors or architectures as they can integrate with their preferred catalog, deploy on-prem, in the cloud, or in a hybrid architecture while maintaining smooth interoperability across platforms to unite analytics without vendor lock-in. 

“Flexibility is essential for modern organizations looking to maximize the value of their data. With expanded Iceberg catalog support across all environments, Dremio empowers businesses to deploy their lakehouse architecture wherever it’s most effective,” said Tomer Shiran, founder, Dremio. “We’re 100% committed to giving customers the freedom to choose the best tools and infrastructure while reducing fears of vendor lock-in.”

Resource:
Blog: Now in Private Preview: Dremio Lakehouse Catalog for Apache Iceberg