Integration of Cloudian HyperStore Object Storage and Cribl Stream for Data Lake-Based Observability Platform for Data Analytics

Cribl, Inc. and Cloudian, Inc. announced a data analytics solution that integrates Cribl Stream and Cloudian HyperStore object storage to provide an observability platform on an on-premises, S3-compatible data lake.

The joint solution enables organizations to ingest, parse, restructure and enrich large data volumes in flight, ensuring they get the right data in the formats they need – all securely behind their firewall. Users can convert logs into metrics, reduce cost, and increase search speed.

Need for observability platform
Monitoring solutions alone only help enterprises answer well-known questions about their infrastructure and environment. To get new signals or opportunities, they also need an observability platform that can ingest and store data from multiple sources with full fidelity and can replay the relevant data into various analytics tools without adding new infrastructure and agents. This allows IT teams to perform advanced analytics, getting full visibility and insight into their environment, from applications to infrastructure assets.

Cribl-Cloudian solution
Cribl Stream is an observability pipeline that collects data from any source and can send and replay data to a Cloudian HyperStore S3-compatible data lake. The data is stored in HyperStore with full fidelity and is always available to search and analyze. HyperStore can scale up to thousands of nodes across multiple data centers, supporting millions of users and hundreds of petabytes of data.

Other solution benefits include:

• Cost-effective, limitless scalability for long-term compliance – Data can be reformatted via a pre-parser, helping users mask, reduce, or restructure and route it to multiple destinations, such as low-cost Cloudian storage (½¢/GB/month or less) for long-term retention.

• Replay data – Customers can replay multiple data formats stored in the data lake to popular analytics and search platforms.

• Hybrid cloud ready – Users can employ policy-based tools to replicate or tier data to AWS, GCP, Azure, or another Cloudian HyperStore cluster for offsite disaster recovery, capacity expansion or data analysis in the cloud, enabling cost-effective storage across environments but managed as a single pool.

• Military-grade security and ransomware protection – Features include AES-256 server-side encryption for data stored at rest, SSL for data in transit (HTTPS), role-based access controls with specified levels of access, audit trail logging and data immutability for protection against ransomware.

• Data resiliency – The solution provides up to 14 nines of resiliency and supports administrator-selectable storage policies based on replication or erasure coding, along with fine grain control of data placement across data centers.

“Giving our customers freedom and control over their data is the core of Cribl,” said Zac Kilpatrick, VP, global channels and alliances, Cribl. “When customers use Cloudian and Cribl together, they can collect the largest amounts of data in any format with Cribl Stream and store and replay full fidelity data leveraging Cloudian HyperStore, saving costs and optimizing performance.”

“Organizations are continually striving to gain deeper insights from their data to drive greater business and operational advancements,” said Larry Meese, VP, products and solutions, Cloudian. “The Cribl-Cloudian solution enables them to easily and cost-effectively apply modern analytics applications to their on-prem data with the same scalability and flexibility of public cloud services while avoiding the cost and performance issues of moving large data volumes to the cloud.”