Next ‘23: Recap of Google Cloud Storage Announcements

By Vivek Saraswat, group product manager, Google Cloud storage, and

Jason Wu, director, product management, Google Cloud Storage

Cloud Storage provides simple, scalable, secure, and cost-effective object storage for customers to store, access, and manage their data.

This year at Google Cloud Next ‘23, the company presented a number of new cloud storage capabilities designed to help you serve your storage needs for mission-critical applications of all kinds, including AI/ML and data analytics workloads. Here’s a recap of all the new features and capabilities announced.

Optimized storage for AI/ML and data analytics
Our customers increasingly rely on object storage when running their data-intensive workloads (AI/ML, batch analytics, streaming analytics, and HPC). To help you with this critical and fast-growing area, the firm announced multiple capabilities in the areas of programmability, performance, and manageability.

Programmability – APIs for easy integration into your business

• Cloud Storage Fuse (generally available): Mount and access a cloud storage bucket as a local filesystem, optimized for AI/ML and GKE workloads (learn more)
• Pub/Sub to Cloud Storage subscriptions (GA): Simplify streaming data ingestion to your object storage buckets with just a few clicks (learn more)
• Transfer for HDFS (preview in 4Q23): Use Storage Transfer Service to easily transfer petabytes of Hadoop/Spark workloads to Google Cloud

Performance – Optimal price/performance for your data-intensive workloads

• Cloud storage client library transfer manager (preview): Improved read/write performance in client libraries by parallelizing uploads and downloads (learn more)
• Anywhere Cache (preview in 4Q23): Elastically scalable zonal SSD read cache to minimize your egress bandwidth costs at predictable low latencies
• gRPC API (preview in 4Q23): New cloud storage API option that provides more efficient routing for analytics workloads, reducing overall execution time
• Improved Hadoop connector (GA in 4Q23): Improved write performance for Hadoop/Spark workloads on cloud storage via parallelization and disk buffering

Manageability – End-to-end data lifecycle tailored for ease of use and security

• Event-driven transfer (GA): Listen to event notifications to trigger Storage Transfer Service (learn more)
• Managed folders (preview in 4Q23): Prefix/folder level fine-grained access control to enable simplicity and security for data-intensive workloads

Built for mission-critical workloads
No matter what workload you leverage on object storage, areas such as security, governance, availability, scalability, and management are important. Here are the capabilities we announced for mission-critical workloads of all kinds:

Intelligent storage – Easily understand and manage your data at scale

• Storage Insights datasets (preview in 4Q23): Visualize and analyze your storage usage and trends within BigQuery. This supplements currently available Storage Insights inventory reports (learn more)
• Autoclass for existing buckets (GA in 4Q23): Automatically transition objects to different storage classes based on last access time, without needing to create a new bucket

Enterprise security and governance: Safeguard assets while meeting security, governance, regulation, and compliance requirements

• Custom audit information (GA): Additional headers to give user, application, and job context to audit logs (learn more)
• Object retention lock (in preview): WORM-compliant immutable object storage by setting retain-until policies on individual objects
• Assured Workloads (GA): Integrated and packaged security product controls to help meet compliance requirements in United States public sector with added support for ITAR and IL5 compliance regimes. Includes controls for data residency, data access and data sovereignty for data at rest, in use and in transit. (learn more)
• Custom organization policy (preview in 4Q23): Custom guardrails authored and enforced by customers on GCP resources (bucket) to meet their organization governance requirements

Resilience and scalability: Enable geo-redundancy while maintaining strict data residency controls, DR, and real-time monitoring

• Soft delete (preview in 4Q23): Quickly recover your data from accidental deletion during a specified retention period
• Replication monitoring (preview in 4Q23): Near real-time replication status is now provided for default and turbo replication in dual-region and multi-region buckets
• Additional dual-regions (GA): Flexible and performant geo-redundancy with new support for Canada and Australia regions (learn more)

To learn more about how to leverage Cloud Storage in your application, try out one of the new Google Cloud Jump Start Solution tutorials. For a list of all of Google Cloud Next ‘23 announcements, check out the Next ‘23 Wrap-Up.