Oracle: Shatter Million IO/s Barrier in Cloud with OCI Block Storage

By Max Verun, senior principal product manager, Oracle International Corp.

We are pleased to announce that you can now achieve up to an aggregate 1.3 million IO/s and up to 12GB/s throughput per OCI Compute instance with the OCI Block Volume Service.

In addition, you can now also attach up to 32 Ultra High Performance (UHP) volumes to a single compute instance. This service update is particularly useful for high performance I/O workloads, such as AI/ML with intensive data processing and GenAI, 3D modeling and simulation, and demanding blockchain processing. In addition to modern workloads, traditional applications such as OTP, data warehousing and analytics, databases with large data sets and extensive complex processing also benefit from this level of performance.

We continue to offer a single, simple and all NVMe SSD based volume type that is only available from OCI, vs. multiple complex disjoint tiers from other cloud providers. You can easily scale I/O performance of your attached block storage any time with a simple dynamic performance slider without the overhead of provisioning additional volumes or compute instances.

1.3 million IO/s/instance with block storage provides 63% increase over our prior 800,000 IO/s limit without any change in OCI storage pricing. This performance update is generally available to use for all your existing and new volumes without additional cost. We continue to guarantee performance with the financially backed SLA. You can configure the maximum IO/s/volume with the now familiar Volume Performance Units per Gigabyte (VPU/GB) using the dynamic performance slider. This setting configures volume performance on demand, and you can continue to adjust it dynamically for optimal performance without detaching, migrating, or otherwise disturbing the attached volumes on your compute instances.

How to get this level of performance
This update lifts the limit of attaching 1 UHP volume per compute instance. Single volume performance remains unchanged: All bare metal instances continue to achieve UHP performance of maximum 300,000 IO/s per volume at sub-millisecond latency. VMs with iSCSI attachments achieve up to 300,000 IO/s/volume, and with paravirtualized attachments achieve up to 150,000 IO/s/volume.

You need to attach multiple volumes to a compute instance to get an aggregate of 1.3 million IO/s block storage performance.

Below is an example storage benchmark fio performance run on a BM.Standard.E4.128 compute instance on one of our regions. By attaching 5 UHP volumes to the compute instance, it shows that you can achieve 1.3 million IO/s or more on the instance. This is predictable and sustained, not burst performance. Compute shapes evolve at a fast rate with newer and faster memory, CPU and network. Now your block storage also scales and aligns with high performance compute shapes for your demanding applications in the cloud.

5x2TB volumes, each with 120 VPU/GB performance setting that guarantees 300,000 IO/s/volume. They are all attached to the same BM.Standard.E4.128 compute instance.

Click to enlarge

Edit Volume Console page shows that each volume is easily configured for 120 VPU/GB that guarantees 300,000 IO/s/volume using the performance slider.

Click to enlarge

Storage performance benchmark tool fio is run on this compute instance across all 5 volumes using random read and write I/O. It shows the predictable and steady performance for this example, with an even distribution of 750,000 read and 750,000 write IO/s totaling to 1.5 million IO/s.

Click to enlarge

OCI Block Volumes performance page provides more detail and how to achieve this level of performance. This depends on memory and network bandwidth configurations of compute shapes. Not all compute shapes can reach up to 1.3 million IO/s for remote block storage.

Block storage performance per instance on bare metal (BM) shapes

Shape	OCPU	Memory (GB)	Max. Network Bandwidth	Max. IO/s per Instance (up to)	Max .Throughput per Instance (Block Volume)
BM.Standard.E5.192	192	2,304	1x100Gb/s	1,300,000	12GB/s
BM.Standard.E4.128	128	2,048	2x50Gb/s	1,300,000	6GB/s
BM.DenseIO.E4.128	128	2,048	2x50Gb/s	1,300,000	6GB/s
BM.Standard3.64	64	1,024	2x50Gb/s	1,300,000	6GB/s
BM.Optimized3.36	36	512	2x50Gb/s 1x100Gb/s RDMA	1,300,000	6 GB/s
BM.GPU.A100-v2.8	128	640	2x50Gb/s 16x100Gb/s RDMA	1,300,000	6GB/s
BM.GPU.A10.4	64	96	2x50Gb/s	1,300,000	6GB/s
BM.GPU4.8	64	GPU: 320GB CPU: 2,048GB	1x50Gb/s 8x200Gb/s RDMA	1,300,000	6GB/s
BM.Standard.A1.160	160	2,048	2 x 50 Gbps	800,000	6 GB/s
BM.GPU3.8	52	GPU: 128GB CPU: 768GB	2 x 50 Gbps	625,000	3 GB/s
BM.GPU2.2	28	GPU: 32GB CPU: 192GB	2 x 25 Gbps	625,000	3 GB/s

Block storage performance per instance on VM shapes

Shape	OCPU	Memory	Max. Network Bandwidth	Max. IO/s per Instance	Max. Throughput per Instance (Block Volume)
VM.Standard.E5.Flex	1 OCPU minimum, 90 OCPU maximum	1GB minimum, 1,049GB maximum	1Gb/s per OCPU, maximum 40Gb/s	20,000 * max network bandwidth in Gb/s (up to 600,000)	120MB/s * max network bandwidth in Gb/s (up to 4,800MB/s)
VM.Standard.E4.Flex	1 OCPU minimum, 64 OCPU maximum	1GB minimum, 1,024 GB maximum	1Gb/s per OCPU, maximum 40Gb/s
VM.DenseIO.E4.Flex	8/16/32	128/256/512	8Gb/s/16Gb/s /32Gb/s
VM.Standard3.Flex	1 OCPU minimum, 32 OCPU maximum	1GB minimum, 512GB maximum	1Gb/s per OCPU, maximum 32Gb/s
VM.Optimized3.Flex	1 OCPU minimum, 18 OCPU maximum	1GB minimum, 256GB maximum	4Gb/s per OCPU, maximum 40Gb/s
VM.Standard.A1.Flex	1 OCPU minimum, 80 OCPU maximum	1GB minimum, 1,024GB maximum	1Gb/s per OCPU, maximum 40Gb/s
VM.GPU.A10.2	30	GPU: 48GB CPU: 480GB	48Gb/s	600,000	5,760MB/s

Heinz Mielimonka, customer success director and cloud architect, Oracle, provides additional insight in his blog post Unimagined Disk Speed for Your Servers. He gives a comparison of block storage performance levels across largest cloud providers. He also provides tooling to try and prove it to yourself that you can reach this level of performance on OCI.

Try it for yourself
We want you to experience this level of cloud storage performance and all the enterprise-grade capabilities that OCI offers. It’s easy to try them out with OCI Free Tier. For more information on taking advantage of these performance updates, see the Block Volume service overview, Ultra High Performance (UHP) volumes, Block Volume performance, dynamic performance scaling with auto-tuning, OCI storage pricing.