Public Cloud Storage Performance Benchmark by Storpool

StorPool Storage announced the results from a public cloud performance measurement report, focusing on shared block-storage services of several providers.

The public clouds which are benchmarked are Amazon AWS, Digital Ocean, DreamHost, OVH, ToggleBox, eApps and other. The tests are comparing storage systems/offerings, not other aspects, like VM sizes and plans.

Random reads and writes-block size 4k-queue depth 64
StorPool performance test
Click to enlarge

Latency under light load-reads and writes
StorPool Performance Test
Click to enlarge

Seven VMs, with identical parameters (8 vCPUs, 16GB RAM) and various pricing (priced $69 to $188), were procured by the company from all tested clouds. Further the firm procured 7 identical block storage volumes from each of these providers and attached them as virtual disks to the VMs that were being tested.

Four different types of tests were performed:

PGBENCH – a PostgreSQL database benchmark, perhaps closest to actual application performance
Sysbench/mysql – a second database benchmark for control of PGBENCH results, MySQL based
fio – a set of synthetic benchmarks – random reads/writes, sequential reads/writes, latency measurements
rsync – copying files and syncing, simulating rapid deployment and backup workloads

Full report

Results summary
The results from the tests show significant difference between the offerings of the benchmarked public clouds, in all meaningful metrics – latency (the number one metric) of any cloud, IO/s, MB/s and price.

Conclusions

Most shared-storage offerings you can get in a public cloud are painfully slow for running databases and demanding apps, with a few exceptions
Offerings which are the same on paper, perform very, very differently. Ranging from ‘is this thing working at all’, to ‘Just WOW’
There is several fold difference in the price of the seemingly same service
Difference in price performance of shared storage services is even higher than the variance in price or the variance in performance alone
Surprisingly, nor the best price, nor the best price-performance ratios are not found in the most popular services on the market

Full measurement report