SPECstorage Solution 2020 Storage Benchmark With AI and Genomics Workloads

The Standard Performance Evaluation Corp. (SPEC) has released a major version of its storage solution benchmark.

Click to enlarge

The statistical visualization mechanism within SPECstorage solution 2020 allows users to extract runtime counters and connect them to a database for visualization.

The SPECstorage solution 2020 benchmark measures performance of an entire storage configuration as it interacts with application-based workloads. This version includes workloads for AI and genomics, expanded custom workload capabilities, massively better scaling, and a statistical visualization mechanism for displaying benchmark results.

“The new SPEC storage benchmark is a valuable tool to help IT managers make better purchasing, configuration and optimization decisions based on actual applications and real-world scenarios,” says Don Capps, chair, storage subcommittee.

Workloads and customization
The AI workload in the SPECstorage solution 2020 benchmark represents AI Tensorflow image processing environments, with traces collected from systems running COCO, Resnet50 and CityScape datasets. The genomics workload comes from commercial and research facilities that perform genetic analysis. Other workloads in the benchmark address video data acquisition (VDA), electronic design automation (EDA), and software build environments.

Capabilities in the benchmark enable users to customize existing workloads or create new ones and test them to gain insight into storage performance issues.

Massive scaling with fewer constraints
The internal IPC mechanisms in SPECstorage solution 2020 reduce the number of TCP ports and DNS lookups, improving scalability, reliability, and start-up and run-time performance. Scaling has been increased from 60,000 load-generating processes in previous versions to more than 4 million processes that can be distributed throughout the world. This efficiency is especially significant for testing cloud-based storage solutions.

“Among cloud providers, TCP ports are a scarce resource, with most VMs able to accommodate only a few thousand at best,” says Capps. “The new benchmark never requires more than 1,000 TCP ports. This enables users to scale up to four million processes without running into cloud constraints.“

Visualization for greater insights
The statistical visualization mechanism within SPECstorage solution 2020 allows users to extract runtime counters and connect them to a database for visualization. This provides greater insight into the behavior and operation details of the system under test.

Available for downloading
The SPECstorage solution 2020 benchmark is available for download on the SPEC website for $2,000, with discounts for qualified research and academic institutions. Results from the benchmark are not comparable to those from past SPEC storage benchmarks.

SPEC corporate members active in the development of the benchmark include Dell, IBM, Intel, iX Systems, NetApp, Pure Storage, and WekaIO. Supporting individual contributors include Udayan Bapat, Sorin Faibish, and Brian Pawlowski.