WekaIO Delivers Record Results on 17 STAC-M3 Tick Analytics Benchmarks Using HPE Apollo Systems

WekaIO, Inc. announced record-breaking performance on the STAC-M3 Benchmark with HPE Apollo 2000 Gen10 systems.

Click to enlarge

The Weka File System (WekaFS) running on these HPE servers delivered record performance at scale for the STAC-M3 ‘Tick Analytics’ Benchmarks-setting 12 STAC-M3 world records for mean query-response times and 5 world records for throughput. These results affirm that an integrated storage solution from Weka and HPE is the choice for algorithmic trading and quantitative analysis workloads common in financial services.

The STAC-M3 benchmark specs are maintained by the STAC Benchmark Council, which consists of over 500 financial institutions and vendor organizations. The Council’s purpose is to discuss technical challenges and solutions in financial services and to develop useful technology benchmark standards for financial organizations. User firms include the largest global banks, brokerage houses, exchanges, hedge funds, proprietary trading shops, and other market participants. The STAC-M3 results were audited by the Securities Technology Analysis Center (STAC), which facilitates the Council.

Key benchmark result highlights include:

• Outperformed all publicly disclosed results in 11 of 24 mean-response time benchmarks in the STAC-M3 Kanaga suite:

• • 100-user 12-day VWAB: all five benchmark years (STAC-M3.β1.100T.YR[n].VWAB-12D-HO.TIME)

  • 50-user 12-day VWAB: benchmark years four and five (STAC-M3.β1.50T.YR4VWAB-12D-HO.TIME and STAC-M3.β1.50T.YR5VWAB-12D-HO.TIME)

  • Multi-year high bid: all 4 multi-year spans (STAC-M3.β1.1T.[n]YRHIBID.TIME)

• Outperformed all publicly disclosed results in all 5 throughput benchmarks in the STAC-M3 Kanaga suite (STAC-M3.β1.1T.*.BPS)

• Outperformed all publicly disclosed results in the 100-user 12-day VWAB (no overlap) benchmark in the STAC-M3 Antuco suite (STAC-M3.β1.100T.VWAB-12D.NO.TIME)

The system under test (SUT) was a Kx kdb+ 3.6 database system distributed across 14 HPE ProLiant XL170r Gen 10 servers with dual Xeon Gold 6134 CPU @3.2GHz and data stored on HPE servers with WekaFS (v3.6.1) software running on 18 HPE ProLiant XL170r Gen 10 servers, each with four NVMe SSDs, using Mellanox SB7790 36-port Non-blocking Managed EDR 100Gb IB switches. The tests were the baseline benchmark suite (Antuco) and the scaling suite (Kanaga), which uses multiple years of simulated stock market data. STAC-M3 Antuco and Kanaga Benchmark Suites are the industry standard for testing solutions that enable high-speed analytics on time series data such as tick database stacks. The SUT performed faster than a solution using NFS-based NAS and four database nodes (KDB190430) on 100% of all the Kanaga multi-year benchmarks and faster than a solution with a single server with direct-attached Optane and 3D NAND on 79% of all Kanaga multi-year mean response time benchmarks. The storage system had a combined performance density of up to 113GB/s available bandwidth to the client nodes, a maximum observed read throughput of 54.7GB/s ^(*), and 8.8 million 4K IO/s in 10U, with the ability to fully distribute data, metAdata: and system services. In addition, the storage system delivered low latency by leveraging high-speed networks and NVMe-optimized storage.

The Weka SDS solution is validated for deployment in HPE environments-including those with ProLiant DL360 and the Apollo Gen10 system, which is optimized and purpose-built to serve storage-centric workloads such as big data analytics and SDS. WekaFS software scales to meet the demanding IO/s and throughput requirements for high-performance data processing, delivering an integrated flash-based parallel file system on HPE servers that can accelerate compute-intensive workloads. For customers, the integrated solution accelerates time-to-market by delivering a low-latency, throughput-optimized environment that provides industry-leading performance, scale, and value.

Barbara Murphy has presented WekaFS for Financial Analytics at Global STAC Live, an online event starting June 2, 2020.

Resources:
Full STAC Report: STACresearch.com/KDB200401 (registration required)
WekaIO landing page at STAC: STACresearch.com/wekaio
Blog: What is the STAC-M3 Benchmark and Why Should You Care?     
WekaFS with HPE Apollo 2000 Reference Architecture    
WekaFS with HPE DL360 Reference Architecture     
WekaFS for Financial Services use case

^(*) 54.7GB/s in Kanaga 5-year high bid throughput benchmark (STAC-M3.β1.1T.5YRHIBID.BPS)

Bill Mannel, VP, business operations for HPC and AI, and GM, commercial HPC, HPE, a win innovation partner, said: “We are pleased to see that our joint hardware and software engineered solutions are validated with impressive results on the STAC-M3 benchmarks. Our integrated solution, which includes the HPE Apollo 2000 Gen10 systems, delivers highly-accelerated storage capabilities to support high-performance data analytics environments. Our solution accomplishes this while eliminating possible I/O bottlenecks that occur as customers dramatically increase compute horsepower and fabric bandwidth.”

Glenn Wright, systems architect, Kx Systems, Inc., said: “We first qualified WekaFS to run kdb+ in Amazon Web Services (AWS) EC2, but we have since seen very good storage performance from WekaFS running kdb+ on bare metal. Kx’s time-series database kdb+ is used for large scale complex analytics on streaming, real time, and historical data. Kdb+ is used in capital markets for algorithmic trading, backtesting, surveillance, regulatory reporting, and research environments as well as in other industries where it is used for high-speed sensor monitoring, fault detection, predictive analytics, and ML. Kx customers have come to expect the very best performance from file systems used to store kdb+ historical analytics data. Kx is pleased to recognize the excellent performance of the Weka file system supporting kdb+ as represented in this recent STAC-M3 benchmark.“

Amita Potnis, analyst, IDC, said: “The industry has seen an influx of new technologies that enable I/O-intensive and extreme technical computing workloads inclusive of high-performance data analytics. And there is increasing widespread adoption of flash storage for higher throughput data movement, but in general, new hardware and networking implementations only rise to their full solution potential when there is a corresponding advancement in software. This philosophy holds true for Weka’s integration with HPE hardware as the distributed and parallel file system optimized for NVMe fully maximizes the benefits of the low-latency, high-throughput capabilities of the system.“

Barbara Murphy, VP, marketing, WekaIO, said: “Weka has seen a surge in adoption by customers seeking cost-effective alternatives to legacy storage systems to support their HPDA workloads at scale. WekaFS has been proven to deliver performance that is 3x that of local file systems and 10x that of traditional NAS or SAN, and it is the only shared file solution that can scale to hundreds of clients with no performance degradation. WekaIO is a firm believer in the value of benchmark testing as a means to guide and inform development in the industry and frame results in a manner that delivers actionable intelligence to customers. In addition to breaking world records for the STAC-M3 benchmarks, Weka currently holds several top positions for the independent SPEC SFS 2014 benchmark and the #1 position on VI4IO’s IO-500 List.“

Read also:
WekaIO AI Accelerates Edge to Core to Cloud Data Pipelines
Expedites time-to-market and to-value while delivering agility and security at scale.
April 28, 2020 | Press Release
Exclusive Interview With WekaIO CEO Liran Zvibel
Growing fast as serious commercial alternative to leading HPC file storage such Lustre and IBM Spectrum Scale
by Philippe Nicolas | April 9, 2020 | News
600% Revenue Growth From 2018 to 2019 for WekaIO
With help of partners Supermicro, Dell EMC, HPE, and Nvidia
January 29, 2020 | Press Release