STAC-M3 Benchmarks WekaIO Matrix Parallel Filesystem, and Server Cluster From Penguin Computing

STAC (Securities Technology Analysis Center LLC) performed STAC-M3 benchmarks on a stack involving kdb+, the WekaIO, Inc.‘s Matrix parallel filesystem, and a server cluster from Penguin Computing.

Click to enlarge

STAC-M3 is the set of industry standard enterprise tick-analytics benchmarks for database software/hardware stacks that manage large time series of market data (tick data). In this project, the company ran both the baseline benchmark suite (code named Antuco) and the optional scaling suite (code named Kanaga).

The stack under test was Kx’s kdb+ 3.6 database system distributed across 7 Penguin Computing Relion XO1132g servers, with data stored in a cluster of 8 Penguin Computing Relion XE1112 servers with a total of 105TB of SSD capacity, all accessed via Matrix 3.2.2 software. WekaIO chose to highlight that this solution:

Outperformed all publicly disclosed results in 8 of 24 Kanaga mean-response time benchmarks (STAC-M3.β1.100T.YR1VWAB-12D-HO.TIME to STAC-M3.β1.100T.YR4VWAB-12D-HO.TIME, plus STAC-M3.β1.1T.2YRHIBID.TIME to STAC-M3.β1.1T.5YRHIBID.TIME)
Outperformed all publicly disclosed results in 4 of 5 Kanaga throughput benchmarks (STAC-M3.β1.1T.2YRHIBID.MBPS through STAC-M3.β1.1T.5YRHIBID.MBPS)
Was faster in 15 of 17 mean-response time benchmarks in the Antuco suite versus a kdb+ solution involving a Lustre-based on-premise cluster (SUT ID KDB150528), including:
11x speedup in year-high bid (STAC-M3.β1.1T.YRHIBID.TIME)
2.1x speedup in market snapshots (STAC-M3.β1.10T.MKTSNAP.TIME)
Was faster in 13 of 17 mean-response time benchmarks in the Antuco suite versus a solution in which kdb+ ran in distributed mode on a public cloud (SUT ID KDB181001), including:
2.15x the speed for STAC-M3.β1.100T.STATS-UI.TIME
2.36x the speed for STAC-M3.β1.1T.VWAB-D.TIME
Achieved 168ms for year-high bid (STAC-M3.β1.1T.YRHIBID.TIME), which is within 3ms of the best publicly disclosed result for this benchmark across all architectures.

The results are available here (registration required)