WekaIO,
the innovation leader in high-performance, scalable file storage for
data-intensive applications, today announced record-breaking performance
on the STAC-M3™ Benchmark from a new integrated solution with Penguin
Computing—the FrostByte™ storage solution featuring the WekaIO File
System. The STAC-M3 Antuco and Kanaga Benchmark Suites are the industry
standard for testing solutions that enable high-speed analytics on time
series data, such as tick database stacks. The WekaIO File System on
Penguin Computing® Relion® servers and FrostByte storage system
delivered record-breaking performance at scale for the STAC-M3 “Tick
Analytics” Benchmark, affirming that the integrated solution is the
ideal choice for algorithmic trading and quantitative analysis workloads
common in financial services.
This press release features multimedia. View the full release here:
https://www.businesswire.com/news/home/20190606005483/en/
Figure 1 (Graphic: Business Wire)
The STAC-M3 benchmark specifications are maintained by the STAC
Benchmark Council, which consists of over 300 financial institutions and
more than 50 vendor organizations whose purpose is to discuss technical
challenges and solutions in financial services and to develop technology
benchmark standards that are useful to financial organizations. User
firms include the largest global banks, brokerage houses, exchanges,
hedge funds, proprietary trading shops, and other market participants.
The STAC-M3 results were audited by the Securities Technology Analysis
Center (STAC®), which facilitates the Council.
The system under test was a Kx kdb+ 3.6 database system distributed
across 7 Penguin Computing Relion XO1132g servers with dual Intel® Xeon®
Gold 6140 CPU @ 2.30 GHz and data stored on Penguin FrostByte with the
WekaIO File System (v3.2.2) Software running on 8-1U RXE1112 server
nodes with 9 NVMe SSDs per server, using Mellanox SB7780 36-port
Non-blocking Managed EDR 100Gb/s InfiniBand Router and InfiniBand EDR/
Ethernet 100Gb 1-port 840QSFP28 Adapters. The tests were run on the
baseline benchmark suite (“Antuco”) and the scaling suite (“Kanaga”), a
test suite with five years of simulated stock market data. The test
system had a combined performance density of up to 87.5 GB/sec available
bandwidth to the client nodes, 40.5 GB/sec of actual throughput*, and
2.5M 4K IOPS in 8U, with the ability to fully distribute data, metadata
and system services, as well as super low file system latency that
leverages high-speed networks and NVMe-optimized storage.
See Figure 1.
The test system delivered record-breaking performance, breaking 8
STAC-M3 world records for mean query-response times and 4 world records
for throughput.
Kx’s time-series database kdb+ is commonly used for large scale complex
analytics on streaming, real time, and historical datasets in industries
ranging from financial services to marketing analytics, and for
Industrial IoT. In the financial industry, typical kdb+ applications are
for algorithmic trading, back testing (trading algorithm validation),
surveillance, regulatory and compliance reporting and research
environments. These workloads can demand a lot from the file system.
Better than block-based solutions for Financial Services, the WekaIO
File System is the only NVMe-optimized, shared, coherent,
POSIX-compliant file system with a fully distributed architecture for
data resiliency. The WekaIO Software eliminates islands of storage,
providing all applications access to a shared data pool with
record-breaking performance. The WekaIO File System delivers 10x better
performance than a local file system by bending the rules on data
locality, distributing the data across the cluster over low latency
InfiniBand or Ethernet networks.
WekaIO is a firm believer in the value of benchmark testing as a means
to guide and inform development in the industry and frame results in a
manner that delivers actionable intelligence to customers. The WekaIO
File System currently holds one of the top positions on the independent SPEC
SFS 2014 benchmark and the #1 position on VI4IO’s 10
Node Challenge list.
For the full STAC Report, which contains an explanation of the benchmark
specifications and the results for this solution, see www.STACresearch.com/news/2019/06/03/KDB190501.
Supporting quotes:
“Traditionally, IT organizations have used block-based storage
solutions for their trading databases, rather than compromise
performance due to network and file system latencies. The result has
been large investments in specialized hardware and software to run
different applications,” said David Hiatt, Director of Business
Development and Alliances at WekaIO. “Now with the STAC-M3 Benchmark
results, we have proven that with the WekaIO File System on Penguin
Computing Relion servers and FrostByte delivers better application
performance for Kx kdb+ by running in distributed mode over a network
than with traditional block storage, with the additional benefit of
simultaneously supporting other applications on the same platform.
That’s a huge leap in application acceleration and simplifying the
storage infrastructure for financial services.”
“The Penguin Computing FrostByte integrated solution featuring the
WekaIO File System is designed, engineered and vetted to deliver a
high-performance storage solution for Kx kdb+,” added Dr. Kevin Tubbs,
Sr. Director of Technology, Advanced Solutions Group at Penguin
Computing. “To break eight STAC-M3 performance records is an incredible
achievement and a testament to the innovative architecture of the WekaIO
File System and leading HPC and AI solutions from Penguin Computing. For
IT supporting different applications and Kx kdb+ time-series databases,
this platform is an unbeatable combination for performance, scale, and
ease of management.”
Glenn Wright, Systems Architecture expert at Kx, said: “Kx customers
demand both high performance and low latency for compute and storage.
The ability to combine both a distributed kdb+ implementation running
against a high-performance shared file system can be attractive to our
customers. With WekaIO as this file system, our customers are able to
choose how and when they distribute their historical data workload to be
accessible by one or more of their analytics workloads.”
“It’s great to see another successful benchmark test outcome when the
WekaIO File System is paired with Mellanox’s network adapters. We have
successfully collaborated with WekaIO on many benchmarks and together,
we continue to set benchmark records,” contributed Gilad Shainer, Senior
Vice President Marketing at Mellanox Technologies. “The Mellanox
InfiniBand EDR/Ethernet 100Gb 1-port 840QSFP28 Adapters provide the
lowest latency available, and that coupled with the WekaIO distributed
and parallel file system satisfies the performance of even the most
demanding I/O intensive application requirements.”
“There’s now a proliferation of software and hardware configurations
aiming to support I/O intensive workloads,” commented David Floyer,
Co-founder and CTO at Wikibon. “But to meet the performance demands of
real-time and historical analytics in kdb+ environments, storage systems
must offer both high performance and low latency. The results from the
STAC-M3 Benchmark should help IT in making an informed decision when
architecting storage systems to support algorithmic trading and
quantitative analytics workloads.”
*As per the benchmark STAC-M3.ß1.1T.5YRHIBID.MAX
About WekaIO
WekaIO helps companies manage, scale and futureproof their data center
so they can solve big problems. WekaIO built the world’s fastest shared
and parallel file system on new architecture optimized for flash that
leapfrogs legacy storage infrastructures by delivering the best
performance density at any scale, for a fraction of the cost. With the
most flexible deployment models, the storage software is a modern file
system designed for modern I/O intensive and extreme technical computing
workloads. For more information go to, https://www.weka.io.
View source version on businesswire.com: https://www.businesswire.com/news/home/20190606005483/en/