Highpoint Technologies Examining Viability of 2.5-Inch and EDSFF NVMe Based Cache Solutions for Cost-Effective HCI Cluster Servers

Summary:

NVMe storage solutions are ideal for HCI and SDS applications, offering performance and versatility for both capacity and cache roles.
2.5″ U.2/U.3 NVMe media are suitable for server caching, providing significant speed advantages (up to 14,000MB/s for Gen5) and large storage capacities (up to 30TB/unit), along with features like power-loss protection (PLP) and high endurance ratings.
EDSFF media, specifically E1.S NVMe SSDs, offer a compact form factor with high capacity and endurance, making them excellent for caching applications. They combine the benefits of M.2 SSDs’ compactness with the performance and reliability of U.2/U.3 drives.
HighPoint’s NVMe solutions, such as the Rocket 1628A PCIe Gen5 and SSD7749E RAID AIC, support high-density configurations, enabling significant storage and performance capabilities for each server, potentially eliminating the need for SAS/SATA storage altogether.

Network servers that rely HCI technology utilize SDS suites to pool, configure and manage storage resources. This article examines how NVMe storage solutions are ideal for these type of workflows, and how different NVMe media classes and form factors are suited to particular use cases.

HCI and SDS applications
Versatile SDS applications such as Microsoft’s Storage Spaces Direct, known as S2D (and designed for Azure and Windows Server environments), or VMware’s vSphere, are ideal for applications that employ HCI technology. This type of software suite enables customers to link multiple cost-effective computing platforms, each with their own internal DAS solution, to function as a collective enterprise grade server (aka cluster server). Storage hosted by the individual systems are pooled and managed as a single resource, which can then be distributed as needed. This storage pool is typically split into 2 general categories; ‘capacity’ (which serves as general server storage) or ‘cache’ (which is exactly that – cache). In general, the ratio of capacity to cache is 2:1 (2 devices assigned to capacity for every one device assigned as cache).

The individual platforms may employ any number of internal storage devices. Typically, platter HDDs are used exclusively in the capacity role. While extremely dense, and inexpensive, they are incredibly slow by modern standards (MB/s in the 100s vs.1,000s for a basic M.2 NVMe SSD), and are simply ill suited for cache. Conventional SSDs (SAS, SATA) and NVMe media is far more versatile, especially the latter, can reliably serve in either role.

To examine the advantages posed by NVMe storage solutions, let’s take a closer look at how 2.5” NVMe SSDs (U.2/U.3) and EDSFF media (specifically, E1.S) can be used for caching purposes in a Microsoft S2D environment for Windows Server 2022.

NVMe media for HCI caching
As touched upon earlier, S2D managed storage, if used for caching purposes, is typically configured in a 2:1 ratio (2 storage drives for every 1 cache drive) for data redundancy (data security). Consider one of our high-density Rocket or SSD series NVMe AICs, which can support up to 8 NVMe SSDs. If you were to devote the entire AIC to caching purposes, you could pair it with a 16-channel SAS/SATA hard disk controller.

2.5” U.2/U.3 media
U.2/U.3 NVMe media is ideal for server caching, especially if the target platform supports the standard 2.5” drive form factor. They are capable of performing much faster than SAS or SATA SSDs; 500 to 1GB/s for SATA/SAS vs. 7GB/s for Gen4 or up to 14,000MB/s for Gen5 (per SSD!). U.2/U.3 media can store considerable quantities of data, upwards of 30TB per unit. In addition, as enterprise-class storage devices, U.2/U.3 NVMe media are often available with PLP (power-loss protection), which will enhance the reliability of the storage pool. PLP can help reduce the risk of data loss during a sudden power outage by flushing data that was stored in the SSD’s cache memory to the drive’s permanent flash memory. U.2/U.3 media also benefits from-enterprise class endurance ratings, which are measured in terms of disk-writes per day, as opposed to total-bytes written (which is associated with standard SSDs).

HighPoint’s Rocket 1628A PCIe Gen5 x16 NVMe Switch AIC can support 8x30TB U.2/U.3 NVMe SSDs, which would net you 120TB of cache capable of performing at 56GB/s. A comparable SATA/SAS adapter could potentially double your storage if your budget can handle it (the largest SAS is currently 64TB), but would perform well below 7GB/s. To put this into perspective, that’s roughly the equivalent of a single Gen4 NVMe SSD.

ESDFF media (E1.S)

E1.S NVMe SSDs are a type of ESDFF media, and are also well suited for caching applications. They combine the compact form-factor of M.2 SSDs (approximately the length of a 22210 M.2 SSD, in 2 thicknesses: 9.5mm and 15mm), with capacity and endurance approaching that of 2.5” U.2/U.3 drives; a sort of best-of-both-worlds approach to the medium. HighPoint’s SSD7749E RAID AIC and RocketAIC 7749E E1.S AIC SSDs, enable customers to support up to 8xE1.S 9.5mm E1.S SSDs, or 4x15mm E1.S SSDs using a single PCIe Gen4 or Ge5 x16 slot.

Currently available in sizes up to 8TB, 8x E1.S can net you over 60TB/PCIe slot, and deliver performance in the range of 28,000MB/s (Gen4 x16). Unlike many M.2 SSDs, they provide an excellent balance of random and sequential I/O performance, and are not prone to handicap transfer speeds during extended write sessions. Most E1.S SSDs are classified as DC class storage, and share enterprise features with many U.2/U.3 models, such as high-endurance (disk writes per day), and are also available with PLP.

Massive storage aggregation
The compact, single-slot AIC and Adapter form factors enables each individual platform to host multiple Rocket/SSD series solutions. Modern platforms can potentially support as many as seven add-in-cards; devoting a little more than half the slots, say 2 Rocket 1608A or SSD7749E cards and 4 SAS/SATA HDD adapters would net you up to 480TB and 112GB/s of cache, and multiple petabytes of general storage, for each server.

And in truth, HighPoint U.2/U.3 and E1.S based solutions may enable you to forgo SAS/SATA storage media altogether. 240TB and 56GB/s of speed from each PCIe slot is a formidable combination.