Storage and IT Infrastructure Predictions for 2018 by StorPool

What is coming in the fields of storage and IT infrastructure in 2018, according to StorPool Ltd.?

2017 was a turbulent year for IT, but this is business as usual now.

2018 promises to be even more turbulent, with increasing levels of consolidation, regulation and global competition.

No matter whether you strive to be at the bleeding edge of IT or not, these 2018 storage predictions and IT infrastructure trends are a starting point to get a competitive edge and make life easier.

Storage

Storage devices – NVMe on the rise
NVMe SSDs are going mainstream, superseding SATA and SAS SSDs. Server OEMs are finally delivering good server platforms for NVMe drives, which will help adoption of NVMe drives tremendously. Since the price of NVMe drives is just 5-10% higher compared to the price of datacenter grade SATA SSDs, we expect adoption to pick up tremendously during 2018. For small margin customers will get 2-3x the performance and half the latency.

Flash and DRAM shortage to alleviate
During 2017 the flash and DRAM shortage was leading to increased costs for computer components which use flash and DRAM, i.e. SSDs and DIMM modules. Delivery times were extended and varied and between vendors – from couple of days to over a month, sometimes longer for larger quantities.

During 2018 new production capacities are coming into operation and we expect the shortage to be relieved. During 2Q and 3Q18 the price of NAND flash will drop moderately. This will further lower the cost for SSDs and flash-based storage systems and accelerate their adoption.

Hybrid storage systems will continue to decline
With the decreasing cost of Flash, SSD-HDD hybrids will make less and less sense. HDDs will be moved down to serve strictly capacity use-cases (where $/GB is more important than performance). Performance use-cases will be satisfied by flash-based systems. Flash-based storage systems will also grow in size and scope, utilizing the newly-found efficiency of low cost NAND flash.

High performance storage systems based on NVMe SSDs will start to gain wider adoption. With the right storage system partner, it now becomes possible to have a shared storage system as fast as or faster than local SSDs.

The divergence of storage systems into two distinct categories will continue. The two categories are systems which focus only on performance (disregarding other aspects) and systems (like StorPool) which provide performance, but also end-to-end data integrity and the full spectrum of data features (HA, scalability, snapshots, multi-site, etc).

SDS (software-defined storage) and DS (distributed storage) accelerated adoption
While in 2016 SDS was the second option in new projects (first were either traditional SAN or all-flash arrays), in 2017 SDS became the first option companies reviewed. We now see spill-over of SDS and DS in the datacenters and it will be 30-50% of new deployments in 2018, with accelerating growth in 2019.

Also SDS is usually adopted in relatively larger projects (50TB+), so more than 50% of deployed capacity in 2018 to be SDS powered.

SDS is also the fundamental storage layer of HCI (hyper-converged infrastructure) solutions, and there it replaces low- to mid-range SANs and all-flash arrays.

SDS consolidation
The concept of ‘unified storage’ (block, file and object in one product) is being demystified. As good as this concept sounds, it is an elusive promise. In practice, storage software needs very different architecture and implementation to be very good in each one of these use cases.

In 2017 we saw initial segmentation, where many vendors started to be more precise in what they actually do and to focus on use cases they are strong at – be it block, file or object. Customers and solution providers seem to now understand that they are likely to end up with 2 or 3 best-of breed solutions for each storage layer, to get the job done well.

This search for ‘best of breed’ will continue in 2018. And as the technology mature, we will start to see some clear leaders on a per use case and per technology stack basis.

DR and workload mobility in growth mode
Integrated multi-site capabilities for DR, workload mobility and for backup to boom in 2018. This was somewhat a surprise, since most companies have backup figured out and one would imagine that DR was part of this plan. However increasing application complexity and business demands are driving a new wave of DR requirements, where storage systems can deliver nearly zero RTO and RPO.

IT infrastructure

Compute
Heterogeneous computing, the concept of using general purpose CPUs alongside compute resources better at the task at hand – GPUs, FPGAs, might start gaining wider adoption outside of the traditional applications it has been used for – video, image processing (and now increasingly bitcoin and cryptocurrency mining, etc.). The main driver for this is machine learning and analytics over large pools of structured and unstructured enterprise data.

Machine learning, AI, IoT and the analytics applications they require will become a major driver for new IT infrastructure.

CPUs
Intel Xeon Scalable is being used in nearly all new IT infrastructure projects (public clouds, on-prem clouds and others). It brings approximately 50% higher compute density (cores per server, GB RAM per server) and considerable power efficiency gains over the previous generation. This in turn brings even better overall economics to virtualized infrastructure.

The AMD Zen microarchitecture (Epyc) are back in the game with the recently announced deployment in Microsoft Azure. We see further adoption in 2018.

Some companies are experimenting with ARM and Power architecture servers. However we do not consider this mass market, yet. So this 2017 prediction of ours came short.

Our expectation is that the use of ARM and Power architecture CPUs will remain very limited to specific applications. ARM servers are better suited for batch processing (throughput-driven) applications than for interactive/online (latency-driven) applications.

Qualcomm has recently entered the server CPU market with their 48-core CPU.

Virtualization stacks
The virtualization stack is ripe for reinvention. On the one hand, KVM is gaining market share on everyone else’s behalf. This trend will accelerate during 2018, with small and large vendors offering HCI and rack-level architecture cloud solutions based on KVM. On the other hand containers, especially managed by Kubernetes are spreading fast. We see potential for reinvention of the entire virtualization stack.

We project the decreasing market share of other hypervisors to accelerate in 2018.

Networks
25/50/100GbE is continuing to gain adoption over 10GbE. We predict that during 2018 more than 50% of new datacenter networks to use 25/50/100GbE instead of the older 10/40GbE standards.

Intel OmniPath has moved from seldom being used to a market leader in HPC interconnects. This is because the OmniPath adapter is now integrated into Intel’s Xeon CPUs. During 2018, we expect that most new RDMA fabrics in HPC will be OmniPath. Some deployments outside of HPC might choose it due to better cost/performance vs. Ethernet, but only for large scale multi-rack cases where the application demands extreme performance between compute nodes.

IB is not going away. It is continuing to be used in new HPC deployments. Outside of HPC, we don’t see a compelling case for IB during 2018.

Data growth and applications
According to a Seagate commissioned study made by IDC by 2025 the global datasphere will grow to 163ZB (that is a trillion gigabytes). That’s ten times the 16.1ZB of data generated in 2016. This growth comes mainly from unstructured data although structured data and core business application continue to add.

As much of this data is created in specialized platforms (social media, specific hardware manufacturers, etc.) we expect the raise of a new crop of large private clouds, which will become good target market for IT infrastructure providers.

Market developments

We expect new wave of consolidation in 2018 especially between server vendors and storage players.

Cloud is increasing its adoption, although still 25.09% of money are spent on building public clouds now, according to a study from IDC. There is an ongoing trend of cross-regional consolidation between second and third tier cloud providers, which we expect to continue.

On the other hand the General Data Protection Rule (GDPR) which will take effect on 25 May 2018 in the European Union will stir the public, private and hybrid-clouds market and will make local markets more segmented. We already see datacenter and cloud providers shifting workloads to newly opened local facilities in bigger markets, so they can save and serve data locally.

Asian vendors are picking up
We already see actions from leading Asian vendors like Alibaba, Huawei, AIC, NEC, etc. to compete head-to-head with their US counterparts. We expect this trend to deepen in 2018, fuelled by significant cash and human resources dedicated to it by Asian vendors and their increasingly global vision.

Summary

2018 promises to be an exciting year for storage and IT infrastructure. There are some master-trends happening in the industry (regulation with GDPR and the increasing strength of Asian vendors), which have the potential to swirl the markets and make 2018 an year, unlike any of the recent ones. So instead of incremental shifts, there may be some tectonic shifts.