Storage Market 2020 Retrospective

2020 will stay in our mind for a long time for sure, very long time. We have a few more days to run and this year will be over and we all hope to start 2021 in a complete different mindset and behavior. Vaccines should help and we saw huge collaboration between countries and many partnerships between therapeutics research centers and biotech laboratories have demonstrated the power of human being. It’s fantastic to realize such rapid progress and human effort and support globally. Let’s stay positive wishing for all a better year 2021.

In the industry, obviously the impact of pandemic is visible like in any other markets with significant erosion of revenue and users adoption and at the same time some confirmations of trends accelerated by the Covid-19.

Kubernetes is clearly a game changer as some giants have been pushed to move fast towards better integration, support or even redesign of their product. This is the case for VMware that refused for a long time to believe in Kubernetes and finally jumped in as the Kubernetes tsunami is uncontrollable even by them. So you have to surf on it and avoid to resist. Several storage vendors also were forced to react fast as the choice was still open. Pure Storage acquired Portworx, Hitachi Vantara swallowed Containership, SUSE targeted Rancher Labs, Veeam absorbed Kasten and DataCore made an investment in MayaData. In other words, supporting CSI is a minimum but doesn’t change your life, almost every player does that. On the other hand, not having CSI eliminates the vendor for market vision, anticipation and strategy. All these moves listed above confirmed that large or established vendors had difficulties to manage Kubernetes market fast growing footprint being almost naked for some of them. And it’s still true for many others, just ask them “what is your Kubernetes strategy?” and wait.

This wave is implicitly associated with open source as several initiatives continued to be strong in 2020 like Ceph, Lustre, BeeGFS, MinIO, TrueNAS even Gluster and of course disk file systems like OpenZFS or XFS beyond Linux, KVM and Kubernetes. Just to illustrate the impact of open source, MinIO has 15 million of instances running globally on-premise and in the cloud. Two open source initiatives from 2 hardware vendors: Intel DAOS (Distributed Asynchronous Object Storage) and Seagate CORTX popping up in a few places.

Public cloud pricing continues to be a real red ocean with the best price we found at $0.0045/GB/month and no traffic fees with RStor followed by Wasabi Technologies. Check our recent article on this topic.

SaaS and consumption model accelerated for on-premise solutions meaning that finally differences reside only on the location of the services and on which infrastructure they are running. Also SaaS enterprise applications data protection have seen some growth with players like OwnBackup, Asigra, Datto, Druva, HYCU, Kaseya/Spanning, Zix/CloudAlly or Veritas.

Cloud providers, especially the big 3, strengthened their leadership with larger storage product portfolios than classic storage vendors. Just visit AWS, Azure or GCP product pages and you realize the footprint. And if you add their marketplace, these wide offerings are unbeatable. Some people argue about cloud storage pricing being expensive but put in perspective all advantages to use instantaneously all services, applications and marketplace products. This unity of usage is what make them powerful without doubt. And even if you can find some very good isolated product, let’s say a good compatible S3 storage, being segregated introduces complexity.

Classic server and storage vendors talked for years about hybrid model accepting the erosion of their reserved on-premise landscape. But they didn’t anticipate on-premises solutions like AWS Outposts or Azure Stack or even the recent Anthos support of bare-metal infrastructure. Finally all workloads escape usual vendors control with the capabilities from cloud giants to span clouds – public and private – elect workload anywhere and migrate data on-demand. The only limitation, if we consider this as a limitation, is regulations that limit where data can reside. It explains difficulties for some historical vendors with a clear software shift and the reality of trojan horses strategy. These products landing on-premise by cloud leaders are just good examples of that.

Cloud represents a good example of the need of SDS as the adherence on hardware is completely removed. SDS is well installed and is here to stay as it represents independence for users. We find 4 types with block, file, object or unified SDS. On the block side, Datera and StorPool are interesting.

On the data access part, unified storage had a very good year with BFO – Block, File and Object – combined access methods. Some vendors combined the 3 like Pavilion Data Systems and we saw more classic BF or FO solutions like Infinidat, StorCentric, Dell PowerScale and PowerStore, StorONE, to list a few. S3 confirmed its leadership position being a de-facto standard and we spoke more and more about S3 storage instead of object storage.

Global file services has demonstrated good penetration as well with CTera Networks, Hammerspace and Nasuni essentially.

HPC has penetrated the enterprise with more commercial oriented offerings like WekaIO and high performance NFS like Qumulo and VAST Data started to appear in the I0500 list demonstrating some levels of convergence. On the same online list, we found Intel initiative with its DAOS approach being a storage foundation layer for more rich storage services. It is also represented by the necessity to run AI and ML applications that require new performance levels that eliminate good enough solutions. Nvidia has developed a new method to interact with the storage and reduce latency with its GPUDirect Storage model solving the DMA transfers between GPU memory and storage. As of today, 3 vendors support this officially: DDN, VAST Data and WekaIO.

QLC is here and have several implementations for capacity oriented offering but also coupled with SCM to deliver high performance (DDN, Dell, Hitachi Vantara, HPE, IBM, Huawei, Infinidat, Lenovo, NetApp, Pavilion Data Systems, Pure Storage, StorCentric, StorONE, VAST Data, etc.).

NVMe and its network companion NVMe-oF entered a new maturity plateau adopted by all the industry in different transport flavors (Lightbits Labs, Pavilion Data Systems, Qumulo, DDN, Kaminario/Silk, StorCentric, Infinidat, StorONE, NetApp, Dell, HPE, IBM, etc.).

This triggers also disaggregated architecture that decouples compute and storage layers and the obvious capability to scale these independently of each other. In that category, DriveScale and Liqid made significant progress. 

Persistent Memory or Storage Class Memory essentially with the most visible one Intel Optane.

Tape with LTO-9 after a long wait but disappointing on the capacity aspect but some good news came from recent IBM and FujiFilm with some research around large capacity and 580TB on a single cartridge. To reach this new record, it implies beating past areal density numbers and this time the number is 317GB/in². It was done on a prototype strontium ferrite (SrFe) particulate magnetic tape developed by FujiFilm. By comparison, it represents 27x more than the areal density used in current state-of-the-art commercial tape drives we can buy for our data centers. Lots of expectations here.

More and more tiering/migration/HSM like techniques were adopted as we all realize the benefits of cloud, S3 and tape storage representing real savings. We saw some S3 to tape library offerings like a modern VTL if I can suggest this analogy.

New players appeared like Nebulon or Fungible with their dedicated data services board that offloads central processors.

As we don’t anticipate any slow-down on the data volume growth, data reduction, advanced protection and energy savings are key topics especially as users accumulate data on their private cloud or for cloud providers. Some research teams and companies unveiled new methods to reduce data footprint with compression and de-dupe but also new erasure coding techniques. On the third dimension, the energy savings, tape is difficult to beat as a passive media but need to be coupled intelligently with fine grain content indexing and flash caching. Archiving on flash could be seen as provocative but the footprint is very small, energy consumption is good and on-demand restart time is adequate except the price… today.

Storage unicorn club lost a few members with Datto, owned by private equity firm Vista Equity Partners, who did an IPO in October with a current market capitalization of $4.5 billion, Actifio swallowed recently by Google and Datrium acquired by VMware at a crazy price.

Snowflake did an incredible IPO last September with current market capitalization of roughly $100 billion, we list them here even if the company is not a storage player leveraging cloud object storage to deliver its cloud data warehouse approach. We can even list also Sumo Logic (NASD: SUMO) for its IPO as well last September.

Some CEO moves occurred like at Nutanix, Igneous, Infinidat, Panzura or Hitachi Vantara. We clearly hesitate to mention Nutanix as it’s not a storage vendor but more a HCI one and we all know that HCI includes storage, network and host applications being a convergence platform. Rubrik finally just acquired the Igneous’ assets. Panzura CEO left following the acquisition by the financial firm Profile Capital Management.

We must mention new VC rounds with for instance $479 million for Snowflake, $250 million for Cohesity, $125 million for Qumulo, $100 million for VAST Data, $50 million for OwnBackup, $37 million for Firebolt, $34 million for SoftIron, $9 million for Model9 and a mysterious round for Infinidat.

Some considerations happened as well triggered or accelerated for some of them by the Covid-19. Normal companies became fragile and fragile ones became on the edge and even got bankrupt. Some companies who exist for more than 10 years, not yet profitable, continue to raise money to survive, and some VCs seem to believe – again – in miracles. This situation generated some opportunities as some companies represent now real bargains, and some potential acquirers love that… We determined around 20 M&As this year for a total value of $100 billion as we include large ones sometimes considered at the edge. But these movements illustrate the business climate and exit opportunities.

By transaction value descending order when we know it, then no specific order:

• Nvidia acquired ARM for $40 billion, completed Mellanox acquisition launched in 2019 and swallowed SwiftStack,
• AMD acquired Xilinx for $35 billion,
• Marvell got Inphi for $10 billion,
• SK Hynix bought Intel memory business for $9 billion,
• Insight Partners acquired its portfolio company Veeam for $5 billion, and Veeam itself acquired Kasten for $150 million, an other Insight portfolio company,
• Pure Storage acquired Portworx for $370 million,
• VMware acquired Datrium,
• Quantum acquired the ActiveScale product line form Western Digital and more recently Square Box Systems for its CatDV software,
• Rubrik acquired Igneous assets,
• Google acquired Actifio,
• Profile Capital Management acquired Panzura,
• StorCentric acquired Violin Systems,
• Zix acquired CloudAlly,
• SUSE acquired Rancher Labs,
• NetApp acquired Spot, Talon Storage and CloudJumper,
• OVHcloud acquired OpenIO and Exten Technologies

Also some other projects confirmed their interest like DNA storage with players like Twist Bioscience or Catalog. And more recently a DNA Alliance was founded by Microsoft, Western Digital, Illumina and Twist Bioscience plus normal members like Ansa Biotechnologies, Catalog, The Claude Nobs Foundation (Montreux Jazz Digital Project), DNA Script, EPFL (École Polytechnique Fédérale de Lausanne), Cultural Heritage & Innovation Center (Montreux Jazz Digital Project), ETH Zurich, The Swiss Federal Institute of Technology in Zurich (Switzerland), imec, Iridia, Molecular Assemblies and Molecular Information Systems Lab at the University of Washington. DNA storage seems to be hot, since January 1, 2020, the Twist Bioscience Nasdaq ticket has been multiplied by 7 in other words +600%.

In the following quarters, we’ll scrutinize some companies in their respective domain of excellence: Atempo, AWS, Backblaze, Catalog, Cohesity, Cloudian, CTera Networks, DDN, Dell, Fungible, Google Cloud, Hitachi Vantara, HYCU, IBM including Red Hat, Infinidat, Intel, Lightbits Labs, Micron, Microsoft Azure, MinIO, Nasuni, Nebulon, NetApp, Nvidia, Pavilion Data Systems, Pure Storage with Portworx, Quantum, Qumulo, Robin.IO, RStor, Rubrik, SoftIron, Spectra Logic, StorCentric, StorONE, StorPool, VAST Data, Veeam, VMware, Wasabi and WekaIO.