Market of Cloud, Hyperscale and Enterprise Storage to Reach $17 billion in 2018

HDD sales to Cloud, Hyperscale, and Enterprise markets will reach $17 billion in 2018 according to a new Trendfocus, Inc.‘s research report, Nearline HDD Long-Term Market Analysis.

A comprehensive SSD report will be available in April 2014.

In the midst of skyrocketing exabyte demand growth, companies including Google, Facebook, and Amazon are redefining traditional methods of storage deployment.

“Skyrocketing demand for storage is shifting the foundation of the industry,” states Mark Geenen, president and founder, Trendfocus.

He says that HDDs will remain the primary bulk storage medium through 2018 while NAND and SSD technologies enable higher performance in critical areas.

John Chen, VP, Trendfocus’ lead analyst for the new service, adds: “For traditional OEMs and the emerging class of ‘do-it-yourself’ (DIY) and commodity hardware data centers, HDDs are the lowest cost technology on a per gigabyte basis and will comprise more than 90% of the enterprise storage exabytes shipped through 2018.”

Traditional storage and server OEMs are also responding to the pressure from a new wave of ‘ODM (Original Design Manufacturer) direct,’ or DIY hyperscale system architectures promoted by companies such as Google, Facebook, and Amazon.

“The entrenched suppliers are responding with their own solutions, promising intensified competition and redefining both HDD and SSD storage solutions. The competition between traditional OEMs and emerging players is driving a revolutionary change in device designs that will provide significant growth opportunities for both HDDs and flash-based storage over the coming decade“, added Chen.

This new market research service includes long-term and quarterly forecasts of HDDs and SSDs used in cloud, hyperscale, and enterprise storage architectures, as well as analysis of technology and market trends.

Preview of Trendfocus’ Cloud, Hyperscale, & Enterprise Storage Service

Data Generation Fuels New Markets, Applications forHDDs and SSDs

The first sub-segmentation analysis of traditional OEM storage and ODM direct hardware sectors and the impact on HDDs, SSDs, and the entire storage value chain – in Trendfocus’ detailed fashion

Key Features
The service tracks, forecasts and analyzes the enterprise storage demand evolution currently underway. Not only will workloadspecific storage devices emerge from this evolution, but also differences in adoption rates of these new devices by the traditional system OEM market versus the ODM direct market market will become apparent over time. This service will track capacity and workload trends as well as storage device costs for each market, resulting in a detailed revenue and ASP picture that will shed new light on the growing, yet diversifying enterprise storage market.

Chart 1: Total Nearline (NL) HDD Unit Shipments by Market

This preview of the Cloud, Hyperscale & Enterprise Storage Service highlights Trendfocus’ first nearline HDD market sub-segmentation analysis and looks at the following:

• 1. Nearline HDD adoption by traditional enterprise OEMs and ODM direct markets (Chart 1)
• 2. Revenue by nearline HDD market and type (Chart 2)
• 3. Disk-per-drive ratio for all nearline HDDs (Chart 3)

Chart 2: Total Nearline HDD Revenue by Market and Nearline HDD Type

Chart 3: Total Nearline HDD Disk per Drive Ratio (Normalized), Media Forecast

Background
Over the past two decades, enterprise datacenters have adopted a configuration of dedicated servers running specific applications connected to networked storage – either NAS or SAN. Networked storage systems provided a large amount of scalable storage available to numerous remote servers, fueling the evolution away from DAS .

Over the past decade, however, large internet-based services have sought out more efficient methods of scaling out large numbers of servers and storage as demand for services grew exponentially. At the same time, virtualization enabled greater utilization of physical server hardware, enabling multiple instances of servers (and their associated applications) to be deployed on each physical piece of hardware. Servers require some local storage to provide higher speed data access, but by virtualizing servers, the need for local storage has also expanded greatly. In addition, the speed of SSDs and flash accelerators is maximized by placing those devices in or attached directly to the server through an expansion shelf. The need for more local storage has caused the re-emergence of DAS in an architecture that turns out to be highly scalable and efficient.

The term coined for this easily scalable architecture is ‘hyperscale’ and it has transformed datacenters, especially in the public cloud space. Large internet service companies, such as Google, Microsoft, Amazon and Facebook may offer similar or differing services, but the key to their ability to scale has been the adoption of hyperscale architectures.

Hyperscale architectures for many internet and public cloud services include building out only a few server types, tailored for specific workloads. Part of the key to scalability for these companies is to deploy common servers/storage racks in volume as end-user demand requires. The relatively few hardware types in most hyperscale datacenters has driven at least three large internet companies – Google, Amazon and Facebook – to design their own hardware and software, bypassing traditional server and storage OEMs and relying on ODMs or contract manufacturers to assemble, integrate and test equipment. These do-it-yourself, or DIY, systems have lowered hardware costs for the companies that can support the hardware and software needs in-house without relying on the traditional OEMs such as EMC, NetApp, HP and Dell, among others. Hyperscale architectures typically rely on replication where data resides in multiple copies across several servers, potentially in geographically disparate datacenters. For individual storage devices – either HDDs or SSDs – write workloads can be significantly lower than in traditional networked storage environments. While both ODM direct hardware and traditional OEMs can design and manufacture systems for the hyperscale, those companies that utilize the former approach have the opportunity to work directly with storage device makers to loosen specifications for well-understood workloads. The goal is to reduce the acquisition costs of the HDD or SSD compared to purchasing the storage as part of an OEM system. OEMs must maintain very tight device specifications in order to ship to numerous customers with highly variable workloads.

Not all hyperscale-type companies have embraced the ODM direct path. Microsoft, as a large example, relies on its partners, namely Dell and HP, for its hyperscale datacenter equipment. Many other companies may shy away from the daunting task of supporting ODM direct hardware and software and will continue to buy from OEMs for good warranties and support.

While most hyperscale datacenters are cloaked in secrecy, Facebook has been open about its hardware and datacenter architectures. The social media giant was and remains the main proponent of Open Compute Project – an industry group that seeks to develop specifications for open-sourced server, storage and datacenter designs. Open source software such as OpenStack promise to complete the picture, possibly resulting in the ability for new hyperscale players to adopt these commodity hardware and software platforms.

HDDs and SSDs designed specifically for less-demanding write workloads can be designed for both ODM direct hardware as well as OEMs deploying hyperscale servers and storage. This has resulted in the HDD market launching workload-limited nearline HDDs with write workloads of only 180TB per year, instead of the 550TB per year of conventional nearline models. SSD vendors are also busy launching more read-intensive SSDs that can utilize lowerendurance NAND and thus lowering cost.

In the HDD space, cold storage provides another tier of nearly write-once devices that provide the absolute lowest cost per gigabyte of any enterprise grade HDD, while still designed to survive operations where dozens, if not hundreds of HDDs occupy datacenter racks. Purpose-built cold storage HDDs are still about a year off, due to the hugely varied specifications requested by different customers, but eventually, these cold storage HDDs will enable cost-reducing archival storage with access times that may obviate the need for tape in some circumstances.

Service Timeline
In March 2014, Trendfocus will publish its latest five-year long-term forecasts (2013 to 2018) for the entire HDD and SSD markets. Following that publication, the first full Cloud, Hyperscale, & Enterprise Storage Service publication will update the nearline HDD sub-segmentation forecast and will provide the following details:

• 1. Storage device forecasts by traditional OEM and ODM direct hardware markets and nearline HDD type (standard nearline, workload-limited and cold HDDs)
• 2. Forecasts by units, revenues, EBs, $/GB
• 3. Media and head forecasts for each nearline HDD type
• 4. Commentary and analysis of the hyperscale market by traditional OEM versus ODM direct hardware markets

In late CQ1 ’14, the service will analyze and forecastenterprise SSDs split by the same market segments as in nearline HDDs (traditional OEM versus ODM direct hardware), but will also include the SSD-specific details including:

• 1. SSD forecasts by units, revenues, EBs, average $/GB
• 2. Market segment forecasts by SSD interface type – SATA, SAS, PCIe
• 3. Impact of interface standards adopted by the market
• 4. NAND flash technology and process node transitions impacting each SSD type and market segment requirements

Each year the service deliverables will include:

• 1. A detailed five year long-term forecast published twice per year for both HDDs and SSDs in the enterprise space
• 2. Quarterly shipment, revenue, $/GB, and EB details by market and device type
• 3. Periodic reports on major developments in the enterprise server, storage and hyperscale markets that may influence the direction of storage devices

Who Should Subscribe?

• Server and storage OEMs interested in the competitive headwinds caused by the ODM direct market and HDD/SSD cost trends
• Companies deploying hyperscale datacenter architectures seeking insight on workload-specific storage device trends in both the OEM and ODM direct markets
• Storage device companies looking for independent analysis of the adoption of OEM versus commodity hardware, interface trends and the emergence of the cold storage market
• Component companies searching for actionable data on ‘data creation’ and ‘the nearline market’ based on Trendfocus’ knowledge of technology transitions and market demand for both HDDs and solid-state storage