R&D: Addressing the Data Storage Crisis – Open Access Article

Communications of the ACM has published an article written by Don Monroe, science and technical writer, Middlebury, Middlebury, Vermont, USA.

Abstract: “Our increasingly digitized world is creating more data every year, including videos from ubiquitous smart phones, observations from billions of sensors and surveillance cameras, output from by artificial intelligence, and much more. Until now, exponential growth in data storage capacity has largely kept pace with the flood of data at a steady cost.“

“This trend may not continue, said John Monroe at Furthur Market Research, who for almost 25 years, until 2022, was an analyst at Gartner. “We’ve just spoiled all the users: Storage is infinitely there and always cheaper, year on year. That isn’t necessarily going to be the case in the near-term future.”

“Hard disk drives (HDDs) have been critical for keeping pace with exploding demand, along with their solid-state equivalents (SSDs), which are widely used for “warm” data that needs to be accessed frequently. The costs of these solutions are no longer falling quickly enough, however. In 2020, Monroe (no relation to this author) produced a Gartner report that projected a growing storage gap this decade, one that will demand “new technologies that can deliver millions of enterprise-grade petabytes at costs approaching $0.001 per gigabyte.”

“In the near term, magnetic tape is likely to be the best option to fill the gap. But much of the growing demand is for long-term archival storage of “cold” data that will be retrieved only rarely – or never. Archiving such data demands long-term robustness and low upfront and maintenance costs.“

“To this end, researchers are exploring techniques such as laser writing of inorganic films or bulk modification of fused silica. Vastly denser long-term storage, however, could potentially be achieved with organic molecules, especially DNA, which can stably store genetic information over centuries. Biology tools for cheaply reading DNA sequences have already vastly improved. More recently, investigators have adapted microelectronics techniques to encode information in the molecules quickly and cheaply, but the techniques are still at an early stage.“