EMC Unveiling ViPR Software-Defined Storage Platform

At EMC World 2013, EMC Corp. introduced ViPR, a software-defined storage platform.

As service provider and enterprise data centers grow to tens and hundreds of petabytes, the cost and complexity associated with managing them becomes untenable. By using ViPR to manage storage infrastructure and the data residing within that infrastructure, organizations can both drive groundbreaking improvements in automation and lay down a modern storage architecture for future application deployments. Using ViPR, service providers and IT departments can drive towards the operational model of web-scale data centers without hiring thousands of technical experts to build a custom environment.

ViPR can:

• Manage both storage infrastructure (called the Control Plane) and the data stored within that infrastructure (called the Data Plane).
• Decouple the Control Plane from the Data Plane, allowing the use of both together – or enabling customers to use only the Control Plane to manage the underlying intelligence of the storage arrays through policy-based automation. This is a departure from prior attempts to virtualize storage.
• Offer the ability to view objects as files and provides file access performance without the latency inherent to object storage.
• Provide the capability to be implemented entirely in software and will run against EMC, non-EMC and commodity hardware.

ViPR – The Control Plane
The ViPR Controller (the Control Plane, which manages the storage infrastructure) can deliver improvements in automation because it virtualizes the underlying storage infrastructure. Common storage management functions, like provisioning or migration, are abstracted so that different storage arrays can be managed as a single pooled resource in exactly the same way. Now for management of these highly complex functions, ViPR delivers a single point-and-click approach. (This is analogous to using a universal remote control in your living room to operate your TV, DVD, streaming device and DVR). Customers no longer need to engage in the complex and all-consuming task of interacting with multiple management interfaces that permeate heterogeneous storage arrays.

Once created, these pools of storage are carved up for consumption by applications. For this task, ViPR provides a self-service portal so application owners can browse the storage service catalog and provision service resources best suited for their needs. This provides IT departments with the frictionless experience the application teams are looking for – and have become accustomed to – in public cloud environments.

For most traditional storage infrastructures, ViPR will purely provide the Control Plane. It discovers storage, creates virtual storage pools, provisions those pools to the application and then gets out of the way; it is not in the data path – that is left to the array itself. In the same way that OSs are smart about detecting the presence of things like graphics co-processors (and offloading processing), the ViPR Controller is smart about what it finds in the underlying storage infrastructure. If it can offload processing to the underlying array and leverage the intelligence there, it will.

ViPR Data Services – The Data Plane
For traditional workloads that utilize file and block, ViPR steps out of the way and lets the underlying array fulfill the role of Data Plane, or the data stored within the storage infrastructure. This represents the majority of application workloads within the data center and EMC estimates those workloads will grow approximately 70% by 2016. But new application workloads are emerging – often operating on vast quantities of data (Big Data) and servicing tens of thousands or millions of users. EMC estimates these workloads will grow approximately 700% by 2016 – an order of magnitude more than traditional.1  

ViPR is able to view objects as files, which provides file access performance without the latency inherent to object storage. In addition, ViPR HDFS Data Service enables customers to perform in-place analytics across the entire heterogeneous storage environment. As a result, the all-consuming task of managing heterogeneous storage environments melts away, leaving IT with the ability to focus on supporting business priorities and innovating.

Because of the nature of these new applications, they are being architected in new ways. The massive scale required is dictating a simpler approach to storage infrastructure – specifically object storage. Access methods are also changing from traditional protocols such as NFS and iSCSI to new protocols such as HDFS, made famous as the underlying foundation of the Hadoop database. To support these new application architectures ViPR providers Object Data Services.

The ViPR Object Data Services will provide Amazon S3 and OpenStack Swift compatible REST APIs and HDFS access methods – existing software applications written to these APIs should run seamlessly. ViPR Object Data Services will support existing EMC Atmos, EMC VNX and Isilon arrays as a persistence layer in addition to third party arrays and commodity hardware. This enables the customer to determine, based on the application workload, what the suitable performance, QoS and cost characteristics should be. Over time, EMC expects to provide a broader set of data services as a part of the ViPR platform.

Integral Part of Software-Defined Data Center

With ViPR, EMC is doing for storage what VMware has already done for servers – abstracting, pooling and automating the infrastructure. Pools of storage created with ViPR will leverage VMware APIs to simply appear as an array within vSphere. In addition, the ViPR Controller will integrate with the overarching VMware SDDC management and orchestration tool vCloud Automation Center as well as vCenter Operations Manager. In this way, storage can be managed by ViPR as an entity in its own right (and surfaced into the Microsoft and OpenStack virtual environments) or as an integral part of the VMware Software-Defined Data Center.

Design Point: The Cloud
ViPR is designed for cloud environments and specifically service providers, however it is also suitable for the enterprise IT departments that are transforming themselves to offer IT-as-a-service and building out internal web-scale clouds. With this in mind, ViPR has a globally distributed architecture – enabling IT departments to avoid moving massive amounts of data across the network and perform functions like ‘in-place’ analytics. The platform scales out as devices and data grows with no single point of failure and provides an entirely self-managing and self-provisioning environment.

Amitabh Srivastava, president, advanced software division, EMC, said: "Building the web-scale data center is critical for service providers and large enterprises. The rise of the Software-Defined Data Center is a groundbreaking step toward delivering the management and performance capabilities needed to protect and leverage data. Only by separating the data center from its underlying hardware can IT truly deliver resources as customizable, on-demand services. As the only solution on the market today, ViPR is able to support IT services in a heterogeneous storage environment while retaining and extending the value of underlying arrays. This is a game-changer for storage."

Raghu Raghuram, EVP, cloud infrastructure and management, VMware, said: "Through the software-defined data center, VMware is extending the benefits of virtualization to all areas of IT infrastructure beyond compute to networking, security, management and storage. With EMC ViPR and VMware solutions, organizations are able to get more value, automation and simplicity out of their storage infrastructure through integrated systems and software innovation."

Gary Budzinski, EVP and GM, global infrastructure services, Computer Sciences Corp., said: "EMC’s new ViPR offering is key to how the storage industry and enterprises will evolve to a software-defined data center. By moving to an IaaS model and virtualizing storage, network and compute environments, CSC has the ability to bring new business value to our clients including faster time to market, reduced complexity and increased innovation. The industry, led by early adopters, will be able to benefit from unprecedented resource utilization through automation and management of resources across heterogeneous enterprise environments."

Vernon Turner, SVP, infrastructure research group, IDC, said: "With the unveiling of ViPR, EMC is sending a clear message that the combination of arrays with a powerful software layer is unbeatable in terms of speed and simplicity. Customers want to extract more value from their storage investments while scaling back on management, and ViPR meets these needs while embracing open architecture and catering to all arrays."

ViPR will be available in the second half of 2013.