WANdisco LiveData Migrator for Data Lake Migration to Cloud

WANdisco plc. announced LiveData Migrator, an automated, self-service solution that democratizes cloud data migration at any scale by enabling companies to start migrating Hadoop data from on-premises to Amazon Web Services (AWS) within minutes, even while the source data sets are under active use.

Click to enlarge

Available as a free trial for up to 5TB, businesses can migrate HDFS data without the expertise of engineers or other consultants – the program can be implemented immediately to enable companies’ digital transformations. LiveData Migrator works without any production system downtime or business disruption while ensuring the migration is complete and continuous and any ongoing data changes are replicated to the target cloud environment.

It allows businesses to migrate HDFS data without the expertise of engineers or other consultants to immediately enable companies’ digital transformation.

“Through 2022, more than 50% of data migration initiatives will exceed their budget and timeline – and potentially harm the business – because of flawed strategy and execution,” according to Gartner in a recent report. ⁽¹⁾ This is despite the fact that nearly 70% of enterprises have started moving data to the cloud as part of their digital transformation.

Migrating large data volumes with traditional approaches, such as transfer devices or DistCp (distributed copy), requires disrupting the operation of on-premises applications and doesn’t cater to data that is modified or created during migration. Reconciliation at scale is costly and cannot guarantee a completely consistent data outcome. The overhead required to achieve non-disruptive, no-downtime big data migration is significant due to repeated scans, systems out of sync and manual intervention for anticipated failures and interruptions.

“Enterprises that want to move their data to the cloud but are concerned about the risks of doing so now have a powerful solution that’s self-service and extremely easy to use,” said David Richards, co-founder and CEO, WANdisco. “Regardless of company size or technical expertise, LiveData Migrator enables businesses to migrate their data risk-free to the cloud on a massive scale without any disruption to business operations. Data can be accommodated without any risk of data loss while modernizing data and applications to stay competitive.“

GoDaddy, Inc. is using LiveData Migrator to migrate 500TB of HDFS data to Amazon Simple Storage Service (Amazon S3). It will operate in a hybrid cloud mode and continue to update S3 data as changes are made to Hadoop content in the source, so that it is available for cloud analytics using Amazon EMR. The first 70TB were migrated within days with zero on-premises system downtime.

“At GoDaddy, deep technical knowledge is in our DNA, and we often build applications in-house to support growth,” said Wayne Peacock, chief data and analytics officer, GoDaddy. “In the use case of a Hadoop to Amazon S3 data migration and replication, we found WANdisco’s LiveData Migrator to be the optimal approach to deliver the best time to value, rather than running a more time-consuming and costly manual migration project internally. With LiveData Migrator, we have completed the first phase, giving us a quick win by fully automating the HDFS migration of 70TB in five days, avoiding disruption of production processes and making the data available in Amazon S3 for testing.“

The company’s LiveData Migrator eliminates data migration risks with immediate, live and scalable migration. Administrators can move data lake content to the cloud while minimizing IT resource involvement with automated migration capabilities. No changes are required to applications, cluster or node configuration or operation, and there is no production system downtime or business disruption, while ensuring their data changes are migrated continuously and completely.

The company is an Advanced Technology Partner in the AWS Partner Network (APN) and has achieved AWS Migration Competency status with the release of LiveData Migrator. It was one of four APN Partners who collaborated with AWS to define the requirements to cover different use cases, including Hadoop, storage and database data migration, and mainframe data integration.

“We are delighted to have APN Partners like WANdisco that exhibit extensive knowledge and proven expertise in migration and replication technology,” said Sabina Joseph, director, Americas ISVs, Amazon Web Services, Inc. “LiveData Migrator is designed to offer customers a platform that automates and safely accelerates large-scale on-premises Hadoop migrations to AWS without business disruption. Customers will benefit from optimizing costs and expanding analytics and insights capabilities leveraging industry leading services like Amazon S3, Amazon EMR and Amazon Athena.“

LiveData Migrator gives enterprises the opportunity to free their systems from the capital constraints of acquiring storage hardware by leveraging Amazon S3 and associated services. It delivers fast time to value for migrating unstructured data into cloud storage to then take advantage of ML powered cloud analytics such as Amazon EMR, Databricks or Snowflake to gain valuable business insights. It also enables the transition to a hybrid architecture, where on-premises and cloud environments are kept consistent for active-active replication capabilities, and sets the foundation for a future multi-cloud architecture.

LiveData Migrator capabilities:

• Complete and continuous data migration
  Migrates any changes made to the source data sets, allowing applications to continue to modify the source system’s data without causing divergence between source and target.

• Rapid availability
  Enables data to become available for use in the target environment as soon as it has been migrated, without having to wait for all data set migrations to complete.

• Any scale
  Migrates any volume of data, from terabytes to exabytes, to cloud storage without needing to stop changes to data at source during migration

• Hadoop and object storage conversion
  Migrates HDFS data to other Hadoop-compatible file systems and cloud storage, including the ongoing changes made to those data before, throughout and after migration.

• Selective migration
  Allows selection of which data sets should be migrated and selectively excludes data from migration to specific clusters in the new environment.

⁽¹⁾ Gartner, Make Data Migration Boring: 10 Steps to Ensure On-Time, High-Quality Delivery by Ted Friedman, December 13, 2019.

Read also:
WANdisco LiveData Platform for Petabyte Scale Cloud Migration to Microsoft Azure
Automates and minimizes risk of Hadoop migration with expected ROI of 3 months.
June 12, 2020 | Press Release