Leverage SFTP Support for Azure Blob Storage to Build Unified Data Lake

By Yash Lunagaria, product manager, Azure, Microsoft Corp.

We are announcing that SSH File Transfer Protocol (SFTP) support for Azure Blob Storage is available. SFTP support for Azure Blob Storage is a managed, scalable SFTP service that enables simple, secure, and easy-to-manage file transfers. This empowers you to modernize your data transfer workflows and eliminate data silos.

The addition of SFTP to Azure Blob Storage, our object storage platform, expands on our vision of multi-protocol access and enables you to run your SFTP workloads with minimal management effort and low infrastructure costs. SFTP support, combined with protocol support for NFS 3.0, Blob REST, and Azure Data Lake Storage, helps customers migrate their applications without any changes. Building on top of the Blob Storage foundation also allows SFTP-enabled accounts to inherit the security, durability, scalability, and cost efficiency of Azure Blob Storage.

This feature is a 1-click enablement solution to transfer files to and from object storage using SFTP without having to monitor or maintain the underlying infrastructure. Customers no longer need to spend resources and time to deploy, manage, scale, and maintain VM–based SFTP servers.

During our public preview, thousands of customers from various industries such as consulting, retail, healthcare, telecom, financial services, and governments have embraced this feature and are eager to deploy their workloads in production. These customers have been using SFTP for a variety of data transfer scenarios such as exchanging data with customers and partners, modernizing legacy data workflows, syncing data across on-premises and cloud, and collecting data from nodes in a network to unlock insights via a unified data lake.

Manage hybrid workloads using SFTP support for Azure Blob Storage
AT&T, a large telecommunications company, has a goal to move the majority of its applications to the cloud using private networks. As an organization, it wants to modernize legacy apps and adopt a hybrid architecture where some critical applications are running on-premises and some on Azure.

“As a part of its hybrid architecture, AT&T transfers data between on-premises to Azure and one of the primary methods used is SFTP. Now, instead of creating, maintaining, and patching VMs to keep an SFTP service running, AT&T leverages Blob SFTP to eliminate these repetitive tasks. By providing one-click enablement to create an SFTP endpoint for our Blob Storage accounts, Azure abstracts the infrastructure complexity and provides a highly available SFTP service. The cherry on top of the cake is Local Users, a lightweight identity that complements SFTP, which is very easy to set up, manage, and allows granular permission setting at container level,” said Chirag Choksi, principal software engineer, AT&T.

Unlock insights via unified data lake
Kraft Heinz is an American multinational food company with many beloved brands across the globe. It has a deep partnership with Microsoft and Azure is its preferred cloud platform for various IT modernization, digital transformation, data science, and analytics workloads that drive its business forward. Its data analytics processes involve collaborating with multiple partners and vendors who share data that needs to be ingested into their ‘Enterprise Data Warehouse’.

“Managing data sharing pipelines with a wide range of data providers, partners, and retailers is extremely complicated and becomes messy if many different tools are used. SFTP is the common denominator that helps us exchange data in a scalable and secure manner with all collaborators, but we have been incurring tech debt by managing our own SFTP VM servers that require constant maintenance. With SFTP support for Azure Blob Storage, we can easily enable an SFTP endpoint for our data lake for both inbound and outbound file transfers without compromising security or creating additional tech debt. This frees up valuable resources which were previously used to maintain our own SFTP servers. Most importantly, it allows us to create a unified data lake that can be used to generate business insights,” said Ashish Agrawal, director, cloud engineering, Kraft Heinz Company.

Migrate business-critical applications to cloud
SNCF Réseau is the French railway network management company that orchestrates traffic on more than 28,000 kilometers of railway lines. It is responsible for 5 million passengers and 250,000 tons of freight every day and strives to provide cost-effective and environmentally friendly mobility solutions. To modernize its billing application, it utilizes SFTP support for Azure Blob Storage.

“SFTP servers enable SNCF to communicate data between Azure cloud and their on-premises data centers. In SFTP support for Azure Blob Storage, SNCF found the perfect fully managed, highly available, massively scalable SFTP PaaS that vastly simplified our data transfer workflows. Blob Storage’s native SFTP solution decreases maintenance overhead, freeing up resources that enable SNCF to focus on their goal to innovate and enrich the lives of millions of travelers,” said Taij Triki, solution architect, SNCF.

Get started
Accelerate migration to the cloud for SFTP workloads with SFTP support for Azure Blob Storage!
Get started by checking out the introductory video and reviewing how to connect to Azure Blob Storage using SFTP.

Resources:
Optimize performance with guidance in performance Considerations for SFTP in Azure Blob Storage.
Assess limitations of known issues with SFTP support for Azure Blob Storage.
Verify host keys by referring to host keys for SFTP support for Azure Blob Storage.