Microsoft Ignite 2024: HPE Unveils Azure Hybrid Cloud Infrastructure Solution for AI Analytics

By Mike Harding, manager, product management, Microsoft Storage solutions, Hewlett Packard Enterprise LP

HPE is previewing a new SQL Server hybrid cloud analytics for AI solution at the Microsoft Ignite 2024 event. It’s built around an on-premises SQL Server 2022 environment that connects with on-prem unstructured data lakes as well as cloud-based sources, then leverages Azure-based services such as Microsoft Fabric for analytics. This means you now have a new way to leverage your on-prem data estate to accomplish valuable AI projects.

Organizations everywhere are investing in AI to gain competitive advantage and reduce operational costs. The Microsoft solution team in the HPE Hybrid Cloud group is sharing a preview of work on optimized infrastructure for SQL Server within an Azure hybrid cloud configuration for running analytics to support AI initiatives. You’ll get the chance to see this in the HPE booth at this year’s Microsoft Ignite show in Chicago, IL, November 18-21.

Solid foundation for hybrid cloud AI analytics
The solution being shared at Ignite uses SQL Server 2022 on premises along with an unstructured data storage environment. SQL Server remains the main instance and point of control, and enables pulling in data from attached data lakes including S3-compatible object storage, for executing integrated analytical queries. That output in turn would then be used with Azure cloud-based resources and services including Microsoft Fabric.

The key value of this approach is that SQL Server customers can avoid the higher and tougher-to-forecast cost of cloud-based data storage – while keeping the aility to access valuable Azure-based resources by leveraging on-prem unstructured data through this SQL Server 2022 solution. This approach parallels work demonstrated by Microsoft at this past HPE Discover 2024 in Las Vegas, NV, which showed hybrid cloud analytics with Azure SQL S3-compliant object storage.

Hybrid cloud analytics builds on a hybrid cloud infrastructure to enable the mixing of on-prem data with cloud-based analytic capabilities, providing you with the optimal mix of cost efficiency, control, and analytics capabilities. It’s a preferred alternative to going either completely in cloud or completely on prem. It gives you the flexibility to work with data regardless of whether it originates – in the cloud, on prem, or somewhere in between. Our work here expands on earlier publications that detailed how to ‘arc-enable’ SQL Server on Windows Server, which provides a relatively easy way to connect on-prem instances to the Azure cloud. 

In this latest project, our intent is to demonstrate how the on-premises data, both structured and unstructured, can be used with cloud resources to enable AI efforts. We expect customers to have the need to use this data in 1 or more of these areas including:

• AI training – using a public cloud’s computing power to train AI algorithms
• AI workload management – managing AI workloads with data storage and computing power across the hybrid cloud infrastructure  
• AI deployment – deploying AI at scale, across the hybrid cloud, leveraging critical data wherever it’s located

Infrastructure making the analysts job easier
In this solution, SQL Server on-prem serves as the master instance providing a single entry point for developers, BI analysts, and data scientists to use for primary access to data. From there they can link to external sources, whether an on-prem data lake, other on-prem business databases, or edge or cloud-based data sources.

Specific solution components include: 

• Microsoft Fabric – Azure cloud based, end-to-end analytics and data platform designed for enterprises. It offers a comprehensive suite of services including data egineering, data factory, data science, real-time analytics, data warehouse, and databases. Customers are using it as the newest way to create and manage AI models, reducing the time data scientists need to deliver value, versus legacy approaches.
• SQL Server 2022 on HPE Storage – HPE has a broad set of SQL Server solutions, with the highest levels of all-NVMe flash performance and data availability up to 100% guaranteed which allows the company to stand out in the SQL Server Infrastructure market.
• Storage for unstructured data – This solution is designed to work with on-prem S3-compliant data storage systems containing unstructured data. This could include enterprise data lakes, digital repositories, and data used for AI. The primary benefits should be around the lower cost and improved control of on-prem Object storage for less performance-oriented use cases like database copies for related ETL and analytics. Look for new upcoming developments in this area from HPE.
• GreenLake – The GreenLake approach is to not just provide the best infrastructure for enterprise computing, but to provide it as an IaaS solution, that is easy to order, sold with and through IT partners, that provides the fastest time to value with a turn-key deployment experience, and is structured so customers only pay for what they use.

With this solution, the SQL Server user (developer, BI analyst, data scientist) can operate from the familiarity of SQL Server, accessing external data sources on-prem or in the cloud, and leveraging cloud-bases analytics services.

It’s all about your connections
You can achieve SQL Server hybrid cloud in multiple ways, depending on the source of the data, and the targeted services. This solution employs the use of mirroring to connect the SQL Server instances to Azure. It takes an on-prem SQL Server and mirrors a database based on change feeds to the fabric, implemented through Azure Arc. Database Mirroring in Fabric (currently in preview) is a low-cost, low-latency solution to bring data from various systems together into a single analytics platform. It replicates databases to fabric with zero ETL and no additional licensing cost. Data is replicated into OneLake and is kept up to date in near real-time. Mirroring protects performance impacts to operational databases from analytical queries.

As described earlier, the object storage serves as an S3 complaint bucket that can be a data lake where unstructured data is stored from various sources that could include CSV, parquet, delta, text. Connectivity between SQL Server (2016 and after) and the object store is via Polybase. PolyBase enables the SQL Server instance to query data with T-SQL directly from the S3-compatible object storage without separately installing client connection software. Additional sources can be other SQL Server databases, Oracle, Teradata, MongoDB, Hadoop clusters, and Cosmos DB. The PolyBase feature enables ‘data virtualization,’ allowing the external data to stay in its original location and format, but it can be queried like any other table in SQL Server. This minimizes the need for ETL processes for data movement.

From the object store to Azure, the means to connect is via S3 shortcuts by creating ‘links’ or a shortcut to an S3-compatible object provider. In this case it’s to the on-prem object store. Shortcuts can also be used to connect to external data sources such as other object stores (e.g. Scality) or even Amazon.

This solution has the SQL Server running on an arc-enabled Windows Server. Azure Arc-enabled servers – physical servers and VMs – can be managed through the Azure portal as if they were native resources. It allows management of all these hybrid resources together, the ability to use Azure services on them, and uniformly apply policy for monitoring, security or governance. And in case you haven’t noticed, there’s a new version out, Window Server 2025, with enhanced hybrid cloud features and more, which you can read more about in the Microsoft preview announcement blog.

See hybrid cloud analytics for AI in action at Ignite
This new SQL Server hybrid cloud analytics for AI solution will be previewed at the HPE booth at the Microsoft Ignite 2024 event, with more formal documentation and recorded demos forthcoming after the show. If you’re not planning to be in Chicago for Ignite, no worries – you can also visit the HPE Ignite sponsor page to watch our event solution video for a glimpse of the solution, as well as see what else is new for Microsoft workload solutions from HPE. With this latest workload optimized SQL Server solution, organizations have a new way to leverage on-prem data to accomplish valuable AI projects.

Resource:
HPE Ignite sponsor page to watch our event solution video and see what else is new for Microsoft workload solutions from HPE.