Evolution of IBM Storage Insights

By Ramakrishna Vadla, storage system management specialist, IBM Corp.

IBM Storage Insights revolutionizes the way enterprise organizations manage and optimize their storage environments by leveraging AIOps and self-healing infrastructure services.

The IBM Storage Insights has been evolved as a storage observability platform providing actionable insights that can be shared with ecosystem integration services. This will enable deeper assessment, correlation, and automation of storage management actions, enhancing overall operational efficiency and proactive issue resolution. This blog explains various ecosystem services such as security, obervability, incident management and productivity services integration with IBM Storage Insights.

IBM Storage Insights APIs
IBM Storage Insights platform APIs support building an interconnected ecosystem such as integrated operational observability dashboards, incident management systems, and even security platforms. The ecosystem services can leverage IBM Storage Insights APIs to consume the day in the following ways,

• REST APIs Support (Pull) – IBM Storage Insights offers a suite of REST APIs that allow seamless integration of storage system, switches and servers observability data – such as configuration, performance, and health metrics – into a variety of external ecosystem applications. (Refer this hyperlink to know more),

• Webhook Support (Push) – IBM Storage Insights allows to configure third-party applications webhook to notify the alerts events data such as ransomware alerts. (Refer this hyperlink to know more),

• Email data ingestion – The alert events data can also be sent via email for ingestion into the data pipeline of ecosystem services.

IBM Storage Insights integration with Ecosystem Services
IBM Storage Insights platform APIs enable building an interconnected ecosystem integration, supporting integrations with security ecosystem services, enterprise observability services, incident management systems, AIOps and GenAI enabled Chatbots, productivity tools and VMWare vSphere plugin. The following section gives the summary of IBM Storage Insights integration with various ecosystem services.

a. IBM Storage Insights integration with Security Ecosystem Services

• IBM Storage Defender
  IBM Storage Defender delivers end to end data resilience by combining backup and storage management solutions from across the IBM Storage portfolio with IBM Business Partner capabilities into one comprehensive package, while still providing the flexibility to choose the capabilities that are right for your enterprise.

  The ransomware alerts generated by IBM Storage Insights Pro for a monitored IBM FlashSystem can be auto forwarded to IBM Storage Defender to trigger cyber resiliency workflows, and hence protect your systems as soon as possible.  For a customer who is having a subscription for IBM Storage Insights Pro and IBM Defender, this will enhance their protection from ransomware attacks.

Refer these hyperlinks to know more: IBM Storage community blog – Double your defense: Protection from ransomware attacks on IBM FlashSystem using IBM Storage Insights and IBM Storage Defender, and IBM Storage Insights docs – Integrating IBM Storage Insights with IBM Storage Defender

• IBM Cloud Pak for Security (CP4S)
  IBM Cloud Pak for Security (CP4S) provides a platform to quickly integrate security tools and generate deeper insights into threats across hybrid and multicloud environments. Threat management capabilities in IBM Cloud Pak for Security deliver robust, open, process-wide visibility, detection, investigation, and response. This platform also offers security, orchestration, automation, and response (SOAR).

  IBM Storage Insights sends email alerts to a specific email address based on ransomware threat events. Further, IBM Cloud Pak for Security detects these alerts and automatically initiates an automated response.

Refer to this hyperlink to know more: IBM Storage Insights docs – Configuring IBM Cloud Pak for Security for real-time ransomware threat alerts

b. IBM Storage Insights integration with Enterprise Observability services

• Grafana
  Many enterprise organizations rely on a variety of vendor-specific observability tools, yet Grafana stands out as the preferred solution for an integrated operations dashboard. Its flexibility and compatibility make it the ideal choice for creating a unified, single-pane view across multi-vendor environments. With the IBM Storage Insights REST APIs and a pre-configured Grafana dashboard template, customers can effortlessly integrate monitoring of IBM storage, switches, and servers into a Grafana-based operational dashboard.

Refer these hyperlinks to know more: Opensource plugin link – IBM-storageinsights-ecosystem-grafana Setup Documentation, IBM Storage community blog – Use IBM Storage Insights REST APIs to integrate IBM Storage observability with Grafana, IBM Storage Insights docs – Grafana, and Storage Guru Demo – Video.

• Dynatrace
  Dynatrace is an open and extensible platform. You can extend the observability data collected out of the box with data provided by observability standards and frameworks such as OpenTelemetry and Prometheus. Dynatrace can ingest data from a wide range of tools available in the Dynatrace Hub, enabling continuous, actionable data flows that help quickly resolve issues, maintain complex systems, enhance code quality, and accelerate digital transformation. It acts as a single source of truth for observability.

The IBM Storage Insights Dynatrace Extension 2.0 enables customers to seamlessly monitor the health, capacity, and performance of their storage systems within IBM Storage Insights, utilizing Dynatrace dashboards for comprehensive visibility.

Refer these links to know more: Opensource plugin link, IBM Storage community blog, IBM Storage Insights docs, and Storage Guru Demo video.

•  Prometheus
  Prometheus is a monitoring system that is well suited for monitoring dynamic systems. It features a dimensional data model and a query language. It provides an integrated ecosystem for instrumentation, gathering of metrics, service discovery, alerting etc. With Prometheus client library, we can instrument applications to add and expose Prometheus metrics.
  IBM Storage Insights can export the storage systems performance and capacity metrics collected into Prometheus. The Prometheus UI provides real-time visualization of metrics. We can execute PromQL (Prometheus Query Language) queries directly in the UI to visualize data, examining metrics as they are collected in real-time. This immediate feedback is crucial for observing the state of systems and debugging issues promptly.

Refer these hyperlinks to know more: Opensource plugin link, and IBM Storage community blog – Visualizing Storage Insights metrics on Prometheus.

• OpenTelemetry
  OTel is a standards body that provides a vendor-neutral solution to instrument and collect telemetry data. It provides a reference architecture that can be adopted and extended by others. The OTel specification is large and made up of multiple components. The entry barrier for adopting OTel can be quite high, especially if you’re new to observability or the specific OTel concepts.
  The systems onboarded to Storage Insights produce a vast amount of telemetry data that can be monitored on Storage Insights GUI. We’ll make use of the REST APIs provided by SI to pull those telemetry data and bring them onto the OTel processing pipeline. Since many observability services support OpenTelemetry for data ingestion, we have provided users with a solution to convert IBM Storage Insights REST API data into the OpenTelemetry format, enabling seamless integration with observability platforms.

Refer this link to know more: IBM Storage community blog – OpenTelemetry Collector for IBM Storage Insights Metrics.

c. IBM Storage Insights integration with Incident Management Services

• ServiceNow
  ServiceNow is a platform-as-a-service, that allows for the operation of enterprise and technical management support systems, such as IT service management and help desk functionality that include management of ‘incident, problem, and change’ IT operational events.
  When IBM Storage Insights is integrated with ServiceNow, triggering critical alerts like ransomware threat alerts in IBM Storage Insights automatically creates an incident in the ServiceNow instance.

Refer this link to know more: IBM Storage Insights docs: Integrating IBM Storage Insights with ServiceNow.

• Jira
  IBM Storage Insights identifies incidents across storage systems that it monitors and send alerts to the users. These incidents can range from minor system glitches to critical security vulnerabilities such as ransomware threats. Once an incident is identified, a Jira bug can be created to track and manage it. For example, a potential ransomware attack needs to be triaged by security analysts by analyzing it and correlating the information from different sources.

Refer these links to know more: IBM Storage community blog – Integrating IBM Storage Insights with Jira for Ransomware Alerts, and Storage Guru Demo video

d. IBM Storage Insights integration with AIOps and GenAI Chatbot

• Gen AI Chatbot using IBM Storage Insights APIs
  This is an open-source chatbot created by the IBM Storage Insights team to enable natural language interactions. It enables users to interact & chat with IBM Storage Insights in natural language form to assist in easy observability and monitoring. By leveraging large language models (LLMs) and advanced prompt engineering, the bot intelligently detects user intent, extracts relevant entities, and seamlessly makes IBM Storage Insights API calls to fulfil user queries. Powered by IBM Watsonx and utilizing both Granite and Llama (Built with Llama) LLMs, this chatbot provides a conversational, intuitive approach for managing and accessing storage data and insights.

Refer these links to know more: Opensource plugin link, IBM Storage community blog – IBM Storage Insights open sources Observability Chat bot, IBM Storage Insights docs, and Storage Guru Demo video.

• IBM Cloud Pak for AIOps (CP4AIOps)
  IBM Storage Insights offers real-time visibility into storage health, performance, and capacity, identifying alerts across the systems it monitors. The CP4AIOps, powered by AI, detects anomalies, correlates alerts, and identifies related tickets. Together, these tools enable proactive storage issue resolution, reduce alert noise, and streamline incident response.

Refer these links to know more: IBM Storage community blog – IBM Storage Insights: Integration with Cloud Pak for AIOps, and Storage Guru Demo video.

e. IBM Storage Insights integration with Productivity & Collaboration Tools

• Slack
  Slack is a collaboration tool designed for teams to communicate and work efficiently. It offers real-time messaging, file sharing, and integration with various apps, enabling streamlined workflows and centralized communication across projects. When IBM Storage Insights is integrated with Slack via a webhook, triggering critical alerts like ransomware alerts in IBM Storage Insights sends an alert message to a designated Slack channel.

Refer these links to know more: IBM Storage community blog – Integrate Slack with IBM Storage Insights for Real-Time Ransomware Notifications, and IBM Storage Insights docs.

f. IBM Storage Insights integration with IBM Storage Virtualize Plugin for vSphere

IBM Storage Virtualize Plugin for vSphere has been integrated with IBM Storage Insights using REST APIs, to monitor the ransomware alerts and key performance such as IOPs, latency and bandwidth metrics of the volumes.

Refer this link to know more: IBM Storage Insights docs – Connecting with IBM Storage Insights.