Terascala Intelligent Storage Bridge

Terascala, Inc. announced the availability of its Intelligent Storage Bridge (ISB), a high-performance workflow manager designed to transfer large data sets quickly and automatically between fast scratch storage and enterprise storage.

The ISB is the first workflow manager that is purpose-built for application users working in HPC environments that use the Lustre parallel file system and need direct access to project data at any point in the application workflow.

The rate at which technical data is collected grows larger every day. Incorporating large volumes of data into research workflows requires many iterations and causes extended application run times and longer time to insight. Users routinely work with gigabyte and terabyte-size active data sets that need to be shared among colleagues now and preserved by the organization later. However, data movement can be tedious, time consuming and error-prone-even when handled by an HPC administrator. It can take days to manually mount source and destination directories, copy files, and move large data sets when using a general-purpose server or workstation. Even script-based automation requires continuous maintenance and monitoring.

The Terascala ISB solves these challenges by automatically moving large data sets on and off scratch storage using the maximum available network, compute and storage resources. The data movement is parallelized, saving time as well as costs because organizations no longer need to over-provision the infrastructure in order to compensate for slow data transfer rates. In addition, the ISB is the first workflow manager that puts primary control of the data in the hands of the user rather than the HPC administrator, removing the need for data management expertise through wizard-based tools and user-defined automation policies.

“Our users are most efficient when they have the data they need, when they need it,” said Kevin Campbell, director of HPC, The Translational Genomics Research Institute (TGen). “The Intelligent Storage Bridge does just that-it gives our user community a tool to automate data movement in and out of our fast scratch environment for long-term storage and protection. It’s a better use of our resources because it lets us keep scratch for what it’s meant for-processing high-performance workflows.”

“Because it is difficult to tie departmental HPC environments to IT, we often find that research data ends up stranded within the workgroup,” said Terri McClure, senior analyst, Enterprise Strategy Group. “That leaves it untracked, unprotected, and more importantly the company can’t leverage it for other projects. Terascala’s ISB makes that connection and can help users centralize data within, across and between departments to make it easily available for new or shared projects, ultimately speeding the time to results.”

The Terascala ISB helps organizations:

• Manage data transfer tasks more easily:  The ISB user interface is fully integrated into the Terascala OS (TeraOS) portal. It gives users the ability to create and monitor the progress of their data transfer tasks, and gives administrators the information they need to ensure that resources are optimally allocated between data transfer and compute jobs.
• Save time, speed processes: Intuitive wizard-based menus allow users and administrators to create and manage jobs and view job history, accelerating and automating the data management process. Polices can be defined by time, event, or file age.
• Ensure HA and redundancy: In the event of an ISB node failure, other nodes will automatically assume the workload, minimizing the impact a hardware failure can have on the workflow. Administrators are automatically notified about all hardware failures.

Once confined to large academic and government laboratories, HPC is becoming more prevalent in small and medium-size companies as compute-intensive applications for genomics sequencing, seismic processing, and product design are more readily available. For HPC to be effective in the enterprise, organizations must optimize their entire HPC workflow by adopting a holistic approach that seamlessly integrates high-performance solutions, such as those provided by Terascala.

“The Terascala Intelligent Storage Bridge fulfills our vision of optimizing HPC workflows for faster time to results,” said Steve Butler, CEO, Terascala. “The ISB eliminates the chasm that exists between HPC and enterprise storage. Now, those two storage environments are connected, which is vital for project data sharing, long-term data protection and regulatory compliance.”

The ISB is integrated with Terascala-powered high-performance storage appliances and is managed via a single user interface. Sold by partners and through Terascala directly, the entry-level ISB is packaged as a three-node, highly available cluster. Additional nodes can be added to achieve higher performance.