Index Engines Achieves 1TB/Hour Indexing

Index Engines, Inc., in enterprise discovery solutions, has joined the EMC Velocity2 Technology and ISV Program. In addition, Index Engines announced jointly conducted test results of indexing performance and capabilities on EMC Data Domain deduplication storage systems.

The Index Engines 3.0 platform is able to perform full content and metadata indexing of backup data stored on EMC Data Domain systems, enabling users to avoid time consuming restores prior to the indexing process. During recent tests conducted by Index Engines with support from EMC, a single indexing node of the Index Engines 3.0 platform achieved sustained rates of over 1 Terabyte per hour on a Data Domain DD690 storage system.

The format of backup data, by design, is optimized for restores versus the indexing process that accompanies eDiscovery efforts. Identification and search efforts associated with eDiscovery are putting pressure on IT managers to leverage backup and recovery infrastructures to discover data that may be relevant to litigation. Typically this has necessitated the full restoration of backup data to enable indexing – an arduous, multi-step process that requires a large cache of disk. With the Index Engines platform and EMC Data Domain deduplication storage systems, backup data can be indexed directly at high speed without having to restore multiple versions of backups to disk. Instead, a comparably small index of relevant data is created that users can evaluate and then choose what to extract. The combined solution is unique in its capability to enable customers to index deduplicated backup data and enables a markedly more efficient approach to the eDiscovery process.

The interoperability of Index Engines and Data Domain storage systems enables users to index, search, classify and extract both backup and archive data stored on Data Domain systems for fast and easy discovery, backup management, storage assessment and reporting. A combined EMC Data Domain – Index Engines environment allows users to quickly find and recover only what they need. Easy access to stored data enables management to better understand the profile of the data in order to make more intelligent decisions. Additionally, users can determine what content is no longer necessary and what needs to be migrated to archives.

“Index Engines’ speed and scale combined with efficient storage platforms from EMC Data Domain enable enterprise IT teams to leverage stored data as an asset,” said George Crump, Senior Analyst from Storage Switzerland. “This combined solution enables backup and archive data to be proactively indexed so that it is ready to be recalled.”

The validated configuration consisted of a DD690 system that was networked, via four GigE connections, to an Index Engines appliance. The Index Engines system was configured to access the backup directory of the DD690 using NFS. The process was configured to perform a full index of the content, however incremental indexing is also available. The DD690 system contained over 1TB of unstructured data and email backed up using Symantec’s NetBackup software. The Index Engines appliance was configured to run four simultaneous indexing jobs, each focused on a specific directory. The resulting indexing speed was 271MB/sec per stream for an aggregated total of 1TB/hour.

"EMC’s market-leading storage platforms address a broad scope of enterprise data and workload types," said Jim McGann, Vice President of Information Discovery at Index Engines. “Indexing deduplicated backup data at rates of 1 TB/hr on Data Domain systems gives the enterprise efficient and rapid access to stored data for litigation, data management and compliance initiatives.”

“EMC is pleased that Index Engines has joined the EMC Velocity2 Technology and ISV Program to demonstrate its commitment to excellence in enterprise-class data discovery,” said Shane Jackson, Senior Director of Product Marketing and Channel Marketing, EMC Backup Recovery Systems division. “We look forward to working with Index Engines to ensure that our mutual customers have the highest level of support possible for their information infrastructure initiatives.”