UK Met Office Installs HPC System to Improve Weather and Climate Data Analysis

Silicon Graphics International Corp. (SGI) together with Bright Computing, Inc. and DataDirect Networks, Inc. (DDN) announce that the Met Office, the UK’s national weather service, selected the three HPC vendors to provide HPC capabilities for its new Scientific Processing and Intensive Compute Environment (SPICE) system.

SPICE will enable weather and climate researchers to reduce time required to analyse massive amounts of climate simulation data.

The Met Office is a weather forecasting and climate prediction organisation that conducts research designed to protect lives and increase prosperity. The institution’s 500 scientists conduct research using data-intensive, high-resolution models to increase forecast accuracy and provide a deeper understanding of climate change. The Met Office required a powerful system for post-processing data and analysis downstream of the primary HPC facility.

As a result, the UK Met chose SGI to power its SPICE initiative and upgrade its Managed Archive Storage System (MASS).

SGI was selected by the Met Office for its value and performance, allowing users to more easily manage multiple servers and increase system utilisation rates. Following the installation in April 2016, the Met Office’s researchers experienced a positive increase in processing capacity, furthering their understanding of meteorology on a global scale.

To support the growth in its MASS which is a critical adjunct to the Met Office’s supercomputer system archive, the Met Office selected SGI’s solution with DDN storage. MASS acts as a repository or archive for the data resulting from scientific research carried out on the supercomputer as well as observational data. By 2020, this crucial storage archive is predicted to grow to about 300PB of weather and climate research data.

To build a well-rounded, turnkey system, the Met Office chose to integrate Bright Cluster Manager for HPC to deploy the new SPICE cluster over bare metal, providing single-pane-of-glass management for the hardware, OS, HPC software, and users.

It also chose to install Bright OpenStack to enable the IT team deployment, provision, and management of its OpenStack-based private-cloud infrastructure.

The fact that Bright’s solutions can be administered from a single point of control was a consideration in the Met Office’s decision-making process. With the combined solution of compute, OpenStack, and storage, the Met Office can scale SPICE storage predictably while delivering high-throughput performance to handle simultaneous data reads/writes.

Using the SGI system, the Met Office’s researchers can spin up VMs and operate their own private virtual environment with control and direct access to their local network. In addition, they can increase the capacity of the virtual environment by adding more servers to the OpenStack environments.

Scientists using SPICE have already noted performance advantages over previous systems, enabling far quicker analysis to support ongoing research. Massive volumes of data are analysed in several hours, rather than days. The improvements support and enhance ongoing development of meteorological and climate change research.

Announcement Highlights

• The UK Met Office has selected SGI to power its new Scientific Processing and Intensive Compute Environment (SPICE), enabling weather and climate researchers to reduce time required for analyzing climate simulation data.
• To upgrade its MASS archive, the Met Office has selected SGI together with DDN’s Storage architecture. Currently, the archive stores about 100TB each day and is expected to increase to 200TB per day by 2017.
• The Met Office has chosen Bright Cluster Manager for HPC to deploy the new SPICE cluster over bare metal, providing single-pane-of-glass management for the hardware, operating system, HPC software, and users. The Met Office has also chosen to install Bright OpenStack, enabling the IT team’s deployment, provision, and management of its OpenStack-based private-cloud infrastructure.
  

Technical Information

• The SGI Rackable system for SPICE has 36 nodes Intel Xeon processor E5-2690 v4, achieving performance of up to 30,000 Gigaflops
• Data access and storage for MASS is provided by 3.5PB of DDN Disk (models GS7K, GS12K) to support disk cache
• ConnectX-3 Pro Adapter with Virtual Protocol Interconnect for both IP and InfiniBand communication
• Bright Cluster Manager for HPC and Bright OpenStack
  

“With the new SPICE system from SGI, we have seen a step-change in performance for our researchers and scientists doing post-processing of weather and climate data,” said Richard Bevan, head of operational technology, Met Office. “Tasks that used to take 1-2 days to complete are now done in a fraction of that time, allowing scientists to perform multiple runs in one day””

According to Bob Maynard, storage team technical lead for the Met Office, DDN’s scalable storage for both archive and post-processing delivers value and unparalleled performance: “No matter how much we grow, DDN will continue to provide a performance step-change that will enable scientists to collect and make sense of massive amounts of data every day for the benefit of mankind and our planet.”

“Everyone who uses Bright technology says it’s impressive, but you really start seeing the power of our technology when you seamlessly connect solutions together,” said Lee Carter, VP EMEA, Bright Computing. “By coupling its HPC and OpenStack environments with Bright management, the Met Office is in a strong position to accelerate research, and provide world leading applications and climate models to both internal as well as external users.”

“The Met Office is driving intensive climate research around the globe, and we’re pleased to provide them with an all-inclusive system to power the SPICE program,” said Gabriel Broner, VP and GM, HPC, SGI. “By combining our SGI Rackable solution with Bright Computing and DDN technologies, we’re able to provide the Met Office a strong HPC system that not only handles massive data workloads, but also allows easier manageability for researchers and IT managers to operate the computer simultaneously.“