© Quantum Corporation. All rights reserved. | Page Feedback
GWDG enables university researchers and Max Planck Society scientists to conduct groundbreaking studies across a broad range of subjects, from genomics to the humanities, studies which nearly double storage needs each year. “Next-generation sequencing, for example, produces huge amounts of data,” says Dr. Philipp Wieder, deputy head at GWDG. “One team needed to store 300TB of data just for one project.”
The center also supports the Göttingen State and University Library—serving the university, the State of Lower Saxony, and the Göttingen Academy of Sciences. “The library is digitizing texts and creating models of physical objects,” says Wieder. “All that data must be stored somewhere.”
With more than 40,000 users, “we needed a file system that could handle billions of files and tremendous data volumes,” says Stefan Teusch, deputy head of IT infrastructure, GWDG.
Creating a 7PB Multi-Tier Storage Environment That Spans Disk and Tape
The GWDG storage system uses multiple tiers. “We now have in StorNext 2.5PB of data on disk and 4.5PB on tape,” says Teusch. “The goal was to make the data accessible from a single file system.” The economics of tape libraries help the center meet the requirement to store data for ten years or more.
The center adopted Quantum’s StorNext more than a decade ago, and has expanded the system over the years. “In our experience, StorNext is the only platform that can handle the number of files and amount of data we manage while also delivering high performance,” says Teusch.
Easily Supporting All the Different Operating Systems That Scientists Use
StorNext enables the center to support all its users’ operating systems. “Many scientists, particularly in the natural sciences, use UNIX or Linux, but other users might prefer Windows or Mac OS. In fact, some departments are Mac-only,” says Teusch. “With StorNext, we can easily support all these operating systems. Users can access the data they need without making any changes to their workflows.”
Enabling HPC Processing with High-Speed Shared Storage over Ethernet
To provide the HPC cluster shared access to data from the StorNext environment, the IT group installed StorNext distributed LAN clients (DLCs) and gateways for load balancing and failover. “The StorNext DLC eliminates the need to connect all HPC nodes with Fibre Channel,” says Teusch. “We can continue to use traditional Ethernet. HPC users requested 2GB-per-second performance. With StorNext DLC, we’re achieving 3.2GB per second,” says Teusch. “Researchers can quickly move large amounts of data into the HPC cluster for processing and out again for long-term storage.”
Preserving Copies of Data for Disaster Recovery
To protect data, the IT group uses StorNext policies to create tape copies of primary data. “If a hardware device fails or we lose a storage system, we still have the copy in the StorNext tape archive, and we can rapidly restore data,” says Teusch. “Users can get right back to work.”
Even as the multi-tier storage environment continues to grow, managing StorNext has remained simple. The IT group can continue to expand without hiring extra staff or pulling existing staff away from other projects.
Future Could Include Network of StorNext Environments
Looking ahead, the center’s IT group is considering new ways to employ StorNext, including building a network of connected StorNext environments across Germany. Overall, GWDG IT sees StorNext as an ideal storage solution that meets the center’s needs for scale, performance, and manageability.
“StorNext has met our needs even as our workflows have become more demanding, our environment has evolved, and our management team has changed,” says Wieder. “We’re confident that StorNext will continue to meet our needs well into the future.”
Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) is a computer center shared by Germany’s Göttingen University and the Max Planck Society. GWDG fulfills the functions of a joint computing and IT competence center, providing a range of basic IT services, data management solutions, and computing facilities to support scientific research and promote professional training. The center has built a large-scale storage environment to store, protect, and provide access to millions of files and large-scale data volumes for tens of thousands of users.