Embedding Archive Into the DNA of the Australian Genome Research Facility

The Australian Genome Research Facility (AGRF) amasses large amounts of genome sequencing data to support academic, industry, and government customers. The team at AGRF needed a storage solution to accommodate rapid data growth and better protect valuable data.

Facing Rapid Growth of Genomic Data

AGRF provides genomic sequencing for a broad range of customers, including academic researchers, healthcare organizations, commercial businesses, government agencies, and more. “Our goal is to process a large amount of data and hand it back to customers within a reasonably fast timeframe,” says Rowan Gronlund, head of information and communications technology (ICT) at AGRF.

Storing that sequencing data presents serious challenges. “The amount of data we store has tripled in the last two and a half years,” says Gronlund.

Data growth had limited the amount of time that AGRF could hold data. “In the past, we could only hold on to data for 200 to 300 days before we had to remove it from our systems to make room for the next data sets,” says Gronlund.

Recognizing the Critical Importance of Protecting Sequencing Data

The AGRF IT group also lacked a way to protect the valuable genomics data produced by researchers. If data was lost, the IT group would have to re-sequence the biological sample. Re-sequencing could take four to six weeks, which could seriously delay existing projects. Data loss could also significantly affect AGRF’s reputation.

Implementing a Quantum Multi-Tier Solution to Store and Archive Data

The ICT group decided to implement a multi-tier storage solution from Quantum. “We had heard very positive feedback from other Quantum customers,” says Gronlund.

The ICT group selected a Quantum Artico NAS appliance, which is powered by StorNext data management software. Artico provides more than 30TB of high-performance disk-based storage as the front end for a multi-tier storage and archive environment. The ICT group also implemented a Quantum Scalar tape library behind the Artico appliance.

Providing Longer-Term Storage for Sequencing Data

With the new solution, AGRF can provide longer-term storage for data produced by sequencers. “Instead of having to remove data within 300 days to make room for new data sets, we can now hold on to it for up to three years,” says Gronlund. “Customers can now request older data, and we can produce it quickly and easily.”

Reducing Costs with Active Vault

The AGRF ICT group uses Active Vault—a feature that allows infrequently used tapes to be stored in a separate partition of the library—to avoid excessive hardware purchases. “By using the Artico solution plus the Scalar tape library with Active Vault, we are able to bring the total cost down to half the cost of a competing solution,” says Gronlund.

Streamlining IT Administration

The ICT group selected the Quantum solution in part because it offered an easier way to manage storage than solutions from other vendors. “The Quantum solution doesn’t require a high level of expertise for management, and that’s perfect for us,” says Gronlund. “Since we completed the initial setup and policy configurations, we really haven’t touched the environment except for purchasing more tapes.”

Protecting Sequencing Data and Facilitating Rapid Recovery

The Quantum solution helps the ICT group better protect valuable sequencing data by enabling more robust backup and disaster recovery (DR) strategies. “By continuously archiving data using the Quantum solution, we can better preserve and protect data,” says Gronlund.

The Quantum solution also facilitates disaster recovery. “Artico allows us to keep an untainted version of the data produced by sequencers,” says Gronlund. “If anything goes wrong with post-sequencing analysis, we can recover original data easily.”

The tape archives provide additional layers of DR protection. “We keep two copies of data on tape—one onsite and another offsite,” says Gronlund. “With the Quantum solution, we can avoid having to re-sequence data and—more importantly—we can minimize the risk of losses that would affect our organization’s reputation.”