Crawford Media Services Revolutionizes Media Archiving with Object Storage

One of the country’s preeminent media service companies, Crawford Media Services provides everything from post-production to asset management and archiving. Its projects cover the gamut: from episodic television stalwarts like The Walking Dead to major motion pictures like Lincoln and Hidden Figures, and its customer list includes industry leaders, such as Coca-Cola, PBS, and Viacom.

“One of the ways that we have stayed successful and kept growing over the last 30 years,” explains Steve Davis, Crawford executive vice president and chief technology officer, “is by constantly being aware of where the industry is moving and making sure that we offer our clients the level and kind of services that will keep them competitive over the long term.” A perfect example is the company’s approach to archiving.

As the company’s old, linear media was aging past its useful life, Crawford realized that thousands and thousands of hours of content would need to be digitized and archived for long-term storage.

Capacity Requirement Tests the Limits of RAID

Crawford already offered archiving for its clients using RAID arrays, digital tape, and their own digital asset manager, Engage. But the massive data influx from preservation-grade media files, ones consuming 100GB or more for every hour of content, was quickly outpacing RAID.

“RAID gives you redundancy at the level of the disk. As you get more disks and they get larger— now 4TB disks are common—the odds of a failure go up. The consequences are massive, and disk rebuilds can last for days. During that time data is vulnerable, and the system’s performance is degraded.”

Investigating Object Storage

Crawford spent almost two years investigating the technology to support its second-generation archive, called AMBER, and were convinced that erasure coding, one technology behind object storage, was the answer. It breaks data into many discrete units, and then spreads it, with parity and redundancy, over many storage nodes. If errors occur or a disk fails, the data is automatically read from the other locations. Disks can be replaced with a broad range of choices, there is virtually no downtime during rebuild, and the integrity of the data is much higher than with RAID.

Leveraging Object Storage for a Scalable, Resilient Archive

After looking at the alternatives, the Crawford team decided that Quantum’s object storage solution was the ideal choice for AMBER.

“We knew the technology behind Lattus and had been very impressed with its maturity, resilience, and scalability,” says Davis. “Quantum also offers an appliance-based implementation, Artico, that integrates Lattus with Quantum’s StorNext data management platform. Quantum’s StorNext File System makes it easy to use the archive, and its API enables integration with Engage. It also supports tape for clients who rely on removable media for content transfer.”

Scalability to Preserve Data and Reduce Costs

Lattus gives users a choice of using SSDs or conventional hard disks as a cache. It scales in small increments and makes it easy to add new technology, integrating it into the existing storage pool. The result is that Crawford can offer AMBER services at substantially lower rates than conventional cloud services.

The new archive was initially sized to accommodate the company’s 1.2PB client archive and provide room for growth. Clients view proxies using Engage, and it sends full-resolution copies to an FTP server or tape. “One of the strengths of the system is that our clients are not limited to recovering whole files,” Davis explains. “StorNext supports partial file recovery, so users can minimize data movement by bringing back specific timecode ranges.”

“Lattus gives us the ability to provide a level of archiving service that we have not seen from anyone else in the industry—100% data integrity, migration to newer technology in the future through non-disruptive scalability, and the option of storing duplicate copies of data in different locations for an additional level of DR protection.”