If you’re looking for an efficient way to share content among creative staff, odds are you’ve researched NAS (network attached storage), virtualization appliances, and even simple network shares for your workstations.  But have you considered a file system?

While it may not be immediately apparent, all the solutions mentioned above leverage file systems.  The overall feature set may vary between products, but at the heart of each is a file system that allows users to share content across multiple systems.  For the most part, these file systems are tuned for office files, not media files.  So the question arises:

Do you need a file system designed especially for sharing media content?

A file system is a structure for storing and organizing content.  Every system from a high end SGI graphics server to a Macintosh laptop has one.  File systems can be looked at from two view points – a user view and a system view.  From a user stand point, file systems like EXT3, HFS, or NTFS allow artists to build directory structures, set access permissions, and store files on their workstations.

From a system view, file systems help manage the disk capacity (internal drives, SAN volumes, etc) used to store creative content.  Whether its raw footage from ingest, digital intermediaries, or a final cut for distribution, a file system is responsible for helping organizing how and where content is stored on disk to enable quick data access and efficient use of capacity.

In general, file systems are mounted and owned by a single workstation at a given time (Windows box ABC owns D:\).  This configuration allows a workstation to read / write content very quickly, but it is only available to that system – another artist cannot easily access the content.  This creates not only a data sharing issue but, in larger environments, a consolidation issue because each user has their own storage which may not be fully utilized or may store content duplicated on other workstations.

When content needs to be utilized by multiple individuals or there is a demand for storage consolidation, a special type of file system called a network file system can be used to provide many artists with shared access to single content repository.  Network file systems accomplish this by using IP communication protocols that enable multiple workstations to “see” an access point on a server and read / write content stored there.

Network file systems come in a variety of forms, but the most commonly used are NFS, a network file system for Unix and Linux, and CIFS, a network file system for Windows.  For simple data sharing, NFS and CIFS are very attractive because they come pre-installed within their respective OS platforms so that all a user typically needs to do to share data with other artists is enable network sharing and configure a mount point for users to access.

While network shares on artists’ workstations may be suitable for infrequent, small content transfers between two artists, they do not scale well over time or provide a centralized repository for content management.  For this, users often employ NAS.  NAS appliances serve as a centralized data share and utilize optimized versions of NFS and CIFS to accomplish the same functions as simple network shares, but with additional valued added services like snapshots and replication.

For small shops NAS can act as a hub where content is stored and shared between artists, but it does not scale well.  As content files get larger (e.g. HD content or large media files), need to be shared among more artists, or require low latency access for streaming media, NFS and CIFS are often insufficient (50 – 70 MB/s generally for NFS, 35 MB/s generally for CIFS over Gigabit Ethernet connections).

For larger files or content that requires low latency, high speed throughput a different solution is required.  To meet these needs shared file systems were developed.  A shared file system is a special version of file system designed specifically for high performance content sharing.  For example, a shared file system could allow 10s to 100s of artists to access a single pool of content at speeds scaling from 100s of MBps to multiple-GBps of throughput per workstation.

Like NAS, shared file systems create a central hub for storing and sharing content, but instead of requiring an appliance through which all data is sent, a shared file system gives every server direct access to a content pool.  This eliminates issues with performance scaling and allows users to utilize different access methods like FC, fibre channel, to get better throughput.

Shared file systems do however require software to be installed on each workstation that will access the centralized content pool.  This software sits on the workstation and coordinates access among all the other workstations sharing content.  The shared file system also provides applications with transparent access.  The application, whether it is from Autodesk, Apple, or another vendor, will access content on the shared file system in the same was it accesses content on a local file system (e.g. NTFS, HFS, EXT3) thus preventing additional complexity.

Simple shared file systems are homogeneous, allowing many servers of a single OS type (usually Linux) to share the file system.  While this can be sufficient in small environments, it lacks scalability within organizations that have servers running different operating systems (Windows, Linux, UNIX, etc.).  This is especially true in media organizations where a workflow will consist of applications running on high end UNIX platforms and a mix of Windows, Linux, and Macs.  Heterogeneous shared file systems enable multiple servers to access a common disk repository regardless of OS type.
Depending on the vendor, there are additional things that a shared file system will provide to enable increased throughput for digital workflows.  These include:

• Clustering for performance and reliability
• Separation of data and metadata
• Steering to different disk types (SATA, SAS, FC) based on performance / reliability requirements
• FC, iSCSI, and IP LAN access
• Extension to digital archiving

For more information on StorNext, a heterogeneous shared file system click here.