Data Deduplication

A data reduction technique in which repeated blocks of data are identified and replaced with references, or "pointers", to reduce the number of bits needed to store and transmit data. Data deduplication typically discovers redundancy at a sub-file level of granularity, between different files, and across time. Data deduplication is most widely used for backing up and archiving data to disk, where data protection procedures often store multiple copies of identical files or several versions of files with minor changes. Data deduplication is almost always combined with data compression, another data reduction technique. Also see fixed-length deduplication and variable-length deduplication.