Comparing fastq compression with gzip, bzip2 and xz

Storage is expensive. Compression is a lossless approach to reduce the storage requirements.

Sébastien Boisvert

2012-11-05

Table 1: Comparison of compression methods on SRR001665_1.fastq -- 10408224 DNA sequences of length 36. Tests were on Fedora 17 x86_64 with a Intel Core i5-3360M processor and a Intel SSDSC2BW180A3 drive. Tests were not run in parallel. The time is the real entry from the time command. Each test was done twice.

Compression	Time	Size (bytes)
none	0	2085533064 (100%)
time cat SRR001665_1.fastq | gzip -9 > SRR001665_1.fastq.gz	7m31.519s 7m20.340s	376373619 (18%)
time cat SRR001665_1.fastq | bzip2 -9 > SRR001665_1.fastq.bz2	3m12.601s 3m25.243s	295000140 (14%)
time cat SRR001665_1.fastq | xz -9 > SRR001665_1.fastq.xz	32m45.933s	257621508 (12%)

Table 2: Decompression tests. Each test was run twice.

Decompression	Time
time cat SRR001665_1.fastq.gz | gunzip > SRR001665_1.fastq	0m14.612s 0m13.247s
time cat SRR001665_1.fastq.bz2 | bunzip2 > SRR001665_1.fastq	1m3.412s 1m4.337s
time cat SRR001665_1.fastq.xz | unxz > SRR001665_1.fastq	0m24.194s 0m23.923s

It is strange that bzip2 is faster than gzip for compression.