Comparing fastq compression with gzip, bzip2 and xz


Storage is expensive. Compression is a lossless approach to reduce the storage requirements.
Sébastien Boisvert
2012-11-05


Table 1: Comparison of compression methods on SRR001665_1.fastq -- 10408224 DNA sequences of length 36. Tests were on Fedora 17 x86_64 with a Intel Core i5-3360M processor and a Intel SSDSC2BW180A3 drive. Tests were not run in parallel. The time is the real entry from the time command. Each test was done twice.
Compression
 Time
 Size (bytes)
none
 0
 2085533064
(100%)
time cat SRR001665_1.fastq | gzip -9 > SRR001665_1.fastq.gz
 7m31.519s
7m20.340s
 376373619
(18%)
time cat SRR001665_1.fastq | bzip2 -9 > SRR001665_1.fastq.bz2
 3m12.601s
3m25.243s
 295000140
(14%)
time cat SRR001665_1.fastq | xz -9 > SRR001665_1.fastq.xz
 32m45.933s



 257621508
(12%)


Table 2: Decompression tests. Each test was run twice.
Decompression
 Time
time cat SRR001665_1.fastq.gz | gunzip > SRR001665_1.fastq
 0m14.612s
0m13.247s
time cat SRR001665_1.fastq.bz2 | bunzip2 > SRR001665_1.fastq
 1m3.412s
1m4.337s
time cat SRR001665_1.fastq.xz | unxz > SRR001665_1.fastq
 0m24.194s
0m23.923s

It is strange that bzip2 is faster than gzip for compression.

Comments

Post a Comment

Popular posts from this blog

My 2022 Calisthenics split routine

Hey my fellow blog readers ! At the beginning of year 2022, I watched some YouTube videos from Tom Merrick, The Bodyweight Warrior . The video 3 Things I Wish I Knew When I Started made me think a lot. Since I began my bodyweight journey in July 2020, I used a full-body routine, 3 times per week. As I became stronger, the volume of strength work became too high. Watching videos from Tom Merrick convinced me to split my full body routine into a split routine. My new Calisthenics routine is a split routine. Splitting the routine allowed me to increase rest time between upper body workouts, and also to increase rest time between lower body workouts. It also allowed me to increase from 3 (full-body) workouts per week to 4 workouts (upper body, lower body + core, upper body, lower body + core) per week. Here what my split routine looks like: Wednesday: Upper Body Workout (1h15min) Pair 1 Ring Pull-Up Pseudo Planche Push-Up  Pair 2 Tuck Assisted Victorian Box Handstand Push-Up Pair 3 Ring R
Read more

Adding ZVOL VIRTIO disks to a guest running on a host with the FreeBSD BHYVE hypervisor

On atlantis.domain (my FreeBSD BHYVE hypervisor host), I use these commands: Install FreeBSD on a 10G disk. vm create -s 10G guestname   vm install FreeBSD-12.1-RELEASE-amd64-memstick.img vm console guestname Add 8 30GB zvol-based disks vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname You will need to do vm poweroff guestname vm start guestname because the guest does not seem to see the 8 30GB without power cycling. (rebooting won't work) *** If you want the boot disk (the one create by "vm create") to be a zvol, you must edit the template: root@atlantis:~ # zfs snapshot tank/vm-bhyve@2020-06-11-2145 root@atlantis:~ # mount -t zfs tank/vm-bhyve@2020-06-11-2145 /tank/vm-bhyve@2020-06-11-2145 r
Read more

Le tissu adipeux brun, la thermogénèse, et les bains froids

Image
Figure 4 de Mapping of human brown adipose tissue in lean and obese young men https://www.pnas.org/content/114/32/8649 BAT = Brown Adipose Tissue    Avec l'hiver à nos portes, je partage ci-dessous mon intérêt pour le tissu adipeux brun. Le tissu adipeux blanc a un rôle d'isolant et de stockage. Le tissu adipeux brun ("gras brun") diffère du traditionnel tissue adipeux blanc. Le tissus adipeux brun a un rôle de plinthe électrique (thermogénèse). Et non, manger de la galvaude ne donne pas plus de gras brun.😄 Selon le livre Human Anatomy & Physiology de Élaine Marieb , 5ème édition, publié en 2001, le tissu adipeux brun est présent seulement chez les bébés, qui ne peuvent pas frissonner. Le tissu adipeux brun fonctionnel a été découvert chez les adultes en 2009 ( New England Journal of Medicine ). Le tissu adipeux brun a été cartographié chez l'humain adulte en 2017 ( Proceedings of the National Academy of Sciences ). J'ai mis la Figure 4 de l'articl
Read more