The impact of link-time optimization

-
Recently, I compared running times with and without link-time optimization. Link-time optimization (LTO) makes programs run faster and it must be done at compilation. Both the GNU Compiler Collection (GCC) and Clang (a frontend for LLVM) call this link-time optimization. The Intel compiler can also do that in a similar process called interprocedural optimization (IPO). This process is called Whole Program Optimization (also needs Link-time Code Generation -- LTCG) in Microsoft Visual Studio

The dataset I used for these tests is ERS006494. It contains 186073978 Illumina DNA sequences. The length of these sequences is 75 nucleotides.

I used 8 nodes, each with 2 Intel Xeon X5560 processors (8 cores per node) and 24 GiB of memory. Storage was served by a Lustre file system. Each job therefore had 64 ranks. The version of GCC was 4.7.2. The version of Open-MPI was 1.6.3. The version of Ray was 606be2a7a710a226. The version of Ray Platform was d78e7ec5037c9c9e8a0. The Host Communication Adapter was Mellanox Technologies MT26428.

The command stript was used to remove useless information from Ray executables. The complete command for link-time optimization with GCC is available here. The template to run the jobs is available here.

Table 1: Comparison of running times with different compilation options.

Compilation options
Running time
-Wall -std=c++98 -O3 -march=native
7 hours, 14 minutes, 33 seconds
-Wall -std=c++98 -Os -march=native -flto -fwhole-program
10 hours, 39 minutes, 26 seconds
-Wall -std=c++98 -O3 -march=native -flto -fwhole-program
7 hours, 8 minutes, 36 seconds


 There is no difference with LTO when running Ray on Infiniband apparently.



Comments

Post a Comment

Popular posts from this blog

A survey of the burgeoning industry of cloud genomics

-
Image
I like lists. Here is a list of companies in the cloud genomics industry. Table 1: Companies acting in the industry of cloud genomics. The list is in no particular order. Company Products People Founded Link GenoSpace, LLC - GenoSpace platform - John Quackenbush (CEO) 2011 https://www.genospace.com/ DNAnexus, Inc. - DNAnexus Platform - DNAnexus Platform SDK (a.k.a. dx-toolkit) - Serafim Batzoglou (co-founder) - Andreas Sundquist (co-founder and CTO) - a bunch of other famous people in the field 2009 https://dnanexus.com/ Seven Bridges Genomics, Inc. - IGOR - IGOR Python SDK - Igor Bogicevic (CTO) 2009 https://www.sbgenomics.com/ BGI - EasyGenomics - Xing Xu (Senior Product Manager) Introduced in 2012 https://www.easygenomics.com/ Illumina, Inc. - BaseSpace™ - BaseSpace API - Alex Dickinson (SVP of Cloud Genomics) Introduced in 2011 https://basespace.illumina.com/ Era7 Information Technologies SLU - BG7...
Read more

Generating neural machine instructions for multi-head attention

-
  An animal can do training and inference every day of its existence until the day of its death. A forward pass is all you need.   Here is the current model configuration that I use for multi-head attention:         let sequence_length = 6;         let vocab_size = 20;         let n_embd = 64;         let num_heads = 8; You can see the full model here: https://github.com/sebhtml/novigrad/blob/8df6192829edd9cda3a221e12affa770fbda2a1b/src/models/mega_man_attention.rs The struct uses Embedding, MultiHeadAttention, Linear, Softmax: struct MegaManAttentionModel {     input_shape: Vec<usize>,     output_shape: Vec<usize>,     vocab_size: usize,     sequence_length: usize,     embedding: Embedding,     multi_head_attention: MultiHeadAttention, ...
Read more

Adding ZVOL VIRTIO disks to a guest running on a host with the FreeBSD BHYVE hypervisor

-
On atlantis.domain (my FreeBSD BHYVE hypervisor host), I use these commands: Install FreeBSD on a 10G disk. vm create -s 10G guestname   vm install FreeBSD-12.1-RELEASE-amd64-memstick.img vm console guestname Add 8 30GB zvol-based disks vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname You will need to do vm poweroff guestname vm start guestname because the guest does not seem to see the 8 30GB without power cycling. (rebooting won't work) *** If you want the boot disk (the one create by "vm create") to be a zvol, you must edit the template: root@atlantis:~ # zfs snapshot tank/vm-bhyve@2020-06-11-2145 root@atlantis:~ # mount -t zfs tank/vm-bhyve@2020-06-11-2145 /tank/vm-bhyve@2020-06-11-2145 r...
Read more