The impact of link-time optimization
Recently, I compared running times with and without link-time optimization. Link-time optimization (LTO) makes programs run faster and it must be done at compilation. Both the GNU Compiler Collection (GCC) and Clang (a frontend for LLVM) call this link-time optimization. The Intel compiler can also do that in a similar process called interprocedural optimization (IPO). This process is called Whole Program Optimization (also needs Link-time Code Generation -- LTCG) in Microsoft Visual Studio The dataset I used for these tests is ERS006494 . It contains 186073978 Illumina DNA sequences. The length of these sequences is 75 nucleotides. I used 8 nodes, each with 2 Intel Xeon X5560 processors (8 cores per node) and 24 GiB of memory. Storage was served by a Lustre file system. Each job therefore had 64 ranks. The version of GCC was 4.7.2. The version of Open-MPI was 1.6.3. The version of Ray was 606be2a7a710a226. The version of Ray Platform was d78e7ec5037c9c9e8a0. The Host Communic...