New "mini-ranks" hybrid programming model.

-


Table 1: Comparison of MPI ranks with mini-ranks on the Colosse
super-computer at Laval University.
+-------+---------------------------------------------------+
| Cores | Average round-trip latency (us)                   |
+-------+-----------------------+---------------------------+
|       | MPI ranks             | mini-ranks                |
|       | (pure MPI)            | (MPI + pthread)           |
+-------+-----------------------+---------------------------+
| 8     | 11.25 +/- 0           | 24.1429 +/- 0             |
| 16    | 35.875 +/- 6.92369    | 43.0179 +/- 8.76275       |
| 32    | 66.3125 +/- 6.76387   | 41.7143 +/- 1.23924       |
| 64    | 90 +/- 16.5265        | 37.75 +/- 6.41984         |
| 128   | 126.562 +/- 25.0116   | 43.0179 +/- 8.76275       |
| 256   | 203.637 +/- 67.4579   | 44.6429 +/- 6.11862       |
| 512   |                       |                           |
+-------+-----------------------+---------------------------+

Comments

Post a Comment

Popular posts from this blog

A survey of the burgeoning industry of cloud genomics

-
Image
I like lists. Here is a list of companies in the cloud genomics industry. Table 1: Companies acting in the industry of cloud genomics. The list is in no particular order. Company Products People Founded Link GenoSpace, LLC - GenoSpace platform - John Quackenbush (CEO) 2011 https://www.genospace.com/ DNAnexus, Inc. - DNAnexus Platform - DNAnexus Platform SDK (a.k.a. dx-toolkit) - Serafim Batzoglou (co-founder) - Andreas Sundquist (co-founder and CTO) - a bunch of other famous people in the field 2009 https://dnanexus.com/ Seven Bridges Genomics, Inc. - IGOR - IGOR Python SDK - Igor Bogicevic (CTO) 2009 https://www.sbgenomics.com/ BGI - EasyGenomics - Xing Xu (Senior Product Manager) Introduced in 2012 https://www.easygenomics.com/ Illumina, Inc. - BaseSpace™ - BaseSpace API - Alex Dickinson (SVP of Cloud Genomics) Introduced in 2011 https://basespace.illumina.com/ Era7 Information Technologies SLU - BG7
Read more

Generating neural machine instructions for multi-head attention

-
  An animal can do training and inference every day of its existence until the day of its death. A forward pass is all you need.   Here is the current model configuration that I use for multi-head attention:         let sequence_length = 6;         let vocab_size = 20;         let n_embd = 64;         let num_heads = 8; You can see the full model here: https://github.com/sebhtml/novigrad/blob/8df6192829edd9cda3a221e12affa770fbda2a1b/src/models/mega_man_attention.rs The struct uses Embedding, MultiHeadAttention, Linear, Softmax: struct MegaManAttentionModel {     input_shape: Vec<usize>,     output_shape: Vec<usize>,     vocab_size: usize,     sequence_length: usize,     embedding: Embedding,     multi_head_attention: MultiHeadAttention,     linear: Linear,     softmax: Softmax, } The instructions generated by Novigrad are below. Notes: - INSTRUCTION    MatMul    t170 t166    t171     means that we do a matrix multiplication between tensors t170 and t166 and the result is wr
Read more

Adding ZVOL VIRTIO disks to a guest running on a host with the FreeBSD BHYVE hypervisor

-
On atlantis.domain (my FreeBSD BHYVE hypervisor host), I use these commands: Install FreeBSD on a 10G disk. vm create -s 10G guestname   vm install FreeBSD-12.1-RELEASE-amd64-memstick.img vm console guestname Add 8 30GB zvol-based disks vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname You will need to do vm poweroff guestname vm start guestname because the guest does not seem to see the 8 30GB without power cycling. (rebooting won't work) *** If you want the boot disk (the one create by "vm create") to be a zvol, you must edit the template: root@atlantis:~ # zfs snapshot tank/vm-bhyve@2020-06-11-2145 root@atlantis:~ # mount -t zfs tank/vm-bhyve@2020-06-11-2145 /tank/vm-bhyve@2020-06-11-2145 r
Read more