Justification pour accéder à un calculateur Microsoft(R) Windows(R) HPC(R)

-
** Problématique

Les séquençeurs d'ADN actuels (comme le Illumina(R) HiSeq(R) 2500) produit plus de 6 000 000 000 séquences d'ADN numériques de longueur entre 100 et 200 lettres (A, T, C, G) en une seule analyse. Un des types d'analyse possible est "l'assemblage de novo de génomes."

** Système logiciel

Mon logiciel s'appelle Ray et est codé en C++ 1998 (Microsoft(R) Visual Studio(R) 2010 supporte complètement ce standard). Une librairie MPI de passage de message est aussi requise. MPICH2 et Open-MPI sont les deux disponibles en distribution binaire sous Microsoft(R) Windows(R).

Ray est un logiciel libre (licence GPLv3) et utilise la librairie parallèle RayPlatform (licence: LGPLv3).

- http://www.ohloh.net/p/ray-assembler
- http://www.ohloh.net/p/rayplatform

- http://denovoassembler.sourceforge.net/


Ray fait de l'assemblage de novo de génomes ou de métagénomes dans l'industrie des sciences de la vie (génomique). Ray "scale" très bien pour les problèmes de "Big Data".

** Plateformes testés (liste incomplète)

- Windows 7 sur Intel(R) Q6600 (Visual Studio 2010, MPICH2)
- Amazon EC2 (MIT StarCluster)
- CentOS sur IBM(R) iDataPlex(R) (Intel Xeon)
- CentOS sur Sun(R) Constellation (Intel Xeon)
- IBM Blue Gene/Q (IBM PowerPC A2)
- Cray Linux sur Cray XE6 (AMD Opteron)
- Ubuntu sur Apple PowerBook G4 (PowerPC G4)
- Debian sur Sparc (Sun SunBlade 100)
- Fedora sur HP ProLient (AMD Opteron)
- Funtoo sur ARMv6j (Raspberry Pi)

** Buts

1. Avoir un distributeur de Ray dans le monde de Microsoft(R) Windows(R)
2. Tester la portabilité du code sur Windows(R)
3. Tester Microsoft(R) MPI
4. Tester la performance sur un système HPC Microsoft(R) / Fujitsu(R) / DMR(R)

** Jeux de donnée

Nom: SRA001125
Description: E. coli / ILLUMINA / 2008-07-01
Adresse: http://trace.ddbj.nig.ac.jp/DRASearch/submission?acc=SRA001125

Nom: SRS011098
Description: Human metagenome sample from G_DNA_Supragingival plaque of a female
participant in the dbGaP study "HMP Core Microbiome Sampling Protocol A (HMP-A)"
Adresse: http://trace.ddbj.nig.ac.jp/DRASearch/sample?acc=SRS011098

** Distributeurs

Courant:

- Geeknet, Inc. (sourceforge)
- GitHub, Inc..
- Calcul Canada, Inc. (Calcul Québec)

Prochainement:

- Software in the Public Interest, Inc. (Debian)
- Canonical, Inc. (Ubuntu)
- Amazon, Inc. (une image AMI de CloudBioLinux)

En cours:

- Cray, Inc
- Red Hat, Inc. (via Fedora(TM))

Comments

Post a Comment

Popular posts from this blog

A survey of the burgeoning industry of cloud genomics

-
Image
I like lists. Here is a list of companies in the cloud genomics industry. Table 1: Companies acting in the industry of cloud genomics. The list is in no particular order. Company Products People Founded Link GenoSpace, LLC - GenoSpace platform - John Quackenbush (CEO) 2011 https://www.genospace.com/ DNAnexus, Inc. - DNAnexus Platform - DNAnexus Platform SDK (a.k.a. dx-toolkit) - Serafim Batzoglou (co-founder) - Andreas Sundquist (co-founder and CTO) - a bunch of other famous people in the field 2009 https://dnanexus.com/ Seven Bridges Genomics, Inc. - IGOR - IGOR Python SDK - Igor Bogicevic (CTO) 2009 https://www.sbgenomics.com/ BGI - EasyGenomics - Xing Xu (Senior Product Manager) Introduced in 2012 https://www.easygenomics.com/ Illumina, Inc. - BaseSpace™ - BaseSpace API - Alex Dickinson (SVP of Cloud Genomics) Introduced in 2011 https://basespace.illumina.com/ Era7 Information Technologies SLU - BG7
Read more

Generating neural machine instructions for multi-head attention

-
  An animal can do training and inference every day of its existence until the day of its death. A forward pass is all you need.   Here is the current model configuration that I use for multi-head attention:         let sequence_length = 6;         let vocab_size = 20;         let n_embd = 64;         let num_heads = 8; You can see the full model here: https://github.com/sebhtml/novigrad/blob/8df6192829edd9cda3a221e12affa770fbda2a1b/src/models/mega_man_attention.rs The struct uses Embedding, MultiHeadAttention, Linear, Softmax: struct MegaManAttentionModel {     input_shape: Vec<usize>,     output_shape: Vec<usize>,     vocab_size: usize,     sequence_length: usize,     embedding: Embedding,     multi_head_attention: MultiHeadAttention,     linear: Linear,     softmax: Softmax, } The instructions generated by Novigrad are below. Notes: - INSTRUCTION    MatMul    t170 t166    t171     means that we do a matrix multiplication between tensors t170 and t166 and the result is wr
Read more

Adding ZVOL VIRTIO disks to a guest running on a host with the FreeBSD BHYVE hypervisor

-
On atlantis.domain (my FreeBSD BHYVE hypervisor host), I use these commands: Install FreeBSD on a 10G disk. vm create -s 10G guestname   vm install FreeBSD-12.1-RELEASE-amd64-memstick.img vm console guestname Add 8 30GB zvol-based disks vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname vm add -d disk -t zvol -s 30G guestname You will need to do vm poweroff guestname vm start guestname because the guest does not seem to see the 8 30GB without power cycling. (rebooting won't work) *** If you want the boot disk (the one create by "vm create") to be a zvol, you must edit the template: root@atlantis:~ # zfs snapshot tank/vm-bhyve@2020-06-11-2145 root@atlantis:~ # mount -t zfs tank/vm-bhyve@2020-06-11-2145 /tank/vm-bhyve@2020-06-11-2145 r
Read more