Snakemake workflow used to deploy and perform basic indexes of genome sequence.

This is done for teaching purpose as an example of FAIR principles applied with Snakemake.

Usage

The usage of this workflow is described in the Snakemake workflow catalog, it is also available locally on a single page.

Results

The expected results of this pipeline are described here.

Material and methods

The tools used in this pipeline are described here textually.

Step by step

Get DNA sequences

| Step | Commands | | -------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Download DNA Fasta from Ensembl | ensembl-sequence | | Remove non-canonical chromosomes | pyfaidx | | Index DNA sequence | samtools | | Creatse sequence Dictionary | picard |

┌────────────────────────────────────────┐ │Download Ensembl Sequence (wget + gzip) │ └──────────────────┬─────────────────────┘ │ │ ┌──────────────────▼────────────────────────┐ │Remove non-canonical chromosomes (pyfaidx) │ └──────────────────┬──────────────────────┬─┘ │ │ │ │ ┌──────────────────▼──────────┐ ┌─▼───────────────────────────────────┐ │Index DNA Sequence (samtools)│ │Create sequence dictionary (Picard) │ └─────────────────────────────┘ └─────────────────────────────────────┘

Get genome annotation (GTF)

| Step | Commands | | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | Download GTF annotation | ensembl-annotation | | Fix format errors | Agat | | Remove non-canonical chromosomes, based on above DNA Fasta | Agat | | Remove <NA> Transcript support levels | Agat | | Convert GTF to GenePred format | gtf2genepred |

┌─────────────────────────────────────────┐ │Download Ensembl Annotation (wget + gzip)│ └─────────────┬───────────────────────────┘ │ │ ┌─────────────▼─────────┐ │Fix format Error (Agat)│ └─────────────┬─────────┘ │ │ ┌─────────────▼─────────────────────────┐ ┌────────────────────────────────────────┐ │Remove non-canonical chromosomes (Agat)◄───────────┤Fasta sequence index (see Get DNA Fasta)│ └─────────────┬─────────────────────────┘ └────────────────────────────────────────┘ │ │ ┌─────────────▼───────────────────────┐ │Remove <NA> transcript levels (Agat) │ └─────────────┬───────────────────────┘ │ │ ┌─────────────▼────────────────┐ │Convert GTF to GenePred (UCSC)│ └──────────────────────────────┘

Get transcripts sequence

| Step | Commands | | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Extract transcript sequences from above DNA Fasta and GTF | gffread | | Index DNA sequence | samtools | | Creatse sequence Dictionary | picard |

┌───────────────────────────────┐ ┌─────────────────────────────┐ │GTF (see get genome annotation)│ │DNA Fasta (See get dna fasta)│ └────────────────────┬──────────┘ └────────┬────────────────────┘ │ │ │ │ ┌──────▼───────────────────────────▼─────┐ │Extract transcripts sequences (gffread) │ └──────┬───────────────────────────┬─────┘ │ │ │ │ ┌────────────────────▼────┐ ┌────────▼───────────────────────────┐ │Index sequence (samtools)│ │Create sequence dictionary (Picard) │ └─────────────────────────┘ └────────────────────────────────────┘

Get cDNA sequences

| Step | Commands | | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Extract coding transcripts from above GTF | Agat | | Extract coding sequences from above DNA Fasta and GTF | gffread | | Index DNA sequence | samtools | | Creatse sequence Dictionary | picard |

┌───────────────────────────────┐ ┌─────────────────────────────┐ │GTF (see get genome annotation)│ │DNA Fasta (See get dna fasta)│ └────────────────────┬──────────┘ └────────┬────────────────────┘ │ │ │ │ ┌──────▼───────────────────────────▼─────┐ │Extract cDNA        sequences (gffread) │ └──────┬───────────────────────────┬─────┘ │ │ │ │ ┌────────────────────▼────┐ ┌────────▼───────────────────────────┐ │Index sequence (samtools)│ │Create sequence dictionary (Picard) │ └─────────────────────────┘ └────────────────────────────────────┘

Get dbSNP variants

| Step | Commands | | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | Download dbSNP variants | ensembl-variation | | Filter non-canonical chromosomes | pyfaidx + BCFTools | | Index variants | tabix |

``` ┌──────────────────────────────────────────┐
│Download dbSNP variants (wget + bcftools) │
└──────────┬───────────────────────────────┘
│
│
┌──────────▼───────────────────────────────────────────┐ │Remove non-canonical chromosomes (bcftools + bedtools)│ └──────────┬───────────────────────────────────────────┘ │
│
┌──────────▼─────────────┐
│Index variants (tabix) │
└────────────────────────┘

```

Get transcriptid, geneid, and gene_name correspondancy

| Step | Commands | | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Extract geneid <-> genename correspondancy | pyroe | | Extract transcriptid <-> geneid <-> gene_name | Agat + XSV |

┌────────────────────────────────┐ │Genome annotation (see get GTF) ├──────────────────┐ └──────┬─────────────────────────┘ │ │ │ │ │ ┌──────▼──────────────────────────────┐ ┌────────▼─────────────────────────────────────────────┐ │Extract gene_id <-> gene_name (pyroe)│ │Extract gene_id <-> gene_name <-> transcript_id (Agat)│ └──────┬──────────────────────────────┘ └────────┬─────────────────────────────────────────────┘ │ │ │ │ ┌──────▼─────┐ ┌────────▼────┐ │Format (XSV)│ │Format (XSV) │ └────────────┘ └─────────────┘

Get blacklisted regions

| Step | Commands | | ---------------------------- | -------------------------------------------------------------------------------------------- | | Download blacklisted regions | Github source | | Merge overlapping intervals | bedtools |

``` ┌────────────────────────────────┐
│Download known blacklists (wget)│
└────────────┬───────────────────┘
│
│
┌────────────▼──────────────────────────┐ │Merge overlapping intervals (bedtools) │ └───────────────────────────────────────┘

```

GenePred format

| Step | Commands | | --------------- | -------------------------------------------------------------------------------------------------- | | GTF to GenePred | UCSC-tools |

``` ┌────────────────────────────────┐
│Genome annotation (see get GTF) │
└────────────┬───────────────────┘
│
│
┌────────────▼──────────────┐ │GTFtoGenePred (UCSC-tools) │ └───────────────────────────┘

```

2bit format

| Step | Commands | | --------------- | -------------------------------------------------------------------------------------------------- | | Fasta to 2bit | UCSC-tools |

``` ┌────────────────────────────────┐
│Genome sequence (see get Fasta) │
└────────────┬───────────────────┘
│
│
┌────────────▼──────────────┐ │FaToTwoBit (UCSC-tools) │ └───────────────────────────┘

```

STAR index

| Step | Commands | | --------------- | -------------------------------------------------------------------------------------------------- | | STAR index | STAR |

``` ┌────────────────────────────────┐
│Genome sequence (see get DNA) │
└────────────┬───────────────────┘
│
│
┌───────▼────┐ │ STAR index │ └────────────┘

```

Bowtie2 index

| Step | Commands | | --------------- | -------------------------------------------------------------------------------------------------- | | Bowtie2 build | Bowtie2 build |

``` ┌────────────────────────────────┐
│Genome sequence (see get DNA) │
└────────────┬───────────────────┘
│
│
┌───────▼────┐ │ STAR index │ └────────────┘

```

Salmon decoy aware gentrome index

| Step | Commands | | --------------- | -------------------------------------------------------------------------------------------------- | | Generate decoy | Bash | | Salmon index | Salmon |

┌─────────────────────────────┐ ┌─────────────────────────────────────┐ │Genome sequence (see get DNA)│ │Transcriptome sequence (see get cDNA)│ └──────────────────────────┬──┘ └─────┬───────────────────────────────┘ │ │ │ │ │ │ ┌────▼─────────────────▼────┐ │Generate decoy and gentrome│ └─────────────┬─────────────┘ │ ┌─────────────────┐ │ ┌───────────────┐ │Gentrome sequence◄────────────────┴─────►Decoy sequences│ └────────────┬────┘ └────┬──────────┘ │ │ │ │ │ ┌──────────────┐ │ └───────► Salmon index ◄─────────┘ └──────────────┘