Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/aashish75/transformer-attention

https://github.com/aashish75/transformer-attention

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (2.4%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: Aashish75
  • Language: Jupyter Notebook
  • Default Branch: main
  • Size: 1.51 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created over 1 year ago · Last pushed about 1 year ago
Metadata Files
Readme

README.md

This project contains the following-

-A loose implementation of the attention mechanism, from the "Attention Is All You Need" paper

-Sequence to sequence translation using a transformer

-Visualising the encoder, decoder and cross attention using bertviz.

The pipeline followed is -

  1. Prepare the data.
  2. Get the embeddings of the training partition.
  3. Write the elemnents of the self-attention mechanism.
  4. Train a transformer model
  5. Evaluate the model.
  6. Visualize the attention mechanism.

Owner

  • Name: Aashish M
  • Login: Aashish75
  • Kind: user

I code

GitHub Events

Total
  • Push event: 6
  • Create event: 2
Last Year
  • Push event: 6
  • Create event: 2