Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/astorfi/sparse-structured-attention

Sparse and structured neural attention mechanisms

https://github.com/astorfi/sparse-structured-attention

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.0%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Sparse and structured neural attention mechanisms

Basic Info
  • Host: GitHub
  • Owner: astorfi
  • License: bsd-3-clause
  • Language: Python
  • Default Branch: master
  • Size: 90.8 KB
Statistics
  • Stars: 0
  • Watchers: 3
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Fork of vene/sparse-structured-attention
Created about 8 years ago · Last pushed over 8 years ago

https://github.com/astorfi/sparse-structured-attention/blob/master/ 

# Sparse and structured attention mechanisms
[![Build Status](https://travis-ci.org/vene/sparse-structured-attention.svg?branch=master)](https://travis-ci.org/vene/sparse-structured-attention)
[![PyPI version](https://badge.fury.io/py/torchsparseattn.svg)](https://badge.fury.io/py/torchsparseattn)




--------------------------------------------------------------------------------

Efficient implementation of structured sparsity inducing
attention mechanisms: fusedmax, oscarmax and sparsemax.

Currently available for pytorch v0.2. Requires python
(3.6, 3.5, or 2.7), cython, numpy, scipy, scikit-learn,
and [lightning](http://contrib.scikit-learn.org/lightning/).

For details, check out our paper:

> Vlad Niculae and Mathieu Blondel
> A Regularized Framework for Sparse and Structured Neural Attention
> In: Proceedings of NIPS, 2017. 
> https://arxiv.org/abs/1705.07704 

See also:

> Andr F. T. Martins and Ramn Fernandez Astudillo
> From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
> In: Proceedings of ICML, 2016
> https://arxiv.org/abs/1602.02068

> X. Zeng and M. Figueiredo,
> The ordered weighted L1 norm: Atomic formulation, dual norm, and projections.
> eprint http://arxiv.org/abs/1409.4271

Owner

  • Name: Sina Torfi
  • Login: astorfi
  • Kind: user
  • Location: San Jose
  • Company: Meta

PhD & Developer working on Deep Learning, Computer Vision & NLP

GitHub Events

Total
Last Year