attentionlayer.jl
Science Score: 67.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 2 DOI reference(s) in README -
✓Academic publication links
Links to: arxiv.org, zenodo.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (12.5%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: DEEPDIP-project
- License: mit
- Language: Julia
- Default Branch: main
- Size: 1.7 MB
Statistics
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 3
- Releases: 5
Metadata Files
README.md
AttentionLayer
This package implements the attention mechanism as a Lux layer. It can then be used for closure modeling.
Install
julia
using Pkg
Pkg.add(url="git@github.com:DEEPDIP-project/AttentionLayer.jl.git")
Usage
You are probably interested in using the attentioncnn model, which is a built-in cnn that uses the attention mechanism.
Here is an example of how to use it:
- first you define the parameters of the model
julia
T = Float32 # the type of the data
N = 16 # size of the input
D = 2 # number of channels
rng = Xoshiro(123) # random number generator
r = [2, 2] # radii of the attention mechanism
c = [4, 2] # number of features of the intermediate layers
σ = [tanh, identity] # activation functions
b = [true, false] # use bias
emb_sizes = [8, 8] # size of the embeddings
patch_sizes = [8, 5] # size of the patches in which the attention mechanism is applied
n_heads = [2, 2] # number of heads of the attention mechanism
use_attention = [true, true] # use the attention at this layer
sum_attention = [false, false] # use attention in sum mode instead of concat mode (BUG)
- then you can call the model
julia
closure, θ, st = attentioncnn(
T = T,
N = N,
D = D,
data_ch = D,
radii = r,
channels = c,
activations = σ,
use_bias = b,
use_attention = use_attention,
emb_sizes = emb_sizes,
patch_sizes = patch_sizes,
n_heads = n_heads,
sum_attention = sum_attention,
rng = rng,
use_cuda = false,
)
Look in test/ for more examples about how to use the package.
How to Cite
If you use AttentionLayer.jl in your work, please cite using the reference given in CITATION.cff.
Contributing
If you want to make contributions of any kind, please first that a look into our contributing guide directly on GitHub or the contributing page on the website
Contributors
 SCiarella 💻 ⚠️ 🚧 |
Owner
- Name: DEEPDIP
- Login: DEEPDIP-project
- Kind: organization
- Email: p.rodriguez-sanchez@esciencecenter.nl
- Location: Netherlands
- Repositories: 8
- Profile: https://github.com/DEEPDIP-project
Citation (CITATION.cff)
# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!
cff-version: 1.2.0
title: AttentionLayer.jl
message: >-
If you use this software, please cite it using the
metadata from this file.
type: software
authors:
- given-names: Simone
family-names: Ciarella
email: simoneciarella@gmail.com
affiliation: Netherlands eScience Center
orcid: 'https://orcid.org/0000-0002-9247-139X'
GitHub Events
Total
- Create event: 26
- Release event: 3
- Issues event: 7
- Delete event: 17
- Member event: 1
- Issue comment event: 10
- Push event: 93
- Pull request event: 48
Last Year
- Create event: 26
- Release event: 3
- Issues event: 7
- Delete event: 17
- Member event: 1
- Issue comment event: 10
- Push event: 93
- Pull request event: 48
Issues and Pull Requests
Last synced: 10 months ago
All Time
- Total issues: 2
- Total pull requests: 21
- Average time to close issues: 25 days
- Average time to close pull requests: 15 days
- Total issue authors: 1
- Total pull request authors: 3
- Average comments per issue: 0.0
- Average comments per pull request: 0.24
- Merged pull requests: 13
- Bot issues: 0
- Bot pull requests: 10
Past Year
- Issues: 2
- Pull requests: 21
- Average time to close issues: 25 days
- Average time to close pull requests: 15 days
- Issue authors: 1
- Pull request authors: 3
- Average comments per issue: 0.0
- Average comments per pull request: 0.24
- Merged pull requests: 13
- Bot issues: 0
- Bot pull requests: 10
Top Authors
Issue Authors
- SCiarella (3)
Pull Request Authors
- github-actions[bot] (15)
- SCiarella (12)
- dependabot[bot] (2)
- allcontributors[bot] (1)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- JuliaRegistries/TagBot v1 composite
- julia-actions/cache v2 composite
- julia-actions/setup-julia v2 composite
- actions/checkout v4 composite
- julia-actions/cache v2 composite
- julia-actions/setup-julia v2 composite
- actions/cache v4 composite
- actions/checkout v4 composite
- actions/setup-python v5 composite
- julia-actions/cache v2 composite
- julia-actions/setup-julia v2 composite
- lycheeverse/lychee-action v2 composite
- actions/checkout v4 composite
- actions/setup-python v5 composite
- peter-evans/create-pull-request v7 composite
- actions/checkout v4 composite
- codecov/codecov-action v4 composite
- julia-actions/cache v2 composite
- julia-actions/julia-buildpkg v1 composite
- julia-actions/julia-processcoverage v1 composite
- julia-actions/julia-runtest v1 composite
- julia-actions/setup-julia v2 composite