https://github.com/anemily-machina/isotropy_transformers
A repoisitory for the Paper: Anisotropy is Not Inherent to Transformers
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.7%) to scientific vocabulary
Repository
A repoisitory for the Paper: Anisotropy is Not Inherent to Transformers
Basic Info
- Host: GitHub
- Owner: anemily-machina
- Language: Python
- Default Branch: main
- Size: 6.21 MB
Statistics
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
Anisotropy is Not Inherent to Transformers
A repository to go along with the NA ACL 2024 paper (Link to paper)
graphs/svd reducation
2D SVD reduction graphs for all models examined in the paper. You can even seen how the grouping changes during trianing.
src/
Currently just two files. One to reproduce the partition function results and a demo file to see how our faster cosine method works.
Will probabily add more here, but if there's anything you'd like me to focus on let me know!
anemily.machina@uwo.ca
References
Please consider citing us if you use any graphs or code from this repository.
@inproceedings{machina-mercer-2024-anisotropy,
title = "Anisotropy is Not Inherent to Transformers",
author = "Machina, Anemily and
Mercer, Robert",
editor = "Duh, Kevin and
Gomez, Helena and
Bethard, Steven",
booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
month = jun,
year = "2024",
address = "Mexico City, Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.naacl-long.274",
pages = "4892--4907",
}
Owner
- Login: anemily-machina
- Kind: user
- Repositories: 1
- Profile: https://github.com/anemily-machina