https://github.com/awslabs/extending-the-context-length-of-open-source-llms
https://github.com/awslabs/extending-the-context-length-of-open-source-llms
Science Score: 26.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.2%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: awslabs
- License: apache-2.0
- Language: Python
- Default Branch: main
- Size: 37.2 MB
Statistics
- Stars: 55
- Watchers: 8
- Forks: 6
- Open Issues: 4
- Releases: 0
Metadata Files
README.md
extendingcontextlengthOSLLMs
This repository contains sample code to make it easier to extend the context length of the Falcon 40B model. Please read the blog post for further instructions.
This code may increase the context length of the Falcon-40b-sft-top-1-560. Before using this code, or the Falcon model, you should perform your own independent assessment, and take measures to ensure that your use would comply with your own specific quality control practices and standards, and that your use would comply with the local rules, laws, regulations, licenses and terms that apply to you, and your content. The author of this code does not make any representations, warranties or guarantees that your use of this code or the model will result in a particular outcome or result.
Getting Started
See the medium blog post:
https://medium.com/@chenwuperth/extend-the-context-length-of-falcon40b-to-10k-85d81d32146f
License
This code is released under the Apache 2.0 software license
Citation
This code is a modification of the layers.py released by HuggingFace under an Apache 2.0 license in the following location:
https://github.com/huggingface/text-generation-inference/blob/v0.9.2/server/textgenerationserver/utils/layers.py
Owner
- Name: Amazon Web Services - Labs
- Login: awslabs
- Kind: organization
- Location: Seattle, WA
- Website: http://amazon.com/aws/
- Repositories: 914
- Profile: https://github.com/awslabs
AWS Labs
GitHub Events
Total
- Issues event: 2
- Watch event: 8
- Push event: 13
- Pull request event: 2
- Fork event: 1
Last Year
- Issues event: 2
- Watch event: 8
- Push event: 13
- Pull request event: 2
- Fork event: 1
Issues and Pull Requests
Last synced: about 2 years ago
All Time
- Total issues: 9
- Total pull requests: 4
- Average time to close issues: 13 days
- Average time to close pull requests: about 22 hours
- Total issue authors: 8
- Total pull request authors: 3
- Average comments per issue: 2.44
- Average comments per pull request: 0.25
- Merged pull requests: 4
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 9
- Pull requests: 4
- Average time to close issues: 13 days
- Average time to close pull requests: about 22 hours
- Issue authors: 8
- Pull request authors: 3
- Average comments per issue: 2.44
- Average comments per pull request: 0.25
- Merged pull requests: 4
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- ontheway-arch (1)
- keyboardAnt (1)
- hyandell (1)
- aasthavar (1)
- timtensor (1)
- ehartye (1)
- chintanckg (1)
- ravilkashyap (1)
- csnelsonchu (1)
Pull Request Authors
- yinsong1986 (4)
- dependabot[bot] (3)
- jkitaok (1)
- yapweiyih (1)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- debian bullseye-slim build
- ghcr.io/huggingface/text-generation-inference 1.0.3 build
- pytorch-install latest build
- ghcr.io/huggingface/text-generation-inference 0.9.2 build
- ghcr.io/huggingface/text-generation-inference 0.9.2 build
- text_generation *