priorisat
Flexible Natural Language-based Image Data Downlink Prioritization for Nanosatellites
Science Score: 49.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 4 DOI reference(s) in README -
✓Academic publication links
Links to: mdpi.com, ieee.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.4%) to scientific vocabulary
Keywords
Repository
Flexible Natural Language-based Image Data Downlink Prioritization for Nanosatellites
Basic Info
- Host: GitHub
- Owner: ezrafielding
- License: mit
- Language: Jupyter Notebook
- Default Branch: main
- Homepage: https://www.mdpi.com/2226-4310/11/11/888
- Size: 8.57 MB
Statistics
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Topics
Metadata Files
README.md
PrioriSat: Flexible Natural Language-Based Image Data Downlink Prioritization for Nanosatellites
Abstract
Nanosatellites increasingly produce more data than can be downlinked within a reasonable time due to their limited bandwidth and power. Therefore, an on-board system is required to prioritize scientifically significant data for downlinking, as described by scientists. This paper determines whether natural language processing can be used to prioritize remote sensing images on CubeSats with more flexibility compared to existing methods. Two approaches implementing the same conceptual prioritization pipeline are compared. The first uses YOLOv8 and Llama2 to extract image features and compare them with text descriptions via cosine similarity. The second approach employs CLIP, fine-tuned on remote sensing data, to achieve the same. Both approaches are evaluated on real nanosatellite hardware, the VERTECS Camera Control Board. The CLIP approach, particularly the ResNet50-based model, shows the best performance in prioritizing and sequencing remote sensing images. This paper demonstrates that on-orbit prioritization using natural language descriptions is viable and allows for more flexibility than existing methods.
Oriented Bounding Box (OBB) Approach
Uses fine-tuned YOLOv8 and Llama2 models as image and text feature extractors, respectively.
Two YOLOv8 models with different image input sizes were trained (960 px and 1280 px).
CLIP Approach
Uses CLIP fine-tuned for RS images.
ResNet50 and ViT-B-16 -based CLIP models were trained.
Datasets
- DOTA-v1.5: https://captain-whu.github.io/DOTA/index.html
- NWPU-Captions: https://ieeexplore.ieee.org/document/9866055
Citation
Fielding, E.; Hanazawa, A. Flexible Natural Language-Based Image Data Downlink Prioritization for Nanosatellites. Aerospace 2024, 11, 888. https://doi.org/10.3390/aerospace11110888
Owner
- Name: Ezra Fielding
- Login: ezrafielding
- Kind: user
- Location: Kitakyushu, Japan
- Company: Kyushu Institute of Technology
- Twitter: ezrafielding
- Repositories: 2
- Profile: https://github.com/ezrafielding
GitHub Events
Total
- Push event: 4
Last Year
- Push event: 4
Dependencies
- ultralytics ==8.0.220