https://github.com/aashish75/blip-pathvqa-peft
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (4.8%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: Aashish75
- Language: Jupyter Notebook
- Default Branch: main
- Size: 521 KB
Statistics
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
blip-PathVQA-PEFT: Fine-Tuning BLIP on PathVQA with LoRA
This repository contains code for fine-tuning the BLIP model on the PathVQA dataset using Parameter-Efficient Fine-Tuning (PEFT) with LoRA (Low-Rank Adaptation). The goal is to optimize BLIP for pathology-based Visual Question Answering (VQA) while reducing computational overhead.
🚀 Project Overview
- Fine-tunes Salesforce/blip-vqa-base on the PathVQA dataset.
- Implements LoRA (Low-Rank Adaptation) instead of full fine-tuning to reduce memory usage while maintaining strong performance.
- Compares LoRA fine-tuning vs. traditional full fine-tuning.
- Evaluates model performance before and after fine-tuning.
📂 Dataset: PathVQA
- The dataset is loaded from the Hugging Face
datasetslibrary: ```python from datasets import loaddataset dataset = loaddataset("flaviagiammarino/path-vqa")
📊 Evaluation Results
The training significantly increased the capabilities of the base BLIP model in pathological visual question answering (yes/no questions).
🔹 After LoRA Fine-Tuning
| Metric | Score | |-------------|----------| | Accuracy | 85.19% | | Precision | 85.89% | | Recall | 86.84% | | F1 Score | 86.36% |
🔹 Before Fine-Tuning (BLIP Base Model)
| Metric | Score | |-------------|----------| | Accuracy | 51.64% | | Precision | 53.44% | | Recall | 81.22% | | F1 Score | 64.47% |
🔹 Overall Improvements
| Metric | Increase | |-------------|-------------| | Accuracy | +33.55% | | Precision | +32.45% | | Recall | +5.62% | | F1 Score | +21.89% |
✅ LoRA significantly improves performance while being computationally efficient!
Owner
- Name: Aashish M
- Login: Aashish75
- Kind: user
- Repositories: 1
- Profile: https://github.com/Aashish75
I code
GitHub Events
Total
- Watch event: 1
- Push event: 16
- Create event: 2
Last Year
- Watch event: 1
- Push event: 16
- Create event: 2