https://github.com/chenliu-1996/brainfm
Science Score: 26.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.6%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: ChenLiu-1996
- Language: Jupyter Notebook
- Default Branch: main
- Size: 585 MB
Statistics
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
Readme.md
Brain Foundation Model
Krishnaswamy Lab, Yale University
Steps to reproduce
- Data preparation.
If you work on Misha, you can find the preprocessed data at
/gpfs/radev/home/cl2482/project/BrainFM/data/Dynamic_Natural_Vision.
1.1 Download Dynamic Natural Vision data.
cd src/data_download
python download_DNV.py
1.2 Unzip (sorry this step is messy) The desired structure is: data/DynamicNaturalVision/{subject1,subject2,subject3,video} Under {subject1,subject2,subject3} there are {fmri/smri} folders. Under {video} there are {seg1.mp4,seg2.mp4,...,test5.mp4}.
1.3 Preprocess the video. This will create a {videoframes} folder. ``` cd src/preprocessing python preprocessDNV_videos.py ```
1.4 Preprocess the fMRI. This will create a {fMRIscans} folder. ``` cd src/preprocessing python preprocessDNV_fmri.py ```
- Train.
cd src/ python main.py --batch-size 4 --max-training-iters 64 --max-validation-iters 64
Dependencies
We developed the codebase in a miniconda environment. How we created the conda environment: ```
Optional: Update to libmamba solver.
conda update -n base conda conda install -n base conda-libmamba-solver conda config --set solver libmamba
conda create --name brainfm pytorch==2.1.0 torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia -c anaconda -c conda-forge -y conda activate brainfm conda install scikit-learn scikit-image pillow matplotlib seaborn tqdm -c pytorch -c anaconda -c conda-forge -y conda install accelerate -c conda-forge -y conda install nibabel -y python -m pip install opencv-python python -m pip install torchgeometric einops python -m pip install git+https://github.com/openai/CLIP.git python -m pip install kornia open-clip-torch omegaconf python -m pip install diffusers["torch"]==0.21.4 transformers huggingfacehub==0.25.2 python -m pip install xformers==0.0.22.post7 python -m pip install dalle2-pytorch==1.15.6 python -m pip install natsort python -m pip install lpips
For NeuroClips
python -m pip install webdataset pytorch-lightning ```
Owner
- Name: Chen Liu
- Login: ChenLiu-1996
- Kind: user
- Location: New Haven
- Company: Yale University
- Website: https://chenliu-1996.github.io/
- Twitter: ChenLiu_1996
- Repositories: 5
- Profile: https://github.com/ChenLiu-1996
CS PhD student at @KrishnaswamyLab, @YaleUniversity. Reviewing Committee member at NeurIPS, ICLR, ICML.
GitHub Events
Total
- Watch event: 1
- Push event: 46
- Public event: 1
Last Year
- Watch event: 1
- Push event: 46
- Public event: 1