Data

Tools

Indexes

Applications

Experiments

Open Source Science

trainllmv3

https://github.com/thistleknot/trainllmv3

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (1.8%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: thistleknot
  • License: apache-2.0
  • Language: Jupyter Notebook
  • Default Branch: main
  • Size: 33.2 MB
Statistics
  • Stars: 1
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created over 2 years ago · Last pushed over 2 years ago
Metadata Files
Readme License Citation

README.md

TrainLLMv3

framework for [eventually] achieving closed-book qa

see 4-phase-training (old document)

atm only doing phase II. Phase I is meant to pre-train

important files

resources/train.json

vars.py

pre-training

Generate_questions.py

writes outputted json to llm dataset format of contexts, questions, answers

preprawdata.py

other files of interest

export LDLIBRARYPATH=$LDLIBRARYPATH:/home/user/env/lib/python3.11/site-packages/nvidia/cusparse/lib/ export WANDBMODE=offline ./mezo-run.sh prepdata.py ./mezo-run.sh mezo-all-phases.py

after training

mezo-inference.py

Owner

  • Name: Turning out data tricks since 2006!
  • Login: thistleknot
  • Kind: user

Data Analyst MS in Data Science from CSUF (2019) BS in Business Analytics from CSUDH (2016)

Citation (CITATION.cff) 

cff-version: 1.2.0
title: "Stack More Layers Differently: High-Rank Training Through Low-Rank Updates"
version: 1.0.0
message: "If you use this software, please cite it as below."
authors:
  - family-names: "Lialin"
    given-names: "Vladislav"
  - family-names: "Shivagunde"
    given-names: "Namrata"
  - family-names: "Muckatira"
    given-names: "Sherin"
  - family-names: "Rumshisky"
    given-names: "Anna"
year: 2023
repository-code: "https://arxiv.org/abs/2307.05695"

GitHub Events

Total
  • Watch event: 1
Last Year
  • Watch event: 1