Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/amazon-science/glean

https://github.com/amazon-science/glean

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.8%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: amazon-science
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Size: 6.74 MB
Statistics
  • Stars: 1
  • Watchers: 3
  • Forks: 0
  • Open Issues: 1
  • Releases: 0
Created over 1 year ago · Last pushed over 1 year ago
Metadata Files
Readme Contributing License Code of conduct

README.md

GLEAN: Generalized Category Discovery with Diverse and Quality-Enhanced LLM Feedback

Pipeline

This repository contains the implementation of the paper:

GLEAN: Generalized Category Discovery with Diverse and Quality-Enhanced LLM Feedback [Paper]
Henry Peng Zou, Siffi Singh, Yi Nian, Jianfeng He, Jason Cai, Saab Mansour, Hang Su

Setup

```bash conda create -n glean python=3.9 -y conda activate glean

install pytorch

conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia

install dependency

pip install -r requirements.txt pip install faiss-gpu==1.7.2 --no-cache-dir ``` To reproduce our paper results, make sure that you have the following package version installed: transformers==4.15.0, pytorch==2.1.0, 2.1.1 or 2.1.2, as we found that model performance may vary across different package versions, particularly with the transformers package.

Running

First, add you OpenAI API key in line 58 of the 'run.sh' file.

Pre-training, training and testing our model through the bash script: bash sh run.sh You can also add or change parameters in run.sh (More parameters are listed in init_parameter.py)

Bugs or Questions

If you have any questions related to the code or the project, feel free to email Henry Peng Zou (pzou3@uic.edu, penzou@amazon.com). If you encounter any problems when using the code, or want to report a bug, please also feel free to reach out to us. Please try to specify the problem with details so we can help you better and quicker!

Acknowledgement

This repo borrows some data and codes from Loop and JointMatch. We appreciate their great works!

Owner

  • Name: Amazon Science
  • Login: amazon-science
  • Kind: organization

GitHub Events

Total
  • Issues event: 4
  • Watch event: 3
  • Delete event: 1
  • Issue comment event: 9
  • Member event: 1
  • Push event: 7
  • Public event: 1
  • Pull request event: 1
  • Fork event: 1
Last Year
  • Issues event: 4
  • Watch event: 3
  • Delete event: 1
  • Issue comment event: 9
  • Member event: 1
  • Push event: 7
  • Public event: 1
  • Pull request event: 1
  • Fork event: 1

Dependencies

requirements.txt pypi
  • huggingface_hub ==0.8.1
  • matplotlib ==3.7.3
  • numpy ==1.23.5
  • openai ==0.28.0
  • pandas ==2.0.3
  • scikit_learn ==1.2.0
  • scipy ==1.9.3
  • seaborn ==0.13.2
  • sentence_transformers ==2.2.0
  • together ==1.4.1
  • tqdm ==4.66.4
  • transformers ==4.15.0