Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/csinva/tree-prompt

Tree prompting: easy-to-use scikit-learn interface for improved prompting.

https://github.com/csinva/tree-prompt

Science Score: 46.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ✓
    Committers with academic emails
    1 of 1 committers (100.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.3%) to scientific vocabulary

Keywords

ai artificial-intelligence classification controllability decision-tree huggingface interpretability language-model language-models llm llm-inference machine-learning prompt-engineering prompting scikit-learn
Last synced: 5 months ago · JSON representation

Repository

Tree prompting: easy-to-use scikit-learn interface for improved prompting.

Basic Info
Statistics
  • Stars: 40
  • Watchers: 3
  • Forks: 4
  • Open Issues: 1
  • Releases: 0
Topics
ai artificial-intelligence classification controllability decision-tree huggingface interpretability language-model language-models llm llm-inference machine-learning prompt-engineering prompting scikit-learn
Created over 2 years ago · Last pushed over 2 years ago
Metadata Files
Readme

readme.md

🌳 Tree Prompting 🌳

Tree Prompting: Efficient Task Adaptation without Fine-Tuning, code for the Tree-prompt paper.

Tree Prompting uses training examples to learn a tree of prompts to make a classification, yielding higher accuracy and better efficiency that baseline ensembles.

Quickstart

Installation: pip install treeprompt (or clone this repo and pip install -e .)

```python from treeprompt.treeprompt import TreePromptClassifier import datasets import numpy as np from sklearn.tree import plot_tree import matplotlib.pyplot as plt

set up data

rng = np.random.defaultrng(seed=42) dsettrain = datasets.loaddataset('rottentomatoes')['train'] dsettrain = dsettrain.select(rng.choice( len(dsettrain), size=100, replace=False)) dsetval = datasets.loaddataset('rottentomatoes')['validation'] dsetval = dsetval.select(rng.choice( len(dset_val), size=100, replace=False))

set up arguments

prompts = [ "This movie is", " Positive or Negative? The movie was", " The sentiment of the movie was", " The plot of the movie was really", " The acting in the movie was", ] verbalizer = {0: " Negative.", 1: " Positive."} checkpoint = "gpt2"

fit model

m = TreePromptClassifier( checkpoint=checkpoint, prompts=prompts, verbalizer=verbalizer, cachepromptfeaturesdir=None, # 'cachepromptfeaturesdir/gp2', ) m.fit(dsettrain["text"], dsettrain["label"])

compute accuracy

preds = m.predict(dsetval['text']) print('\nTree-Prompt acc (val) ->', np.mean(preds == dsetval['label'])) # -> 0.7

compare to accuracy for individual prompts

for i, prompt in enumerate(prompts): print(i, prompt, '->', m.promptaccs[i]) # -> 0.65, 0.5, 0.5, 0.56, 0.51

visualize decision tree

plottree( m.clf, fontsize=10, featurenames=m.featurenames, classnames=list(verbalizer.values()), filled=True, ) plt.show() ```

Reference: r @misc{morris2023tree, title={Tree Prompting: Efficient Task Adaptation without Fine-Tuning}, author={John X. Morris and Chandan Singh and Alexander M. Rush and Jianfeng Gao and Yuntian Deng}, year={2023}, eprint={2310.14034}, archivePrefix={arXiv}, primaryClass={cs.CL} }

Reproducing experiments

See the full code for reproducing all experiments in the paper at https://github.com/csinva/tree-prompt-experiments

Owner

  • Name: Chandan Singh
  • Login: csinva
  • Kind: user
  • Location: Microsoft research
  • Company: Senior researcher

Senior researcher @Microsoft interpreting ML models in science and medicine. PhD from UC Berkeley.

GitHub Events

Total
  • Watch event: 10
  • Fork event: 1
Last Year
  • Watch event: 10
  • Fork event: 1

Committers

Last synced: 10 months ago

All Time
  • Total Commits: 5
  • Total Committers: 1
  • Avg Commits per committer: 5.0
  • Development Distribution Score (DDS): 0.0
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Chandan Singh c****h@b****u 5
Committer Domains (Top 20 + Academic)
berkeley.edu: 1

Issues and Pull Requests

Last synced: 8 months ago

All Time
  • Total issues: 1
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 2.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • srush (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 9 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 1
  • Total maintainers: 1
pypi.org: treeprompt

Tree prompting

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 9 Last month
Rankings
Dependent packages count: 7.3%
Stargazers count: 25.1%
Forks count: 30.0%
Average: 32.7%
Dependent repos count: 68.5%
Maintainers (1)
csinva
Last synced: 6 months ago