Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/bentoml/bentollamacpp

BentoML + llama.cpp

https://github.com/bentoml/bentollamacpp

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (2.7%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

BentoML + llama.cpp

Basic Info
  • Host: GitHub
  • Owner: bentoml
  • Language: Python
  • Default Branch: main
  • Size: 15.6 KB
Statistics
  • Stars: 0
  • Watchers: 3
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created about 1 year ago · Last pushed about 1 year ago
Metadata Files
Readme

README.md

Installation

```bash uv venv -p 3.11

For M1 Mac

CMAKEARGS="-DCMAKEOSXARCHITECTURES=arm64 -DCMAKEAPPLESILICONPROCESSOR=arm64 -DGGML_METAL=on" pip install --upgrade --verbose --force-reinstall --no-cache-dir llama-cpp-python

For Mac

CMAKEARGS="-DGGMLMETAL=on" pip install --upgrade --verbose --force-reinstall --no-cache-dir llama-cpp-python

uv pip install -r pyproject.toml

bentoml serve ```

If you want to use different models:

bash bentoml serve -f qwq.yaml

It will use Gemma 3 by default here.

Deploy

bash bentoml deploy

Owner

  • Name: BentoML
  • Login: bentoml
  • Kind: organization
  • Location: San Francisco

The most flexible way to serve AI models in production

GitHub Events

Total
  • Public event: 1
  • Push event: 9
  • Pull request event: 1
  • Create event: 2
Last Year
  • Public event: 1
  • Push event: 9
  • Pull request event: 1
  • Create event: 2

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 7
  • Total Committers: 2
  • Avg Commits per committer: 3.5
  • Development Distribution Score (DDS): 0.143
Past Year
  • Commits: 7
  • Committers: 2
  • Avg Commits per committer: 3.5
  • Development Distribution Score (DDS): 0.143
Top Committers
Name Email Commits
Aaron Pham c****t@a****z 6
jianshen92 j****2@g****m 1
Committer Domains (Top 20 + Academic)
aarnphm.xyz: 1

Issues and Pull Requests

Last synced: about 1 year ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

pyproject.toml pypi
  • bentoml ==1.4.8
  • fastapi >=0.115.12
  • huggingface-hub >=0.30.2
  • llama-cpp-python *
  • openai >=1.74.0