Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/bentoml/llm-inference-handbook

Everything you need to know about LLM inference

https://github.com/bentoml/llm-inference-handbook

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (5.3%) to scientific vocabulary

Keywords

inference-handbook inference-infrastructure inference-optimization llm llm-inference
Last synced: 10 months ago · JSON representation

Repository

Everything you need to know about LLM inference

Basic Info
  • Host: GitHub
  • Owner: bentoml
  • License: mit
  • Language: TypeScript
  • Default Branch: main
  • Homepage: http://www.bentoml.com/llm
  • Size: 10.7 MB
Statistics
  • Stars: 223
  • Watchers: 2
  • Forks: 21
  • Open Issues: 3
  • Releases: 0
Topics
inference-handbook inference-infrastructure inference-optimization llm llm-inference
Created 12 months ago · Last pushed 10 months ago
Metadata Files
Readme License

README.md

📖 LLM Inference Handbook

This repository contains the source content for LLM Inference Handbook, a practical guide for understanding, optimizing, scaling, and operating LLM inference.

Twitter Community

🔧 Local preview

To preview the site locally:

bash pnpm install pnpm start

It will be running at http://localhost:3000/llm/.

🤝 Contributing

Contributions are welcome! Feel free to open issues, suggest improvements, or submit pull requests.

📄 License

This project is licensed under the MIT License.

Owner

  • Name: BentoML
  • Login: bentoml
  • Kind: organization
  • Location: San Francisco

The most flexible way to serve AI models in production

GitHub Events

Total
  • Watch event: 41
  • Push event: 15
  • Pull request review comment event: 1
  • Pull request review event: 5
  • Pull request event: 27
  • Fork event: 3
Last Year
  • Watch event: 41
  • Push event: 15
  • Pull request review comment event: 1
  • Pull request review event: 5
  • Pull request event: 27
  • Fork event: 3

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 0
  • Total pull requests: 20
  • Average time to close issues: N/A
  • Average time to close pull requests: about 5 hours
  • Total issue authors: 0
  • Total pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 13
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 20
  • Average time to close issues: N/A
  • Average time to close pull requests: about 5 hours
  • Issue authors: 0
  • Pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 13
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • Sherlock113 (14)
  • jinyang1994 (4)
Top Labels
Issue Labels
Pull Request Labels