https://github.com/aaltorse/llminferencepods
Repo for Kubernetes Docker Files and pods for AaltoSciComp LLM deployment
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (3.2%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
Repo for Kubernetes Docker Files and pods for AaltoSciComp LLM deployment
Basic Info
- Host: GitHub
- Owner: AaltoRSE
- Language: Shell
- Default Branch: main
- Size: 64.5 KB
Statistics
- Stars: 0
- Watchers: 4
- Forks: 0
- Open Issues: 0
- Releases: 0
Created over 2 years ago
· Last pushed over 1 year ago
Metadata Files
Readme
README.md
Kubernetes pod instructions for LLM deployment
This repository will contain all set up and Dockerfile routines for the LLM deployment. Each subfolder contains a type of pod that should be deployable along with some instructions for it.
The nginxPod folder contains some settings for the up-front nginx server, which will in the end perform an Auth check. The compose_test is a simple testing approach for the interaction between the nginx pod and the FastChat.
Owner
- Name: AaltoRSE
- Login: AaltoRSE
- Kind: organization
- Repositories: 38
- Profile: https://github.com/AaltoRSE
GitHub Events
Total
- Push event: 7
Last Year
- Push event: 7
Dependencies
FastChat/Dockerfile
docker
- python 3.10-slim-bullseye build
llama-cpp-python/Dockerfile
docker
- python 3-slim-bullseye build
nginxPod/Dockerfile
docker
- openresty/openresty alpine-apk build
FastChat/requirements.txt
pypi
- fschat *
llama-cpp-python/environment.yml
pypi
- llama-cpp-python *
llama-cpp-python/requirements.txt
pypi
- fschat *