Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/andstor/hpc-openai-server

https://github.com/andstor/hpc-openai-server

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.0%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: andstor
  • Language: Python
  • Default Branch: main
  • Size: 31.3 KB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created almost 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme

README.md

hpc-openai-server

Requirements

pip install -r requirements.txt

cp .env.template .env

Add your ngrok auth token to the newly created .env file.

Usage

Server

usage: main.py [-h] [--port PORT] [--basicauth BASICAUTH] [--ngrokauthtoken NGROKAUTHTOKEN] [--ngrokdomain NGROKDOMAIN]

Run the FastAPI server

optional arguments: -h, --help show this help message and exit --port PORT Port to run the FastAPI server --basicauth BASICAUTH Basic authentication "username:password" --ngrokauthtoken NGROKAUTHTOKEN Ngrok token --ngrokdomain NGROKDOMAIN Ngrok domain

Start the server with the following command: python main.py

This will start a local server and expose it to the internet using ngrok. The username and password for the server will be printed to the console. You can now access the server from any device with an internet connection by visiting the ngrok URL provided.

Client

The server is OpenAI comapatible and can be used with the OpenAI client.

```python from openai import OpenAI import base64

authkey = base64.b64encode("username1:password1".encode()).decode() client = OpenAI( baseurl="http://localhost:8000/v1", apikey="None", defaultheaders={"Authorization": f"Basic {auth_key}"}, )

messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is 2+2?"}]

res = client.chat.completions.create( messages=messages, model="Qwen/Qwen2.5-0.5B-Instruct", maxcompletiontokens=50, temperature=1, seed=422, n=1, stop=[], )

print(res.choices[0].message.content)

>>> 2 + 2 is equal to 4.

```

Owner

  • Name: André Storhaug
  • Login: andstor
  • Kind: user
  • Location: Trondheim 🇳🇴
  • Company: NTNU

🎓 CS PhD student @ Norwegian University of Science and Technology (NTNU)

GitHub Events

Total
  • Push event: 8
Last Year
  • Push event: 8

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements.txt pypi
  • accelerate *
  • duckduckgo_search *
  • fastapi *
  • gradio *
  • gradio_client *
  • huggingface-hub *
  • loguru *
  • ngrok *
  • openai *
  • pydantic *
  • python-dotenv *
  • scikit-image *
  • scipy *
  • transformers *
  • uvicorn *