LLMlight

LLMlight is a Python package for running Large Language Models (LLMs) locally with minimal dependencies. It provides a simple interface to interact with various LLM models, including support for GGUF models and local API endpoints.

🌟 Key Features

• Local LLM Support: Run LLMs locally with minimal dependencies
• Full Promp Control:
  • Query
  • Instructions
  • System
  • Context
  • Response Format
  • Automatic formatting
  • Temperature
  • Top P
• Single Endpoint will Connect All Local Models: Compatible with various models including:
  • Hermes-3-Llama-3.2-3B
  • Mistral-7B-Grok
  • OpenHermes-2.5-Mistral-7B
  • Gemma-2-9B-IT
• Flexible Embedding Methods: Support for multiple embedding approaches:
  • TF-IDF for structured documents
  • Bag of Words (BOW)
  • BERT for free text
  • BGE-Small
• Advanced Retrieval Methods:
  • Naive RAG with fixed chunking
  • RSE (Relevant Segment Extraction)
• Advanced Preprocessing Methods: Advanced reasoning capabilities for complex queries.
  • Global-reasoning
  • chunk-wise
• Local Memory:
  • Video memory for storage
• PDF Processing: Built-in support for reading and processing PDF documents

📚 Documentation & Resources

• Documentation
• Blog Posts
• GitHub Issues

🚀 Quick Start

Installation

```bash

Install from PyPI

pip install LLMlight

Install from GitHub

pip install git+https://github.com/erdogant/LLMlight ```

Basic Usage with Endpoint

```python from LLMlight import LLMlight

Initialize with default settings

client = LLMlight(endpoint='http://localhost:1234/v1/chat/completions')

Run a simple query

response = client.prompt('What is the capital of France?', context='The capital of France is Amsterdam.', instructions='Do not argue with the information in the context. Only return the information from the context.') print(response)

According to the provided context, the capital of France is Amsterdam.

```

📊 Examples

1. Basic Usage with Local GGUF

```python from LLMlight import LLMlight

Use with a local GGUF client

client = LLMlight(endpoint='path/to/your/client.gguf')

Run a simple query

response = client.prompt('What is the capital of France?', context='The capital of France is Amsterdam.', instructions='Do not argue with the information in the context. Only return the information from the context.') print(response)

According to the provided context, the capital of France is Amsterdam.

```

2. Using with LM Studio

```python from LLMlight import LLMlight

Initialize with LM Studio endpoint

client = LLMlight(endpoint="http://localhost:1234/v1/chat/completions")

Run queries

response = client.prompt('Explain quantum computing in simple terms') ```

3. Check Available Models at Endpoint

```python from LLMlight import LLMlight

Initialize client

from LLMlight import LLMlight client = LLMlight(verbose='info')

modelnames = client.getavailablemodels(validate=False) print(modelnames)

```

3. Query against PDF files

```python from LLMlight import LLMlight

Initialize client

client = LLMlight()

Read PDF

context = client.readpdf(r'path/to/document.pdf', returntype='string')

Query the document

response = client.prompt('Summarize the main points of this document', context=context)

print(response)

```

4. Global Reasoning

```python from LLMlight import LLMlight

Initialize client

client = LLMlight(preprocessing='global_reasoning')

Read PDF

context = client.readpdf(r'path/to/document.pdf', returntype='string')

Query about the document

response = client.prompt('Summarize the main points of this document', context=context, instructions='Do not argue with the information in the context. Only return the information from the context.')

print(response)

```

5. Creating Local Memory Database

```python

Import library

from LLMlight import LLMlight

Initialize with default settings

client = LLMlight(preprocessing=None, retrieval_method=None)

Load existing video memory

client.memoryinit(pathtomemory="knowledgebase.mp4")

Append more documents: PDF/txt/etc files

filepaths = [r'c://pathtoyourfiles//article1.pdf', r'c://pathtoyourfiles//myfile.txt'] client.memoryadd(inputfiles=filepaths)

Add text chunks if you like

client.memory_add(text=['Apes like USB sticks', 'Trees are mainly yellow'])

Save Memory to disk. You can either create new one or overwite existing one.

client.memorysave(filepath="knowledgebasewithmore_data.mp4", overwrite=False)

Run a simple query

response = client.prompt('What do apes like?', instructions='Only return the information from the context. Answer with maximum of 3 words, and starts with "Apes like: "') print(response)

response = client.prompt('What is the capital of France?', context='The capital of France is Amsterdam.', instructions='Do not argue with the information in the context. Only return the information from the context.') print(response)

response = client.prompt('Provide a summary of HyperSpectral from the pdf or text file.', instructions='Do not argue with the information in the context. Only return the information from the context.') print(response)

```

6. Load Local Memory Database

```python

Import library

from LLMlight import LLMlight

Initialize with default settings

client = LLMlight(preprocessing=None, retrievalmethod=None, pathtomemory="knowledgebase.mp4")

Create queries

response = client.prompt('What do apes like?', instructions='Only return the information from the context. Answer with maximum of 3 words, and starts with "Apes like: "') print(response)

```

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

👥 Contributors

👨‍💻 Maintainer

• Erdogan Taskesen (@erdogant)

☕ Support

This library is free and open source. If you find it useful, consider supporting its development:

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.