Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/deepset-ai/rag-with-nvidia-nims

🚀 Use NVIDIA NIMs with Haystack pipelines

https://github.com/deepset-ai/rag-with-nvidia-nims

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.9%) to scientific vocabulary

Keywords

ai deployment haystack llm
Last synced: 4 months ago · JSON representation

Repository

🚀 Use NVIDIA NIMs with Haystack pipelines

Basic Info
Statistics
  • Stars: 30
  • Watchers: 0
  • Forks: 8
  • Open Issues: 2
  • Releases: 0
Topics
ai deployment haystack llm
Created almost 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme

README.md

Build Air-Gapped RAG with Nvidia NIMs and Haystack

📚 This repository is accompanied by our article "Building RAG Applications with NVIDIA NIM and Haystack on K8s"

Info: This repo is set up to use models hosted and accessible via https://build.nvidia.com/

These models are already available and you can use them by creating yourself API keys through the platform. The project is set up so that you can change these models to NIM deployments by setting the model name and api_url in the NvidiaGenerator, NvidiaDocumentEmbedder and NvidiaTextEmbedder components.

👩🏻‍🍳 We also provide a notebook on Haystack Cookbooks that provide the same code and setup, only expecting self-hosted NIMs

Open In Colab

Run with Docker

  1. pip install -r requirements.txt
  2. Create a .env file and add NVIDIA_API_KEY (if you're using hosted models via https://build.nvidia.com/)
  3. docker-compose up
  4. hayhooks deploy rag.yaml
  5. Go to localhost:1416/docs to interact with your RAG pipeline

File Structure

  • indexing.py: This script preproecesses, embeds and writes ChipNemo.pdf into a Qdrant database
  • rag.py: This scripts runs a RAG pipeline with a NIM LLM and retrieval model.
  • Dockerfile: This is used by the docker-compose file to install dependencies
  • docker-compose.yml: This is the docker compose file we use to spin up a container for hayhooks (Haystack pipeline deployment) and Qdrant
  • rag.yaml: This is the serialized RAG pipeline which is the same as rag.py in YAML. We use this to deploy our pipeline with hayhooks
  • Open In Colab: This notebook shows you how you can set up your components to use self-hosted NIMs.

Owner

  • Name: deepset
  • Login: deepset-ai
  • Kind: organization
  • Email: hello@deepset.ai
  • Location: Berlin, Germany

Building enterprise search systems powered by latest NLP & open-source.

GitHub Events

Total
  • Issues event: 1
  • Watch event: 3
  • Fork event: 2
Last Year
  • Issues event: 1
  • Watch event: 3
  • Fork event: 2

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 1
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • skitsanos (1)
  • priyankapvernekar (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements.txt pypi
  • chroma-haystack *
  • haystack-ai *
  • nvidia-haystack *
  • pydantic ==1.9.0
Dockerfile docker
  • deepset/hayhooks v0.0.15 build
docker-compose.yml docker
  • qdrant/qdrant latest