Data

Tools

Indexes

Applications

Experiments

Open Source Science

fast-rag

https://github.com/kingjupyter/fast-rag

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.8%) to scientific vocabulary
Last synced: 9 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: KingJupyter
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Size: 20.5 MB
Statistics
  • Stars: 3
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created almost 2 years ago · Last pushed almost 2 years ago
Metadata Files
Readme Contributing License Citation Security

README.md

---

Build and explore efficient retrieval-augmented generative models and applications

![PyPI - Version](https://img.shields.io/pypi/v/fastrag) ![PyPI - Downloads](https://img.shields.io/pypi/dm/fastrag) :round_pushpin: Installation • :rocket: Components • :books: Examples • :red_car: Getting Started • :pill: Demos • :pencil2: Scripts • :bar_chart: Benchmarks

fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers with a comprehensive tool-set for advancing retrieval augmented generation.

Comments, suggestions, issues and pull-requests are welcomed! :heart:

[!IMPORTANT] Now compatible with Haystack v2+. Please report any possible issues you find.

:mega: Updates

  • 2024-05: fastRAG V3 is Haystack 2.0 compatible :fire:
  • 2023-12: Gaudi2 and ONNX runtime support; Optimized Embedding models; Multi-modality and Chat demos; REPLUG text generation.
  • 2023-06: ColBERT index modification: adding/removing documents; see IndexUpdater.
  • 2023-05: RAG with LLM and dynamic prompt synthesis example.
  • 2023-04: Qdrant DocumentStore support.

Key Features

  • Optimized RAG: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
  • Optimized for Intel Hardware: Leverage Intel extensions for PyTorch (IPEX), 🤗 Optimum Intel and 🤗 Optimum-Habana for running as optimal as possible on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
  • Customizable: fastRAG is built using Haystack and HuggingFace. All of fastRAG's components are 100% Haystack compatible.

:rocket: Components

For a brief overview of the various unique components in fastRAG refer to the Components Overview page.

LLM Backends
Intel Gaudi Accelerators Running LLMs on Gaudi 2
ONNX Runtime Running LLMs with optimized ONNX-runtime
OpenVINO Running quantized LLMs using OpenVINO
Llama-CPP Running RAG Pipelines with LLMs on a Llama CPP backend
Optimized Components
Embedders Optimized int8 bi-encoders
Rankers Optimized/sparse cross-encoders
RAG-efficient Components
ColBERT Token-based late interaction
Fusion-in-Decoder (FiD) Generative multi-document encoder-decoder
REPLUG Improved multi-document decoder
PLAID Incredibly efficient indexing engine

:round_pushpin: Installation

Preliminary requirements:

  • Python 3.8 or higher.
  • PyTorch 2.0 or higher.

To set up the software, install from pip or clone the project for the bleeding-edge updates. Run the following, preferably in a newly created virtual environment:

bash pip install fastrag

Extra Packages

There are additional dependencies that you can install based on your specific usage of fastRAG:

```bash

Additional engines/components

pip install fastrag[intel] # Intel optimized backend [Optimum-intel, IPEX] pip install fastrag[openvino] # Intel optimized backend using OpenVINO pip install fastrag[elastic] # Support for ElasticSearch store pip install fastrag[qdrant] # Support for Qdrant store pip install fastrag[colbert] # Support for ColBERT+PLAID; requires FAISS pip install fastrag[faiss-cpu] # CPU-based Faiss library pip install fastrag[faiss-gpu] # GPU-based Faiss library ```

To work with the latest version of fastRAG, you can install it using the following command:

bash pip install .

Development tools

bash pip install .[dev]

License

The code is licensed under the Apache 2.0 License.

Disclaimer

This is not an official Intel product.

Owner

  • Name: AI OK
  • Login: KingJupyter
  • Kind: user

AI is Strong

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Izsak"
  given-names: "Peter"
  orcid: "https://orcid.org/0000-0001-8354-6823"
- family-names: "Berchansky"
  given-names: "Moshe"
  orcid: "https://orcid.org/0000-0001-9227-8939"
- family-names: "Fleischer"
  given-names: "Daniel"
  orcid: "https://orcid.org/0000-0003-4031-4410"
- family-names: "Laperdon"
  given-names: "Ronen"
title: "fastRAG: Efficient Retrieval Augmentation and Generation Framework"
version: 1.0
license: Apache-2.0
date-released: 2023-02-16
url: "https://github.com/IntelLabs/fastrag"

GitHub Events

Total
Last Year

Dependencies

scripts/optimizations/embedders/requirements.txt pypi
  • aim *
  • evaluate *
  • intel-extension-for-pytorch *
  • mpi4py *
  • mteb *
  • optimum *
setup.py pypi