tantivy
Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust
Science Score: 54.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
1 of 162 committers (0.6%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.3%) to scientific vocabulary
Keywords
Keywords from Contributors
Repository
Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust
Basic Info
Statistics
- Stars: 13,687
- Watchers: 146
- Forks: 794
- Open Issues: 377
- Releases: 48
Topics
Metadata Files
README.md

Fast full-text search engine library written in Rust
If you are looking for an alternative to Elasticsearch or Apache Solr, check out Quickwit, our distributed search engine built on top of Tantivy.
Tantivy is closer to Apache Lucene than to Elasticsearch or Apache Solr in the sense it is not an off-the-shelf search engine server, but rather a crate that can be used to build such a search engine.
Tantivy is, in fact, strongly inspired by Lucene's design.
Benchmark
The following benchmark breaks down the performance for different types of queries/collections.
Your mileage WILL vary depending on the nature of queries and their load.

Details about the benchmark can be found at this repository.
Features
- Full-text search
- Configurable tokenizer (stemming available for 17 Latin languages) with third party support for Chinese (tantivy-jieba and cang-jie), Japanese (lindera, Vaporetto, and tantivy-tokenizer-tiny-segmenter) and Korean (lindera + lindera-ko-dic-builder)
- Fast (check out the :racehorse: :sparkles: benchmark :sparkles: :racehorse:)
- Tiny startup time (<10ms), perfect for command-line tools
- BM25 scoring (the same as Lucene)
- Natural query language (e.g.
(michael AND jackson) OR "king of pop") - Phrase queries search (e.g.
"michael jackson") - Incremental indexing
- Multithreaded indexing (indexing English Wikipedia takes < 3 minutes on my desktop)
- Mmap directory
- SIMD integer compression when the platform/CPU includes the SSE2 instruction set
- Single valued and multivalued u64, i64, and f64 fast fields (equivalent of doc values in Lucene)
&[u8]fast fields- Text, i64, u64, f64, dates, ip, bool, and hierarchical facet fields
- Compressed document store (LZ4, Zstd, None)
- Range queries
- Faceted search
- Configurable indexing (optional term frequency and position indexing)
- JSON Field
- Aggregation Collector: histogram, range buckets, average, and stats metrics
- LogMergePolicy with deletes
- Searcher Warmer API
- Cheesy logo with a horse
Non-features
Distributed search is out of the scope of Tantivy, but if you are looking for this feature, check out Quickwit.
Getting started
Tantivy works on stable Rust and supports Linux, macOS, and Windows.
- Tantivy's simple search example
- tantivy-cli and its tutorial -
tantivy-cliis an actual command-line interface that makes it easy for you to create a search engine, index documents, and search via the CLI or a small server with a REST API. It walks you through getting a Wikipedia search engine up and running in a few minutes. - Reference doc for the last released version
How can I support this project?
There are many ways to support this project.
- Use Tantivy and tell us about your experience on Discord or by email (paul.masurel@gmail.com)
- Report bugs
- Write a blog post
- Help with documentation by asking questions or submitting PRs
- Contribute code (you can join our Discord server)
- Talk about Tantivy around you
Contributing code
We use the GitHub Pull Request workflow: reference a GitHub ticket and/or include a comprehensive commit message when opening a PR. Feel free to update CHANGELOG.md with your contribution.
Tokenizer
When implementing a tokenizer for tantivy depend on the tantivy-tokenizer-api crate.
Clone and build locally
Tantivy compiles on stable Rust. To check out and run tests, you can simply run:
bash
git clone https://github.com/quickwit-oss/tantivy.git
cd tantivy
cargo test
Companies Using Tantivy
FAQ
Can I use Tantivy in other languages?
- Python → tantivy-py
- Ruby → tantiny
You can also find other bindings on GitHub but they may be less maintained.
What are some examples of Tantivy use?
- seshat: A matrix message database/indexer
- tantiny: Tiny full-text search for Ruby
- lnx: adaptable, typo tolerant search engine with a REST API
- and more!
On average, how much faster is Tantivy compared to Lucene?
- According to our search latency benchmark, Tantivy is approximately 2x faster than Lucene.
Does tantivy support incremental indexing?
- Yes.
How can I edit documents?
- Data in tantivy is immutable. To edit a document, the document needs to be deleted and reindexed.
When will my documents be searchable during indexing?
- Documents will be searchable after a
commitis called on anIndexWriter. ExistingIndexReaders will also need to be reloaded in order to reflect the changes. Finally, changes are only visible to newly acquiredSearcher.
Owner
- Name: Quickwit OSS
- Login: quickwit-oss
- Kind: organization
- Email: hey@quickwit.io
- Website: https://quickwit.io
- Twitter: Quickwit_Inc
- Repositories: 36
- Profile: https://github.com/quickwit-oss
Quickwit OSS Project
Citation (CITATION.cff)
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- alias: Quickwit Inc.
website: "https://quickwit.io"
title: "tantivy"
version: 0.22.0
doi: 10.5281/zenodo.13942948
date-released: 2024-10-17
url: "https://github.com/quickwit-oss/tantivy"
Committers
Last synced: 9 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Paul Masurel | p****l@g****m | 1,564 |
| Pascal Seitz | p****z@g****m | 664 |
| trinity-1686a | t****y@q****o | 63 |
| dependabot[bot] | 4****] | 62 |
| Laurentiu Nicola | l****a@d****o | 53 |
| Adam Reichold | a****d | 38 |
| Paul Masurel | p****l@i****m | 29 |
| Kat Lim Ruiz | k****m@r****i | 26 |
| Evance Souamoro | e****0@g****m | 25 |
| Michael J. Curry | c****j@f****m | 24 |
| Antoine G | 3****h | 22 |
| Bruce Mitchener | b****r@g****m | 22 |
| boraarslan | b****n@o****m | 20 |
| k-yomo | k****y@g****m | 19 |
| Ashley Mannix | a****x@l****u | 18 |
| petr-tik | p****k | 17 |
| Dru Sellers | d****u@d****m | 17 |
| Adrien Guillo | a****o@g****m | 16 |
| François Massot | f****t@g****m | 16 |
| barrotsteindev | b****n@g****m | 16 |
| Vignesh Sarma K (വിഘ്നേഷ് ശ൪മ കെ) | v****a@g****m | 12 |
| Shikhar Bhushan | s****r@s****t | 11 |
| dependabot-preview[bot] | 2****] | 11 |
| Jason Wolfe | m****e@g****m | 9 |
| Rihards Krišlauks | r****s@g****m | 9 |
| Kryesh | 2****h | 8 |
| fdb-hiroshima | 3****a | 7 |
| lyj | s****1@g****m | 6 |
| Remi Dettai | r****i@g****m | 5 |
| Pasha Podolsky | p****y@m****m | 5 |
| and 132 more... | ||
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 294
- Total pull requests: 675
- Average time to close issues: 5 months
- Average time to close pull requests: 24 days
- Total issue authors: 141
- Total pull request authors: 79
- Average comments per issue: 2.28
- Average comments per pull request: 1.05
- Merged pull requests: 450
- Bot issues: 1
- Bot pull requests: 73
Past Year
- Issues: 74
- Pull requests: 224
- Average time to close issues: 4 days
- Average time to close pull requests: 14 days
- Issue authors: 51
- Pull request authors: 39
- Average comments per issue: 0.64
- Average comments per pull request: 0.59
- Merged pull requests: 108
- Bot issues: 1
- Bot pull requests: 46
Top Authors
Issue Authors
- PSeitz (49)
- fulmicoton (40)
- trinity-1686a (10)
- fmassot (10)
- inzanez (9)
- soasada (4)
- MochiXu (4)
- GeeWee (3)
- xrl (3)
- xingtanzjr (3)
- anshulgoel27 (3)
- jpountz (3)
- cyccbxhl (3)
- Barre (3)
- inboxsphere (3)
Pull Request Authors
- PSeitz (278)
- dependabot[bot] (73)
- trinity-1686a (48)
- fulmicoton (46)
- adamreichold (39)
- fmassot (11)
- PSeitz-dd (9)
- rdettai (8)
- ChillFish8 (7)
- kryesh (6)
- stuhood (6)
- philippemnoel (5)
- waywardmonkeys (5)
- LebranceBW (5)
- b41sh (4)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 30
-
Total downloads:
- cargo 62,628,663 total
- pypi 212,957 last-month
- Total docker downloads: 214,479,502
-
Total dependent packages: 161
(may contain duplicates) -
Total dependent repositories: 2,311
(may contain duplicates) - Total versions: 208
- Total maintainers: 8
crates.io: tantivy
Search engine library
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy/
- License: MIT
-
Latest release: 0.24.2
published 7 months ago
Rankings
Maintainers (2)
crates.io: ownedbytes
Expose data as static slice
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/ownedbytes/
- License: MIT
-
Latest release: 0.9.0
published 11 months ago
Rankings
Maintainers (2)
crates.io: tantivy-common
common traits and utility functions used by multiple tantivy subcrates
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy-common/
- License: MIT
-
Latest release: 0.10.0
published 6 months ago
Rankings
Maintainers (2)
crates.io: tantivy-query-grammar
Search engine library
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy-query-grammar/
- License: MIT
-
Latest release: 0.25.0
published 6 months ago
Rankings
Maintainers (2)
crates.io: tantivy-bitpacker
Tantivy-sub crate: bitpacking
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy-bitpacker/
- License: MIT
-
Latest release: 0.9.0
published 6 months ago
Rankings
Maintainers (2)
crates.io: fastfield_codecs
Fast field codecs used by tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/fastfield_codecs/
- License: MIT
-
Latest release: 0.3.1
published about 3 years ago
Rankings
Maintainers (2)
crates.io: tantivy-tokenizer-api
Tokenizer API of tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy-tokenizer-api/
- License: MIT
-
Latest release: 0.6.0
published 6 months ago
Rankings
Maintainers (1)
crates.io: tantivy-sstable
sstables for tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy-sstable/
- License: MIT
-
Latest release: 0.6.0
published 6 months ago
Rankings
Maintainers (2)
crates.io: tantivy-stacker
term hashmap used for indexing
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy-stacker/
- License: MIT
-
Latest release: 0.6.0
published 6 months ago
Rankings
Maintainers (1)
crates.io: tantivy-columnar
column oriented storage for tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/tantivy-columnar/
- License: MIT
-
Latest release: 0.6.0
published 6 months ago
Rankings
Maintainers (1)
proxy.golang.org: github.com/quickwit-oss/tantivy
- Documentation: https://pkg.go.dev/github.com/quickwit-oss/tantivy#section-documentation
- License: mit
-
Latest release: v0.24.0
published 11 months ago
Rankings
pypi.org: tantivy
Official Python bindings for the Tantivy search engine
- Documentation: https://tantivy.readthedocs.io/
- License: MIT
-
Latest release: 0.24.0
published 10 months ago
Rankings
crates.io: summavy-common
common traits and utility functions used by multiple tantivy subcrates
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/summavy-common/
- License: MIT
-
Latest release: 0.5.0
published about 3 years ago
Rankings
Maintainers (1)
crates.io: summavy-bitpacker
Tantivy-sub crate: bitpacking
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/summavy-bitpacker/
- License: MIT
-
Latest release: 0.3.0
published about 3 years ago
Rankings
Maintainers (1)
crates.io: summavy-ownedbytes
Expose data as static slice
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/summavy-ownedbytes/
- License: MIT
-
Latest release: 0.5.0
published about 3 years ago
Rankings
Maintainers (1)
crates.io: summavy_fastfield_codecs
Fast field codecs used by tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/summavy_fastfield_codecs/
- License: MIT
-
Latest release: 0.3.1
published about 3 years ago
Rankings
Maintainers (1)
crates.io: lb-tantivy-sstable
sstables for tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/lb-tantivy-sstable/
- License: MIT
-
Latest release: 0.5.0
published 7 months ago
Rankings
Maintainers (1)
crates.io: lb-tantivy-columnar
column oriented storage for tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/lb-tantivy-columnar/
- License: MIT
-
Latest release: 0.5.0
published 7 months ago
Rankings
Maintainers (1)
crates.io: lb-tantivy
Search engine library
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/lb-tantivy/
- License: MIT
-
Latest release: 0.24.0
published 7 months ago
Rankings
Maintainers (1)
pypi.org: tantivy-fork
- Documentation: https://tantivy-fork.readthedocs.io/
- License: MIT
- Status: removed
-
Latest release: 0.22.1
published 12 months ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy
Search engine library
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy/
- License: MIT
-
Latest release: 0.25.1
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy-columnar
column oriented storage for tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy-columnar/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy-query-grammar
Search engine library
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy-query-grammar/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy-bitpacker
Tantivy-sub crate: bitpacking
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy-bitpacker/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy-common
common traits and utility functions used by multiple tantivy subcrates
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy-common/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy-tokenizer-api
Tokenizer API of tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy-tokenizer-api/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy-sstable
sstables for tantivy
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy-sstable/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-ownedbytes
Expose data as static slice
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-ownedbytes/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: izihawa-tantivy-stacker
term hashmap used for indexing
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/izihawa-tantivy-stacker/
- License: MIT
-
Latest release: 0.25.0
published about 1 year ago
Rankings
Maintainers (1)
crates.io: paradedb-tantivy
Search engine library
- Homepage: https://github.com/quickwit-oss/tantivy
- Documentation: https://docs.rs/paradedb-tantivy/
- License: MIT
-
Latest release: 0.21.0
published over 2 years ago
Rankings
Maintainers (1)
Dependencies
- Swatinem/rust-cache v2 composite
- actions/checkout v4 composite
- codecov/codecov-action v3 composite
- taiki-e/install-action cargo-llvm-cov composite
- actions-rs/toolchain v1 composite
- actions/checkout v4 composite
- Swatinem/rust-cache v2 composite
- actions-rs/clippy-check v1 composite
- actions-rs/toolchain v1 composite
- actions/checkout v4 composite
- taiki-e/install-action nextest composite
- env_logger 0.10.0 development
- fail 0.5.0 development
- futures 0.3.21 development
- maplit 1.0.2 development
- matches 0.1.9 development
- more-asserts 0.3.1 development
- paste 1.0.11 development
- pretty_assertions 1.2.1 development
- proptest 1.0.0 development
- rand 0.8.5 development
- rand_distr 0.4.3 development
- test-log 0.2.10 development
- aho-corasick 1.0
- arc-swap 1.5.0
- async-trait 0.1.53
- base64 0.21.0
- bitpacking 0.8.4
- byteorder 1.4.3
- census 0.4.0
- columnar 0.2
- common 0.6
- crc32fast 1.3.2
- crossbeam-channel 0.5.4
- downcast-rs 1.2.0
- fail 0.5.0
- fastdivide 0.4.0
- fs4 0.6.3
- futures-util 0.3.28
- htmlescape 0.3.1
- itertools 0.11.0
- levenshtein_automata 0.2.1
- log 0.4.16
- lru 0.11.0
- lz4_flex 0.11
- measure_time 0.8.2
- memmap2 0.7.1
- murmurhash32 0.3.0
- num_cpus 1.13.1
- once_cell 1.10.0
- oneshot 0.1.5
- query-grammar 0.21.0
- rayon 1.5.2
- regex 1.5.5
- rust-stemmers 1.2.0
- rustc-hash 1.1.0
- serde 1.0.136
- serde_json 1.0.79
- sketches-ddsketch 0.2.1
- smallvec 1.8.0
- sstable 0.2
- stacker 0.2
- tantivy-bitpacker 0.5
- tantivy-fst 0.4.0
- tempfile 3.3.0
- thiserror 1.0.30
- time 0.3.10
- tokenizer-api 0.2
- uuid 1.0.0
- zstd 0.12
- proptest 1 development
- rand 0.8 development
- bitpacking 0.8
- more-asserts 0.3.1 development
- proptest 1 development
- rand 0.8 development
- common 0.6
- fastdivide 0.4.0
- fnv 1.0.7
- itertools 0.11.0
- serde 1.0.152
- sstable 0.2
- stacker 0.2
- tantivy-bitpacker 0.5
- proptest 1.0.0 development
- rand 0.8.4 development
- async-trait 0.1
- byteorder 1.4.3
- ownedbytes 0.6
- serde 1.0.136
- time 0.3.10
- criterion 0.5 development
- names 0.14 development
- proptest 1 development
- rand 0.8 development
- common 0.6
- tantivy-fst 0.4
- zstd 0.12
- proptest 1.2.0 development
- rand 0.8.5 development
- rustc-hash 1.1.0 development
- zipf 7.0.0 development
- ahash 0.8.3
- common 0.6
- murmurhash32 0.3