Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/crowdstrike/apbf

Go package implementing Age-Partitioned Bloom Filters (APBF)

https://github.com/crowdstrike/apbf

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, springer.com
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (5.9%) to scientific vocabulary

Keywords

apbf bloom bloom-filter go
Last synced: 5 months ago · JSON representation

Repository

Go package implementing Age-Partitioned Bloom Filters (APBF)

Basic Info
  • Host: GitHub
  • Owner: CrowdStrike
  • License: mit
  • Language: Go
  • Default Branch: master
  • Size: 10.7 KB
Statistics
  • Stars: 31
  • Watchers: 8
  • Forks: 6
  • Open Issues: 0
  • Releases: 0
Topics
apbf bloom bloom-filter go
Created almost 6 years ago · Last pushed over 3 years ago
Metadata Files
Readme License

README.md

Age-Partitioned Bloom Filters

Age-Partitioned Bloom Filters (APBF) is a novel approach for duplicate detection in sliding windows over an unbounded stream of items described in Age-Partitioned Bloom Filters: Ariel Shtul, Carlos Baquero and Paulo Sérgio Almeida, 2020.

The implementation employs the enhanced double hashing technique for fast index computation introduced in Bloom Filters in Probabilistic Verification: Peter C. Dillinger and Panagiotis Manolios, 2004.

Example

```golang // create a filter with k=10, l=7, and g=1000 filter := apbf.New(10, 7, 1000)

item := []byte("test item") filter.Add(item)

if filter.Query(item) { fmt.Println("item was found") } ```

Installation

Use go get to add the project to your workspace: bash go get -u github.com/CrowdStrike/apbf

Benchmarks

The following results show the performance of main filter operations Add and Query with and without refresh enabled for a small and large filter. Benchmarks were executed on a MacBook Pro 2017 dev laptop.

BenchmarkSmallFilterAdd-8 20000000 103 ns/op 0 B/op 0 allocs/op BenchmarkSmallFilterAddWithRefresh-8 10000000 177 ns/op 0 B/op 0 allocs/op BenchmarkSmallFilterQuery-8 10000000 133 ns/op 0 B/op 0 allocs/op BenchmarkSmallFilterQueryWithRefresh-8 10000000 206 ns/op 0 B/op 0 allocs/op BenchmarkLargeFilterAdd-8 5000000 252 ns/op 0 B/op 0 allocs/op BenchmarkLargeFilterAddWithRefresh-8 5000000 325 ns/op 0 B/op 0 allocs/op BenchmarkLargeFilterQuery-8 3000000 431 ns/op 0 B/op 0 allocs/op BenchmarkLargeFilterQueryWithRefresh-8 2000000 543 ns/op 0 B/op 0 allocs/op

Contributors

Bogdan-Ciprian Rusu - Author/Maintainer

License

The project is licensed under the MIT License.

Owner

  • Name: CrowdStrike
  • Login: CrowdStrike
  • Kind: organization
  • Email: github@crowdstrike.com
  • Location: United States of America

GitHub Events

Total
  • Watch event: 4
Last Year
  • Watch event: 4

Committers

Last synced: 9 months ago

All Time
  • Total Commits: 2
  • Total Committers: 2
  • Avg Commits per committer: 1.0
  • Development Distribution Score (DDS): 0.5
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Nicolae Sinea n****a@c****m 1
brusu b****u@c****m 1
Committer Domains (Top 20 + Academic)
crowdstrike.com: 2

Issues and Pull Requests

Last synced: 9 months ago

All Time
  • Total issues: 0
  • Total pull requests: 1
  • Average time to close issues: N/A
  • Average time to close pull requests: 1 day
  • Total issue authors: 0
  • Total pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • cs-sinea (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads: unknown
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 1
proxy.golang.org: github.com/CrowdStrike/apbf
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Stargazers count: 4.7%
Forks count: 5.0%
Average: 6.5%
Dependent packages count: 7.0%
Dependent repos count: 9.3%
Last synced: 7 months ago

Dependencies

go.mod go
  • github.com/gogo/protobuf v1.3.2
  • github.com/spaolacci/murmur3 v1.1.0
go.sum go
  • github.com/gogo/protobuf v1.3.2
  • github.com/kisielk/errcheck v1.5.0
  • github.com/kisielk/gotool v1.0.0
  • github.com/spaolacci/murmur3 v1.1.0
  • github.com/yuin/goldmark v1.1.27
  • github.com/yuin/goldmark v1.2.1
  • golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
  • golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
  • golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
  • golang.org/x/mod v0.2.0
  • golang.org/x/mod v0.3.0
  • golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3
  • golang.org/x/net v0.0.0-20190620200207-3b0461eec859
  • golang.org/x/net v0.0.0-20200226121028-0de0cce0169b
  • golang.org/x/net v0.0.0-20201021035429-f5854403a974
  • golang.org/x/sync v0.0.0-20190423024810-112230192c58
  • golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
  • golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9
  • golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
  • golang.org/x/sys v0.0.0-20190412213103-97732733099d
  • golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f
  • golang.org/x/text v0.3.0
  • golang.org/x/text v0.3.3
  • golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
  • golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e
  • golang.org/x/tools v0.0.0-20200619180055-7c47624df98f
  • golang.org/x/tools v0.0.0-20210106214847-113979e3529a
  • golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7
  • golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898
  • golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
  • golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1