Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/jmoralez/window_ops

Fast window operations

https://github.com/jmoralez/window_ops

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.4%) to scientific vocabulary

Keywords

expanding numba numpy online rolling

Keywords from Contributors

interactive projection sequences archival genomics observability autograding hacking shellcodes modular
Last synced: 6 months ago · JSON representation

Repository

Fast window operations

Basic Info
Statistics
  • Stars: 44
  • Watchers: 2
  • Forks: 6
  • Open Issues: 2
  • Releases: 1
Topics
expanding numba numpy online rolling
Created over 5 years ago · Last pushed over 1 year ago
Metadata Files
Readme Contributing License

README.md

Window ops

This library is intended to be used as an alternative to pd.Series.rolling and pd.Series.expanding to gain a speedup by using numba optimized functions operating on numpy arrays. There are also online classes for more efficient updates of window statistics.

Install

PyPI

pip install window-ops

conda

conda install -c conda-forge window-ops

How to use

Transformations

For a transformations n_samples -> n_samples you can use [seasonal_](rolling|expanding)_(mean|max|min|std) on an array.

Benchmarks

python pd.__version__ 

'1.3.5'

python n_samples = 10_000 # array size window_size = 8 # for rolling operations season_length = 7 # for seasonal operations execute_times = 10 # number of times each function will be executed

Average times in milliseconds.

python times.applymap('{:.2f}'.format)

window_ops pandas
rolling_mean 0.03 0.43
rolling_max 0.14 0.57
rolling_min 0.14 0.58
rolling_std 0.06 0.54
expanding_mean 0.03 0.31
expanding_max 0.05 0.76
expanding_min 0.05 0.47
expanding_std 0.09 0.41
seasonal_rolling_mean 0.05 3.89
seasonal_rolling_max 0.18 4.27
seasonal_rolling_min 0.18 3.75
seasonal_rolling_std 0.08 4.38
seasonal_expanding_mean 0.04 3.18
seasonal_expanding_max 0.06 3.29
seasonal_expanding_min 0.06 3.28
seasonal_expanding_std 0.12 3.89

python speedups = times['pandas'] / times['window_ops'] speedups = speedups.to_frame('times faster') speedups.applymap('{:.0f}'.format)

times faster
rolling_mean 15
rolling_max 4
rolling_min 4
rolling_std 9
expanding_mean 12
expanding_max 15
expanding_min 9
expanding_std 4
seasonal_rolling_mean 77
seasonal_rolling_max 23
seasonal_rolling_min 21
seasonal_rolling_std 52
seasonal_expanding_mean 78
seasonal_expanding_max 52
seasonal_expanding_min 51
seasonal_expanding_std 33

Online

If you have an array for which you want to compute a window statistic and then keep updating it as more samples come in you can use the classes in the window_ops.online module. They all have a fit_transform method which take the array and return the transformations defined above but also have an update method that take a single value and return the new statistic.

Benchmarks

Average time in milliseconds it takes to transform the array and perform 100 updates.

python times.to_frame().applymap('{:.2f}'.format)

average time (ms)
RollingMean 0.12
RollingMax 0.23
RollingMin 0.22
RollingStd 0.32
ExpandingMean 0.10
ExpandingMax 0.07
ExpandingMin 0.07
ExpandingStd 0.17
SeasonalRollingMean 0.28
SeasonalRollingMax 0.35
SeasonalRollingMin 0.38
SeasonalRollingStd 0.42
SeasonalExpandingMean 0.17
SeasonalExpandingMax 0.14
SeasonalExpandingMin 0.15
SeasonalExpandingStd 0.23

Owner

  • Name: José Morales
  • Login: jmoralez
  • Kind: user
  • Location: México
  • Company: @nubank

GitHub Events

Total
  • Watch event: 6
  • Issue comment event: 1
  • Pull request review event: 2
  • Pull request review comment event: 1
  • Pull request event: 1
  • Fork event: 1
Last Year
  • Watch event: 6
  • Issue comment event: 1
  • Pull request review event: 2
  • Pull request review comment event: 1
  • Pull request event: 1
  • Fork event: 1

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 45
  • Total Committers: 4
  • Avg Commits per committer: 11.25
  • Development Distribution Score (DDS): 0.111
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
José Morales j****s@g****m 40
dependabot[bot] 4****] 3
maironchaves 4****s 1
capybara 6****z 1
Committer Domains (Top 20 + Academic)
grupoabraxas.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 1
  • Total pull requests: 11
  • Average time to close issues: 16 days
  • Average time to close pull requests: 7 days
  • Total issue authors: 1
  • Total pull request authors: 5
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.36
  • Merged pull requests: 7
  • Bot issues: 0
  • Bot pull requests: 3
Past Year
  • Issues: 0
  • Pull requests: 2
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.5
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • rpanai (1)
Pull Request Authors
  • jmoralez (5)
  • dependabot[bot] (3)
  • tempoxylophone (2)
  • maironchaves (2)
  • MarkusSagen (2)
Top Labels
Issue Labels
Pull Request Labels
dependencies (3)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 212,221 last-month
  • Total docker downloads: 388
  • Total dependent packages: 1
  • Total dependent repositories: 2
  • Total versions: 15
  • Total maintainers: 1
pypi.org: window-ops

Implementations of window operations such as rolling and expanding.

  • Versions: 15
  • Dependent Packages: 1
  • Dependent Repositories: 2
  • Downloads: 212,221 Last month
  • Docker Downloads: 388
Rankings
Downloads: 0.9%
Docker downloads count: 3.6%
Dependent packages count: 4.8%
Average: 11.1%
Dependent repos count: 11.5%
Stargazers count: 16.1%
Forks count: 29.8%
Maintainers (1)
capybara
Last synced: 6 months ago

Dependencies

.github/workflows/deploy.yaml actions
  • fastai/workflows/quarto-ghp master composite
.github/workflows/test.yaml actions
  • fastai/workflows/nbdev-ci master composite
.github/workflows/release.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • pypa/gh-action-pypi-publish release/v1 composite
setup.py pypi