Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/whylabs/whylogs

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

https://github.com/whylabs/whylogs

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (4.6%) to scientific vocabulary

Keywords

ai-pipelines analytics approximate-statistics calculate-statistics constraints data-constraints data-pipeline data-quality data-science dataops dataset logging machine-learning ml-pipelines mlops model-performance python statistical-properties

Keywords from Contributors

distributed agents distributed-computing hyperparameter-optimization interpretability standardization parallel animal hack autograder
Last synced: 5 months ago · JSON representation

Repository

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

Basic Info
  • Host: GitHub
  • Owner: whylabs
  • License: apache-2.0
  • Language: Jupyter Notebook
  • Default Branch: mainline
  • Homepage: https://whylogs.readthedocs.io/
  • Size: 181 MB
Statistics
  • Stars: 2,749
  • Watchers: 33
  • Forks: 130
  • Open Issues: 0
  • Releases: 173
Topics
ai-pipelines analytics approximate-statistics calculate-statistics constraints data-constraints data-pipeline data-quality data-science dataops dataset logging machine-learning ml-pipelines mlops model-performance python statistical-properties
Created over 5 years ago · Last pushed about 1 year ago
Metadata Files
Readme Contributing License Code of conduct

README.md

python/README.md

Owner

  • Name: WhyLabs
  • Login: whylabs
  • Kind: organization
  • Email: github@whylabs.ai
  • Location: United States of America

Observability for AI pipelines and applications. Instrument data pipelines, analyze data quality and drift, catch deviations before they cause model failures.

GitHub Events

Total
  • Create event: 23
  • Issues event: 12
  • Release event: 5
  • Watch event: 102
  • Delete event: 16
  • Member event: 7
  • Issue comment event: 26
  • Push event: 92
  • Pull request review comment event: 31
  • Pull request review event: 58
  • Pull request event: 42
  • Fork event: 9
Last Year
  • Create event: 23
  • Issues event: 12
  • Release event: 5
  • Watch event: 102
  • Delete event: 16
  • Member event: 7
  • Issue comment event: 26
  • Push event: 92
  • Pull request review comment event: 31
  • Pull request review event: 58
  • Pull request event: 42
  • Fork event: 9

Committers

Last synced: 9 months ago

All Time
  • Total Commits: 854
  • Total Committers: 27
  • Avg Commits per committer: 31.63
  • Development Distribution Score (DDS): 0.686
Past Year
  • Commits: 60
  • Committers: 5
  • Avg Commits per committer: 12.0
  • Development Distribution Score (DDS): 0.433
Top Committers
Name Email Commits
Jamie Broomall 8****6 268
Felipe Adachi f****i@g****m 176
richard-rogers 9****s 136
murilommen m****n@g****m 87
Andy Dang 2****g 62
Melly Burns 9****e 27
Anthony Naddeo a****y@w****i 24
naddeoa a****o@g****m 14
github-actions[bot] 4****] 13
Daniel David Leybzon d****n@g****m 11
Han Wang h****g@l****m 6
Bernease Herman b****e@g****m 5
Jakob Homan j****n 5
Andy Dang a****y@w****i 3
Sage Elliott h****o@s****m 2
rayreed137 1****7 2
whylabs-automator 1****r 2
TheMellyBee i****o@m****t 2
Abdou Aziz Diop a****z@g****m 1
Alessya Visnjic a****l@g****m 1
Andre Elizondo e****e@l****m 1
Ashwin Kamath 1****n 1
Cate Lewison 6****2 1
Chris Warth c****h@g****m 1
Darlan Tódero ten Caten d****c 1
Mateus Pereira m****s@o****m 1
Scott Graham 5****n 1
Committer Domains (Top 20 + Academic)
whylabs.ai: 2 melanieburns.net: 1 sageelliott.com: 1 lyft.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 110
  • Total pull requests: 377
  • Average time to close issues: 3 months
  • Average time to close pull requests: 10 days
  • Total issue authors: 17
  • Total pull request authors: 23
  • Average comments per issue: 1.49
  • Average comments per pull request: 0.54
  • Merged pull requests: 323
  • Bot issues: 0
  • Bot pull requests: 3
Past Year
  • Issues: 7
  • Pull requests: 34
  • Average time to close issues: about 1 month
  • Average time to close pull requests: 9 days
  • Issue authors: 3
  • Pull request authors: 5
  • Average comments per issue: 0.29
  • Average comments per pull request: 0.32
  • Merged pull requests: 26
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • jamie256 (51)
  • FelipeAdachi (24)
  • andyndang (10)
  • richard-rogers (7)
  • naddeoa (2)
  • christinedraper (2)
  • JanPalasek (2)
  • cswarth (2)
  • murilommen (2)
  • andrewelizondo (2)
  • drajsel (1)
  • snarasimhan1997 (1)
  • gabohc (1)
  • lukeduongdnanexus (1)
  • matsair (1)
Pull Request Authors
  • jamie256 (166)
  • richard-rogers (143)
  • FelipeAdachi (47)
  • naddeoa (37)
  • lalmei (21)
  • MilenaTrajanoska (13)
  • murilommen (11)
  • pecop2 (9)
  • andyndang (8)
  • w0-automator (6)
  • cswarth (4)
  • github-actions[bot] (2)
  • abdouaziz (2)
  • phantomgoose (2)
  • bernease (2)
Top Labels
Issue Labels
stale :zzz: (47) maintenance (5) bug (5) enhancement (4) performance (3) feature (3) testing (1) workflow (1) documentation (1)
Pull Request Labels
automated pr (68) stale :zzz: (12) bug (12) maintenance (10) documentation (9) enhancement (7) feature (7) experimental (3) dependencies (3) performance (2) work-in-progress (1) integration (1) java (1)

Packages

  • Total packages: 5
  • Total downloads: unknown
  • Total dependent packages: 8
    (may contain duplicates)
  • Total dependent repositories: 1
    (may contain duplicates)
  • Total versions: 218
proxy.golang.org: github.com/whylabs/whylogs
  • Versions: 178
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Stargazers count: 1.4%
Forks count: 2.6%
Average: 3.8%
Dependent packages count: 5.4%
Dependent repos count: 5.7%
Last synced: 6 months ago
repo1.maven.org: ai.whylabs:whylogs-java-core

The core library for WhyLogs

  • Versions: 22
  • Dependent Packages: 6
  • Dependent Repositories: 1
Rankings
Stargazers count: 7.3%
Dependent packages count: 9.7%
Average: 14.3%
Forks count: 19.2%
Dependent repos count: 20.8%
Last synced: 6 months ago
repo1.maven.org: ai.whylabs:whylogs-java-spark_3.1.1-scala_2.12

Spark integration for WhyLogs

  • Versions: 15
  • Dependent Packages: 2
  • Dependent Repositories: 0
Rankings
Stargazers count: 5.6%
Forks count: 13.3%
Average: 18.3%
Dependent packages count: 22.4%
Dependent repos count: 32.0%
Last synced: 6 months ago
repo1.maven.org: ai.whylabs:whylogs-spark-bundle_3.1.1-scala_2.12

spark bundle library for WhyLogs

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Stargazers count: 5.6%
Forks count: 13.3%
Average: 24.9%
Dependent repos count: 32.0%
Dependent packages count: 48.9%
Last synced: 6 months ago
repo1.maven.org: ai.whylabs:whylogs-spark_3.1.1-scala_2.12

spark bundle library for WhyLogs

  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Stargazers count: 5.6%
Forks count: 13.3%
Average: 24.9%
Dependent repos count: 32.0%
Dependent packages count: 48.9%
Last synced: 6 months ago

Dependencies

.github/workflows/codeql.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • github/codeql-action/analyze v1 composite
  • github/codeql-action/init v1 composite
.github/workflows/markdown-link-check.yml actions
  • actions/checkout master composite
  • gaurav-nelson/github-action-markdown-link-check v1 composite
.github/workflows/push-release.yml actions
  • abatilo/actions-poetry v2.0.0 composite
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • ad-m/github-push-action master composite
  • arduino/setup-protoc v1 composite
  • peter-evans/create-pull-request v4.1.1 composite
.github/workflows/release-drafter.yml actions
  • release-drafter/release-drafter v5 composite
.github/workflows/stale.yml actions
  • actions/stale v3 composite
.github/workflows/test-notebook.yml actions
  • abatilo/actions-poetry v2.0.0 composite
  • actions/cache v3 composite
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • arduino/setup-protoc v1 composite
.github/workflows/whylogs-ci.yml actions
  • JamesIves/github-pages-deploy-action 4.0.0 composite
  • abatilo/actions-poetry v2.0.0 composite
  • actions/cache v3 composite
  • actions/checkout v3 composite
  • actions/setup-java v1 composite
  • actions/setup-python v4 composite
  • arduino/setup-protoc v1 composite
  • codecov/codecov-action v2 composite
  • eskatos/gradle-command-action v1 composite
  • gradle/gradle-build-action v2 composite
  • nikeee/setup-pandoc v1 composite
python/examples/integrations/flask_streaming/Dockerfile docker
  • python ${PYTHON_VERSION} build
java/build.gradle.kts maven
java/core/build.gradle.kts maven
  • ai.whylabs:sketching-java 3.2.0-SNAPSHOT api
  • com.google.code.findbugs:jsr305 3.0.2 api
  • com.google.guava:guava 19.0 api
  • com.google.protobuf:protobuf-java 3.20.1 api
  • org.apache.commons:commons-lang3 3.10 api
  • org.apache.datasketches:datasketches-memory 2.0.0 api
  • org.slf4j:slf4j-api 1.7.27 api
  • org.projectlombok:lombok 1.18.20 compileOnly
  • org.projectlombok:lombok 1.18.20 testCompileOnly
  • com.google.protobuf:protobuf-java-util 3.11.0 testImplementation
  • org.apache.commons:commons-lang3 3.10 testImplementation
  • org.apache.commons:commons-math3 3.6.1 testImplementation
  • org.hamcrest:hamcrest 2.2 testImplementation
  • org.testng:testng 6.8 testImplementation
java/core-bundle/build.gradle.kts maven
  • com.google.protobuf:protobuf-java-util 3.20.1 compileOnly
java/smoketest/build.gradle maven
  • org.projectlombok:lombok 1.18.20 compileOnly
  • ai.whylabs:core 1.0.0-b1-DEV implementation
  • com.google.protobuf:protobuf-java 3.13.0 implementation
  • org.apache.datasketches:datasketches-java 1.3.0-incubating implementation
java/spark/build.gradle.kts maven
  • org.slf4j:slf4j-api 1.7.27 api
  • org.projectlombok:lombok 1.18.20 compileOnly
  • ai.whylabs:whylabs-api-client 0.1.7 implementation
  • com.squareup.okhttp3:okhttp 4.9.1 implementation
  • org.projectlombok:lombok 1.18.20 testCompileOnly
  • org.testng:testng 6.8 testImplementation
  • com.vladsch.flexmark:flexmark-profile-pegdown 0.36.8 testRuntimeOnly
  • org.slf4j:slf4j-log4j12 1.7.30 testRuntimeOnly
java/spark-bundle/build.gradle.kts maven
  • ai.whylabs:whylabs-api-client 0.1.7 implementation
python/examples/integrations/bentoml/requirements.txt pypi
  • bentoml >=1.0.0
  • numpy *
  • scikit-learn *
  • whylogs *
python/examples/integrations/fastapi/requirements.txt pypi
  • fastapi *
  • joblib *
  • pandas *
  • scikit-learn *
  • uvicorn *
  • whylogs *
python/examples/integrations/flask_streaming/requirements.txt pypi
  • Flask ==2.0.1
  • Flask-Pydantic ==0.8.0
  • flasgger ==0.9.5
  • flask_cors ==3.0.10
  • gunicorn ==20.1.0
  • joblib ==1.0.1
  • pandas ==1.3.2
  • pydantic ==1.8.2
  • python-dotenv ==0.19.0
  • requests ==2.22.0
  • scikit-learn ==1.1.1
  • utils ==1.0.1
  • whylogs >=1
python/poetry.lock pypi
  • 213 dependencies
python/pyproject.toml pypi
  • Pillow ^9.2.0
  • boto3 ^1.22.13
  • faster-fifo ^1.4.5
  • fugue ^0.8.1
  • furo ^2022.3.4
  • google-cloud-storage ^2.5.0
  • importlib-metadata <4.3
  • ipython *
  • ipython_genutils ^0.2.0
  • mlflow-skinny --- - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "^2.5.0" python: ">=3.8" optional: true - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "<2.0.1" python: "<3.8" optional: true
  • myst-parser ^0.17.2
  • nbconvert ^7.0.0
  • nbsphinx ^0.8.9
  • numpy --- - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "*" python: "<3.11" optional: true - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: ">=1.23.2" python: ">=3.11" optional: true
  • orjson ^3.8.10
  • pandas *
  • platformdirs ^3.5.0
  • protobuf >=3.19.4
  • pyarrow >=8.0.0, <13
  • pybars3 ^0.9
  • pyspark ^3.0.0
  • python >=3.7.1, <4
  • requests ^2.27
  • scikit-learn --- - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "^1.0.2" python: "<3.11" optional: true - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: ">=1.1.2, <2" python: ">=3.11" optional: true
  • scipy --- - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: ">=1.5" python: "<3.11" optional: true - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: ">=1.9.2" python: ">=3.11" optional: true
  • sphinx *
  • sphinx-autoapi *
  • sphinx-autobuild ^2021.3.14
  • sphinx-copybutton ^0.5.0
  • sphinx-inline-tabs *
  • sphinxext-opengraph ^0.6.3
  • types-requests ^2.30.0.0
  • typing-extensions >=3.10
  • whylabs-client ^0.5.6
  • whylogs-sketching >=3.4.1.dev3
.github/workflows/docker-test.yml actions
python/examples/integrations/flask_streaming/environment.yml pypi
  • anyio ==3.3.1
  • api *
  • appnope ==0.1.2
  • argcomplete ==1.12.3
  • argon2-cffi ==21.1.0
  • attrs ==21.2.0
  • babel ==2.9.1
  • backcall ==0.2.0
  • bleach ==4.1.0
  • cffi ==1.14.6
  • charset-normalizer ==2.0.6
  • debugpy ==1.4.3
  • decorator ==5.1.0
  • defusedxml ==0.7.1
  • entrypoints ==0.3
  • idna ==3.2
  • importlib-metadata ==4.8.1
  • ipykernel ==6.4.1
  • ipython ==7.27.0
  • ipython-genutils ==0.2.0
  • jedi ==0.18.0
  • jinja2 ==3.0.1
  • joblib ==1.0.1
  • json5 ==0.9.6
  • jsonschema ==3.2.0
  • jupyter-client ==7.0.3
  • jupyter-core ==4.8.1
  • jupyter-server ==1.11.0
  • jupyterlab ==3.1.12
  • jupyterlab-pygments ==0.1.2
  • jupyterlab-server ==2.8.1
  • markupsafe ==2.0.1
  • matplotlib-inline ==0.1.3
  • mistune ==0.8.4
  • nbclassic ==0.3.2
  • nbclient ==0.5.4
  • nbconvert ==6.1.0
  • nbformat ==5.1.3
  • nest-asyncio ==1.5.1
  • notebook ==6.4.4
  • numpy ==1.21.2
  • packaging ==21.0
  • pandas ==1.3.3
  • pandocfilters ==1.5.0
  • parso ==0.8.2
  • pexpect ==4.8.0
  • pickleshare ==0.7.5
  • prometheus-client ==0.11.0
  • prompt-toolkit ==3.0.20
  • ptyprocess ==0.7.0
  • pycparser ==2.20
  • pygments ==2.10.0
  • pyparsing ==2.4.7
  • pyrsistent ==0.18.0
  • python-dateutil ==2.8.2
  • pytz ==2021.1
  • pyzmq ==22.3.0
  • requests ==2.26.0
  • requests-unixsocket ==0.2.0
  • scikit-learn ==1.1.1
  • scipy ==1.7.1
  • send2trash ==1.8.0
  • six ==1.16.0
  • sniffio ==1.2.0
  • terminado ==0.12.1
  • testpath ==0.5.0
  • threadpoolctl ==2.2.0
  • tornado ==6.1
  • traitlets ==5.1.0
  • typing-extensions ==3.10.0.2
  • urllib3 ==1.26.6
  • wcwidth ==0.2.5
  • webencodings ==0.5.1
  • websocket-client ==1.2.1
  • zipp ==3.5.0