Data

Tools

Indexes

Applications

Experiments

Open Source Science

deepdiff

DeepDiff: Deep Difference and search of any Python object/data. DeepHash: Hash of any object based on its contents. Delta: Use deltas to reconstruct objects by adding deltas together.

https://github.com/seperman/deepdiff

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    7 of 94 committers (7.4%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.1%) to scientific vocabulary

Keywords

comparison deep-search deepdiff deephash delta diff difference distance distance-calculation hash hashing nested python reconstruction recursive repetition report-repetition tree

Keywords from Contributors

yapf pre-commit-hook gofmt formatter cryptocurrencies codeformatter pde autopep8 test-data-generator test-data
Last synced: 6 months ago · JSON representation ·

Repository

DeepDiff: Deep Difference and search of any Python object/data. DeepHash: Hash of any object based on its contents. Delta: Use deltas to reconstruct objects by adding deltas together.

Basic Info
  • Host: GitHub
  • Owner: seperman
  • License: other
  • Language: Python
  • Default Branch: master
  • Homepage: http://zepworks.com
  • Size: 2.72 MB
Statistics
  • Stars: 2,365
  • Watchers: 27
  • Forks: 246
  • Open Issues: 91
  • Releases: 36
Topics
comparison deep-search deepdiff deephash delta diff difference distance distance-calculation hash hashing nested python reconstruction recursive repetition report-repetition tree
Created over 11 years ago · Last pushed 6 months ago
Metadata Files
Readme Changelog Funding License Citation Support Authors

README.md

DeepDiff v 8.6.1

Downloads Python Versions License Build Status codecov

Modules

  • DeepDiff: Deep Difference of dictionaries, iterables, strings, and ANY other object.
  • DeepSearch: Search for objects within other objects.
  • DeepHash: Hash any object based on their content.
  • Delta: Store the difference of objects and apply them to other objects.
  • Extract: Extract an item from a nested Python object using its path.
  • commandline: Use DeepDiff from commandline.

Tested on Python 3.9+ and PyPy3.

What is new?

Please check the ChangeLog file for the detailed information.

DeepDiff 8-6-1 - Patched security vulnerability in the Delta class which was vulnerable to class pollution via its constructor, and when combined with a gadget available in DeltaDiff itself, it could lead to Denial of Service and Remote Code Execution (via insecure Pickle deserialization).

DeepDiff 8-6-0

  • Added Colored View thanks to @mauvilsa
  • Added support for applying deltas to NamedTuple thanks to @paulsc
  • Fixed test_delta.py with Python 3.14 thanks to @Romain-Geissler-1A
  • Added python property serialization to json
  • Added ip address serialization
  • Switched to UV from pip
  • Added Claude.md
  • Added uuid hashing thanks to @akshat62
  • Added ignore_uuid_types flag to DeepDiff to avoid type reports when comparing UUID and string.
  • Added comprehensive type hints across the codebase (multiple commits for better type safety)
  • Added support for memoryview serialization
  • Added support for bytes serialization (non-UTF8 compatible)
  • Fixed bug where group_by with numbers would leak type info into group path reports
  • Fixed bug in _get_clean_to_keys_mapping without explicit significant digits
  • Added support for python dict key serialization
  • Enhanced support for IP address serialization with safe module imports
  • Added development tooling improvements (pyright config, .envrc example)
  • Updated documentation and development instructions

DeepDiff 8-5-0

  • Updating deprecated pydantic calls
  • Switching to pyproject.toml
  • Fix for moving nested tables when using iterablecomparefunc. by
  • Fix recursion depth limit when hashing numpy.datetime64
  • Moving from legacy setuptools use to pyproject.toml

DeepDiff 8-4-2

  • fixes the type hints for the base
  • fixes summarize so if json dumps fails, we can still get a repr of the results
  • adds ipaddress support

Installation

Install from PyPi:

pip install deepdiff

If you want to use DeepDiff from commandline:

pip install "deepdiff[cli]"

If you want to improve the performance of DeepDiff with certain functionalities such as improved json serialization:

pip install "deepdiff[optimize]"

Install optional packages: - yaml - tomli (python 3.10 and older) and tomli-w for writing - clevercsv for more rubust CSV parsing - orjson for speed and memory optimized parsing - pydantic

Documentation

https://zepworks.com/deepdiff/current/

A message from Sep, the creator of DeepDiff

👋 Hi there,

Thank you for using DeepDiff! As an engineer, I understand the frustration of wrestling with unruly data in pipelines. That's why I developed a new tool - Qluster to empower non-engineers to control and resolve data issues at scale autonomously and stop bugging the engineers! 🛠️

If you are going through this pain now, I would love to give you early access to Qluster and get your feedback.

ChangeLog

Please take a look at the CHANGELOG file.

Survey

:mega: Please fill out our fast 5-question survey so that we can learn how & why you use DeepDiff, and what improvements we should make. Thank you! :dancers:

Local dev

  1. Clone the repo
  2. Switch to the dev branch
  3. Create your own branch
  4. Install dependencies
- Method 1: Use [`uv`](https://github.com/astral-sh/uv) to install the dependencies:  `uv sync --all-extras`.
- Method 2: Use pip: `pip install -e ".[cli,coverage,dev,docs,static,test]"`
  1. Build flit build

Contribute

  1. Please make your PR against the dev branch
  2. Please make sure that your PR has tests. Since DeepDiff is used in many sensitive data driven projects, we strive to maintain around 100% test coverage on the code.

Please run pytest --cov=deepdiff --runslow to see the coverage report. Note that the --runslow flag will run some slow tests too. In most cases you only want to run the fast tests which so you wont add the --runslow flag.

Or to see a more user friendly version, please run: pytest --cov=deepdiff --cov-report term-missing --runslow.

Thank you!

Authors

Please take a look at the AUTHORS file.

Owner

  • Name: Sep Dehpour
  • Login: seperman
  • Kind: user
  • Location: Los Angeles
  • Company: @zepworks @qlustered

I build data tools.

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Dehpour"
  given-names: "Sep"
  orcid: "https://orcid.org/0009-0009-5828-4345"
title: "DeepDiff"
version: 8.6.1
date-released: 2024
url: "https://github.com/seperman/deepdiff"

GitHub Events

Total
  • Create event: 10
  • Commit comment event: 1
  • Release event: 7
  • Issues event: 64
  • Watch event: 283
  • Issue comment event: 139
  • Push event: 106
  • Pull request review comment event: 5
  • Pull request review event: 18
  • Pull request event: 49
  • Fork event: 30
Last Year
  • Create event: 10
  • Commit comment event: 1
  • Release event: 7
  • Issues event: 64
  • Watch event: 283
  • Issue comment event: 139
  • Push event: 106
  • Pull request review comment event: 5
  • Pull request review event: 18
  • Pull request event: 49
  • Fork event: 30

Committers

Last synced: 8 months ago

All Time
  • Total Commits: 1,061
  • Total Committers: 94
  • Avg Commits per committer: 11.287
  • Development Distribution Score (DDS): 0.254
Past Year
  • Commits: 112
  • Committers: 14
  • Avg Commits per committer: 8.0
  • Development Distribution Score (DDS): 0.232
Top Committers
Name Email Commits
Seperman s****p@z****m 792
Victor Hahn v****n@f****t 54
seperman s****r@t****m 30
Michał Karaś m****s@1****m 11
Bernhard Thiel t****l@t****t 10
Eras Notra x****d@y****m 8
Florian Klien f****n@s****m 7
sunao.626 s****6@b****m 6
Chris Hamill c****l@e****g 5
Uwe Fladrich u****h@s****e 5
Dustin Lorres d****s@s****o 5
Dustin Torres d****s@g****m 4
Enji Cooper y****a@g****m 4
Hugo h****k 4
Håvard Thom h****1@h****m 4
Lyz l****z@r****t 4
Martin Mokry m****s 4
Mathis Chenuet 9****t 4
Bryce Jasmer b****e@j****m 3
Leo Sin l****n@m****k 3
Markus Gruber r****g@w****e 3
William Jamieson w****n@s****u 3
dependabot[bot] 4****] 3
wangfenjin w****n@b****m 3
Joachim Langenbach j****h@e****e 3
Brendan Moloney m****y@a****u 3
kor4ik 5****k 2
Yael Mintz y****3@g****m 2
Todd Calhoun t****b@s****m 2
Ivan Piskunov v****0@m****u 2
and 64 more...
Committer Domains (Top 20 + Academic)
bytedance.com: 2 arconics.com: 1 bmat.com: 1 radarservices.com: 1 netflix.com: 1 kyriba.com: 1 medigate.io: 1 mail.ru: 1 statefarm.com: 1 airclogin-a1.ohsu.edu: 1 engsas.de: 1 stsci.edu: 1 my.cityu.edu.hk: 1 jasmer.com: 1 riseup.net: 1 surefyre.co: 1 smhi.se: 1 ets.org: 1 smaxtec.com: 1 tbi.univie.ac.at: 1 tufts.edu: 1 imt.uni-marburg.de: 1 diamond.ac.uk: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 178
  • Total pull requests: 125
  • Average time to close issues: 4 months
  • Average time to close pull requests: 16 days
  • Total issue authors: 162
  • Total pull request authors: 49
  • Average comments per issue: 2.35
  • Average comments per pull request: 1.92
  • Merged pull requests: 93
  • Bot issues: 0
  • Bot pull requests: 3
Past Year
  • Issues: 55
  • Pull requests: 56
  • Average time to close issues: 24 days
  • Average time to close pull requests: 16 days
  • Issue authors: 52
  • Pull request authors: 19
  • Average comments per issue: 0.89
  • Average comments per pull request: 1.63
  • Merged pull requests: 37
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • dtorres-sf (2)
  • chriswyatt (2)
  • behrm (2)
  • DominicOram (2)
  • emonsler (2)
  • explorigin (2)
  • GugelRobin (2)
  • tpcalhoun (2)
  • codingbutstillalive (2)
  • martin-kokos (2)
  • maggelus (2)
  • jbones001 (2)
  • Okroshiashvili (2)
  • yurivict (2)
  • sant527 (2)
Pull Request Authors
  • seperman (39)
  • dtorres-sf (7)
  • vmatt (4)
  • sf-tcalhoun (4)
  • ngie-eign (3)
  • akshat62 (3)
  • dependabot[bot] (3)
  • doronbehar (2)
  • mauvilsa (2)
  • stebranchi (2)
  • 3schwartz (2)
  • leoslf (2)
  • sherjeelshabih (2)
  • bmorck (2)
  • DominicOram (2)
Top Labels
Issue Labels
bug (21) enhancement (7) help wanted (3)
Pull Request Labels
dependencies (3)

Packages

  • Total packages: 19
  • Total downloads:
    • pypi 35,750,909 last-month
  • Total docker downloads: 817,048,902
  • Total dependent packages: 393
    (may contain duplicates)
  • Total dependent repositories: 2,086
    (may contain duplicates)
  • Total versions: 182
  • Total maintainers: 4
  • Total advisories: 1
pypi.org: deepdiff

Deep Difference and Search of any Python object/data. Recreate objects by adding adding deltas to each other.

  • Versions: 87
  • Dependent Packages: 378
  • Dependent Repositories: 2,058
  • Downloads: 35,569,365 Last month
  • Docker Downloads: 817,048,902
Rankings
Downloads: 0.1%
Dependent packages count: 0.1%
Docker downloads count: 0.1%
Dependent repos count: 0.2%
Average: 1.0%
Stargazers count: 1.6%
Forks count: 3.7%
Maintainers (1)
seperman
Advisories (1)
Last synced: 6 months ago
pypi.org: deepdiff6

Deep Difference and Search of any Python object/data. Recreate objects by adding adding deltas to each other.

  • Versions: 3
  • Dependent Packages: 5
  • Dependent Repositories: 17
  • Downloads: 181,544 Last month
Rankings
Downloads: 1.0%
Dependent packages count: 1.1%
Stargazers count: 1.6%
Average: 2.2%
Dependent repos count: 3.6%
Forks count: 3.6%
Maintainers (1)
erasmose
Last synced: 6 months ago
alpine-v3.18: py3-deepdiff-pyc

Precompiled Python bytecode for py3-deepdiff

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 3.2%
Stargazers count: 6.0%
Forks count: 7.0%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.18: py3-deepdiff

Deep Difference and Search of any Python object/data

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 3.3%
Stargazers count: 6.0%
Forks count: 7.0%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
proxy.golang.org: github.com/seperman/deepdiff
  • Versions: 20
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Stargazers count: 1.6%
Forks count: 1.9%
Average: 6.0%
Dependent packages count: 9.6%
Dependent repos count: 10.8%
Last synced: 6 months ago
alpine-v3.17: py3-deepdiff

Deep Difference and Search of any Python object/data

  • Versions: 1
  • Dependent Packages: 2
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Stargazers count: 6.2%
Average: 6.4%
Forks count: 6.9%
Dependent packages count: 12.7%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-edge: py3-deepdiff-pyc

Precompiled Python bytecode for py3-deepdiff

  • Versions: 12
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Stargazers count: 6.4%
Average: 7.2%
Forks count: 8.3%
Dependent packages count: 14.3%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-edge: py3-deepdiff

Deep Difference and Search of any Python object/data

  • Versions: 14
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Average: 7.6%
Stargazers count: 7.6%
Forks count: 8.1%
Dependent packages count: 14.6%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
conda-forge.org: deepdiff

Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes.

  • Versions: 23
  • Dependent Packages: 5
  • Dependent Repositories: 11
Rankings
Stargazers count: 9.9%
Dependent packages count: 10.4%
Dependent repos count: 10.6%
Average: 10.9%
Forks count: 12.9%
Last synced: 6 months ago
spack.io: py-deepdiff

Deep Difference and Search of any Python object/data..

  • Versions: 10
  • Dependent Packages: 2
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Stargazers count: 5.9%
Forks count: 7.2%
Average: 17.6%
Dependent packages count: 57.3%
Maintainers (1)
adamjstewart
Last synced: about 1 year ago
anaconda.org: deepdiff

Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes.

  • Versions: 2
  • Dependent Packages: 1
  • Dependent Repositories: 0
Rankings
Dependent packages count: 51.0%
Average: 55.4%
Dependent repos count: 59.9%
Last synced: 6 months ago
alpine-v3.21: py3-deepdiff

Deep Difference and Search of any Python object/data

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.22: py3-deepdiff-pyc

Precompiled Python bytecode for py3-deepdiff

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.20: py3-deepdiff

Deep Difference and Search of any Python object/data

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.22: py3-deepdiff

Deep Difference and Search of any Python object/data

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.21: py3-deepdiff-pyc

Precompiled Python bytecode for py3-deepdiff

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.20: py3-deepdiff-pyc

Precompiled Python bytecode for py3-deepdiff

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.19: py3-deepdiff-pyc

Precompiled Python bytecode for py3-deepdiff

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago
alpine-v3.19: py3-deepdiff

Deep Difference and Search of any Python object/data

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
alpine@ptrcnull.me
Last synced: 6 months ago

Dependencies

requirements-cli.txt pypi
  • clevercsv ==0.7.1
  • click ==8.0.3
  • pyyaml ==5.4.1
  • toml ==0.10.2
requirements-dev.txt pypi
  • Sphinx ==4.2.0 development
  • bump2version ==1.0.1 development
  • coverage ==6.0.2 development
  • flake8 ==4.0.1 development
  • ipdb ==0.13.9 development
  • jsonpickle ==2.0.0 development
  • numpy ==1.21.2 development
  • pytest ==6.2.5 development
  • pytest-cov ==3.0.0 development
  • python-dotenv ==0.19.1 development
  • sphinx-sitemap ==2.2.0 development
  • watchdog ==2.1.6 development
  • wheel ==0.37.0 development
requirements.txt pypi
  • ordered-set >=4.0.2,<4.2.0
.github/workflows/main.yaml actions
  • actions/cache v2 composite
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • codecov/codecov-action v1 composite
requirements-dev-3.7.txt pypi
  • bump2version ==1.0.1 development
  • ipdb ==0.13.9 development
  • jsonpickle ==2.2.0 development
  • numpy ==1.21.6 development
  • pytest ==7.1.2 development
  • python-dateutil ==2.8.2 development
  • python-dotenv ==0.20.0 development
  • wheel ==0.37.0 development
requirements-optimize.txt pypi
  • orjson *
setup.py pypi