https://github.com/datafold/data-diff - v0.11.1

What's Changed

clarify that SQL Server support is limited by @leoebfolsom in https://github.com/datafold/data-diff/pull/864
formatting update to readme by @leoebfolsom in https://github.com/datafold/data-diff/pull/865
Loosen the restrictions of dependencies versions: >= instead of ^ and = by @nolar in https://github.com/datafold/data-diff/pull/867

Full Changelog: https://github.com/datafold/data-diff/compare/v0.11.0...v0.11.1

- Python
Published by nolar about 2 years ago

https://github.com/datafold/data-diff - v0.11.0

What's Changed

Fix github action that doesn't work with forked branches by @sungchun12 in https://github.com/datafold/data-diff/pull/815
Ignore columns at runtime on request (e.g. with too many diffs in them) by @nolar in https://github.com/datafold/data-diff/pull/822
[to#811]Fix special characters in PG url and Mysql connection reconnect by @idling11 in https://github.com/datafold/data-diff/pull/812
yield list vs. indiv tuples by @dlawin in https://github.com/datafold/data-diff/pull/805
Retrieve collations from the schema (and refactor the column info structures) by @nolar in https://github.com/datafold/data-diff/pull/814
Make InfoTree classes overrideable by @nolar in https://github.com/datafold/data-diff/pull/824
Cast to unlimited varchars in MS SQL (especially UUDS) by @nolar in https://github.com/datafold/data-diff/pull/825
Do not detect MD5s as UUIDs, and preserve UUID casing for UUID PKs by @nolar in https://github.com/datafold/data-diff/pull/813
Fix precision fetching for DataBricks by @nolar in https://github.com/datafold/data-diff/pull/826
Tolerate column type mismatches in non-strict mode by @nolar in https://github.com/datafold/data-diff/pull/829
Update ruff formatter to work differently between forks and native branches by @sungchun12 in https://github.com/datafold/data-diff/pull/821
keep precision, scale info for databricks decimal by @dlawin in https://github.com/datafold/data-diff/pull/830
Fix redshift schema ingest by @teraamp in https://github.com/datafold/data-diff/pull/835
fix typo -> ignoredcolumns2=self.ignoredcolumns2 by @dlawin in https://github.com/datafold/data-diff/pull/832
Fix databricks schema handling by @teraamp in https://github.com/datafold/data-diff/pull/836
Fix precision recognition by @vvkh in https://github.com/datafold/data-diff/pull/823
Group rows by all columns of composite PKs by @nolar in https://github.com/datafold/data-diff/pull/838
Update README to be ergonomic and excite new users by @sungchun12 in https://github.com/datafold/data-diff/pull/816
Type annotate some things ("no-brainers") by @nolar in https://github.com/datafold/data-diff/pull/827
remove local computer piece by @sungchun12 in https://github.com/datafold/data-diff/pull/843
Ability to install all database adapters. by @sar009 in https://github.com/datafold/data-diff/pull/842
Microsoft SQL Server Support Docs by @sungchun12 in https://github.com/datafold/data-diff/pull/845
Send event json errors by @dlawin in https://github.com/datafold/data-diff/pull/847
LAB-271 Redshift Spectrum type mapping by @teraamp in https://github.com/datafold/data-diff/pull/848
Detect duplicate rows on each side by @nolar in https://github.com/datafold/data-diff/pull/850
Improve error reporting for PK type mismatch by @vvkh in https://github.com/datafold/data-diff/pull/851
Fix mssql, databricks date types by @dlawin in https://github.com/datafold/data-diff/pull/853
Fix date and time formatting by @teraamp in https://github.com/datafold/data-diff/pull/854
closing connection once data diff is executed by @sar009 in https://github.com/datafold/data-diff/pull/785
Minor readme edits by @elliotgunn in https://github.com/datafold/data-diff/pull/855
Fix motherduck bug (#858) by @sungchun12 in https://github.com/datafold/data-diff/pull/861
Evolve dbt data diff output by @sungchun12 in https://github.com/datafold/data-diff/pull/857

New Contributors

@idling11 made their first contribution in https://github.com/datafold/data-diff/pull/812
@teraamp made their first contribution in https://github.com/datafold/data-diff/pull/835
@elliotgunn made their first contribution in https://github.com/datafold/data-diff/pull/855

Full Changelog: https://github.com/datafold/data-diff/compare/v0.10.1...v0.11.0

- Python
Published by dlawin about 2 years ago

https://github.com/datafold/data-diff - v0.10.1

What's Changed

add vhol callout by @sungchun12 in https://github.com/datafold/data-diff/pull/808
Better Action Next Step when using CLI by @sungchun12 in https://github.com/datafold/data-diff/pull/809

Full Changelog: https://github.com/datafold/data-diff/compare/v0.10.0...v0.10.1

- Python
Published by dlawin about 2 years ago

https://github.com/datafold/data-diff - v0.10.0

What's Changed

Make local dbt data diffs concurrent by @sungchun12 in https://github.com/datafold/data-diff/pull/776
Accept either key file path or file itself in Snowflake by @vvkh in https://github.com/datafold/data-diff/pull/792
mssql normalize_uuid by @dlawin in https://github.com/datafold/data-diff/pull/793
Revert "mssql normalize_uuid" and fix the overall problem by @dlawin in https://github.com/datafold/data-diff/pull/794
Remove Unneeded Logging by @sungchun12 in https://github.com/datafold/data-diff/pull/798
Tolerate empty samples & allow custom database schemas by @nolar in https://github.com/datafold/data-diff/pull/802
redshift: also try to get schema from svv_columns by @dlawin in https://github.com/datafold/data-diff/pull/803
Track data-diff usage in MotherDuck by @sungchun12 in https://github.com/datafold/data-diff/pull/800
fix unreachable code branches by @dlawin in https://github.com/datafold/data-diff/pull/791

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.17...v0.10.0

- Python
Published by dlawin about 2 years ago

https://github.com/datafold/data-diff - v0.10.0rc0

What's Changed

Make local dbt data diffs concurrent by @sungchun12 in https://github.com/datafold/data-diff/pull/776
Accept either key file path or file itself in Snowflake by @vvkh in https://github.com/datafold/data-diff/pull/792
mssql normalize_uuid by @dlawin in https://github.com/datafold/data-diff/pull/793
Revert "mssql normalize_uuid" and fix the overall problem by @dlawin in https://github.com/datafold/data-diff/pull/794

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.17...v0.10.0rc0

- Python
Published by dlawin about 2 years ago

https://github.com/datafold/data-diff - v0.9.17

What's Changed

Chiel fix link readme by @cfernhout in https://github.com/datafold/data-diff/pull/783
Handle full timestamp ranges for redshift, postgres by @dlawin in https://github.com/datafold/data-diff/pull/787
databricks does not require "database" by @dlawin in https://github.com/datafold/data-diff/pull/788

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.16...v0.9.17

- Python
Published by dlawin about 2 years ago

https://github.com/datafold/data-diff - v0.9.16

What's Changed

revert databricks information_schema by @dlawin in https://github.com/datafold/data-diff/pull/782

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.15...v0.9.16

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.15

What's Changed

Normalize schema info in databricks and redshift by @dlawin in https://github.com/datafold/data-diff/pull/781

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.14...v0.9.15

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.14

What's Changed

add TIMESTAMP_NTZ for databricks by @dlawin in https://github.com/datafold/data-diff/pull/780

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.13...v0.9.14

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.13

What's Changed

feat: support TOP operator by @pik94 in https://github.com/datafold/data-diff/pull/756
Add pre-commit automation for faster, safer development by @sungchun12 in https://github.com/datafold/data-diff/pull/771
Prevent type overflow by @pik94 in https://github.com/datafold/data-diff/pull/757

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.12...v0.9.13

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.12

What's Changed

[issue 772] open up accepted mashumaro versions by @dlawin in https://github.com/datafold/data-diff/pull/774

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.11...v0.9.12

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.11

What's Changed

Add Prod Flag Options for dbt data diffs by @sungchun12 in https://github.com/datafold/data-diff/pull/765
Block merge on formatter error by @dlawin in https://github.com/datafold/data-diff/pull/768
tracking should be optional in dbt configuration by @eschrock in https://github.com/datafold/data-diff/pull/769
validate dbt 1.7.0 works by @sungchun12 in https://github.com/datafold/data-diff/pull/767

New Contributors

@eschrock made their first contribution in https://github.com/datafold/data-diff/pull/769

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.10...v0.9.11

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.10

What's Changed

update str type_reprs for mssql, databricks by @dlawin in https://github.com/datafold/data-diff/pull/760

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.9...v0.9.10

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.9

What's Changed

BQ: Unexpected keyword impersonateserviceaccount by @dlawin @Stochastic-Squirrel in https://github.com/datafold/data-diff/pull/759

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.8...v0.9.9

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.8

What's Changed

Simplify: Squash database-specialised mixins into their database-specialised dialects by @nolar in https://github.com/datafold/data-diff/pull/751
Correct a few misleading type annotations by @nolar in https://github.com/datafold/data-diff/pull/752
Simplify: Remove unused code & mixins by @nolar in https://github.com/datafold/data-diff/pull/753
Simplify: Squash the MD5 & NormalizerValue & OptimizerHints mixins into the base dialect by @nolar in https://github.com/datafold/data-diff/pull/754

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.7...v0.9.8

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.7

What's Changed

Fix: Presto Basic authentication error by @pppsunil in https://github.com/datafold/data-diff/pull/742
add checksum offset to avoid bigint overflow by @vvkh in https://github.com/datafold/data-diff/pull/746
Fix a few things here & there by @nolar in https://github.com/datafold/data-diff/pull/740

New Contributors

@pppsunil made their first contribution in https://github.com/datafold/data-diff/pull/742

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.6...v0.9.7

- Python
Published by nolar over 2 years ago

https://github.com/datafold/data-diff - v0.9.6

What's Changed

Mark data-diff as type-annotated by @nolar in https://github.com/datafold/data-diff/pull/744
Bigquery dbt impersonation by @ivan-toriya in https://github.com/datafold/data-diff/pull/715

New Contributors

@ivan-toriya made their first contribution in https://github.com/datafold/data-diff/pull/715

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.5...v0.9.6

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.5

What's Changed

Make DuckDB data diffs work better by @sungchun12 in https://github.com/datafold/data-diff/pull/716
AbstractMixinMD5->MixinMD5 to override pg version by @dlawin in https://github.com/datafold/data-diff/pull/737

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.4...v0.9.5

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.4

What's Changed

Support missing dbt connection attributes aliases by @sar009 in https://github.com/datafold/data-diff/pull/690
unfreeze differ classes by @vvkh in https://github.com/datafold/data-diff/pull/736

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.3...v0.9.4

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.3

What's Changed

Check if test_metadata Exists First by @sungchun12 in https://github.com/datafold/data-diff/pull/734

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.2...v0.9.3

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.2

What's Changed

Unhide the implemented methods for ordering & repring the arith-texts by @nolar in https://github.com/datafold/data-diff/pull/728
Fix the missing fields in database classes by @nolar in https://github.com/datafold/data-diff/pull/729
Call parent's init before field initialisation, not after by @nolar in https://github.com/datafold/data-diff/pull/730
add keepalives for pg by @dlawin in https://github.com/datafold/data-diff/pull/731
adjust DatafoldAPI for attrs by @dlawin in https://github.com/datafold/data-diff/pull/732

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.1...v0.9.2

- Python
Published by nolar over 2 years ago

https://github.com/datafold/data-diff - v0.9.1

What's Changed

'PostgreSQL' object has no attribute '_conn' by @dlawin in https://github.com/datafold/data-diff/pull/726

Full Changelog: https://github.com/datafold/data-diff/compare/v0.9.0...v0.9.1

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.9.0

What's Changed

Initialise mutable fields of dataclasses safely for reuse by @nolar in https://github.com/datafold/data-diff/pull/697
Support MSSQL for cross-database diffs by @nolar in https://github.com/datafold/data-diff/pull/696
Refactor Artifacts Parser to be Native so it's less brittle with each dbt version change by @sungchun12 in https://github.com/datafold/data-diff/pull/688
Make PyODBC for MS SQL optional via extras by @nolar in https://github.com/datafold/data-diff/pull/698
Annotate types for self-cloning factories as per PEP-673 by @nolar in https://github.com/datafold/data-diff/pull/704
Simplify by replacing the self-made WeakCache with the builtin WeakValueDict by @nolar in https://github.com/datafold/data-diff/pull/703
[MySQL] Added mediumint by @yecnj in https://github.com/datafold/data-diff/pull/702
Normalise all imports to fully qualified names by @nolar in https://github.com/datafold/data-diff/pull/707
Remove unused sqeleton's CLI & modules by @nolar in https://github.com/datafold/data-diff/pull/708
Remove sqeleton's runtime mixins (used only in tests) by @nolar in https://github.com/datafold/data-diff/pull/709
Squash sqeleton into data_diff by @nolar in https://github.com/datafold/data-diff/pull/710
Compile all AST elements always via dialects, never directly by @nolar in https://github.com/datafold/data-diff/pull/713
Squash the redundant abstract classes into their "base" counterparts by @nolar in https://github.com/datafold/data-diff/pull/714
14 → 12 checksum digits to support mssql by @vvkh in https://github.com/datafold/data-diff/pull/718
Fix the aftermath of refactoring by @nolar in https://github.com/datafold/data-diff/pull/722
Convert to attrs, remove runtype by @nolar in https://github.com/datafold/data-diff/pull/723
remove python 3.7 support by @dlawin in https://github.com/datafold/data-diff/pull/700
add python 3.11 support, update mashumaro dep by @dlawin in https://github.com/datafold/data-diff/pull/724
add xml col type by @dlawin in https://github.com/datafold/data-diff/pull/717

New Contributors

@yecnj made their first contribution in https://github.com/datafold/data-diff/pull/702

Full Changelog: https://github.com/datafold/data-diff/compare/v0.8.4...v0.9.0

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.8.5rc1

What's Changed

Refactor Artifacts Parser to be Native so it's less brittle with each dbt version change by @sungchun12 in https://github.com/datafold/data-diff/pull/688
Make PyODBC for MS SQL optional via extras by @nolar in https://github.com/datafold/data-diff/pull/698

Full Changelog: https://github.com/datafold/data-diff/compare/v0.8.5rc0...v0.8.5rc1

- Python
Published by nolar over 2 years ago

https://github.com/datafold/data-diff - v0.8.5rc0

What's Changed

Initialise mutable fields of dataclasses safely for reuse by @nolar in https://github.com/datafold/data-diff/pull/697
Support MSSQL for cross-database diffs by @nolar in https://github.com/datafold/data-diff/pull/696

Full Changelog: https://github.com/datafold/data-diff/compare/v0.8.4...v0.8.5rc0

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.8.4

What's Changed

clarify getting started for dbt vs xdb by @leoebfolsom in https://github.com/datafold/data-diff/pull/678
rudimentary support for --stats in --dbt --json mode by @stefankeidel in https://github.com/datafold/data-diff/pull/647
partial --select support for dbt < 1.5 by @dlawin in https://github.com/datafold/data-diff/pull/682
bump parser and data-diff versions by @dlawin in https://github.com/datafold/data-diff/pull/685
Adding support for custom bigquery client credentials by @sar009 in https://github.com/datafold/data-diff/pull/680
Switch from cx_Oracle to oracledb by @nolar in https://github.com/datafold/data-diff/pull/681

New Contributors

@stefankeidel made their first contribution in https://github.com/datafold/data-diff/pull/647
@sar009 made their first contribution in https://github.com/datafold/data-diff/pull/680

Full Changelog: https://github.com/datafold/data-diff/compare/v0.8.3...v0.8.4

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.8.3

What's Changed

Beautify readme by @sungchun12 in https://github.com/datafold/data-diff/pull/671
Fix for more than 50 fields in Postgres by @kindly in https://github.com/datafold/data-diff/pull/662

New Contributors

@sungchun12 made their first contribution in https://github.com/datafold/data-diff/pull/671
@kindly made their first contribution in https://github.com/datafold/data-diff/pull/662

Full Changelog: https://github.com/datafold/data-diff/compare/v0.8.2...v0.8.3

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.8.2

What's Changed

add a one time message about vscode extension by @dlawin in https://github.com/datafold/data-diff/pull/665
Remove duckdb version constrain by @JCZuurmond in https://github.com/datafold/data-diff/pull/659
check for interactive session before email prompt by @dlawin in https://github.com/datafold/data-diff/pull/664
Fix type handling of key_columns in toml by @Attsun1031 in https://github.com/datafold/data-diff/pull/653
Add VS Code Link by @glebmezh in https://github.com/datafold/data-diff/pull/666
fix poetry.lock hash by @dlawin in https://github.com/datafold/data-diff/pull/668

New Contributors

@JCZuurmond made their first contribution in https://github.com/datafold/data-diff/pull/659
@Attsun1031 made their first contribution in https://github.com/datafold/data-diff/pull/653

Full Changelog: https://github.com/datafold/data-diff/compare/v0.8.1...v0.8.2

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.8.1

What's Changed

support dbt 1.6 by @dlawin in https://github.com/datafold/data-diff/pull/661

Full Changelog: https://github.com/datafold/data-diff/compare/v0.8.0...v0.8.1

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.8.0

What's Changed

poll more frequently when using --cloud by @dlawin in https://github.com/datafold/data-diff/pull/644
add --dbt support for --columns by @dlawin in https://github.com/datafold/data-diff/pull/643
fix profile_dir argument overwritten by legacy dir for DBT < 1.3.0 by @sebaap in https://github.com/datafold/data-diff/pull/651
add TRIGGERED_BY env variable support by @vvkh in https://github.com/datafold/data-diff/pull/654
get prod alias from manifest file when provided by @sebaap in https://github.com/datafold/data-diff/pull/652

New Contributors

@sebaap made their first contribution in https://github.com/datafold/data-diff/pull/651

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.14...v0.8.0

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.14

What's Changed

fix crashing when parsing column names including _a, _b by @vvkh in https://github.com/datafold/data-diff/pull/636
exempt issues/prs labeled "stale_immune" by @dlawin in https://github.com/datafold/data-diff/pull/639
add debugging detail for VSCode by @dlawin in https://github.com/datafold/data-diff/pull/641
Add column types to json output by @vvkh in https://github.com/datafold/data-diff/pull/638

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.13...v0.7.14

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.13

What's Changed

fix KeyError exception for --json diffs by @vvkh in https://github.com/datafold/data-diff/pull/635

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.12...v0.7.13

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.12

What's Changed

fix reversal of rows added and removed by @leoebfolsom in https://github.com/datafold/data-diff/pull/630
use a different endpoint for ds metadata by @dlawin in https://github.com/datafold/data-diff/pull/628
include only rows with equal PK to values diff stats in --json output by @vvkh in https://github.com/datafold/data-diff/pull/631
check prod table for DNE/access exception by @dlawin in https://github.com/datafold/data-diff/pull/632

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.11...v0.7.12

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.11

What's Changed

remove prompt to create API key and data source, send user straight to docs by @leoebfolsom in https://github.com/datafold/data-diff/pull/613
set upper limit on dsnparse version by @dlawin in https://github.com/datafold/data-diff/pull/617
Housekeeping: split tests and cleanup imports by @dlawin in https://github.com/datafold/data-diff/pull/619
add formatter + annotation step to PRs by @dlawin in https://github.com/datafold/data-diff/pull/622
add --dbt support for --where/-w by @dlawin in https://github.com/datafold/data-diff/pull/620
only render jinja in the specified profile target by @dlawin in https://github.com/datafold/data-diff/pull/627
log improvements and display diff status by @dlawin in https://github.com/datafold/data-diff/pull/614
black -l 120 . by @dlawin in https://github.com/datafold/data-diff/pull/623

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.10...v0.7.11

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.10

What's Changed

add json format for dbt diff values by @vvkh in https://github.com/datafold/data-diff/pull/612

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.9...v0.7.10

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.9

What's Changed

Add custom exceptions for raises in dbt_parser by @dlawin in https://github.com/datafold/data-diff/pull/589
set close reason to not_planned for stale issues by @dlawin in https://github.com/datafold/data-diff/pull/602
expand ~ in dbt paths by @vvkh in https://github.com/datafold/data-diff/pull/599
snowflake-connector-python>=3.0.2 by @dlawin in https://github.com/datafold/data-diff/pull/611
--dbt option to sign up for email notifications by @dlawin in https://github.com/datafold/data-diff/pull/605
add --state flag feature to dbt integration by @dlawin in https://github.com/datafold/data-diff/pull/600
--dbt add support for BQ service-account by @dlawin in https://github.com/datafold/data-diff/pull/609

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.8...v0.7.9

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.8

What's Changed

show column type changes and continue diff by @dlawin in https://github.com/datafold/data-diff/pull/584
warn and continue when PK is null by @dlawin in https://github.com/datafold/data-diff/pull/585
fix Redshift view support by @alexnikitchuk in https://github.com/datafold/data-diff/pull/592
add warning when using outdated data-diff version by @dlawin in https://github.com/datafold/data-diff/pull/587
add support for custom database config by @dlawin in https://github.com/datafold/data-diff/pull/586
Add snowflake-connector-python support for >=3.0.0 by @dlawin in https://github.com/datafold/data-diff/pull/593

New Contributors

@alexnikitchuk made their first contribution in https://github.com/datafold/data-diff/pull/592

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.7...v0.7.8

- Python
Published by dlawin over 2 years ago

https://github.com/datafold/data-diff - v0.7.7

What's Changed

specify urllib3<2 by @dlawin in https://github.com/datafold/data-diff/pull/578
Use proper casing policy for --cloud diffs by @vvkh in https://github.com/datafold/data-diff/pull/579
output cols added/removed/types changed by @dlawin in https://github.com/datafold/data-diff/pull/541
enhance cloud event metadata by @dlawin in https://github.com/datafold/data-diff/pull/547
support "port" in oracle connection by @qsliu2017 in https://github.com/datafold/data-diff/pull/548
bug label for bug template by @kylemcnair in https://github.com/datafold/data-diff/pull/576

New Contributors

@qsliu2017 made their first contribution in https://github.com/datafold/data-diff/pull/548

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.6...v0.7.7

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.7.6

What's Changed

add database-specific installation instructions to readme by @leoebfolsom in https://github.com/datafold/data-diff/pull/559
add support for two part id by @dlawin in https://github.com/datafold/data-diff/pull/531
support include/exclude meta config by @dlawin in https://github.com/datafold/data-diff/pull/535
add triage labels workflow by @kylemcnair in https://github.com/datafold/data-diff/pull/562
link to docs.datafold.com for XDB by @leoebfolsom in https://github.com/datafold/data-diff/pull/565
Create stale.yml by @kylemcnair in https://github.com/datafold/data-diff/pull/566
Update stale.yml by @kylemcnair in https://github.com/datafold/data-diff/pull/567
stale yml error by @kylemcnair in https://github.com/datafold/data-diff/pull/568
update --select dbt version error message by @leoebfolsom in https://github.com/datafold/data-diff/pull/569
add triage label to unstale issues/PRs by @kylemcnair in https://github.com/datafold/data-diff/pull/570
Create label-update_awaiting-response-to-triage.yml by @kylemcnair in https://github.com/datafold/data-diff/pull/573
Event based triage label by @kylemcnair in https://github.com/datafold/data-diff/pull/572
remove optional flag from --dbt dependencies by @dlawin in https://github.com/datafold/data-diff/pull/544

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.5...v0.7.6

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.7.5

What's Changed

focus readme on the dbt use case by @leoebfolsom in https://github.com/datafold/data-diff/pull/551
center the header by @leoebfolsom in https://github.com/datafold/data-diff/pull/552
Follow-up the sqeleton-to-datadiff embedding by @nolar in https://github.com/datafold/data-diff/pull/543
Json matching & tests for sqeleton PR #15 by @nicolasaldecoa in https://github.com/datafold/data-diff/pull/383
Fix & extend the tests for JSON/JSONB columns by @nolar in https://github.com/datafold/data-diff/pull/553
Allow dbt selectors by @dave-connors-3 in https://github.com/datafold/data-diff/pull/511
Hotfix: Correct early return in diffsareequiv_jsons not including the set object by @nicolasaldecoa in https://github.com/datafold/data-diff/pull/554
Simplify JSON column types by @nolar in https://github.com/datafold/data-diff/pull/545
Compare JSON, ARRAY, STRUCT types in BigQuery (simplistically) by @nolar in https://github.com/datafold/data-diff/pull/533
instantiate MACRO_DEBUGGING global for dbt by @dlawin in https://github.com/datafold/data-diff/pull/555
if dbt ls throws an exception, raise that first by @dlawin in https://github.com/datafold/data-diff/pull/556

New Contributors

@nicolasaldecoa made their first contribution in https://github.com/datafold/data-diff/pull/383
@dave-connors-3 made their first contribution in https://github.com/datafold/data-diff/pull/511

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.4...v0.7.5

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.7.4

What's Changed

remove duplicated consts and methods by @dlawin in https://github.com/datafold/data-diff/pull/521
add a doc link when vars do not exist by @dlawin in https://github.com/datafold/data-diff/pull/528

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.3...v0.7.4

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.7.3

What's Changed

Embed sqeleton with git history preserved by @nolar in https://github.com/datafold/data-diff/pull/485
Cloud diffs in progress by @dlawin in https://github.com/datafold/data-diff/pull/510
Make temp schema optional by @pik94 in https://github.com/datafold/data-diff/pull/509
warn when using recent dbt-core version by @dlawin in https://github.com/datafold/data-diff/pull/517
Add logs to indicate current artifact being parsed in debug mode by @MalanB in https://github.com/datafold/data-diff/pull/508
support datadiff meta filter by @dlawin in https://github.com/datafold/data-diff/pull/522 ## New Contributors
@nolar made their first contribution in https://github.com/datafold/data-diff/pull/485
@MalanB made their first contribution in https://github.com/datafold/data-diff/pull/508

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.2...v0.7.3

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.7.2

What's Changed

issue 505: add clientsessionkeep_alive by @dlawin in https://github.com/datafold/data-diff/pull/506

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.1...v0.7.2

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.7.1

What's Changed

HOTFIX issue 501: instantiate dbt variables by @dlawin in https://github.com/datafold/data-diff/pull/502

Full Changelog: https://github.com/datafold/data-diff/compare/v0.7.0...v0.7.1

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.7.0

What's Changed

raise when project vars not found by @dlawin in https://github.com/datafold/data-diff/pull/478
dbt events by @kylemcnair in https://github.com/datafold/data-diff/pull/488
reduce repetition in print statements by @dlawin in https://github.com/datafold/data-diff/pull/464
getuniquecolumns: continue on None or empty list by @dlawin in https://github.com/datafold/data-diff/pull/477
do not infer source pks from tests by @dlawin in https://github.com/datafold/data-diff/pull/490
add snowflake sso by @dlawin in https://github.com/datafold/data-diff/pull/487
Datafold database credentials from cli by @pik94 in https://github.com/datafold/data-diff/pull/466
allow pass: key for redshift by @dlawin in https://github.com/datafold/data-diff/pull/482
save and get the api key from the system keyring service by @pik94 in https://github.com/datafold/data-diff/pull/492
Grab credentials from profiles.yml by @pik94 in https://github.com/datafold/data-diff/pull/491
fix up links in readme by @leoebfolsom in https://github.com/datafold/data-diff/pull/493
cleaner rows added/removed by @dlawin in https://github.com/datafold/data-diff/pull/497
support insecure_mode key for snowflake profile by @dlawin in https://github.com/datafold/data-diff/pull/496
Tidying up duplication between /docs and docs.datafold.com by @leoebfolsom in https://github.com/datafold/data-diff/pull/495
handle all custom schemas scenarios by @dlawin and @CoreyGaunt in https://github.com/datafold/data-diff/pull/498

Possible breaking change when custom schemas are used

This release changes how prod schema names are determined when a "custom schema" is used via a new variable prod_custom_schema. The boolean variable custom_schemas can be removed. See the updated doc here: https://docs.datafold.com/developmenttesting/opensource

Full Changelog: https://github.com/datafold/data-diff/compare/v0.6.0...v0.7.0

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.6.0

What's Changed

Align PK support with Datafold SaaS by @dlawin in https://github.com/datafold/data-diff/pull/446
Adding user id to tracking by @kylemcnair in https://github.com/datafold/data-diff/pull/455
adds a basic e2e test for --dbt using duckdb by @dlawin in https://github.com/datafold/data-diff/pull/443
increment dbt supported version to include 1.4.5 by @dlawin in https://github.com/datafold/data-diff/pull/463
cloud api token flow by @pik94 in https://github.com/datafold/data-diff/pull/462
expand --cloud output by polling for results by @dlawin in https://github.com/datafold/data-diff/pull/467
add basic snowflake key/pair support by @dlawin in https://github.com/datafold/data-diff/pull/468
Snowflake support privatekeypassphrase by @dlawin in https://github.com/datafold/data-diff/pull/469

Full Changelog: https://github.com/datafold/data-diff/compare/v0.5.0...v0.6.0

- Python
Published by dlawin almost 3 years ago

https://github.com/datafold/data-diff - v0.5.0

What's Changed

Make DBT dependency optional by @erezsh in https://github.com/datafold/data-diff/pull/421
Compound keys implementation, using product order by @erezsh in https://github.com/datafold/data-diff/pull/375
revise columns added-removed output by @kylemcnair in https://github.com/datafold/data-diff/pull/424
Issue 417 error handling by @dlawin in https://github.com/datafold/data-diff/pull/419
Fix multiple Diffs Complete! messages by @dlawin in https://github.com/datafold/data-diff/pull/394
always use model.alias for table path by @dlawin in https://github.com/datafold/data-diff/pull/400
bump to sqeleton v0.0.7 by @williebsweet in https://github.com/datafold/data-diff/pull/432
add redshift, postgres, databricks support by @dlawin in https://github.com/datafold/data-diff/pull/382
support combo pks in --dbt local_diff by @dlawin in https://github.com/datafold/data-diff/pull/431
Support connections to DuckDB for --dbt by @dbeatty10 in https://github.com/datafold/data-diff/pull/408
issue 425 parse and use threads by @dlawin in https://github.com/datafold/data-diff/pull/435
render jinja in entire selected profile by @dlawin in https://github.com/datafold/data-diff/pull/395
Use the same logic as dbt-core for the path for the project and profiles by @dbeatty10 in https://github.com/datafold/data-diff/pull/415
support custom schemas by @dlawin in https://github.com/datafold/data-diff/pull/437
Fix --dbt for dbt-core >= 1.4.2 by @dlawin in https://github.com/datafold/data-diff/pull/440

New Contributors

@dbeatty10 made their first contribution in https://github.com/datafold/data-diff/pull/408

Full Changelog: https://github.com/datafold/data-diff/compare/v0.4.1...v0.5.0

- Python
Published by nolar almost 3 years ago

https://github.com/datafold/data-diff - v0.4.1

What's Changed

reverse order of dev and prod tables in cli printout and cloud diffs by @leoebfolsom in https://github.com/datafold/data-diff/pull/389
fix issue #391: incorrect raise, add tests by @dlawin in https://github.com/datafold/data-diff/pull/392

Full Changelog: https://github.com/datafold/data-diff/compare/v0.4.0...v0.4.1

- Python
Published by nolar about 3 years ago

https://github.com/datafold/data-diff - v0.4.0 - dbt support

What's Changed

Add env variables support in .toml config by @vvkh in https://github.com/datafold/data-diff/pull/356
add dbt integration by @dlawin in https://github.com/datafold/data-diff/pull/364
Add events for dbt feature by @dlawin in https://github.com/datafold/data-diff/pull/380
spruce up CLI output in dbt context by @leoebfolsom in https://github.com/datafold/data-diff/pull/381

Full Changelog: https://github.com/datafold/data-diff/compare/v0.3.2...v0.4.0

- Python
Published by nolar about 3 years ago

https://github.com/datafold/data-diff - v0.3.2 - Small features and fixes

Overview

Added support for 3-part identifiers in BigQuery and Snowflake (can now joindiff across different projects)
Added 'where' argument to Python API
Swapped sqeleton implementation to the external library
Added support for redshift external tables
Small bugfixes

What's Changed

Small refactor in main and tests; Better test coverage for CLI by @erezsh in https://github.com/datafold/data-diff/pull/338
support 3 part identifer for snowflake by @dlawin in https://github.com/datafold/data-diff/pull/335
Fix BigQuery code; Upgrade dependencies to latest versions. by @erezsh in https://github.com/datafold/data-diff/pull/339
add usage type tracking by @kylemcnair in https://github.com/datafold/data-diff/pull/346
Adjustments to PR #341 by @erezsh in https://github.com/datafold/data-diff/pull/345
Add redshift external tables by @danieldiamond in https://github.com/datafold/data-diff/pull/341
Adjustment to PR #348 by @erezsh in https://github.com/datafold/data-diff/pull/350
allow joindiff across bigquery projects by @segv in https://github.com/datafold/data-diff/pull/348
CI: Added testing for Redshift by @erezsh in https://github.com/datafold/data-diff/pull/347
Add 'where' argument to Python API by @leoebfolsom in https://github.com/datafold/data-diff/pull/342
Swap sqeleton implementation to the external library by @erezsh in https://github.com/datafold/data-diff/pull/352
Fix typos by @kianmeng in https://github.com/datafold/data-diff/pull/353
Bump Sqeleton version to 0.0.5 by @erezsh in https://github.com/datafold/data-diff/pull/367
Bugfix: Add brackets around WHERE clause by @erezsh in https://github.com/datafold/data-diff/pull/369

New Contributors

@danieldiamond made their first contribution in https://github.com/datafold/data-diff/pull/341
@segv made their first contribution in https://github.com/datafold/data-diff/pull/348
@kianmeng made their first contribution in https://github.com/datafold/data-diff/pull/353

Full Changelog: https://github.com/datafold/data-diff/compare/v0.3.1...v0.3.2

- Python
Published by erezsh about 3 years ago

https://github.com/datafold/data-diff - Quickfix for v0.3.0

What's Changed

Added tests for current_timestamp + fixes for some of the dbs by @erezsh in https://github.com/datafold/data-diff/pull/336

Full Changelog: https://github.com/datafold/data-diff/compare/v0.3.0...v0.3.1

- Python
Published by erezsh about 3 years ago

https://github.com/datafold/data-diff - v0.3.0 - New algorithm for in-db diffing (joindiff) + tons of new features and bugfixes!

Big points: - Added a new algorithm for in-db diffing that uses OUTER JOIN, called "joindiff". - Much faster than the original "hashdiff" algorithm! - Automatically chosen if both dbs are the same - Validates that the key column is unique and contains no NULLs (joindiff only) - Explicitly switch between algorithms using the --algorithm parameter. - New feature to materialize joindiff results to DB - New feature that diffs the schemas when both dbs are the same - Added DuckDB support (thanks @jardayn!) - Better support for alphanumerics - Better support for boolean types - Added --version switch - New and improved database and query interface, named "sqeleton" - Tons of bugfixes and improvements!

What's Changed

Join-diff (in-db) + new query builder by @erezsh in https://github.com/datafold/data-diff/pull/242
Bugfix: Joindiff crashed when no numeric columns were used. by @erezsh in https://github.com/datafold/data-diff/pull/255
Deprecate use of FixedAlphanum by @erezsh in https://github.com/datafold/data-diff/pull/254
Refactor tests oct2022 by @erezsh in https://github.com/datafold/data-diff/pull/253
General tests now include Presto, Trino & Vertica; Includes small fixes by @erezsh in https://github.com/datafold/data-diff/pull/256
Added --materialize-all-rows switch + tests by @erezsh in https://github.com/datafold/data-diff/pull/258
Various small fixes and refactors by @erezsh in https://github.com/datafold/data-diff/pull/260
Downgrade mysql-connector-python to 8.0.29 by @erezsh in https://github.com/datafold/data-diff/pull/262
Update documentation link by @williebsweet in https://github.com/datafold/data-diff/pull/263
Small changes by @erezsh in https://github.com/datafold/data-diff/pull/264
Added link on how to get a slack invite by @jardayn in https://github.com/datafold/data-diff/pull/265
link to docs and incorporate roman/gerard feedback by @leoebfolsom in https://github.com/datafold/data-diff/pull/266
Tiny Cleanup by @erezsh in https://github.com/datafold/data-diff/pull/267
tests for unique key constraints (if possible) instead of always actively validating (+ tests) by @erezsh in https://github.com/datafold/data-diff/pull/257
Attempt to fix PR #269 by @erezsh in https://github.com/datafold/data-diff/pull/272
Contrib improvements + Fixed Test by @jardayn in https://github.com/datafold/data-diff/pull/269
Refactor dialect by @erezsh in https://github.com/datafold/data-diff/pull/271
Tests: Improvements to CI flow + fixes by @erezsh in https://github.com/datafold/data-diff/pull/274
Bugfix in alphanums (reported by Guarav Singh) by @erezsh in https://github.com/datafold/data-diff/pull/277
Fix databricks by @pik94 in https://github.com/datafold/data-diff/pull/273
Added support for Boolean types by @erezsh in https://github.com/datafold/data-diff/pull/282
Fixed broken "How To Use" links in README. by @daniel-leicht in https://github.com/datafold/data-diff/pull/290
Fix for issue #286 by @erezsh in https://github.com/datafold/data-diff/pull/291
Materialize: rename and reorder columns by @erezsh in https://github.com/datafold/data-diff/pull/287
Revised CLI output to be more understandable and detailed by @erezsh in https://github.com/datafold/data-diff/pull/292
New DB Driver guide update by @jardayn in https://github.com/datafold/data-diff/pull/288
Duckdb driver for Issue #176 by @jardayn in https://github.com/datafold/data-diff/pull/276
Update typing of TableSegment().count() by @MattDelac in https://github.com/datafold/data-diff/pull/293
Refactor common database interface into Sqeleton (databases, queries) by @erezsh in https://github.com/datafold/data-diff/pull/285
Added DDB as an extra by @jardayn in https://github.com/datafold/data-diff/pull/296
More Sqeleton refactoring by @erezsh in https://github.com/datafold/data-diff/pull/295
Added InfoTree as a more descriptive alternative to .stats by @erezsh in https://github.com/datafold/data-diff/pull/297
Refactor tests to use insertrowsin_batches(), instead of internally… by @erezsh in https://github.com/datafold/data-diff/pull/299
CLI: Better errors + tiny bugfix by @erezsh in https://github.com/datafold/data-diff/pull/303
Rudderstack poc by @kylemcnair in https://github.com/datafold/data-diff/pull/298
add databases we support to readme by @leoebfolsom in https://github.com/datafold/data-diff/pull/309
Nov22 sqeleton refactor by @erezsh in https://github.com/datafold/data-diff/pull/308
Fix readme link by @dlawin in https://github.com/datafold/data-diff/pull/310
List tables from schema by @erezsh in https://github.com/datafold/data-diff/pull/311
Tests: Set bisectionfactor=2 for much faster tests; Fix random failures in teststring_keys by @erezsh in https://github.com/datafold/data-diff/pull/312
Nov24 - Small fixes to tests by @erezsh in https://github.com/datafold/data-diff/pull/313
Adjustments for PR #314 by @erezsh in https://github.com/datafold/data-diff/pull/315
return all duplicated rows by @pik94 in https://github.com/datafold/data-diff/pull/314
Cleanup by @erezsh in https://github.com/datafold/data-diff/pull/320
Added version and --version switch (issue #318) by @erezsh in https://github.com/datafold/data-diff/pull/319
data-diff now uses database A's now instead of cli's now. by @erezsh in https://github.com/datafold/data-diff/pull/306
extract methods for stats by @dlawin in https://github.com/datafold/data-diff/pull/300
connect(): Added support for shared connection; Database.is_closed property by @erezsh in https://github.com/datafold/data-diff/pull/323
Better error messages in databases; Default database in clickhouse is now 'default'. by @erezsh in https://github.com/datafold/data-diff/pull/325
diff_tables() now accepts all JoinDiffer params by @erezsh in https://github.com/datafold/data-diff/pull/326
CLI: Automatically choose joindiff is dbs are the same (don't rely just on syntax) by @erezsh in https://github.com/datafold/data-diff/pull/328
Add version module and add version to tracking by @kylemcnair in https://github.com/datafold/data-diff/pull/327
Dec2 cleanup by @erezsh in https://github.com/datafold/data-diff/pull/329
fix link to docs by @leoebfolsom in https://github.com/datafold/data-diff/pull/330
Fix normalizetable_path to always return a pair by @erezsh in https://github.com/datafold/data-diff/pull/333

New Contributors

@williebsweet made their first contribution in https://github.com/datafold/data-diff/pull/263
@jardayn made their first contribution in https://github.com/datafold/data-diff/pull/265
@daniel-leicht made their first contribution in https://github.com/datafold/data-diff/pull/290
@MattDelac made their first contribution in https://github.com/datafold/data-diff/pull/293
@kylemcnair made their first contribution in https://github.com/datafold/data-diff/pull/298
@dlawin made their first contribution in https://github.com/datafold/data-diff/pull/310

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.8...v0.3.0

Let us know what you think in Discussions!

- Python
Published by erezsh about 3 years ago

https://github.com/datafold/data-diff - v0.3.0rc2 - New algorithm for in-db diffing (joindiff) + features and bugfixes

Big points

Add new algorithm for in-db diffing that uses OUTER JOIN, called "joindiff".
New feature to materialize joindiff results to DB
A bunch of bugfixes and improvements

What's Changed

Join-diff (in-db) + new query builder by @erezsh in https://github.com/datafold/data-diff/pull/242
Bugfix: Joindiff crashed when no numeric columns were used. by @erezsh in https://github.com/datafold/data-diff/pull/255
Deprecate use of FixedAlphanum by @erezsh in https://github.com/datafold/data-diff/pull/254
Refactor tests oct2022 by @erezsh in https://github.com/datafold/data-diff/pull/253
General tests now include Presto, Trino & Vertica; Includes small fixes by @erezsh in https://github.com/datafold/data-diff/pull/256
Added --materialize-all-rows switch + tests by @erezsh in https://github.com/datafold/data-diff/pull/258
Various small fixes and refactors by @erezsh in https://github.com/datafold/data-diff/pull/260
Downgrade mysql-connector-python to 8.0.29 by @erezsh in https://github.com/datafold/data-diff/pull/262
Update documentation link by @williebsweet in https://github.com/datafold/data-diff/pull/263
Small changes by @erezsh in https://github.com/datafold/data-diff/pull/264
Added link on how to get a slack invite by @jardayn in https://github.com/datafold/data-diff/pull/265
link to docs and incorporate roman/gerard feedback by @leoebfolsom in https://github.com/datafold/data-diff/pull/266
Tiny Cleanup by @erezsh in https://github.com/datafold/data-diff/pull/267
tests for unique key constraints (if possible) instead of always actively validating (+ tests) by @erezsh in https://github.com/datafold/data-diff/pull/257
Attempt to fix PR #269 by @erezsh in https://github.com/datafold/data-diff/pull/272
Contrib improvements + Fixed Test by @jardayn in https://github.com/datafold/data-diff/pull/269
Refactor dialect by @erezsh in https://github.com/datafold/data-diff/pull/271
Tests: Improvements to CI flow + fixes by @erezsh in https://github.com/datafold/data-diff/pull/274
Bugfix in alphanums (reported by Guarav Singh) by @erezsh in https://github.com/datafold/data-diff/pull/277
Fix databricks by @pik94 in https://github.com/datafold/data-diff/pull/273

New Contributors

@jardayn made their first contribution in https://github.com/datafold/data-diff/pull/265

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.8...v0.3.0rc2

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.8 - Bugfix in algorithm for an edge-case

What's Changed

Bugfix in algorithm: Trigger download if the segment space is smaller than the bisection factor by @erezsh in https://github.com/datafold/data-diff/pull/249
v0.2.8 - Release PR by @erezsh in https://github.com/datafold/data-diff/pull/251

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.7...v0.2.8

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.7 - Better alphanumerics, better threading, and small fixes

What's Changed

Support for varying alphanums, with special characters by @erezsh in https://github.com/datafold/data-diff/pull/235
Re-wrote threading to use a thread-pool + priority queue. by @erezsh in https://github.com/datafold/data-diff/pull/238
Added support for specifying db-name in CLI instead of URI when using --conf by @erezsh in https://github.com/datafold/data-diff/pull/248
Added validation for UUID columns (Also fixes issue #245) by @erezsh in https://github.com/datafold/data-diff/pull/247

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.6...v0.2.7

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.6 - Support for Clickhouse, Vertica, and various bugfixes

Support of Clickhouse by @pik94 in https://github.com/datafold/data-diff/pull/217
add support of Vertica db by @pik94 in https://github.com/datafold/data-diff/pull/231
Fix for pip extras (e.g. pip install data-diff[snowflake]) by @erezsh in https://github.com/datafold/data-diff/pull/232
Fixed support for diffing columns of different names by @erezsh in https://github.com/datafold/data-diff/pull/230
Bugfix in TableSegment: Sampling now respects the 'where' clause (issue #221) by @erezsh in https://github.com/datafold/data-diff/pull/224

Other changes

Better error messages. Move some parsing to before the connects. Tests now only connect if being run. by @erezsh in https://github.com/datafold/data-diff/pull/222
Small bugfixes and refactor by @erezsh in https://github.com/datafold/data-diff/pull/223
Refactors and fixes by @erezsh in https://github.com/datafold/data-diff/pull/227

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.5...v0.2.6

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.5 - Alphanum key columns; Certificate auth in snowflake & presto

New features

Support for alphanumeric key columns
Support certificate authentication in snowflake and presto
Various bugfixes

What's Changed

Fixed docstring in diff_tables() (Issue #182) by @nklsw in https://github.com/datafold/data-diff/pull/183
Bugfix for Oracle - didn't properly handle .rounds attribute. by @erezsh in https://github.com/datafold/data-diff/pull/184
Added support for auto-detecting mutual columns, and using patterns in -c by @erezsh in https://github.com/datafold/data-diff/pull/185
Added new guide for implementing a database driver by @erezsh in https://github.com/datafold/data-diff/pull/189
Update issue templates by @erezsh in https://github.com/datafold/data-diff/pull/192
Create CODEOFCONDUCT.md by @erezsh in https://github.com/datafold/data-diff/pull/193
Update README.md by @kning in https://github.com/datafold/data-diff/pull/181
Bugfix for mutual columns feature (6a4c4438537) by @erezsh in https://github.com/datafold/data-diff/pull/198
[Tests] now using connect() instead of connecttouri(); refactor by @erezsh in https://github.com/datafold/data-diff/pull/202
Refactor - nicer regexp parsing; Trino now inherits from Presto by @erezsh in https://github.com/datafold/data-diff/pull/205
Add extra documentation on installing drivers for postgresql by @cfernhout in https://github.com/datafold/data-diff/pull/206
Update README.md by @glebmezh in https://github.com/datafold/data-diff/pull/209
Various fixes (issue #211, #208) by @erezsh in https://github.com/datafold/data-diff/pull/212
Fix for merging PR #187 by @erezsh in https://github.com/datafold/data-diff/pull/214
Cleanup by @erezsh in https://github.com/datafold/data-diff/pull/215
Added optional tracking by @erezsh in https://github.com/datafold/data-diff/pull/213
Cleanup by @erezsh in https://github.com/datafold/data-diff/pull/216
Presto snowflake enhancement by @matthiasekundayo-eb in https://github.com/datafold/data-diff/pull/187
Fix tests for BigQuery by @erezsh in https://github.com/datafold/data-diff/pull/218

New Contributors

@nklsw made their first contribution in https://github.com/datafold/data-diff/pull/183
@kning made their first contribution in https://github.com/datafold/data-diff/pull/181
@matthiasekundayo-eb made their first contribution in https://github.com/datafold/data-diff/pull/187

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.4...v0.2.5

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.4

Main changes

New features:
- Added arbitrary 'where' option (issue #86) by @erezsh in https://github.com/datafold/data-diff/pull/165
- [api] Function connecttotable() now accepts a dict (Issue #156) by @erezsh in https://github.com/datafold/data-diff/pull/157
New drivers:
- Support for Databricks by @pik94 in https://github.com/datafold/data-diff/pull/55 and @erezsh in https://github.com/datafold/data-diff/pull/160
- Support for Trino by @danthelion in https://github.com/datafold/data-diff/pull/155 and @erezsh in https://github.com/datafold/data-diff/pull/163
Optimization:
- Run faster on local to cloud diffs - Start running as soon as first min/max query returns (Issue #80) by @erezsh in https://github.com/datafold/data-diff/pull/174
Reliability :
- Now separating the columns with a '|' before concatting them by @erezsh in https://github.com/datafold/data-diff/pull/171
Bugfixes and other fixes:
- Fix for the occasional failure in tests in 3.7 by @erezsh in https://github.com/datafold/data-diff/pull/153
- Removed snowflake from list of dependencies (only a dev dep) by @erezsh in https://github.com/datafold/data-diff/pull/161
- Update Preql version to 0.2.16 by @erezsh in https://github.com/datafold/data-diff/pull/166
- Create CONTRIBUTING.md by @erezsh in https://github.com/datafold/data-diff/pull/164
- Initial support for running the tests for multiple databases (replacing TestWithConnection) by @erezsh in https://github.com/datafold/data-diff/pull/167
- Tests now cover oracle, Redshift, snowflake and bigquery; Various fixes to said drivers. by @erezsh in https://github.com/datafold/data-diff/pull/170
- Small fix for Oracle, for when a database isn't specified. by @erezsh in https://github.com/datafold/data-diff/pull/173
- Fix for CLI + tests for CLI (issue #175) by @erezsh in https://github.com/datafold/data-diff/pull/177
- Print configuration during debug, but with passwords redacted by @erezsh in https://github.com/datafold/data-diff/pull/172

New Contributors

@danthelion made their first contribution in https://github.com/datafold/data-diff/pull/155

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.3...v0.2.4

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.3 - Config files; Better UUID support.

Added support for config files - specify the arguments to data-diff using a TOML file
Added support for native UUIDs in Postgresql

What's Changed

Fixed tests; bisection_threshold can now be inf by @erezsh in https://github.com/datafold/data-diff/pull/134
tests: parallel + snowflake, presto in CI + benchmark scripts by @sirupsen in https://github.com/datafold/data-diff/pull/135
Update README.md to include authenticator in Snowflake connection string by @franloza in https://github.com/datafold/data-diff/pull/142
Fix tests for PRs from contributors who don't have access to 'secrets'. by @erezsh in https://github.com/datafold/data-diff/pull/147
Corrections for PR #144 - fix UUID things by @erezsh in https://github.com/datafold/data-diff/pull/148
Fix UUID things by @pik94 in https://github.com/datafold/data-diff/pull/144
Added support for native UUIDs in postgresql. by @erezsh in https://github.com/datafold/data-diff/pull/149
Specify data-diff arguments using config files by @erezsh in https://github.com/datafold/data-diff/pull/143
Small Fixes by @erezsh in https://github.com/datafold/data-diff/pull/151

New Contributors

@franloza made their first contribution in https://github.com/datafold/data-diff/pull/142

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.2...v0.2.3

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.2 - Support for UUIDs; Oracle schemas.

What's Changed

Main -

Oracle: Added support for schemas (Issue #115) by @erezsh in https://github.com/datafold/data-diff/pull/117
[MySQL] Added varbinary by @erezsh in https://github.com/datafold/data-diff/pull/132
Support for UUID key column by @erezsh in https://github.com/datafold/data-diff/pull/119
Fix UUIDs + small fix for presto by @erezsh in https://github.com/datafold/data-diff/pull/133

Also -

Split Integer from Decimal to reduce casts in SQL. Added FractionalType. by @erezsh in https://github.com/datafold/data-diff/pull/111
tests: add bigint/int by @sirupsen in https://github.com/datafold/data-diff/pull/126
benchmark: add suite by @sirupsen in https://github.com/datafold/data-diff/pull/125

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.1...v0.2.2

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.1 - Small fix for MySQL; Refactors

What's Changed

Move common ABCs and types to database_types.py; Fix type annotations by @erezsh in https://github.com/datafold/data-diff/pull/98
Refactor database.py -> databases/*.py, each db gets a file. by @erezsh in https://github.com/datafold/data-diff/pull/101
Small fix for issue #102 by @erezsh in https://github.com/datafold/data-diff/pull/104
readme: update screenshot with BigQuery, remove redundant pair by @sirupsen in https://github.com/datafold/data-diff/pull/97

Full Changelog: https://github.com/datafold/data-diff/compare/v0.2.0...v0.2.1

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.2.0 - Bridging between column types of different format and precision.

What's Changed

Fix for BigQuery: Table-name no longer needs dataset, takes it from URI by @erezsh in https://github.com/datafold/data-diff/pull/77
Added support for Numeric types with arbitrary precision by @erezsh in https://github.com/datafold/data-diff/pull/74
Refactor Normalize-types into normalizetimestamp() normalizenumber() by @erezsh in https://github.com/datafold/data-diff/pull/75
readme: add performance graph by @sirupsen in https://github.com/datafold/data-diff/pull/88
Better errors for missing imports by @erezsh in https://github.com/datafold/data-diff/pull/76
Fix: Only parse relevant columns. Only warn on relevant columns. by @erezsh in https://github.com/datafold/data-diff/pull/92
cli: output diff as jsonl, stats as json by @sirupsen in https://github.com/datafold/data-diff/pull/90

Full Changelog: https://github.com/datafold/data-diff/compare/v0.0.8...v0.2.0

- Python
Published by erezsh over 3 years ago

https://github.com/datafold/data-diff - v0.0.8 - Datetime types supported for all dbs, with arbitrary precision

- Python
Published by erezsh over 3 years ago

Recent Releases of https://github.com/datafold/data-diff

https://github.com/datafold/data-diff - v0.11.1

What's Changed

https://github.com/datafold/data-diff - v0.11.0

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.10.1

What's Changed

https://github.com/datafold/data-diff - v0.10.0

What's Changed

https://github.com/datafold/data-diff - v0.10.0rc0

What's Changed

https://github.com/datafold/data-diff - v0.9.17

What's Changed

https://github.com/datafold/data-diff - v0.9.16

What's Changed

https://github.com/datafold/data-diff - v0.9.15

What's Changed

https://github.com/datafold/data-diff - v0.9.14

What's Changed

https://github.com/datafold/data-diff - v0.9.13

What's Changed

https://github.com/datafold/data-diff - v0.9.12

What's Changed

https://github.com/datafold/data-diff - v0.9.11

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.9.10

What's Changed

https://github.com/datafold/data-diff - v0.9.9

What's Changed

https://github.com/datafold/data-diff - v0.9.8

What's Changed

https://github.com/datafold/data-diff - v0.9.7

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.9.6

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.9.5

What's Changed

https://github.com/datafold/data-diff - v0.9.4

What's Changed

https://github.com/datafold/data-diff - v0.9.3

What's Changed

https://github.com/datafold/data-diff - v0.9.2

What's Changed

https://github.com/datafold/data-diff - v0.9.1

What's Changed

https://github.com/datafold/data-diff - v0.9.0

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.8.5rc1

What's Changed

https://github.com/datafold/data-diff - v0.8.5rc0

What's Changed

https://github.com/datafold/data-diff - v0.8.4

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.8.3

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.8.2

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.8.1

What's Changed

https://github.com/datafold/data-diff - v0.8.0

What's Changed

New Contributors

https://github.com/datafold/data-diff - v0.7.14

What's Changed

https://github.com/datafold/data-diff - v0.7.13

What's Changed

https://github.com/datafold/data-diff - v0.7.12

What's Changed

https://github.com/datafold/data-diff - v0.7.11

What's Changed

https://github.com/datafold/data-diff - v0.7.10

What's Changed