Repository navigation
Conversation
5499798 to
df7bebd
Compare
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #612 +/- ##
==========================================
+ Coverage 95.10% 95.47% +0.36%
==========================================
Files 17 18 +1
Lines 4900 5278 +378
==========================================
+ Hits 4660 5039 +379
+ Misses 240 239 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
dae5993 to
4016686
Compare
Add an opt-in JSON collection comparison facade with path-scoped identity matching, sorting, filtering, normalization, diagnostics, compatibility boundaries, tests, and documentation.
067b7a2 to
a5faa5d
Compare
seperman
left a comment
There was a problem hiding this comment.
Static review: #612 Add path-scoped JSON collection strategies
Summary: This PR adds a new public module, deepdiff/json_diff.py, with a DeepJSONDiff facade and a CollectionStrategy config, and exports both from the package root. Each input is first rewritten into a "canonical" structure: lists picked by $-style selectors become identity-keyed dicts, sorted lists, or filtered/normalized lists. A plain DeepDiff then compares the two canonical structures. The code is readable and validates its configuration carefully, and the static review found nothing suspicious. The main concern is that the default missing-identity mode appears to drop records from the comparison without any warning. Several other points need attention: some forwarded DeepDiff options handle identity keys inconsistently, much of the feature overlaps existing options, and it introduces a second path syntax.
Overlap with existing features / other PRs
- No other open PR overlaps this one. Two related open issues are not linked: #393 asks for nested
group_bykeyed by a DeepDiff path (group_by={"root['_embedded']['messages']": "id"}), and #307 coversiterable_compare_funcon nested lists. group_by+group_by_sort_keyalready do identity keying, duplicate grouping and sorting, but only at the root (diff.pyon dev,if group_by and self.is_root, ~L387).match_by+DuplicateIdentityPolicy.GROUP+sort_byis in effect a nested version of that.ignore_order_func(level)already gives path-scoped order-insensitivity, and addingreport_repetition=Truemakes it a multiset comparison, which is whatcompare_as_setprovides.iterable_compare_funcalready pairs items by identity in selected lists.- The glob
exclude_pathsfrom #590 (e.g.root['orders'][*]['items'][*]['generatedAt']) already coversexclude_fields, nested fields included. _parse_pattern(json_diff.py:88-146) is a second wildcard path parser next toGlobPathMatcher/_path_to_elementsindeepdiff/path.py, and it uses a different syntax:$.groups[*].usersversusroot['groups'][*]['users']. Results come back inroot[...]syntax andget_stats()keys use$syntax, so one call involves several path dialects.
Security
No security concerns found in static review.
- The code does no file, network, subprocess or environment access and uses no dynamic imports,
eval/execor pickle.object.__setattr__is only called with fixed dataclass field names (json_diff.py:205-211). - Quoted selector keys are parsed with
ast.literal_eval(json_diff.py:80).deepdiff/path.pydoes the same, it does not execute code, and selectors are developer configuration rather than input data. - The diff contains no hidden, bidi or zero-width Unicode. Nothing in the code, docs or PR description addresses reviewers. The tests don't touch the filesystem or the network.
- Resource use:
- Canonicalization always copies both inputs in full, even when no strategy matches.
- Each item is deep-copied when
filter_funcornormalizersare set (json_diff.py:440-460). - This is an extra recursive pass before DeepDiff's own, so very deeply nested input hits
RecursionErroras it already does in DeepDiff. - Identity grouping uses a dict, so there is no quadratic matching.
deepdiff/__init__.py:15-23imports the module eagerly. It only uses stdlib modules deepdiff already imports (ast,json,enum,math,copy,dataclasses) and has no import-time side effects, so the extra cost is just creating its classes and dataclasses.
Correctness
- Fallback records may never be compared under the default policy.
- Records without an identity are stored under
"__deepdiff_fallback__"(json_diff.py:586-587), andFALLBACKis the default (json_diff.py:160). - DeepDiff's
_diff_dictskips string keys starting with__whenignore_private_variables=True, which is the default (diff.py,_diff_dict, ~L744-746 on dev). - From reading the code,
DeepJSONDiff({"items": [{"name": "a"}]}, {"items": [{"name": "b"}]}, collection_strategies=[CollectionStrategy(path="$.items", match_by=("id",))])should report no differences. That contradictsdocs/json_collection_strategies.rst:105-107. tests/test_json_diff.py:268-278uses different fallback records on each side but only asserts stats, so it would not catch this. This was not executed; a test would confirm it.
- Records without an identity are stored under
GROUPoutput changes shape with the count (json_diff.py:573-584). An identity with duplicates becomes a list, while a single record stays a dict. If one side has 2 records for an id and the other has 1, the diff reports a list→dicttype_changesinstead of an added or removed record.- Index selectors under an identity-matched parent use each side's original positions. Nested values are canonicalized with the original index (
json_diff.py:488,510-511).- Example:
$.ordersis matched byidand a second strategy targets$.orders[0].items. If the orders are simply reordered, the inner strategy applies to a different order on each side. - The same order then compares a dict against a list (
type_changes). - For the same reason,
get_stats()keys such as$.orders[0].itemscan't be matched to the label-keyed result paths.
- Example:
- Forwarded options treat identity keys inconsistently. Identity labels are JSON strings (
json_diff.py:311-320), and DeepDiff's key cleaning (_get_clean_to_keys_mapping) handles them as strings:ignore_string_casedoes apply. Ids"ABC"and"abc"merge into one key, and if both exist on one side DeepDiff logs a warning and keeps only one, so the other record is never compared.ignore_numeric_type_changesandsignificant_digitsnever apply. Ids1and1.0always show up as one record removed and one added.- Ordering has the same split.
_stable_valueplaces all ints before all floats (json_diff.py:331-340), socompare_as_seton[1, 2.0]vs[2, 1.0]yields twovalues_changedeven withignore_numeric_type_changes=True.
compare_as_setandsort_bystill produce positional diffs. The sorted lists are compared by index (json_diff.py:518-532).[1, 2, 3]vs[1, 3]is reported asvalues_changedat[1]plus a removal at[2], not as "2 removed". This is the cascading-diff problem the PR description sets out to solve.threshold_to_diff_deeper(default 0.33) applies to the identity maps. If less than a third of identities overlap, the whole collection is reported as onevalues_changedwhose values are the internal label-keyed dicts.exclude_fieldsonly removes top-level keys (json_diff.py:483-486), whilematch_byandsort_byaccept dotted paths (json_diff.py:266-283).exclude_fields=("meta.generatedAt",)does nothing and raises no error.- The list of rejected kwargs is incomplete (
json_diff.py:30-39).include_obj_callbackandinclude_obj_callback_strictreceive(obj, path)just like the rejectedexclude_obj_callback*options. DeepDiff's own multiprocessing gate (~L1601-1617 on dev) treats all four as path-sensitive. to_jsonbehaves differently fromDeepDiff.to_json.json_diff.py:624-627silently switches to the builtin json whensort_keysis truthy.serialization.json_dumpsdeliberately raises aTypeErrorwith guidance in that case when orjson is installed, andsort_keys=Falsestill reaches that error here.- Minor points:
- Changes a
filter_funcmakes to its argument carry over into the compared record (json_diff.py:460-466). The test's normalizer has to undo the filter's change (tests/test_json_diff.py:150-157), and the docs (:62-64) could say this. _extractaccepts negative indexes such asversions.-1(json_diff.py:278), which isn't documented.- Some malformed selectors raise a plain
ValueErrororTypeErrorinstead ofCollectionStrategyError.$[²]passesstr.isdigit()but failsint()(json_diff.py:133-134), andliteral_evalcan raiseTypeError(json_diff.py:79-82).
- Changes a
Design / API
- The module is a pre-processing layer over the public
DeepDiffAPI and needs no internals. Its main ideas (nested identity matching, per-path normalizers, strict missing/duplicate policies) could also be offered as smaller extensions of existing options. One example is a path-keyedgroup_by(#393) using the existing glob syntax, instead of a second entry point with its own grammar. - Using
root[...]selectors andGlobPathMatcherwould keep selectors, result paths and stats in one syntax and reuse code that is already tested. - Result paths look like
root['users']['[["int",2]]']. They are deterministic, but users must JSON-decode the label to get the record id. Results describe the canonical structures, soDelta(result.diff)can't be applied to the original inputs; worth documenting. get_stats()has the same name asDeepDiff.get_stats(), which returns internal run statistics, but returns something unrelated. A distinct name such asget_strategy_stats()would avoid confusion.- Rejecting
exclude_paths,include_pathsandexclude_regex_pathsoutright also removes path exclusion for parts of the data that no strategy touches. - The PR adds seven names to the package root (
deepdiff/__init__.py:15-23). Importing them fromdeepdiff.json_diffonly would keep the top-level namespace stable while the API settles. - Nits:
PathToken = str | int | objectis effectivelyobject(json_diff.py:27). The nameDeepJSONDiffsuggests the inputs are JSON strings.
Tests & docs
- Most tests assert
not resultor check only which report types appear (tests/test_json_diff.py:31-35). None check the actual result paths or values for identity-matched changes, which are the feature's main contract. - Missing test cases:
- differences between fallback records (this would catch the
__key issue); GROUPwith different duplicate counts on each side;compare_as_setwith different counts;- forwarded kwargs (
ignore_string_case,ignore_numeric_type_changes,threshold_to_diff_deeper,view='tree',ignore_order); - rejection of
include_obj_callback*.
- differences between fallback records (this would catch the
tests/test_json_diff.py:176-189passesnot resultonly because both sides share the samefloat('nan')object, and DeepDiff short-circuits whenlevel.t1 is level.t2. NaNs parsed separately would be reported unlessignore_nan_inequality=True.tests/test_json_diff_merge_contracts.pycould be folded intotest_json_diff.py; its name doesn't describe what it tests.- Other doc pages live in
deepdiff/docstrings/with a symlink indocs/, but this one is a regular file indocs/only. deepdiff/docstrings/index.rstincludes unrelated whitespace edits (lines 33, 35, ~75) and a new "Unreleased" changelog section.
Suggestions
- Use a fallback key that doesn't start with
__, and add a test showing that differences between fallback records are reported. - Always store a list under
DuplicateIdentityPolicy.GROUP. - Reject
include_obj_callbackandinclude_obj_callback_stricttoo. Decide and document how the case, numeric-type and precision options apply to identities and to sort order. - Consider reusing
deepdiff.pathglob parsing and theroot[...]syntax for selectors. - For
compare_as_set, consider scoping DeepDiff's ownignore_order+report_repetitionto those paths with a generatedignore_order_func, instead of sorting and comparing by position. - Retarget the PR to
dev, where feature work (including #590) lands; it merges intoorigin/devwith no conflicts.
AI-assisted static review; the code was not executed.
- Fallback records under a key starting with "__" were silently skipped by DeepDiff's default ignore_private_variables=True; rename to deepdiff_fallback and add a regression test showing fallback differences are reported. - DuplicateIdentityPolicy.GROUP now always stores a list so a duplicate-count change diffs record-by-record instead of as a list-to-dict type change. - Reject include_obj_callback / include_obj_callback_strict (path-sensitive, same shape as the already-rejected exclude_obj_callback* options). - Reject dotted exclude_fields instead of silently ignoring them. - Bracket selectors with unicode digits no longer leak a plain ValueError; literal_eval failures also catch TypeError. - to_json delegates unchanged to DeepDiff.to_json (no silent builtin-json switch for sort_keys). - Rename get_stats to get_strategy_stats to disambiguate from DeepDiff.get_stats run statistics. - Docs: move to deepdiff/docstrings with docs/ symlink per repo convention, document identity-label interactions with forwarded numeric/case options, top-level-only exclude_fields, GROUP list shape, filter_func mutation carry-over, negative list indexes, concrete-index selectors under identity-matched parents, Delta limitation, and positional compare_as_set diffs. Revert unrelated whitespace edits in index.rst. - Tests: assert exact identity-keyed result paths and values, GROUP and compare_as_set count changes, int/float identity contract with ignore_numeric_type_changes, forwarded ignore_order/view, and the new rejections. NaN sort test no longer relies on shared object identity.
|
Thanks for the thorough review. Fixed in two follow-up commits (coverage + this review); every claim below was reproduced against the code before changing it. Fixed
Not changed (judgment calls, happy to discuss)
|
seperman
left a comment
There was a problem hiding this comment.
Thanks for the detailed follow-up. I found a correctness problem in the core identity-matching path, plus a smaller related one.
1. Identity-matched collections collapse into a single values_changed when few identities overlap
_index_by_identity turns each matched list into a dict keyed by the encoded identity label, and that dict is passed to DeepDiff as an ordinary dictionary. DeepDiff._diff_dict has the threshold_to_diff_deeper shortcut (default 0.33): when len(shared keys) / len(all keys) < 0.33, it stops descending and reports the whole dict as one values_changed.
For identity-matched collections this happens exactly when matching by identity should help most: when several records are added or removed together. Running DeepDiff on the canonical structure that DeepJSONDiff builds for match_by=("id",):
left = [{"id": 1, "n": "a"}, {"id": 2}, {"id": 3}]
right = [{"id": 1, "n": "CHANGED"}, {"id": 4}, {"id": 5}]
# canonical: {"users": {'[["int",1]]': {...}, '[["int",2]]': {...}, '[["int",3]]': {...}}} vs ids 1, 4, 5Result:
{'values_changed': {"root['users']": {
'new_value': {'[["int",1]]': {'id': 1, 'n': 'CHANGED'}, '[["int",4]]': {'id': 4}, '[["int",5]]': {'id': 5}},
'old_value': {'[["int",1]]': {'id': 1, 'n': 'a'}, '[["int",2]]': {'id': 2}, '[["int",3]]': {'id': 3}}}}}- The whole collection is reported as one change, with the internal encoded labels exposed as keys.
- The real field change on record
1is not reported on its own. - A single replaced record (
[{"id": 1}]vs[{"id": 2}]) gives the same one-blob result. - Plain
DeepDiffon the raw lists does better here: it reports three separate field changes.
With threshold_to_diff_deeper=0, the same canonical inputs give the intended result:
{'dictionary_item_added': ['root[\'users\'][\'[["int",4]]\']', 'root[\'users\'][\'[["int",5]]\']'],
'dictionary_item_removed': ['root[\'users\'][\'[["int",2]]\']', 'root[\'users\'][\'[["int",3]]\']'],
'values_changed': {'root[\'users\'][\'[["int",1]]\'][\'n\']': {'new_value': 'CHANGED', 'old_value': 'a'}}}test_identity_matching_ignores_reordering_and_reports_real_changes doesn't catch this because its ids {1,2} vs {2,3} give a ratio of exactly 1/3, just above the threshold.
Suggested fix: have DeepJSONDiff default threshold_to_diff_deeper to 0 unless the caller passes it explicitly. Add tests for "one record replaced" and "most records replaced" that assert per-record dictionary_item_added / dictionary_item_removed and the nested field change.
2. ignore_string_case silently drops records
The docs mention this, but the effect is silent data loss rather than a softer matching mode. _get_clean_to_keys_mapping lowercases the label keys. When two labels collide, it keeps the first and only logs a warning. Example: ids "A" and "a" on both sides, and the record with id "a" changes v from 1 to 999.
- Without the option, the change is reported.
- With
ignore_string_case=True, the result is{}.
Since the other options that change meaning on the canonical structure are already rejected, ignore_string_case should probably be rejected too. Alternatively, raise when two identity labels collide after case folding.
Note on how this was checked: I ran the repository's DeepDiff on the canonical dicts described in _index_by_identity, not the PR branch itself. Please confirm with DeepJSONDiff directly when adding the tests.
|
Thank you for all the work on this, and for the careful, thorough responses to the earlier review. It's clear a lot of thought went into the design, the docs and the tests. After looking at it closely, we've decided not to merge this approach. Matching list records by a business key is a real need, and we want DeepDiff to support it. But we'd like to build it into DeepDiff's existing list matching rather than as a separate layer on top of it. The main reasons:
We've written up the approach we'd like to take in #664: a new DeepDiff parameter that maps glob paths to a key spec. Matching happens in O(n+m) inside the existing list matching, and results keep real indexes, so it works with A lot of your work carries over directly: composite and nested keys, duplicate and missing-key handling, and many of your test scenarios map onto the acceptance criteria in #664. If you'd like to take #664 on, you'd be very welcome, and we'd be glad to review it. Thanks again for pushing this forward. |
Problem statement
DeepDiff compares arbitrary Python and JSON-like structures, but nested API collections are often unordered and keyed by business identity rather than list position. Positional comparison can create cascading false positives when records are inserted, removed, or reordered.
This PR adds an opt-in, reusable normalization layer for declaring collection-specific semantics without changing existing
DeepDiffbehavior.Proposed solution
The PR introduces
DeepJSONDiffandCollectionStrategy. Both inputs are converted into canonical caller-isolated views, then compared by the existingDeepDiffengine.The existing
DeepDiffconstructor and comparison semantics remain unchanged.Selector semantics
Supported selectors include:
[*]matches exactly one array index..*matches exactly one object key.prioritywins.JSON object keys are expected to be strings. Diagnostic paths use the same quoted-key syntax and can be reused as selectors.
Collection behavior
Identity matching
match_bysupports single, composite, and nested fields. Identity values must resolve after normalization to finite JSON scalars:None, booleans, integers, finite floats, or strings.Identity labels use canonical type-preserving encoding, so integer
1, float1.0, and string"1"remain distinct and composite identities cannot collide through delimiter characters.Filtering, normalization, sorting, and exclusion
Filters and normalizers receive defensive copies and may mutate them without modifying caller-owned payloads. Copies are created only when callbacks are configured.
Processing order is:
Identity and sort fields may therefore be excluded from the compared record while still controlling matching or ordering.
sort_byprovides a total, type-stable order for JSON-compatible values. Integers and floats remain distinct. Finite values, infinities, and NaN values have deterministic positions without heterogeneous comparison failures.Structured sort keys are restricted to JSON lists and mappings with string keys. Unsupported values are rejected instead of being ordered through unstable
repr()output.Order-insensitive scalar arrays
compare_as_set=Trueperforms multiset/bag comparison:match_byorsort_by.Missing and duplicate identities
Missing identity policies:
fallback: retain records in an ordered fallback collection;exclude: omit them;error: raiseIdentityExtractionErrorwith input side and path context.Duplicate policies:
error: raiseDuplicateIdentityError;group: retain every record under the identity and optionally order the group withsort_by.Enum members and string values are accepted.
Diagnostics
get_strategy_stats()separates diagnostics by input side:Each entry reports the selected strategy, input item count, filtered count, missing-identity count, and duplicate-group count. Separating sides prevents unrelated parent records from being conflated after parent reordering.
DeepDiff keyword compatibility
Identity matching changes selected arrays into canonical mappings. Options whose behavior depends on caller-visible paths, object paths, or iterable positions are rejected instead of being interpreted against a different structure:
include_pathsexclude_pathsexclude_regex_pathsignore_order_funciterable_compare_funccustom_operatorsexclude_obj_callbackexclude_obj_callback_strictOther keyword arguments are forwarded to the underlying
DeepDiffinstance.Result interface
DeepJSONDiffis a composition-based facade rather than a completeDeepDiffsubclass. It implements the read-onlyMappinginterface, delegatesto_dict()andto_json(), and exposes the underlying result throughresult.diff.Backward compatibility
The feature is fully opt-in:
DeepDiffbehavior is unchanged;DeepJSONDiff;Files changed
deepdiff/json_diff.py— facade, strategy model, selector parsing, canonicalization, identity matching, sorting, policies, diagnostics, and compatibility boundaries.deepdiff/__init__.py— public exports.tests/test_json_diff.py— primary functional, regression, and edge-case coverage.tests/test_json_diff_merge_contracts.py— final sorting/exclusion and callback-compatibility contract tests.docs/json_collection_strategies.rst— public behavior and API documentation.deepdiff/docstrings/index.rst— documentation navigation and unreleased note.No custom GitHub Actions workflow is included.
Validation performed
The implementation is validated across Python 3.10–3.14 and includes focused tests for:
Scope and limitations
This PR provides deterministic exact reconciliation. It does not infer business identity or perform fuzzy matching. The selector grammar is intentionally smaller than full JSONPath. Relative identity and sort fields use dotted extraction and numeric list indexes.
For identity-matched collections, result paths contain canonical identity keys rather than unstable original list indexes.