Skip to content

Add path-scoped JSON collection strategies - #612

Closed
akshat62 wants to merge 4 commits into
qlustered:devfrom
akshat62:akshat62/path-scoped-collection-strategies
Closed

akshat62 wants to merge 4 commits into
qlustered:devfrom
akshat62:akshat62/path-scoped-collection-strategies

Conversation

@akshat62

@akshat62 akshat62 commented Jul 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem statement

DeepDiff compares arbitrary Python and JSON-like structures, but nested API collections are often unordered and keyed by business identity rather than list position. Positional comparison can create cascading false positives when records are inserted, removed, or reordered.

This PR adds an opt-in, reusable normalization layer for declaring collection-specific semantics without changing existing DeepDiff behavior.

Proposed solution

The PR introduces DeepJSONDiff and CollectionStrategy. Both inputs are converted into canonical caller-isolated views, then compared by the existing DeepDiff engine.

from deepdiff import CollectionStrategy, DeepJSONDiff

result = DeepJSONDiff(
    expected,
    actual,
    collection_strategies=[
        CollectionStrategy(
            path="$.orders[*].items",
            match_by=("product.id", "warehouse.code"),
            filter_func=lambda item: item.get("active", True),
            exclude_fields=("generatedAt", "requestId"),
        )
    ],
)

The existing DeepDiff constructor and comparison semantics remain unchanged.

Selector semantics

Supported selectors include:

$.users
$.groups[0].users
$.groups[*].users
$.*.users
$['a-b']
  • [*] matches exactly one array index.
  • .* matches exactly one object key.
  • Quoted bracket selectors address keys that cannot be represented safely in dot form.
  • Wildcards do not cross path levels.
  • Higher priority wins.
  • At equal priority, the pattern with more exact tokens wins.
  • Equally specific matches are rejected as ambiguous.

JSON object keys are expected to be strings. Diagnostic paths use the same quoted-key syntax and can be reused as selectors.

Collection behavior

Identity matching

match_by supports single, composite, and nested fields. Identity values must resolve after normalization to finite JSON scalars: None, booleans, integers, finite floats, or strings.

Identity labels use canonical type-preserving encoding, so integer 1, float 1.0, and string "1" remain distinct and composite identities cannot collide through delimiter characters.

Filtering, normalization, sorting, and exclusion

Filters and normalizers receive defensive copies and may mutate them without modifying caller-owned payloads. Copies are created only when callbacks are configured.

Processing order is:

  1. apply the filter;
  2. apply normalizers;
  3. capture identity and sort fields;
  4. remove excluded fields;
  5. canonicalize nested content.

Identity and sort fields may therefore be excluded from the compared record while still controlling matching or ordering.

sort_by provides a total, type-stable order for JSON-compatible values. Integers and floats remain distinct. Finite values, infinities, and NaN values have deterministic positions without heterogeneous comparison failures.

Structured sort keys are restricted to JSON lists and mappings with string keys. Unsupported values are rejected instead of being ordered through unstable repr() output.

Order-insensitive scalar arrays

compare_as_set=True performs multiset/bag comparison:

  • order is ignored;
  • duplicate counts remain significant;
  • integer and float values remain type-distinct;
  • only finite JSON scalar values are accepted;
  • it cannot be combined with match_by or sort_by.

Missing and duplicate identities

Missing identity policies:

  • fallback: retain records in an ordered fallback collection;
  • exclude: omit them;
  • error: raise IdentityExtractionError with input side and path context.

Duplicate policies:

  • error: raise DuplicateIdentityError;
  • group: retain every record under the identity and optionally order the group with sort_by.

Enum members and string values are accepted.

Diagnostics

get_strategy_stats() separates diagnostics by input side:

stats["left"]["$.orders[0].items"]
stats["right"]["$.orders[0].items"]

Each entry reports the selected strategy, input item count, filtered count, missing-identity count, and duplicate-group count. Separating sides prevents unrelated parent records from being conflated after parent reordering.

DeepDiff keyword compatibility

Identity matching changes selected arrays into canonical mappings. Options whose behavior depends on caller-visible paths, object paths, or iterable positions are rejected instead of being interpreted against a different structure:

  • include_paths
  • exclude_paths
  • exclude_regex_paths
  • ignore_order_func
  • iterable_compare_func
  • custom_operators
  • exclude_obj_callback
  • exclude_obj_callback_strict

Other keyword arguments are forwarded to the underlying DeepDiff instance.

Result interface

DeepJSONDiff is a composition-based facade rather than a complete DeepDiff subclass. It implements the read-only Mapping interface, delegates to_dict() and to_json(), and exposes the underlying result through result.diff.

Backward compatibility

The feature is fully opt-in:

  • existing DeepDiff behavior is unchanged;
  • no strategy is applied unless the caller constructs DeepJSONDiff;
  • caller-owned inputs are not mutated;
  • existing views, Delta behavior, serialization, and multiprocessing paths remain untouched.

Files changed

  • deepdiff/json_diff.py — facade, strategy model, selector parsing, canonicalization, identity matching, sorting, policies, diagnostics, and compatibility boundaries.
  • deepdiff/__init__.py — public exports.
  • tests/test_json_diff.py — primary functional, regression, and edge-case coverage.
  • tests/test_json_diff_merge_contracts.py — final sorting/exclusion and callback-compatibility contract tests.
  • docs/json_collection_strategies.rst — public behavior and API documentation.
  • deepdiff/docstrings/index.rst — documentation navigation and unreleased note.

No custom GitHub Actions workflow is included.

Validation performed

The implementation is validated across Python 3.10–3.14 and includes focused tests for:

  • reordered nested collections;
  • additions, removals, and modifications together;
  • exact one-level wildcards and quoted-key selectors;
  • collision-safe composite identities;
  • scalar identity enforcement;
  • identity and sort extraction before exclusion;
  • callback isolation;
  • finite and non-finite numeric sorting;
  • type-stable integer/float multiset comparison;
  • structured JSON sorting;
  • duplicate-group ordering before exclusion;
  • side-separated nested diagnostics;
  • missing and duplicate policies on both inputs;
  • invalid strategy rejection;
  • complete path-sensitive DeepDiff option rejection;
  • deterministic serialization and mapping behavior.

Scope and limitations

This PR provides deterministic exact reconciliation. It does not infer business identity or perform fuzzy matching. The selector grammar is intentionally smaller than full JSONPath. Relative identity and sort fields use dotted extraction and numeric list indexes.

For identity-matched collections, result paths contain canonical identity keys rather than unstable original list indexes.

@akshat62 akshat62 changed the title Akshat62/path scoped collection strategies Add path-scoped JSON collection strategies Jul 18, 2026
@akshat62
akshat62 force-pushed the akshat62/path-scoped-collection-strategies branch from 5499798 to df7bebd Compare July 18, 2026 08:18
@codecov-commenter

codecov-commenter commented Jul 18, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.47%. Comparing base (79e4379) to head (f192d41).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #612      +/-   ##
==========================================
+ Coverage   95.10%   95.47%   +0.36%     
==========================================
  Files          17       18       +1     
  Lines        4900     5278     +378     
==========================================
+ Hits         4660     5039     +379     
+ Misses        240      239       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@akshat62
akshat62 force-pushed the akshat62/path-scoped-collection-strategies branch 2 times, most recently from dae5993 to 4016686 Compare July 18, 2026 09:13
Add an opt-in JSON collection comparison facade with path-scoped identity matching, sorting, filtering, normalization, diagnostics, compatibility boundaries, tests, and documentation.
@akshat62
akshat62 force-pushed the akshat62/path-scoped-collection-strategies branch from 067b7a2 to a5faa5d Compare July 18, 2026 10:08
@akshat62
akshat62 marked this pull request as ready for review July 18, 2026 10:15

@seperman seperman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review: #612 Add path-scoped JSON collection strategies

Summary: This PR adds a new public module, deepdiff/json_diff.py, with a DeepJSONDiff facade and a CollectionStrategy config, and exports both from the package root. Each input is first rewritten into a "canonical" structure: lists picked by $-style selectors become identity-keyed dicts, sorted lists, or filtered/normalized lists. A plain DeepDiff then compares the two canonical structures. The code is readable and validates its configuration carefully, and the static review found nothing suspicious. The main concern is that the default missing-identity mode appears to drop records from the comparison without any warning. Several other points need attention: some forwarded DeepDiff options handle identity keys inconsistently, much of the feature overlaps existing options, and it introduces a second path syntax.

Overlap with existing features / other PRs

  • No other open PR overlaps this one. Two related open issues are not linked: #393 asks for nested group_by keyed by a DeepDiff path (group_by={"root['_embedded']['messages']": "id"}), and #307 covers iterable_compare_func on nested lists.
  • group_by + group_by_sort_key already do identity keying, duplicate grouping and sorting, but only at the root (diff.py on dev, if group_by and self.is_root, ~L387). match_by + DuplicateIdentityPolicy.GROUP + sort_by is in effect a nested version of that.
  • ignore_order_func(level) already gives path-scoped order-insensitivity, and adding report_repetition=True makes it a multiset comparison, which is what compare_as_set provides. iterable_compare_func already pairs items by identity in selected lists.
  • The glob exclude_paths from #590 (e.g. root['orders'][*]['items'][*]['generatedAt']) already covers exclude_fields, nested fields included.
  • _parse_pattern (json_diff.py:88-146) is a second wildcard path parser next to GlobPathMatcher / _path_to_elements in deepdiff/path.py, and it uses a different syntax: $.groups[*].users versus root['groups'][*]['users']. Results come back in root[...] syntax and get_stats() keys use $ syntax, so one call involves several path dialects.

Security

No security concerns found in static review.

  • The code does no file, network, subprocess or environment access and uses no dynamic imports, eval/exec or pickle. object.__setattr__ is only called with fixed dataclass field names (json_diff.py:205-211).
  • Quoted selector keys are parsed with ast.literal_eval (json_diff.py:80). deepdiff/path.py does the same, it does not execute code, and selectors are developer configuration rather than input data.
  • The diff contains no hidden, bidi or zero-width Unicode. Nothing in the code, docs or PR description addresses reviewers. The tests don't touch the filesystem or the network.
  • Resource use:
    • Canonicalization always copies both inputs in full, even when no strategy matches.
    • Each item is deep-copied when filter_func or normalizers are set (json_diff.py:440-460).
    • This is an extra recursive pass before DeepDiff's own, so very deeply nested input hits RecursionError as it already does in DeepDiff.
    • Identity grouping uses a dict, so there is no quadratic matching.
  • deepdiff/__init__.py:15-23 imports the module eagerly. It only uses stdlib modules deepdiff already imports (ast, json, enum, math, copy, dataclasses) and has no import-time side effects, so the extra cost is just creating its classes and dataclasses.

Correctness

  • Fallback records may never be compared under the default policy.
    • Records without an identity are stored under "__deepdiff_fallback__" (json_diff.py:586-587), and FALLBACK is the default (json_diff.py:160).
    • DeepDiff's _diff_dict skips string keys starting with __ when ignore_private_variables=True, which is the default (diff.py, _diff_dict, ~L744-746 on dev).
    • From reading the code, DeepJSONDiff({"items": [{"name": "a"}]}, {"items": [{"name": "b"}]}, collection_strategies=[CollectionStrategy(path="$.items", match_by=("id",))]) should report no differences. That contradicts docs/json_collection_strategies.rst:105-107.
    • tests/test_json_diff.py:268-278 uses different fallback records on each side but only asserts stats, so it would not catch this. This was not executed; a test would confirm it.
  • GROUP output changes shape with the count (json_diff.py:573-584). An identity with duplicates becomes a list, while a single record stays a dict. If one side has 2 records for an id and the other has 1, the diff reports a list→dict type_changes instead of an added or removed record.
  • Index selectors under an identity-matched parent use each side's original positions. Nested values are canonicalized with the original index (json_diff.py:488, 510-511).
    • Example: $.orders is matched by id and a second strategy targets $.orders[0].items. If the orders are simply reordered, the inner strategy applies to a different order on each side.
    • The same order then compares a dict against a list (type_changes).
    • For the same reason, get_stats() keys such as $.orders[0].items can't be matched to the label-keyed result paths.
  • Forwarded options treat identity keys inconsistently. Identity labels are JSON strings (json_diff.py:311-320), and DeepDiff's key cleaning (_get_clean_to_keys_mapping) handles them as strings:
    • ignore_string_case does apply. Ids "ABC" and "abc" merge into one key, and if both exist on one side DeepDiff logs a warning and keeps only one, so the other record is never compared.
    • ignore_numeric_type_changes and significant_digits never apply. Ids 1 and 1.0 always show up as one record removed and one added.
    • Ordering has the same split. _stable_value places all ints before all floats (json_diff.py:331-340), so compare_as_set on [1, 2.0] vs [2, 1.0] yields two values_changed even with ignore_numeric_type_changes=True.
  • compare_as_set and sort_by still produce positional diffs. The sorted lists are compared by index (json_diff.py:518-532). [1, 2, 3] vs [1, 3] is reported as values_changed at [1] plus a removal at [2], not as "2 removed". This is the cascading-diff problem the PR description sets out to solve.
  • threshold_to_diff_deeper (default 0.33) applies to the identity maps. If less than a third of identities overlap, the whole collection is reported as one values_changed whose values are the internal label-keyed dicts.
  • exclude_fields only removes top-level keys (json_diff.py:483-486), while match_by and sort_by accept dotted paths (json_diff.py:266-283). exclude_fields=("meta.generatedAt",) does nothing and raises no error.
  • The list of rejected kwargs is incomplete (json_diff.py:30-39). include_obj_callback and include_obj_callback_strict receive (obj, path) just like the rejected exclude_obj_callback* options. DeepDiff's own multiprocessing gate (~L1601-1617 on dev) treats all four as path-sensitive.
  • to_json behaves differently from DeepDiff.to_json. json_diff.py:624-627 silently switches to the builtin json when sort_keys is truthy. serialization.json_dumps deliberately raises a TypeError with guidance in that case when orjson is installed, and sort_keys=False still reaches that error here.
  • Minor points:
    • Changes a filter_func makes to its argument carry over into the compared record (json_diff.py:460-466). The test's normalizer has to undo the filter's change (tests/test_json_diff.py:150-157), and the docs (:62-64) could say this.
    • _extract accepts negative indexes such as versions.-1 (json_diff.py:278), which isn't documented.
    • Some malformed selectors raise a plain ValueError or TypeError instead of CollectionStrategyError. $[²] passes str.isdigit() but fails int() (json_diff.py:133-134), and literal_eval can raise TypeError (json_diff.py:79-82).

Design / API

  • The module is a pre-processing layer over the public DeepDiff API and needs no internals. Its main ideas (nested identity matching, per-path normalizers, strict missing/duplicate policies) could also be offered as smaller extensions of existing options. One example is a path-keyed group_by (#393) using the existing glob syntax, instead of a second entry point with its own grammar.
  • Using root[...] selectors and GlobPathMatcher would keep selectors, result paths and stats in one syntax and reuse code that is already tested.
  • Result paths look like root['users']['[["int",2]]']. They are deterministic, but users must JSON-decode the label to get the record id. Results describe the canonical structures, so Delta(result.diff) can't be applied to the original inputs; worth documenting.
  • get_stats() has the same name as DeepDiff.get_stats(), which returns internal run statistics, but returns something unrelated. A distinct name such as get_strategy_stats() would avoid confusion.
  • Rejecting exclude_paths, include_paths and exclude_regex_paths outright also removes path exclusion for parts of the data that no strategy touches.
  • The PR adds seven names to the package root (deepdiff/__init__.py:15-23). Importing them from deepdiff.json_diff only would keep the top-level namespace stable while the API settles.
  • Nits: PathToken = str | int | object is effectively object (json_diff.py:27). The name DeepJSONDiff suggests the inputs are JSON strings.

Tests & docs

  • Most tests assert not result or check only which report types appear (tests/test_json_diff.py:31-35). None check the actual result paths or values for identity-matched changes, which are the feature's main contract.
  • Missing test cases:
    • differences between fallback records (this would catch the __ key issue);
    • GROUP with different duplicate counts on each side;
    • compare_as_set with different counts;
    • forwarded kwargs (ignore_string_case, ignore_numeric_type_changes, threshold_to_diff_deeper, view='tree', ignore_order);
    • rejection of include_obj_callback*.
  • tests/test_json_diff.py:176-189 passes not result only because both sides share the same float('nan') object, and DeepDiff short-circuits when level.t1 is level.t2. NaNs parsed separately would be reported unless ignore_nan_inequality=True.
  • tests/test_json_diff_merge_contracts.py could be folded into test_json_diff.py; its name doesn't describe what it tests.
  • Other doc pages live in deepdiff/docstrings/ with a symlink in docs/, but this one is a regular file in docs/ only.
  • deepdiff/docstrings/index.rst includes unrelated whitespace edits (lines 33, 35, ~75) and a new "Unreleased" changelog section.

Suggestions

  • Use a fallback key that doesn't start with __, and add a test showing that differences between fallback records are reported.
  • Always store a list under DuplicateIdentityPolicy.GROUP.
  • Reject include_obj_callback and include_obj_callback_strict too. Decide and document how the case, numeric-type and precision options apply to identities and to sort order.
  • Consider reusing deepdiff.path glob parsing and the root[...] syntax for selectors.
  • For compare_as_set, consider scoping DeepDiff's own ignore_order + report_repetition to those paths with a generated ignore_order_func, instead of sorting and comparing by position.
  • Retarget the PR to dev, where feature work (including #590) lands; it merges into origin/dev with no conflicts.

AI-assisted static review; the code was not executed.

akshat62 and others added 3 commits November 10, 2026 08:31
- Fallback records under a key starting with "__" were silently skipped by
  DeepDiff's default ignore_private_variables=True; rename to
  deepdiff_fallback and add a regression test showing fallback differences
  are reported.
- DuplicateIdentityPolicy.GROUP now always stores a list so a duplicate-count
  change diffs record-by-record instead of as a list-to-dict type change.
- Reject include_obj_callback / include_obj_callback_strict (path-sensitive,
  same shape as the already-rejected exclude_obj_callback* options).
- Reject dotted exclude_fields instead of silently ignoring them.
- Bracket selectors with unicode digits no longer leak a plain ValueError;
  literal_eval failures also catch TypeError.
- to_json delegates unchanged to DeepDiff.to_json (no silent builtin-json
  switch for sort_keys).
- Rename get_stats to get_strategy_stats to disambiguate from
  DeepDiff.get_stats run statistics.
- Docs: move to deepdiff/docstrings with docs/ symlink per repo convention,
  document identity-label interactions with forwarded numeric/case options,
  top-level-only exclude_fields, GROUP list shape, filter_func mutation
  carry-over, negative list indexes, concrete-index selectors under
  identity-matched parents, Delta limitation, and positional compare_as_set
  diffs. Revert unrelated whitespace edits in index.rst.
- Tests: assert exact identity-keyed result paths and values, GROUP and
  compare_as_set count changes, int/float identity contract with
  ignore_numeric_type_changes, forwarded ignore_order/view, and the new
  rejections. NaN sort test no longer relies on shared object identity.
@akshat62

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review. Fixed in two follow-up commits (coverage + this review); every claim below was reproduced against the code before changing it.

Fixed

  • Fallback records silently un-compared (the big one): confirmed — the fallback key started with __, which DeepDiff skips by default via ignore_private_variables=True, so under the default FALLBACK policy different fallback records produced no diff at all. Renamed to deepdiff_fallback and added a regression test asserting the fallback difference is reported.
  • GROUP shape depends on count: confirmed — 2-vs-1 duplicates reported type_changes. GROUP now always stores a list, so a count change diffs record-by-record (iterable_item_removed); tested.
  • include_obj_callback options not rejected: added to the rejected set (same path-sensitive shape as exclude_obj_callback*); tested.
  • Dotted exclude_fields silently no-op: now rejected at validation with a clear error; documented as top-level-only; tested.
  • Plain ValueError/TypeError escapes: $[²] (unicode digit passes isdigit(), fails int()) now raises CollectionStrategyError; literal_eval failures also catch TypeError; tested.
  • to_json diverged from DeepDiff.to_json: the silent builtin-json switch for sort_keys is gone; it delegates unchanged now.
  • get_stats name collision: renamed to get_strategy_stats() (pre-merge, so cheap now).
  • Index selectors under identity-matched parents: documented — they resolve against each side's original order, so use [*] wildcards there.
  • Docs: moved to deepdiff/docstrings/ + docs/ symlink per repo convention; reverted the unrelated whitespace edits in index.rst; documented the decisions on forwarded options (identity labels are type-preserving encoded strings: ignore_numeric_type_changes/significant_digits never merge int/float identities — contract-tested; ignore_string_case folds case in the labels and can merge/lose records, so it should not be relied on for identity fields), filter_func mutation carry-over, negative list indexes, the Delta limitation (results describe canonical structures), and positional compare_as_set/sort diffs.
  • Tests: added exact identity-keyed path/value assertions (the main contract), GROUP and compare_as_set count changes, forwarded ignore_order/view, and the NaN sort test no longer passes by shared object identity (separate NaNs + ignore_nan_inequality). Module coverage stays at 100%.

Not changed (judgment calls, happy to discuss)

  • Reusing GlobPathMatcher/root[...] for selectors and the ignore_order + report_repetition scoping for compare_as_set — real ideas, but both are redesigns of the selector/normalization core rather than review fixes; the small self-contained grammar is deliberate (quoted keys, one-level wildcards) and documented.
  • Keeping the seven top-level exports: the PR description and docs advertise from deepdiff import CollectionStrategy, DeepJSONDiff, so I would rather settle the API shape first.
  • The two open issues (Ignoring order for deeply nested lists of dictionaries. #393, iterable_compare_func does not seem to work on nested lists #307) and the retarget-to-dev suggestion — process items; can retarget once this round is addressed.

@seperman
seperman changed the base branch from master to dev November 16, 2026 19:51

@seperman seperman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed follow-up. I found a correctness problem in the core identity-matching path, plus a smaller related one.

1. Identity-matched collections collapse into a single values_changed when few identities overlap

_index_by_identity turns each matched list into a dict keyed by the encoded identity label, and that dict is passed to DeepDiff as an ordinary dictionary. DeepDiff._diff_dict has the threshold_to_diff_deeper shortcut (default 0.33): when len(shared keys) / len(all keys) < 0.33, it stops descending and reports the whole dict as one values_changed.

For identity-matched collections this happens exactly when matching by identity should help most: when several records are added or removed together. Running DeepDiff on the canonical structure that DeepJSONDiff builds for match_by=("id",):

left  = [{"id": 1, "n": "a"}, {"id": 2}, {"id": 3}]
right = [{"id": 1, "n": "CHANGED"}, {"id": 4}, {"id": 5}]
# canonical: {"users": {'[["int",1]]': {...}, '[["int",2]]': {...}, '[["int",3]]': {...}}} vs ids 1, 4, 5

Result:

{'values_changed': {"root['users']": {
    'new_value': {'[["int",1]]': {'id': 1, 'n': 'CHANGED'}, '[["int",4]]': {'id': 4}, '[["int",5]]': {'id': 5}},
    'old_value': {'[["int",1]]': {'id': 1, 'n': 'a'}, '[["int",2]]': {'id': 2}, '[["int",3]]': {'id': 3}}}}}
  • The whole collection is reported as one change, with the internal encoded labels exposed as keys.
  • The real field change on record 1 is not reported on its own.
  • A single replaced record ([{"id": 1}] vs [{"id": 2}]) gives the same one-blob result.
  • Plain DeepDiff on the raw lists does better here: it reports three separate field changes.

With threshold_to_diff_deeper=0, the same canonical inputs give the intended result:

{'dictionary_item_added': ['root[\'users\'][\'[["int",4]]\']', 'root[\'users\'][\'[["int",5]]\']'],
 'dictionary_item_removed': ['root[\'users\'][\'[["int",2]]\']', 'root[\'users\'][\'[["int",3]]\']'],
 'values_changed': {'root[\'users\'][\'[["int",1]]\'][\'n\']': {'new_value': 'CHANGED', 'old_value': 'a'}}}

test_identity_matching_ignores_reordering_and_reports_real_changes doesn't catch this because its ids {1,2} vs {2,3} give a ratio of exactly 1/3, just above the threshold.

Suggested fix: have DeepJSONDiff default threshold_to_diff_deeper to 0 unless the caller passes it explicitly. Add tests for "one record replaced" and "most records replaced" that assert per-record dictionary_item_added / dictionary_item_removed and the nested field change.

2. ignore_string_case silently drops records

The docs mention this, but the effect is silent data loss rather than a softer matching mode. _get_clean_to_keys_mapping lowercases the label keys. When two labels collide, it keeps the first and only logs a warning. Example: ids "A" and "a" on both sides, and the record with id "a" changes v from 1 to 999.

  • Without the option, the change is reported.
  • With ignore_string_case=True, the result is {}.

Since the other options that change meaning on the canonical structure are already rejected, ignore_string_case should probably be rejected too. Alternatively, raise when two identity labels collide after case folding.

Note on how this was checked: I ran the repository's DeepDiff on the canonical dicts described in _index_by_identity, not the PR branch itself. Please confirm with DeepJSONDiff directly when adding the tests.

@seperman

Copy link
Copy Markdown
Member

Thank you for all the work on this, and for the careful, thorough responses to the earlier review. It's clear a lot of thought went into the design, the docs and the tests.

After looking at it closely, we've decided not to merge this approach. Matching list records by a business key is a real need, and we want DeepDiff to support it. But we'd like to build it into DeepDiff's existing list matching rather than as a separate layer on top of it. The main reasons:

  • Lists should stay lists. Rebuilding inputs into identity-keyed dicts means:

    • results use encoded keys (root['users']['[["int",2]]']) instead of real paths;
    • Delta can't be applied to the original inputs;
    • ten DeepDiff options have to be rejected.

    It's also the root cause of the two bugs in my earlier comment, where dict-only behaviour (threshold_to_diff_deeper, ignore_string_case) leaks into list comparison.

  • One path syntax. We'd rather reuse the root[...] glob syntax from exclude_paths (thank you again for that one) than add a second selector language.

We've written up the approach we'd like to take in #664: a new DeepDiff parameter that maps glob paths to a key spec. Matching happens in O(n+m) inside the existing list matching, and results keep real indexes, so it works with Delta and the existing options. While scoping it we also found #665, a bug in the current iterable_compare_func matching that the new feature will build on.

A lot of your work carries over directly: composite and nested keys, duplicate and missing-key handling, and many of your test scenarios map onto the acceptance criteria in #664. If you'd like to take #664 on, you'd be very welcome, and we'd be glad to review it. Thanks again for pushing this forward.

@seperman

Copy link
Copy Markdown
Member

Closing in favour of #664, as explained above. Thanks again for the contribution. If you'd like to take #664 on, we'd be glad to review it.

@seperman seperman closed this Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants