Evaluate replacing the hand-written context walker with pyld

[LukasOro]

src/oold/validation/context_resolution.py hand-walks an OO-LD @context chain in about 300 lines to produce a flattened context. Its module docstring justified this by claiming a JSON-LD processor cannot consume these entries. That claim is false and has been corrected in #114; the code was left in place.

Evidence the premise was wrong

Pointing pyld at tests/data/oold/Person.schema.json as a remote context, with a document loader mapping the fixture directory:

doc = {"@context": "https://example.org/Person.schema.json", "name": "Ada"}
jsonld.expand(doc)
# -> [{"http://schema.org/name": [{"@value": "Ada"}]}]

It resolved, following the relative sibling Thing.schema.json and picking up the schema: prefix defined several hops up the chain. Per JSON-LD 1.1 a remote context document only needs a top-level @context; other entries are ignored. That is what OOLD-CMP-b926 guarantees by requiring an OO-LD schema to be usable as a context without further processing.

What is actually needed

The callers do not want expansion. They want the effective context as a value: which terms a schema defines and which hop each came from, for term reporting and the per-property attribution in src/oold/validation/predicates.py. pyld does return the flattened active context, through JsonLdProcessor.process_context - see #118 (comment) and the confirmation below it. What it does not carry is per-term provenance: mappings records no source document, so "which hop defined this term" has to be recovered by processing the chain a hop at a time and diffing.

Work

• Determine whether the effective context can be obtained from pyld without relying on private API, for example by expanding a probe document built from the schema's declared properties and reading back the mapping
• Compare the result against resolve_context() across the whole fixture corpus, including the scoped-context form {"@id": ..., "@context": "Other.schema.json"}
• If it holds, replace the walker; if it does not, record which specific capability is missing so the module has a justification that survives review

Constraint

rule.* and context.* verdicts must not move. The parity tests (OOLD_SCHEMA_DIR=../oold-schema uv run pytest -m parity) are the gate.

Raised from review of #114 (comment).

[simontaurus]

Checked: pyld does expose the flattened active context, through the public JsonLdProcessor.process_context.

p.process_context(p._get_initial_context({}), ctx_url, opts)['mappings']
  id         -> {"@id": "@id"}
  name       -> {"@id": "http://schema.org/name"}
  schema     -> {"@id": "http://schema.org/", "_prefix": true}
  type       -> {"@id": "@type"}
  works_for  -> {"@id": "http://schema.org/worksFor", "@type": "@id"}

Against tests/data/oold/Person.schema.json, which declares only schema and works_for: id, name and type are inherited from the relative sibling Thing.schema.json. So the chain is resolved and flattened, with @type, @container, protected, reverse and prefix flags per term.

Provenance is the only gap - mappings records no source document. That is obtainable by processing the chain one hop at a time and diffing, which is bookkeeping rather than a second implementation:

term provenance by diffing each hop:
  id           first defined by Thing.schema.json
  name         first defined by Thing.schema.json
  works_for    first defined by Person.schema.json

So the remaining justification does not hold for "which terms are in scope", only for "where each came from", and that half is ~10 lines on top of pyld. Worth replacing the walker rather than keeping a parallel implementation of context processing.

[LukasOro]

Confirmed independently, and you are right. Ran it against tests/data/oold/Person.schema.json with a loader mapping the fixture directory:

keys: ['_uuid', 'mappings', 'processingMode']
  id         -> {'@id': '@id',                            '_prefix': False, 'protected': False, 'reverse': False}
  name       -> {'@id': 'http://schema.org/name',         '_prefix': False, ...}
  schema     -> {'@id': 'http://schema.org/',             '_prefix': True,  ...}
  type       -> {'@id': '@type',                          '_prefix': False, ...}
  works_for  -> {'@id': 'http://schema.org/worksFor', '@type': '@id', ...}

Person.schema.json declares only schema and works_for, so id, name and type did come through the relative sibling Thing.schema.json. The chain is resolved and flattened, with prefix flags and @type per term.

I have corrected the issue body: the sentence claiming pyld exposes no public API for this was wrong, and leaving it would have justified the walker on a false premise.

Three things still to settle before the walker goes, none of them fatal:

• One of the two calls is still private. process_context is public, _get_initial_context is not. Either find a public route to the initial context, construct it directly, or accept the private call with a pinned pyld and a test that fails loudly on an upgrade. Worth deciding deliberately rather than discovering it in a release.
• x-oold-context is not a JSON-LD construct, so pyld will not see it. 7875b51 promotes those terms into the resolved context so a property mapped only there is not reported as dropped (OOLD-EXT-4966). That promotion stays on top of pyld whatever happens underneath.
• Scoped contexts in the {"@id": ..., "@context": "Other.schema.json"} form need checking against the corpus, as the issue already says.

On sequencing: after #120 rather than before it. 120 changes what context.predicates reports, and this issue's acceptance criterion is that verdicts do not move, so it needs 120's verdicts to be the settled ones. Then 119 after this, as you say there.

[LukasGold]

Evaluated. The walker stays, and the reason is narrower and better evidenced than the module previously claimed. #150 corrects the docstring; no code path changes.

The premise correction holds, and then some

Diffed the walker's context against JsonLdProcessor.process_context across all 13 top-level fixtures, using Resolver and DocumentLoader exactly as pipeline.py does. Every term set matched. Where they differed, pyld was better: it reports pre-expanded IRIs where the walker reports compact CURIEs, and on Organization.schema.json's employees, a @reverse term, the walker's naive .get("@id") yields None while pyld surfaces the reverse target.

So "which terms are in scope" is fully answered by pyld, as you said.

The blocker is scoped contexts, not provenance

process_context never resolves a term's scoped @context. It leaves the value exactly as authored and defers it to expansion, so {"@id": ..., "@context": "Pet.schema.json"} comes back as an unresolved string. Two committed fixtures use that form: Organization.schema.json (address to Address.schema.json) and PersonWithPet.schema.json (pets to Pet.schema.json). Contact.schema.json and ContactSeparateKeys.schema.json use an inline scoped context, which is unaffected.

Deferring to expansion would be fine if expansion could resolve it. It cannot, here: pipeline.py:487 calls check_predicates with no options, and check_predicates defaults options to None, so there is no document loader and no base IRI by the time the reference is reached. On Organization.schema.json's real context:

raw @context, no document loader     -> ValueError: Found invalid relative IRI 'Thing.schema.json'
walker's resolved context, no loader -> expand OK, identical predicates
raw @context + a document loader     -> expand OK, identical predicates

Note the third line: threading a loader through would make pyld's own lazy resolution work. That is a real route, but it is a predicates.py and pipeline.py contract change with its own blast radius, since roundtrip.py shares classify_property's pattern. It is not a context_resolution.py change, and it is not what this issue asked for.

Your three open points, resolved

• _get_initial_context is not a blocker. pyld 3.0.0's implementation returns {'_uuid': ..., 'processingMode': ..., 'mappings': {}} and nothing else, so the initial context is trivially constructible. If this module is ever rebuilt on pyld, construct it directly rather than pinning around a private call.
• x-oold-context promotion sits on top either way, unchanged.
• Scoped contexts are the blocker, per above.

Two things found in passing

A second docstring claim I could not reproduce. context_graph.py:11 states that on a cyclic scoped context "jsonld.js exhausts the heap and PyLD raises RecursionError", and CYCLIC_NOTE says neither can process one. Against a synthetic cyclic pair, jsonld.expand succeeded: it is lazy and data-driven, and roundtrip.py's frame is shallow. The narrower true statement is that a processor gives a caller no protection against re-entering it across a cycle, since successive calls share no memory of prior hops, which is what _walk_reference's stack and max_scoped_depth provide. I have not changed that docstring, since I could only refute the claim at the depths I could construct, not prove it never happens. Worth someone deciding deliberately.

cut_cycles and truncated_scoped_contexts have no consumer anywhere outside context_resolution.py, nor does ResolvedContext.to_dict(). Grepped across src/ and tests/. That is #126's territory, not this one.

A narrower opportunity that survives this outcome: terms() merges term definitions by hand, and could be recomputed from process_context per hop instead, gaining accurate @type, @container, protected, reverse and prefix flags for free. That replaces the merge without touching the walk, so neither blocker applies. Not opened as an issue; say if it is worth one.

On the gate

Parity passed, 6 passed with both OOLD_SCHEMA_DIR and OOLD_JS_DIR, and the deselected count confirms it ran rather than skipping. But per #148 that is agreement, not correctness, so it was not treated as sufficient evidence on its own. Nothing here moves a verdict regardless: the only change is a docstring.