Skip to content

fix: preserve metadata key named "meta" in Document.from_dict()#11952

Open
Otis0408 wants to merge 1 commit into
deepset-ai:mainfrom
Otis0408:fix/document-to-dict-meta-key-collision
Open

fix: preserve metadata key named "meta" in Document.from_dict()#11952
Otis0408 wants to merge 1 commit into
deepset-ai:mainfrom
Otis0408:fix/document-to-dict-meta-key-collision

Conversation

@Otis0408

Copy link
Copy Markdown
Contributor

What

Document.to_dict() defaults to flatten=True, merging every metadata key up into the top-level dict. Document.from_dict() reverses this by popping the top-level "meta" key back out as the meta constructor parameter. This breaks when a document's metadata legitimately contains a JSON-serializable key literally named "meta": from_dict() pops that value as the meta parameter and splats it with {**meta}. When the value is not a mapping this raises TypeError: 'str' object is not a mapping, so a normal document cannot be serialized and read back through the default round-trip.

Fix

In from_dict(), after popping the "meta" key, check its type. A value that is not a mapping cannot be a valid meta parameter (which must be a dictionary); it can only be a flattened metadata key named "meta". Treat it as flattened metadata instead of splatting it, so the value is preserved and the round-trip succeeds. A dict value under "meta" continues to be treated as the meta parameter, preserving existing behavior (test_from_dict_with_parameters, test_from_dict_with_flat_and_non_flat_meta).

Before / after

from haystack.dataclasses import Document

doc = Document(content="hi", meta={"meta": "value"})

# Before: TypeError: 'str' object is not a mapping
# After:  round-trips cleanly
restored = Document.from_dict(doc.to_dict())
assert restored == doc
assert restored.meta == {"meta": "value"}

Tests

Added test_to_dict_from_dict_roundtrip_with_meta_key_named_meta. It fails on the pre-fix source with the TypeError above and passes with the fix. The full test/dataclasses/test_document.py suite passes (30 tests).

A metadata key literally named "meta" produced by the default to_dict(flatten=True)
was popped as the meta constructor parameter and splatted with {**meta}, raising
TypeError when its value was not a mapping. A non-mapping "meta" value can only be a
flattened metadata key, so treat it as one and preserve it through the round-trip.
@Otis0408 Otis0408 requested a review from a team as a code owner July 10, 2026 09:26
@Otis0408 Otis0408 requested review from sjrl and removed request for a team July 10, 2026 09:26
@vercel

vercel Bot commented Jul 10, 2026

Copy link
Copy Markdown

@Otis0408 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Jul 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/dataclasses
  document.py
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant