fix: preserve metadata key named "meta" in Document.from_dict()#11952
Open
Otis0408 wants to merge 1 commit into
Open
fix: preserve metadata key named "meta" in Document.from_dict()#11952Otis0408 wants to merge 1 commit into
Otis0408 wants to merge 1 commit into
Conversation
A metadata key literally named "meta" produced by the default to_dict(flatten=True)
was popped as the meta constructor parameter and splatted with {**meta}, raising
TypeError when its value was not a mapping. A non-mapping "meta" value can only be a
flattened metadata key, so treat it as one and preserve it through the round-trip.
|
@Otis0408 is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
Coverage reportClick to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Document.to_dict()defaults toflatten=True, merging every metadata key up into the top-level dict.Document.from_dict()reverses this by popping the top-level"meta"key back out as themetaconstructor parameter. This breaks when a document's metadata legitimately contains a JSON-serializable key literally named"meta":from_dict()pops that value as themetaparameter and splats it with{**meta}. When the value is not a mapping this raisesTypeError: 'str' object is not a mapping, so a normal document cannot be serialized and read back through the default round-trip.Fix
In
from_dict(), after popping the"meta"key, check its type. A value that is not a mapping cannot be a validmetaparameter (which must be a dictionary); it can only be a flattened metadata key named"meta". Treat it as flattened metadata instead of splatting it, so the value is preserved and the round-trip succeeds. A dict value under"meta"continues to be treated as themetaparameter, preserving existing behavior (test_from_dict_with_parameters,test_from_dict_with_flat_and_non_flat_meta).Before / after
Tests
Added
test_to_dict_from_dict_roundtrip_with_meta_key_named_meta. It fails on the pre-fix source with theTypeErrorabove and passes with the fix. The fulltest/dataclasses/test_document.pysuite passes (30 tests).