Fix LazyString corruption on copy, deepcopy and pickle under Python 3.11+ - #131
Open
DerevenetsArtyom wants to merge 1 commit into
Open
Fix LazyString corruption on copy, deepcopy and pickle under Python 3.11+#131DerevenetsArtyom wants to merge 1 commit into
DerevenetsArtyom wants to merge 1 commit into
Conversation
….11+ Python 3.11 added `object.__getstate__`, so `__getstate__` now appears in `dir(str)` and `lazy_str_meta` wraps it like any other `str` method. The wrapper reports the evaluated text as the object's serialization state, so `copy.copy`, `copy.deepcopy` and `pickle` each reconstruct a `LazyString` whose `_func` attribute is a plain string, and evaluating that copy raises `TypeError: 'str' object is not callable`. The exclusion set already protects the rest of the serialization protocol (`__reduce__`, `__reduce_ex__`, `__getnewargs__`), but it predates 3.11 and so could not list `__getstate__`. Adding it restores the inherited `object.__getstate__`. `__getstate__` is the only name added to `dir(str)` between 3.10 and 3.12, so this single entry covers the regression. This is straightforward to hit through Django: forms deepcopy their fields on every instantiation and `ChoiceField` deepcopies its choices, so a lazy string used as a choice label is corrupted on every request that builds such a form.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #130.
Python 3.11 added
object.__getstate__, so__getstate__now appears indir(str)andlazy_str_metawraps it like any otherstrmethod. The wrapper reports the evaluated text as the object's serialization state, socopy.copy,copy.deepcopyandpickleeach reconstruct aLazyStringwhose_funcattribute is a plain string, and evaluating that copy raisesTypeError: 'str' object is not callable.The exclusion set already protects the rest of the serialization protocol (
__reduce__,__reduce_ex__,__getnewargs__), but it predates 3.11 and so could not list__getstate__. Adding it restores the inheritedobject.__getstate__.__getstate__is the only name added todir(str)between 3.10 and 3.12, so this single entry covers the regression rather than being the first of several.Test
tests/common/test_strings.py::TestLazyString::test_copy_deepcopy_and_picklecovers all three operations. It fails without the one-line change, with the sameTypeError: 'str' object is not callableraised fromstrings.py, and passes with it.Ran locally using the CI images:
PYTHON_VERSION=3.12 DJANGO_VERSION=4.2— 263 passedPYTHON_VERSION=3.9 DJANGO_VERSION=3.2— 263 passedNote on
make code_qualityflake8reportstransifex/common/strings.py:137:24: B023 Function definition does not bind loop variable 'func'on this branch. That warning is pre-existing ondevel— it appears at line 136 there, and this change shifts it by one line — and it refers to the intentional double-wrapping inlazy_str_meta. Becausecode_qualitylints only changed files, touching this file brings it into scope.I left it untouched to keep the PR to a single concern, but I am happy to add a
# noqa: B023if you would rather see the check green.