Avoid the deprecated generic timedelta unit in timedelta conversion - #2826
Open
sujeito-operator wants to merge 1 commit into
Open
Conversation
sum() starts from the integer 0, so the first addition in maybe_convert_python_timedelta_to_numpy is `0 + np.timedelta64(...)` -- numpy's deprecated 'generic' timedelta unit. Raised from inside parcels.*, the "error:::parcels.*" filter escalates it and the except clause reports it as `Could not convert <timedelta> to np.timedelta64`. functools.reduce(operator.add, dts) adds the parts to each other with no integer start value, and reproduces the existing result units exactly -- a typed start for sum() would promote timedelta64[D] to the start's unit. Turns three existing tests in tests/utils/test_time.py green on numpy 2.5.2. Reported in Parcels-code#2824.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Three tests in
tests/utils/test_time.pyfail onmaintoday, before any change, onnumpy 2.5.2 / Python 3.12.3:
The cause is one line of
src/parcels/_core/utils/time.py:sum()starts from the integer0, so the first addition is0 + np.timedelta64(...)—numpy's deprecated 'generic' timedelta unit, the same deprecation as #2824. Because
it is raised from inside a
parcels.*module, the"error:::parcels.*"rule inpyproject.tomlturns it into an error, theexcept Exceptionaround it catches that,and the caller sees:
which blames the input for a bug in the conversion.
CI on
mainis green today only because it has not resolved to numpy 2.5.2 yet.pyproject.tomlpinsnumpy >=2.1.0with no upper bound, so it will.The fix
functools.reduce(operator.add, dts)— the parts are added to each other, with nointeger start value. I did not use
sum(dts, start=...), because a typed start changesthe result's unit:
sum(dts, start=np.timedelta64(0, "us"))turnstimedelta(days=5)into
timedelta64[us]where the current code returnstimedelta64[D], and neartimedelta.maxthat promotion overflowsint64.reducereproduces the existing unitsexactly.
functoolsandoperatorare both already used elsewhere in the package(
_decorators.py,_core/particle.py).What I ran
pytest tests/utils/test_time.pyonmainatc603cc3, unpatched: 3 failed, 22 passed.-W error::DeprecationWarning: 25 passed — so the deprecation is gone,not merely demoted back to a warning.
2.5.2 made it raise:
days=5→numpy.timedelta64(5,'D'),days=5,seconds=30→numpy.timedelta64(432030,'s'),seconds=-2→numpy.timedelta64(-2,'s'),microseconds=1→numpy.timedelta64(1,'us'),days=999999999→numpy.timedelta64(999999999,'D'), andnumpy.timedelta64input returned unchanged.pytest tests/, the whole suite, unpatched onmainatc603cc3:4 failed, 623 passed, 95 skipped, 10 xfailed, 2 warnings in 449.27s.pytest tests/with this change:1 failed, 626 passed, 95 skipped, 10 xfailed, 2 warnings in 410.09s— the three above and nothing else.ruff format --checkandruff checkclean on the changed file.The one failure that survives is not mine and is not fixed here:
tests/test_fieldset.py::test_fieldset_describe_backendsis anImportErrorfor an optional backend I do not have installed.The 2 warnings are the same in both runs, and they are worth a sentence. They are
tests/test_particlefile.py::test_subsecond_outputdt[100]and[200]— the same numpydeprecation, from a test rather than from
src/, which is what #2824 is for. Theystay warnings because pytest attributes them to
numpy/_core/numeric.py, so"error:::parcels.*"does not match them. This one it does match, which is the wholedifference between a warning and three red tests.
What this does not do, which is the part I would not want you to miss
maybe_convert_python_timedelta_to_numpyhas no caller insrc/. It is defined insrc/parcels/_core/utils/time.pyand imported in exactly one other place —tests/utils/test_time.py, its own test. So this is nota user-visible bug today. It is a broken helper, and a build scheduled to go red on a
numpy release you have not pinned against.
That also means repair may not be what you want here. If the helper is left over from the
v4 rewrite and nothing is going to call it, deleting it and its test is the smaller tree,
and this PR should be closed in favour of that. I have no way to tell which from outside,
so I have fixed it rather than removed it — removing something on the assumption it is
dead is the more expensive mistake to be wrong about.
No test is added: the three tests above are yours, they already cover this exactly, and
they are currently red. Nothing outside
src/parcels/_core/utils/time.pyis touched.Checklist
not fix it
mainfor normal development,v3-supportfor v3 support)AI Disclosure
wrote all of the above. It built a venv from source, ran
pytest tests/utils/test_time.pyonmainto see the three failures, traced them to
sum()'s integer start value, made thechange, re-ran, and controlled it with
-W error::DeprecationWarning. It comparedthe returned unit for each branch of the helper against the pre-deprecation
behaviour, and it grepped the tree for callers before writing the paragraph above
saying there are none.