Fix DAG processor treating non-.zip files as DAG bundles (#71125) - #71166
Open
bujjibabukatta wants to merge 1 commit into
Open
Fix DAG processor treating non-.zip files as DAG bundles (#71125)#71166bujjibabukatta wants to merge 1 commit into
bujjibabukatta wants to merge 1 commit into
Conversation
bujjibabukatta
requested review from
ephraimbuddy and
jedcunningham
as code owners
August 5, 2026 09:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DAG processor treats any zip-format file (.jar, .pptx, .docx, .xlsx, etc.) as a potential DAG bundle, not just .zip
Problem
find_dag_file_paths(airflow-core/src/airflow/utils/file.py) decided whether to attemptDAG discovery on a file using
zipfile.is_zipfile()alone — a content sniff for the PK zipmagic bytes, not an extension check. The same unguarded pattern existed in
PythonDagImporter.list_dag_files(airflow-core/src/airflow/dag_processing/importers/python_importer.py),the newer importer-based discovery path used by
DagBag.Since the zip container format underlies many common file types beyond
.zipitself —.jar,.pptx,.docx,.xlsx,.apk,.epub,.odt,.whl, etc. — any of thesedropped into a DAGs folder (a build artifact, a supporting doc, a packaged dependency) would
pass this check and get opened and scanned via
might_contain_dag, purely because it sharesthe underlying container format with Airflow's own zipped-DAG-bundle feature. At minimum this
is wasted work on every DAG processor cycle; depending on the archive's contents it can also
produce confusing log noise.
This was reported previously in #45718 with a
.pptxfile, but that issue was closed asinvalid because the specific symptom reported there turned out to be an unrelated bug. The
underlying zip-detection design issue itself was never fixed.
Fix
Gate both zip-bundle branches on
path.suffix == ".zip"in addition to the existing contentsniff, so only files actually named
.zipare treated as DAG zip bundles:airflow-core/src/airflow/utils/file.py—find_dag_file_pathsairflow-core/src/airflow/dag_processing/importers/python_importer.py—PythonDagImporter.list_dag_filesI traced every other
zipfile.is_zipfile()call site in the DAG processing code(
dag_processing/manager.py's_get_observed_filelocs,python_importer.py'simport_file,and
utils/file.py'scorrect_maybe_zipped/open_maybe_zipped) and confirmed each is onlyever reached after this extension check upstream (or is already gated by a
.zip-anchoredregex), so no other locations required changes.
Tests
Added a regression test for each fixed code path, each building a real zip-format file named
.jar(viazipfile.ZipFile, so it genuinely sniffs as a zip) alongside a.pyfile and acorrectly-named
.zipbundle, and asserting the.jaris excluded while the other two arestill discovered:
airflow-core/tests/unit/utils/test_file.py::TestListPyFilesPath::test_list_py_file_paths_ignores_non_zip_zip_format_filesairflow-core/tests/unit/dag_processing/importers/test_python_importer.py::TestPythonDagImporterListDagFiles::test_list_dag_files_ignores_non_zip_zip_format_files(new file)Also added
airflow-core/newsfragments/71125.bugfix.rst.closes: #71125
Was generative AI tooling used to co-author this PR?
Generated-by: Claude (Anthropic), used to draft the code fix, the two regression test cases,
this PR description, and the newsfragment, following the guidelines.
All generated code and tests were reviewed and verified by me before submission.