Skip to content

[python] Apply query authorization to the Daft blob-table native read - #10026

Merged
JingsongLi merged 1 commit into
apache:masterfrom
plusplusjiajia:fix/pypaimon-daft-blob-auth
Sep 21, 2026
Merged

JingsongLi merged 1 commit into
apache:masterfrom
plusplusjiajia:fix/pypaimon-daft-blob-auth

Conversation

@plusplusjiajia

Copy link
Copy Markdown
Member

Problem

A blob table never reaches Daft's native parquet reader through use_native_reader: _reader_routing rules it out on _has_blob_columns before it ever looks at has_auth, so that refusal is dead code for such a table. It reaches the native reader through _blob_table_native_files, which hands Daft the covering parquet files when no BLOB column is projected, and that path never looked at query authorization.

A blob table carrying a column mask or a row filter, read with a scalar-only projection, therefore returned raw parquet: both live in pypaimon's reader, which was skipped entirely.

Change

_blob_table_native_files takes has_auth and returns None for it, the way it already does for deletion vectors. The parameter has no default, so a new call site has to answer for it rather than silently bypassing the check. Both existing call sites, task generation and explain, pass it.

Keeping an authorized split off the native reader exposed a second defect. A count or a constant projection pushes columns=[], and the fallback task cannot build a Daft record batch from zero arrays. That already broke such a query on an ordinary authorized table; the blob table only avoided it by bypassing authorization. The fallback now carries the row count in a placeholder column and projects it away.

@plusplusjiajia
plusplusjiajia force-pushed the fix/pypaimon-daft-blob-auth branch 2 times, most recently from d0c6232 to 322e04d Compare September 20, 2026 17:32
@plusplusjiajia
plusplusjiajia force-pushed the fix/pypaimon-daft-blob-auth branch from 322e04d to f10140c Compare September 20, 2026 17:35

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requirement fit: SUPPORTED (triage: GO)\nImplementation: CLEAN\n\nThis has direct security value: scalar-only BLOB reads can no longer route authorized queries through Daft native parquet and bypass row filters or masks. I reviewed both native-file call sites, the PyPaimon fallback, zero-column row-count preservation, and the authorization tests and found no actionable P0/P1/P2 issue.

@JingsongLi
JingsongLi merged commit e2949f8 into apache:master Sep 21, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants