Skip to content

Per-catalog auth state is shared: commit_table mutates the session, and cached S3FileSystem instances share a signer #3970

Description

@sungwy

Two places where authentication state configured for one catalog or table persists onto another.

1. commit_table mutates the shared session's headers

headers = self._session.headers            # the live session mapping, not a copy
if table_token := table.config.get(TOKEN):
    headers[AUTHORIZATION_HEADER] = f"{BEARER_PREFIX} {table_token}"

self._session.headers is the session's own mapping, so assigning into it persists the table-scoped token on the session. Every subsequent request from that RestCatalog carries it, including requests for other tables.

2. fsspec-cached S3FileSystem instances share one signer registration

fs = S3FileSystem(**s3_fs_kwargs)
for event_name, event_function in register_events.items():
    fs.s3.meta.events.unregister(event_name, unique_id=1925)
    fs.s3.meta.events.register_last(event_name, event_function, unique_id=1925)

The signer is registered as a botocore event handler after construction, under a fixed unique_id. It is not part of s3_fs_kwargs, and fsspec caches filesystem instances by constructor arguments (skip_instance_cache is not set). Two catalogs whose anon / client_kwargs / config_kwargs match therefore receive the same S3FileSystem object, and the second unregister + register_last replaces the first catalog's signer with its own.

Note the per-thread lru_cache in get_fs is not involved — _thread_locals is an instance attribute, so that cache is already per-FileIO. The sharing comes from fsspec's instance cache.


Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions