Skip to content

A source module says where the publisher path and asset routes fetch from - #1380

Draft
jwrosewell wants to merge 5 commits into
feature/38-middleware-settingsfrom
feature/39-source-modules
Draft

jwrosewell wants to merge 5 commits into
feature/38-middleware-settingsfrom
feature/39-source-modules

Conversation

@jwrosewell

Copy link
Copy Markdown
Collaborator

Summary

Changes

Area Change
crates/trusted-server-core/src 17 files, 2603 lines added, 916 removed
docs/guide 6 files, 368 lines added, 185 removed
docs/superpowers 1 file, 87 lines added, 0 removed
crates/trusted-server-adapter-fastly/src 5 files, 69 lines added, 23 removed
crates/trusted-server-adapter-cloudflare/src 2 files, 36 lines added, 6 removed
trusted-server.example.toml 1 file, 34 lines added, 16 removed
crates/trusted-server-adapter-axum/tests 2 files, 24 lines added, 4 removed
crates/trusted-server-adapter-cloudflare/tests 1 file, 24 lines added, 4 removed
crates/trusted-server-adapter-spin/src 2 files, 24 lines added, 4 removed
crates/trusted-server-adapter-axum/src 2 files, 18 lines added, 3 removed
crates/trusted-server-adapter-spin/tests 1 file, 18 lines added, 3 removed
crates/trusted-server-integration-tests/tests 3 files, 13 lines added, 3 removed
crates/tag/google-tag-manager 1 file, 12 lines added, 2 removed
crates/testing/seam-probe 1 file, 12 lines added, 2 removed
CHANGELOG.md 1 file, 11 lines added, 0 removed
crates/trusted-server-integration-tests/src 1 file, 10 lines added, 2 removed
crates/trusted-server-cli/src 1 file, 7 lines added, 4 removed
crates/auction/prebid-server 1 file, 6 lines added, 1 removed
crates/auction/prebid 1 file, 6 lines added, 1 removed
crates/trusted-server-integration-tests/fixtures 1 file, 6 lines added, 1 removed
.github/actions 1 file, 3 lines added, 3 removed
scripts/template-cache-local-test.sh 1 file, 2 lines added, 2 removed
.github/workflows 1 file, 1 line added, 1 removed
crates/trusted-server-cli/tests 1 file, 1 line added, 1 removed
scripts/attestation-local-test.sh 1 file, 1 line added, 1 removed

Closes

Closes #1332

Test plan

CI: 16 of 19 checks green, the fork's CI on d17b184 (jwrosewell#34).

  • cargo test-fastly && cargo test-axum
  • cargo clippy-fastly && cargo clippy-axum
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: the full workflow set of this repository, run on the pull request named above

Checklist

  • Changes follow AGENTS.md conventions
  • No unwrap() in production code (clippy denies unwrap_used in CI)
  • Uses log macros as AGENTS.md asks (not println!)
  • New code has tests
  • No secrets or credentials committed

On top of the middleware settings table (#33). Issue: #1332.

What this adds

What changes.

A source module says where the publisher path and the asset routes fetch from. [source] selects the publisher's source by name, and each [[proxy.asset_routes]] entry selects its own in a source table of the same shape. Core supplies origin, an HTTP origin, and a module a build links offers another through with_source_module.

[publisher]
domain = "www.example.com"
cookie_domain = "example.com"

[source]
module = "origin"

[source.origin]
url = "https://origin.example.com"
host_header = "www.example.com"
directory_index = "index.html"
auth = { type = "s3_sigv4", region = "eu-west-1", bucket = "pages" }

How it works.

  • One answer per request. A source answers a SourceLocation for each request, being the address and folder, the Host to send, the signing and the index file. The publisher path and the asset routes fetch, cache, rewrite the origin host and follow the redirect rule from that answer, so a source that answers a different origin for each request is kept apart everywhere.
  • [publisher] keeps who the publisher is. The origin fields leave it for [source.origin], where host_header, directory_index and auth are optional. A document writing one where it was is refused naming the new place, and a document with no [source] is refused with the shape to write.
  • An asset route says where its content comes from the same way. [proxy.asset_routes.source] and the source's own table replace a route's own origin address and signing. A path in a route's url names a folder the target path is added under, a target path with a dot segment under a folder is answered 400, and a route applies directory_index as the publisher does.
  • A module sees the prefix request addressing removed. Request addressing (Rebuild when environment variable change #30) takes a prefix off the path and keeps it as a request extension, and a module read the continuing path only, so a module naming things after the tenant a path addressed could not tell which one it was. ModuleRequest::of now reads the removed prefix, removed_prefix() gives it, and a held request keeps it.
  • Nothing stored changes. Stored page and asset keys are unchanged for an existing deployment. The configuration page masks the origin's url and host header by default, and the signing secrets are declared at their new paths.

Why

Where content comes from was written twice, in [publisher] and in every asset route, and fetched by code that knew one kind of source. A deployment that wants a page from assets built into its binary, from a store its platform offers, or from a service of its own had no place for that code but core. One interface core declares, with the origin fetch as the source core supplies, gives each of those a crate of its own and keeps one table for who the publisher is and one for where the content comes from.

What breaks

A [publisher] table carrying origin_url, origin_host_header_override, origin_auth, directory_index or the origin query rule is refused at startup, with the [source] shape to write instead, and an asset route carrying its own origin address or signing is refused the same way. Every deployment's settings document changes shape, so the build and the documents go out together. The integration test environment and the attestation script set TRUSTED_SERVER__SOURCE__ORIGIN__URL in place of the previous variable, because the overlay ignores a variable for a key the document lacks and the old name would silently leave a build on the placeholder origin.

The commits

  1. A source module says where the publisher path and asset routes fetch from.
  2. A module sees the prefix request addressing removed.

The design page is docs/superpowers/specs/2026-10-14-source-modules-design.md, and the same page goes to the design documents pull request (#1084) so every design page stays in one place.

What is not in this pull request

A source other than the HTTP origin. The interface and the selection are here, a source that answers another address is a module of its own, and a source that holds its content itself, with no network origin, is a later extension of the interface.

How it was verified

On the series' head: core (2,968 tests), the Fastly adapter under Viceroy on wasm32 (277), the Axum, Cloudflare and Spin adapters, the Prebid, Prebid Server, Google Tag Manager, seam probe and modules crates (19 suites), a native workspace check with all targets, and clippy for Fastly, Axum, Cloudflare and Spin native with -D warnings, all clean. The ts command's suites ran on the previous base, where the CLI is unchanged. CI on the fork is the verdict.

…from

Where content comes from was written twice, in [publisher] and in every
asset route, and fetched by code that knew one kind of source. A source
module now answers it. [source] selects the publisher's, and each
[[proxy.asset_routes]] entry selects its own in a source table of the
same shape. Core supplies origin, an HTTP origin, and a module a build
links offers another through with_source_module.

For each request a source answers a SourceLocation, being the address
and folder, the Host to send, the signing and the index file. The
publisher path and the asset routes fetch, cache, rewrite the origin
host and follow the redirect rule from that answer, so a source that
answers a different origin for each request is kept apart everywhere.

The origin fields move into [source.origin] and
[proxy.asset_routes.source.origin]. A document writing one where it was
is refused naming the new place, and a document with no [source] is
refused. A path in an asset route's url names a folder the target path
is added under, a target path with a dot segment under a folder is
answered 400, and a route applies directory_index as the publisher does.
Stored page and asset keys are unchanged for an existing deployment. The
configuration page masks the origin's url and host_header by default,
and the signing secrets are declared at their new paths.

Design: docs/superpowers/specs/2026-10-14-source-modules-design.md
Request addressing takes a prefix off the path and keeps it as a request
extension, so a source can say where content comes from. A module was
handed the continuing path only, so a module that names things after the
tenant a path addressed, such as a prefix module naming cookies after the
publisher, could not tell which one it was.

ModuleRequest::of now reads the removed prefix and removed_prefix() gives
it, with_removed_prefix sets it, and ResolvedRequest keeps it, so a held
request still has it. A request addressing nothing carries none.
A route reads its request through its module call, and the call carried the
request as resolved, the settings and the request state, but no evidence. A
route that asks for the request as evidence, the client address and headers a
module reads, was refused with "there is no request evidence where this
module runs", so every such route answered 500 in production while its own
tests, which build the context by hand, passed.

handle_proxy now takes the evidence before the route takes the request, in
the form the permission path reads it: the normalized client address, the
headers after the caller's EC header is removed, and the address asked for.
The test goes through handle_proxy, so it fails if the evidence is dropped
again.

(cherry picked from commit efad1ece83c3ec214e7d60c3e0754fb1053c5b41)
A document without [source] is refused since the source series, and the
test built its registry from default settings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant