Skip to content

The publisher origin can be a folder of signed object storage - #1377

Draft
jwrosewell wants to merge 14 commits into
feature/35-request-addressingfrom
feature/36-object-storage
Draft

jwrosewell wants to merge 14 commits into
feature/35-request-addressingfrom
feature/36-object-storage

Conversation

@jwrosewell

Copy link
Copy Markdown
Collaborator

Summary

Changes

Area Change
crates/trusted-server-core/src 8 files, 1399 lines added, 84 removed
docs/superpowers 1 file, 107 lines added, 0 removed
docs/guide 1 file, 85 lines added, 1 removed
trusted-server.example.toml 1 file, 17 lines added, 1 removed
CHANGELOG.md 1 file, 1 line added, 0 removed

Closes

Closes #1329

Test plan

CI: 19 of 19 checks green, the fork's CI on f5f8ac2 (jwrosewell#31).

  • cargo test-fastly && cargo test-axum
  • cargo clippy-fastly && cargo clippy-axum
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: the full workflow set of this repository, run on the pull request named above

Checklist

  • Changes follow AGENTS.md conventions
  • No unwrap() in production code (clippy denies unwrap_used in CI)
  • Uses log macros as AGENTS.md asks (not println!)
  • New code has tests
  • No secrets or credentials committed

On top of a deployment addressing a request by its path (#30). Issue: #1329.

What this adds

What changes.

A publisher whose site is static files in object storage is served by the ordinary publisher proxy, so its pages pass through the fetch and serve middleware, the template cache and the making of a missing file, all of which an asset route skips. Three optional [publisher] settings do it, each off unless set.

[publisher]
domain = "publisher.example"
cookie_domain = ".publisher.example"
origin_url = "https://storage.example/bucket/site"
proxy_secret = "publisher_proxy_secret"
directory_index = "index.html"

[publisher.origin_auth]
type = "s3_sigv4"
region = "example-region-1"
access_key_id = "publisher_origin_access_key_id"
secret_access_key = "publisher_origin_secret_access_key"
A request for Is sent to the origin as
/ /bucket/site/index.html, signed
/news/ /bucket/site/news/index.html, signed
/news /bucket/site/news, signed
/style.css?v=2 /bucket/site/style.css?v=2, signed
/news/../../other-site/page Not sent. Answered 400
A POST, or any method but GET and HEAD Not sent. Answered 405 with Allow: GET, HEAD

How it works.

  • The origin path. A path in origin_url is kept and the request path is added after it with exactly one /, whichever side carries one. The query is kept. An origin_url with no path sends the request path as it arrived, which is what every existing document does. The address is built in one place, so the streamed page, the buffered page, the template cache key, a HEAD and every other method take the same one.
  • Signing. origin_auth is the table an asset route's auth takes, read as the same type with the same fields, defaults and secret store key references, and it signs through the same function, so there is one signing path. The request is signed last, once its address and Host are final. Only GET and HEAD are signed and any other method is answered 405, so the publisher's credentials are only used to read. A signed request carries none of the reader's Cookie, Authorization, Proxy-Authorization, Referer, conditional validators or Range, and none of the signing scheme's own headers. origin_url must use https when origin_auth is set.
  • No request reaches outside the folder. With a path in origin_url, a request path with a . or .. segment, as written or after up to four rounds of percent-decoding, is answered 400 and nothing is sent.
  • The index file. directory_index names one file, added to a request path ending in / for the origin request only. The reader's address and the path the middleware see do not change.
  • Refused when the settings load, with the setting named. A malformed origin_auth, an origin_auth beside an origin_url that is not https, an empty key reference (by ts config validate and ts config push), and a directory_index that is not one file name.
  • The credentials are proven masked in the configuration view, never served by it, and kept out of debug output.

Why

The page pipeline, the template cache, the making of a file the origin does not have and the cache policy for a path all hang off the publisher path. A second path for a site kept in storage would carry each of them again, and a page would behave differently according to where its bytes are kept. Each of the three settings changes the request core already makes to the one origin [publisher] names, none is a vendor's, and the signer is the one core already has for asset routes.

What breaks

Nothing for a document that sets none of the three. It sends the origin request it sent before and serializes to the same bytes. A document that sets one is refused by a build that predates them, because [publisher] refuses a field it does not know, so the build is deployed before the document is pushed. Publisher gains two public fields, origin_auth and directory_index, so Rust code outside the crate that builds Publisher with a struct literal stops compiling until it adds them or uses ..Default::default().

The commits

Commit What it does
a53a456 The three settings, the origin path, the signing and the index file, with the guide
c1aed90 The design page, and the credentials proven masked
e8280f1 The merge of #30, after the merges of #28 as its head moved
7a23749 Shorter docs on origin_auth, s3_sigv4 and READER_HEADERS_NOT_SIGNED
359af06 The dot-segment check and the closed-path check share one decode loop
bc0c390 The masked secret fields are listed once
ddec562 The signed-origin tests share one assertion

CI being read on f5f8ac2.

What is not in this pull request

  • Validators and ranges on a signed origin. A browser revalidating a file through a signed publisher origin is sent the file again, and a range request is answered with the whole file. An asset route forwards both and stays the place for large media.
  • A missing object read as a missing file. Some object storage answers a missing object with 403, and with 404 only where the credentials may also list the bucket. No storage service was called in testing this change.
  • Addresses in a page that name the storage host. An absolute address naming it is rewritten to the reader's host as before, and in an HTML document the folder stays in the path, because only the host is matched there.
  • A second signing scheme.

The design page is docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md.

jwrosewell and others added 14 commits October 13, 2026 22:06
A publisher whose site is static files in object storage had to be served
by an asset route, which skips the page pipeline. Three optional
[publisher] settings let the ordinary publisher proxy serve it, so its
pages pass through the fetch and serve middleware. Each is off unless set,
and a document without them sends the origin request it sent before and
serializes to the same bytes.

A path in origin_url is kept, and the request path is added after it with
one slash. With a path set, a request path with a dot segment in any
spelling is refused with 400, so no request reaches outside the folder.

origin_auth takes the table an asset route's auth takes and signs through
the same function. Only GET and HEAD are signed and any other method is
refused with 405, so the publisher's credentials are only used to read. A
signed request carries none of the reader's cookies, credentials, referrer,
conditional validators or range, and none of the signing namespace's
headers. Its key references resolve from the secret store as an asset
route's do.

directory_index names a file added to a request path ending in a slash,
for the origin request only.

A malformed origin_auth or directory_index is refused when the settings
load, with the setting named. The asset route refusals keep their wording.
…proven masked

The design page for the publisher origin in object storage follows the
shape of the stack's design documents, and the changelog and the guide's
list of static secret references name the new settings.

The configuration view's test of every declared secret now holds the
three key references of publisher.origin_auth, which it requires of any
declared secret, so they are proven masked in the view and absent from
what it serves. A further test keeps them out of debug output.

The comments on the signer and on the auth type say the publisher origin
uses them too.
The publisher origin check and the closed path check share one bounded percent-decoding helper. A closed path still decoding after the last round is now closed, as a dot segment already was.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants