diff --git a/CHANGELOG.md b/CHANGELOG.md index 101087b8d..ce3904fea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- `rewrite.exclude_domains` matching is now case-insensitive and ignores surrounding whitespace. Entries written with uppercase letters previously never matched and now take effect, so those hosts stop being proxied and click-wrapped, and `/first-party/sign` now rejects them with `502`; it used to sign them, or return `403` when the host was also outside `proxy.allowed_domains`. Empty and bare `"*"` `exclude_domains` entries, which never matched a host, are dropped at load with a warning. Absolute `http(s)` creative URLs that cannot be parsed (for example, a host containing a space) are now left untouched; previously the attribute was re-quoted and a link also gained `data-tsclick`. Audit `exclude_domains` for mixed-case entries before upgrading. - `[auction].allowed_context_keys` now serializes in sorted, deduplicated order, so ESI template-cache fingerprints and `ts config diff`/`push` envelope hashes are stable across loads. Template fingerprints also sort object keys independently of `serde_json/preserve_order`. Existing envelopes may show a one-time allowlist reorder after upgrading; push once to settle it. The updated fingerprint format causes one template-cache miss per cached page after deployment. - TSJS-generated envelopes now send `trustedServer.params.storedRequest: false`, preventing accidental PBS stored lookups without suppressing eligible non-PBS demand. PBS filters unusable impressions after overrides; explicit `true` and omission in valid envelopes retain inline-first stored fallback. A malformed envelope disables stored fallback for the entire slot, including independent direct demand left unusable after overrides. Publisher intent survives repeated and refresh auctions. Deploy compatible server admission everywhere before serving the new JS, and retain it during rollback while cached clients remain. See the Prebid deployment guide. - Protocol-relative creative URLs now honor `rewrite.exclude_domains`, so excluded creative assets stay direct and excluded absolute or protocol-relative URLs submitted to `/first-party/sign` are rejected. @@ -40,7 +41,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added -- Added the `[auction].rewrite_creatives` (default `true`) and `[auction].sanitize_creatives` (default `false`) options. `rewrite_creatives` rewrites winning-bid adm to first-party endpoints across `POST /auction` and publisher SSAT/page-bids delivery (proxy/click URL conversion, bidder `` removal; creative TSJS injection on `POST /auction` only). Enabling `sanitize_creatives` strips executable markup from winning-bid adm before delivery. +- Added `[auction].rewrite_clicks` to control creative click-through wrapping (``/`` → signed `/first-party/click`) independently of asset rewriting. Unset (the default) follows `rewrite_creatives` for `POST /auction` and SSAT/page-bids and keeps wrapping links in HTML fetched through `/first-party/proxy`, so existing configs behave as before; an explicit value applies to every path. `rewrite_creatives` now governs asset URLs only; bidder `` removal and creative TSJS injection run when either setting is on. Upgrading: deploy the binary first, then push a config that sets `rewrite_clicks`. Rolling back: remove any explicit `rewrite_clicks` (and its environment override), push the resulting config, then roll back the binary. Older binaries tie click wrapping to `rewrite_creatives` in both directions, so rolling back turns clicks back on for `rewrite_creatives = true` with `rewrite_clicks = false`, and off for `rewrite_creatives = false` with `rewrite_clicks = true`. +- Added the `[auction].rewrite_creatives` (default `true`) and `[auction].sanitize_creatives` (default `false`) options. `rewrite_creatives` rewrites winning-bid adm to first-party endpoints across `POST /auction` and publisher SSAT/page-bids delivery (asset URL conversion to `/first-party/proxy`, bidder `` removal; creative TSJS injection on `POST /auction` only). Click-through wrapping is controlled by `[auction].rewrite_clicks`, which follows `rewrite_creatives` when unset. Enabling `sanitize_creatives` strips executable markup from winning-bid adm before delivery. - `creative_opportunities.slot.gam_unit_path` is now a template supporting `{network_id}`, `{slot_id}`, and `{section}`, so a publisher whose ad unit varies by site section expresses it in one slot rule instead of one per (slot × section). `{section}` derives from the request path: `[creative_opportunities].section_segment` selects which path segment names the section (0-based, default `0`; set `1` for locale-prefixed URLs), and `section_root` supplies the value for paths with no such segment. `section_root` is required when a template uses `{section}`. Existing static and absent `gam_unit_path` configs are unchanged. Startup rejects a blank `gam_network_id` only when an absent/default path or `{network_id}` template consumes it. Trusted Server conservatively caps whole rendered dynamic paths at 100 UTF-8 bytes, informed by Google's 100-character per-ad-unit-code limit; an over-limit request-specific path omits that slot without failing the response. During typed/startup finalization, every placeholder-bearing template that omits `section_segment` materializes `section_segment = 0`, so an older binary rejects the blob loudly. Static and absent paths remain legacy-schema compatible only when both `section_root` and `section_segment` are omitted. Before rolling back below this feature, replace or remove dynamic paths, remove both keys, re-push and finalize the config, then roll back the binary. - Added opt-in APS HTTP debug metadata for controlled test sites, exposing the direct request and response under `/auction` provider metadata using the Prebid Server `debug.httpcalls` shape. - Added typed APS renderer transport for direct auctions and GAM/Prebid Universal Creative, using a minimized one-bid envelope, a fragment-bound nonce, and an opaque sandboxed renderer endpoint. diff --git a/crates/trusted-server-cli/tests/config_env_overlay.rs b/crates/trusted-server-cli/tests/config_env_overlay.rs index d81b0366a..06cc3c058 100644 --- a/crates/trusted-server-cli/tests/config_env_overlay.rs +++ b/crates/trusted-server-cli/tests/config_env_overlay.rs @@ -29,6 +29,7 @@ ids = ["trusted_server_secrets"] "#; const REWRITE_ENV: &str = "TRUSTED_SERVER__AUCTION__REWRITE_CREATIVES"; const SANITIZE_ENV: &str = "TRUSTED_SERVER__AUCTION__SANITIZE_CREATIVES"; +const CLICKS_ENV: &str = "TRUSTED_SERVER__AUCTION__REWRITE_CLICKS"; const GAM_ATTRIBUTION_ENV: &str = "TRUSTED_SERVER__INTEGRATIONS__GPT__GAM_ATTRIBUTION_ENABLED"; const AD_TEMPLATES_ENABLED_ENV: &str = "TRUSTED_SERVER__CREATIVE_OPPORTUNITIES__ENABLED"; const PROVIDER_ENDPOINT_ENV: &str = "TRUSTED_SERVER__AUCTION__PROVIDERS__PBS-MAIN__ENDPOINT"; @@ -82,6 +83,76 @@ fn validate_with_overlay(project: &MigratedProject, raw_value: &str) -> Output { .expect("should run ts config validate") } +fn pushed_auction_with_env( + project: &MigratedProject, + key: &str, + raw_value: &str, +) -> serde_json::Value { + let output = Command::new(env!("CARGO_BIN_EXE_ts")) + .args(["config", "push", "--adapter", "axum", "--manifest"]) + .arg(&project.manifest_path) + .arg("--app-config") + .arg(&project.config_path) + .args(["--yes", "--no-diff"]) + .current_dir(project.directory.path()) + .env(key, raw_value) + .output() + .expect("should run ts config push"); + assert!( + output.status.success(), + "config push should succeed: {}", + String::from_utf8_lossy(&output.stderr) + ); + + let local_store_path = project + .directory + .path() + .join(".edgezero/local-config-trusted_server_config.json"); + let local_store: serde_json::Value = serde_json::from_str( + &fs::read_to_string(local_store_path).expect("should read pushed local config"), + ) + .expect("should parse local config store"); + let envelope_json = local_store + .as_object() + .and_then(|entries| entries.values().next()) + .and_then(serde_json::Value::as_str) + .expect("should contain a blob envelope"); + let envelope: serde_json::Value = + serde_json::from_str(envelope_json).expect("should parse blob envelope"); + envelope["data"]["auction"].clone() +} + +#[test] +fn rewrite_clicks_environment_override_applies_when_leaf_present() { + let project = migrated_project(); + let mut document = fs::read_to_string(&project.config_path) + .expect("should read migrated config") + .parse::() + .expect("should parse migrated config"); + document["auction"]["rewrite_clicks"] = value(true); + fs::write(&project.config_path, document.to_string()).expect("should write config"); + + let auction = pushed_auction_with_env(&project, CLICKS_ENV, "false"); + + assert_eq!( + auction["rewrite_clicks"], + serde_json::Value::Bool(false), + "pushed config should contain the rewrite_clicks environment override" + ); +} + +#[test] +fn rewrite_clicks_environment_override_is_ignored_without_leaf() { + let project = migrated_project(); + + let auction = pushed_auction_with_env(&project, CLICKS_ENV, "false"); + + assert!( + auction.get("rewrite_clicks").is_none(), + "an override for a missing leaf should be ignored and the unset default omitted" + ); +} + #[test] fn config_validate_explains_legacy_provider_list_migration() { let project = migrated_project(); diff --git a/crates/trusted-server-core/src/auction/README.md b/crates/trusted-server-core/src/auction/README.md index 8a745eabc..7b0ae585b 100644 --- a/crates/trusted-server-core/src/auction/README.md +++ b/crates/trusted-server-core/src/auction/README.md @@ -108,8 +108,10 @@ as `pbs-main`. response. - `sanitize_creatives = true` strips executable markup. It is opt-in. -- `rewrite_creatives = true` rewrites eligible URLs through first-party routes - and removes bidder `` elements. It is enabled by default. +- `rewrite_creatives = true` rewrites eligible asset URLs through first-party + routes and removes bidder `` elements. It is enabled by default. +- `rewrite_clicks` wraps creative links in `/first-party/click`. Unset, it + follows `rewrite_creatives`. - The publisher inline delivery path uses absolute first-party URLs without injecting the direct endpoint's creative runtime. - Creatives over the configured hard cap are rejected. diff --git a/crates/trusted-server-core/src/auction/endpoints.rs b/crates/trusted-server-core/src/auction/endpoints.rs index ab3585e3d..10600e945 100644 --- a/crates/trusted-server-core/src/auction/endpoints.rs +++ b/crates/trusted-server-core/src/auction/endpoints.rs @@ -84,10 +84,14 @@ const MAX_AUCTION_BODY_SIZE: usize = 256 * 1024; /// ## Response /// /// Returns an `OpenRTB 2.x` response. Creative HTML is inlined in each bid's -/// `adm` field after mandatory server-side sanitization. First-party resource -/// and click URL rewriting plus creative TSJS injection are enabled by default; -/// setting [`auction.rewrite_creatives`][`crate::auction_config_types::AuctionConfig::rewrite_creatives`] -/// to `false` skips only that rewrite pass. +/// `adm` field after optional sanitization +/// ([`auction.sanitize_creatives`][`crate::auction_config_types::AuctionConfig::sanitize_creatives`]). +/// First-party asset rewriting +/// ([`auction.rewrite_creatives`][`crate::auction_config_types::AuctionConfig::rewrite_creatives`]) +/// and click wrapping +/// ([`auction.rewrite_clicks`][`crate::auction_config_types::AuctionConfig::rewrite_clicks`]) +/// are enabled by default. Bidder `` removal and creative TSJS injection +/// run when either is on. /// /// ## Scroll, refresh, and SPA navigation /// diff --git a/crates/trusted-server-core/src/auction/formats.rs b/crates/trusted-server-core/src/auction/formats.rs index a0df63d94..956e81e32 100644 --- a/crates/trusted-server-core/src/auction/formats.rs +++ b/crates/trusted-server-core/src/auction/formats.rs @@ -316,14 +316,16 @@ pub(crate) struct OpenRtbResponseConversion { /// /// Creative HTML in the `adm` field is optionally sanitized and optionally /// rewritten according to the auction configuration -/// ([`AuctionConfig::sanitize_creatives`], opt-in, and -/// [`AuctionConfig::rewrite_creatives`], default-on); with both disabled the -/// creative ships exactly as the bidder returned it, subject to the 1 MiB -/// per-creative cap. Typed renderers are serialized in the response extension +/// ([`AuctionConfig::sanitize_creatives`], opt-in; +/// [`AuctionConfig::rewrite_creatives`] for assets, default-on; and +/// [`AuctionConfig::rewrite_clicks`] for links, following `rewrite_creatives` +/// when unset); with all disabled the creative ships exactly as the bidder +/// returned it, subject to the 1 MiB per-creative cap. Typed renderers are serialized in the response extension /// instead of entering that pipeline at all. /// /// [`AuctionConfig::sanitize_creatives`]: crate::auction_config_types::AuctionConfig::sanitize_creatives /// [`AuctionConfig::rewrite_creatives`]: crate::auction_config_types::AuctionConfig::rewrite_creatives +/// [`AuctionConfig::rewrite_clicks`]: crate::auction_config_types::AuctionConfig::rewrite_clicks /// /// # Errors /// @@ -373,9 +375,10 @@ pub(crate) fn convert_to_openrtb_response_with_report( let height = to_openrtb_i32(bid.height, "height", &bid_context); // Ordinary markup goes through the configured creative processing: - // sanitization is opt-in, rewriting is on by default, and with both - // disabled the creative ships exactly as the bidder returned it. A typed - // renderer is serialized separately and never enters that pipeline. + // sanitization is opt-in, asset rewriting and click wrapping are on by + // default, and with all three disabled the creative ships exactly as + // the bidder returned it. A typed renderer is serialized separately and + // never enters that pipeline. let serialize_renderer = |renderer: &BidRenderer| { (BidExt { trusted_server: BidTrustedServerExt { renderer }, @@ -398,12 +401,13 @@ pub(crate) fn convert_to_openrtb_response_with_report( let processed = creative::process_auction_creative(settings, raw_creative); log::debug!( - "Processed creative for auction {} slot {} bidder {} (sanitize {}, rewrite {}, raw {} bytes, output {} bytes)", + "Processed creative for auction {} slot {} bidder {} (sanitize {}, rewrite {}, clicks {}, raw {} bytes, output {} bytes)", auction_request.id, slot_id, bid.bidder, settings.auction.sanitize_creatives, rewrite_creatives, + settings.auction.rewrites_auction_clicks(), raw_creative.len(), processed.len() ); @@ -1413,6 +1417,37 @@ mod tests { ); } + #[test] + fn convert_to_openrtb_response_wraps_clicks_without_rewriting_assets() { + let mut settings = make_settings(); + settings.auction.sanitize_creatives = false; + settings.auction.rewrite_creatives = false; + settings.auction.rewrite_clicks = Some(true); + let auction_request = make_auction_request(); + let result = make_result(make_complete_creative_bid()); + + let response = convert_to_openrtb_response(&result, &settings, &auction_request, false) + .expect("should convert creative with click rewriting only"); + let adm = response_adm(response); + + assert!( + adm.contains("/first-party/click?tsurl=") && adm.contains("data-tsclick"), + "should wrap the landing link: {adm}" + ); + assert!( + !adm.contains("/first-party/proxy?tsurl="), + "should not proxy any asset: {adm}" + ); + assert!( + adm.contains(r#"src="https://cdn.example.com/ad.png""#), + "should keep the image URL direct: {adm}" + ); + assert!( + adm.contains("tsjs-unified.min.js"), + "should inject the creative runtime for the click guard: {adm}" + ); + } + #[test] fn sanitize_creatives_defaults_to_disabled() { let config = crate::auction_config_types::AuctionConfig::default(); diff --git a/crates/trusted-server-core/src/auction/orchestrator.rs b/crates/trusted-server-core/src/auction/orchestrator.rs index 204201e60..3d2044e32 100644 --- a/crates/trusted-server-core/src/auction/orchestrator.rs +++ b/crates/trusted-server-core/src/auction/orchestrator.rs @@ -4696,6 +4696,7 @@ mod tests { enabled: true, sanitize_creatives: true, rewrite_creatives: true, + rewrite_clicks: None, providers: AuctionConfig::legacy_provider_map(&[]), bidders: Default::default(), mediator: None, diff --git a/crates/trusted-server-core/src/auction_config_types.rs b/crates/trusted-server-core/src/auction_config_types.rs index 56171e1ea..42aab07ef 100644 --- a/crates/trusted-server-core/src/auction_config_types.rs +++ b/crates/trusted-server-core/src/auction_config_types.rs @@ -36,8 +36,13 @@ pub struct AuctionConfig { )] pub sanitize_creatives: bool, - /// Rewrite winning-bid creative HTML to first-party endpoints (applied - /// after sanitization when [`Self::sanitize_creatives`] is enabled). + /// Rewrite winning-bid creative asset URLs (images, scripts, styles, + /// media, iframes, CSS `url()`) to first-party `/first-party/proxy` + /// endpoints, applied after sanitization when + /// [`Self::sanitize_creatives`] is enabled. + /// + /// Bidder `` removal and creative TSJS injection run whenever this + /// or click rewriting ([`Self::rewrites_auction_clicks`]) is on. /// /// The default stays omitted from serialized config blobs to avoid adding /// this field when it has no effect. Any rollback across schema versions @@ -49,6 +54,18 @@ pub struct AuctionConfig { )] pub rewrite_creatives: bool, + /// Wrap creative click-through links (``, ``) in signed + /// `/first-party/click` redirects. + /// + /// Unset keeps each path's existing behavior: auction creatives follow + /// [`Self::rewrite_creatives`], and HTML fetched through + /// `/first-party/proxy` keeps wrapping. An explicit value applies to every + /// path. Unset is omitted from serialized config blobs, so older binaries + /// keep loading them; any explicit value is serialized and rejected by + /// binaries that predate this field. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub rewrite_clicks: Option, + /// Operator-defined bidder-provider instances, keyed by provider ID. #[serde(default, deserialize_with = "deserialize_provider_map")] pub providers: BTreeMap, @@ -87,6 +104,7 @@ impl Default for AuctionConfig { enabled: false, sanitize_creatives: default_sanitize_creatives(), rewrite_creatives: default_rewrite_creatives(), + rewrite_clicks: None, providers: BTreeMap::new(), bidders: BTreeMap::new(), mediator: None, @@ -160,6 +178,25 @@ fn default_allowed_context_keys() -> BTreeSet { } impl AuctionConfig { + /// Whether auction creatives (`POST /auction` and inline SSAT/page-bids) + /// wrap click-through links. + /// + /// Unset [`Self::rewrite_clicks`] follows [`Self::rewrite_creatives`]. + #[must_use] + pub fn rewrites_auction_clicks(&self) -> bool { + self.rewrite_clicks.unwrap_or(self.rewrite_creatives) + } + + /// Whether HTML fetched through `/first-party/proxy` wraps click-through + /// links. + /// + /// Unset [`Self::rewrite_clicks`] keeps wrapping, as before the setting + /// existed. + #[must_use] + pub fn rewrites_proxied_clicks(&self) -> bool { + self.rewrite_clicks.unwrap_or(true) + } + #[cfg(test)] pub(crate) fn legacy_provider_map(names: &[&str]) -> BTreeMap { names @@ -255,6 +292,72 @@ mod tests { ); } + #[test] + fn default_rewrite_clicks_is_unset_and_not_serialized() { + let config = AuctionConfig::default(); + + assert_eq!( + config.rewrite_clicks, None, + "should leave click rewriting unset by default" + ); + let serialized = serde_json::to_value(&config).expect("should serialize defaults"); + assert!( + serialized.get("rewrite_clicks").is_none(), + "should omit the unset click setting from serialized config" + ); + } + + #[test] + fn explicit_rewrite_clicks_is_serialized() { + for value in [true, false] { + let config = AuctionConfig { + rewrite_clicks: Some(value), + ..AuctionConfig::default() + }; + + let serialized = + serde_json::to_value(config).expect("should serialize explicit click setting"); + + assert_eq!( + serialized.get("rewrite_clicks"), + Some(&serde_json::Value::Bool(value)), + "should serialize explicit rewrite_clicks = {value}" + ); + } + } + + #[test] + fn click_rewriting_resolves_per_entry_point() { + // (rewrite_creatives, rewrite_clicks, auction clicks, proxied clicks) + let cases = [ + (true, None, true, true), + (false, None, false, true), + (true, Some(false), false, false), + (false, Some(true), true, true), + (true, Some(true), true, true), + (false, Some(false), false, false), + ]; + + for (rewrite_creatives, rewrite_clicks, auction, proxied) in cases { + let config = AuctionConfig { + rewrite_creatives, + rewrite_clicks, + ..AuctionConfig::default() + }; + + assert_eq!( + config.rewrites_auction_clicks(), + auction, + "auction clicks for rewrite_creatives={rewrite_creatives} rewrite_clicks={rewrite_clicks:?}" + ); + assert_eq!( + config.rewrites_proxied_clicks(), + proxied, + "proxied clicks for rewrite_creatives={rewrite_creatives} rewrite_clicks={rewrite_clicks:?}" + ); + } + } + #[test] fn default_sanitize_creatives_is_not_serialized() { let serialized = diff --git a/crates/trusted-server-core/src/config_payload.rs b/crates/trusted-server-core/src/config_payload.rs index c9690b01f..eee5d7ee8 100644 --- a/crates/trusted-server-core/src/config_payload.rs +++ b/crates/trusted-server-core/src/config_payload.rs @@ -749,6 +749,50 @@ mod tests { ); } + #[test] + fn legacy_blob_without_rewrite_clicks_follows_rewrite_creatives() { + for rewrite_creatives in [true, false] { + let mut original = test_settings(); + original.auction.rewrite_creatives = rewrite_creatives; + let data = serde_json::to_value(&original).expect("should serialize settings to JSON"); + assert!( + data["auction"].get("rewrite_clicks").is_none(), + "should omit unset rewrite_clicks from the payload" + ); + + let reconstructed = load_settings(&envelope_json(&original)) + .expect("should reconstruct settings without rewrite_clicks"); + + assert_eq!( + reconstructed.auction.rewrite_clicks, None, + "should load a blob without rewrite_clicks as unset" + ); + assert_eq!( + reconstructed.auction.rewrites_auction_clicks(), + rewrite_creatives, + "should follow rewrite_creatives = {rewrite_creatives} when unset" + ); + } + } + + #[test] + fn explicit_rewrite_clicks_survives_blob_round_trip() { + for (rewrite_creatives, rewrite_clicks) in [(true, false), (false, true)] { + let mut original = test_settings(); + original.auction.rewrite_creatives = rewrite_creatives; + original.auction.rewrite_clicks = Some(rewrite_clicks); + + let reconstructed = load_settings(&envelope_json(&original)) + .expect("should reconstruct explicit rewrite_clicks"); + + assert_eq!( + reconstructed.auction.rewrite_clicks, + Some(rewrite_clicks), + "should preserve rewrite_clicks = {rewrite_clicks} with rewrite_creatives = {rewrite_creatives}" + ); + } + } + #[test] fn strings_that_look_like_json_scalars_round_trip_as_strings() { let mut original = test_settings(); diff --git a/crates/trusted-server-core/src/creative.rs b/crates/trusted-server-core/src/creative.rs index 578d86d68..074e5f35d 100644 --- a/crates/trusted-server-core/src/creative.rs +++ b/crates/trusted-server-core/src/creative.rs @@ -3,10 +3,11 @@ //! Goals: //! - Normalize external asset fetches in ad creatives (HTML/CSS) to a single //! first-party proxy endpoint so the publisher can control egress. +//! - Route click-through links through a signed first-party click redirect. //! - Leave relative URLs and non-network schemes untouched. //! //! Key behaviors: -//! - Absolute and protocol-relative URLs (http/https or `//`) are proxied to +//! - Absolute and protocol-relative asset URLs (http/https or `//`) are proxied to //! `/first-party/proxy?tsurl=&&tstoken=` across these locations: //! - ``, `data-src`, `[srcset]`, `[imagesrcset]` //! - `
AdExcludedMail"#; + const MATRIX_DEFAULT_INLINE_OUTPUT: &str = r#"
AdExcludedMail"#; + + #[test] + fn default_settings_rewrite_matrix_fixture_byte_for_byte_as_before() { + let settings = matrix_settings(true, None); + + for (path, expected) in [ + (CreativePath::Auction, MATRIX_DEFAULT_AUCTION_OUTPUT), + (CreativePath::Inline, MATRIX_DEFAULT_INLINE_OUTPUT), + ] { + assert_eq!( + process_for_path(&settings, path, MATRIX_FIXTURE_WITH_BODY), + expected, + "{path:?}: default settings should match the pre-rewrite_clicks output exactly" + ); + } + } + + fn normalized(raw: &str) -> Option { + normalize_creative_url(raw).map(|url| url.as_str().to_owned()) + } + + #[test] + fn normalize_creative_url_conversions() { + for (raw, expected) in [ + ("//cdn.example/x", Some("https://cdn.example/x")), + ("HTTPS://cdn.example/x", Some("https://cdn.example/x")), + ("http://cdn.example/x", Some("http://cdn.example/x")), + (" //cdn.example/y ", Some("https://cdn.example/y")), + (" https://cdn.example/a ", Some("https://cdn.example/a")), + ( + "//cdn.example.com:8080/asset.js", + Some("https://cdn.example.com:8080/asset.js"), + ), + ( + "//cdn.example.com:9443/img.png", + Some("https://cdn.example.com:9443/img.png"), + ), + ("/local/x", None), + ("", None), + ("data:image/png;base64,abcd", None), + ("javascript:alert(1)", None), + ("mailto:test@example.com", None), + ("blob:xyz", None), + ("tel:+123", None), + ("about:blank", None), + ("https://exa mple.example/x", None), + ] { + assert_eq!( + normalized(raw).as_deref(), + expected, + "should normalize `{raw}` to {expected:?}" + ); + } } #[test] @@ -3416,18 +3697,6 @@ b{background:url(\"https://cdn.example/c.png\")}"; } } - #[test] - fn to_abs_additional_cases() { - let settings = crate::test_support::tests::create_test_settings(); - assert_eq!( - to_abs(&settings, " https://cdn.example/a "), - Some("https://cdn.example/a".to_owned()) - ); - assert_eq!(to_abs(&settings, "blob:xyz"), None); - assert_eq!(to_abs(&settings, "tel:+123"), None); - assert_eq!(to_abs(&settings, "about:blank"), None); - } - #[test] fn rewrites_lazy_img_data_src_and_data_srcset() { let settings = crate::test_support::tests::create_test_settings(); @@ -3443,62 +3712,86 @@ b{background:url(\"https://cdn.example/c.png\")}"; } #[test] - fn to_abs_respects_exclude_domains() { + fn proxy_if_abs_respects_exclude_domains() { let mut settings = crate::test_support::tests::create_test_settings(); - settings.rewrite.exclude_domains = vec!["trusted-cdn.example.com".to_owned()]; + settings.rewrite.exclude_domains = vec![ + "trusted-cdn.example.com".to_owned(), + "*.example.org".to_owned(), + ]; - // Excluded domain should return None (not proxied) - assert_eq!( - to_abs(&settings, "https://trusted-cdn.example.com/lib.js"), - None - ); + for excluded in [ + "https://trusted-cdn.example.com/lib.js", + "//trusted-cdn.example.com/lib.js", + "https://example.org/cdn.js", + "//cdnjs.example.org/lib.js", + ] { + assert_eq!( + proxy_if_abs(&settings, excluded, ""), + None, + "should leave excluded URL `{excluded}` unproxied" + ); + } + for proxied in [ + "https://other-cdn.example.com/lib.js", + "//other-cdn.example.com/lib.js", + "https://notexample.org/lib.js", + ] { + assert!( + proxy_if_abs(&settings, proxied, "") + .is_some_and(|url| url.starts_with("/first-party/proxy?tsurl=")), + "should proxy non-excluded URL `{proxied}`" + ); + } + } - assert_eq!( - to_abs(&settings, "//trusted-cdn.example.com/lib.js"), - None, - "should exclude a protocol-relative URL by exact domain" + #[test] + fn unparseable_absolute_url_is_left_byte_identical() { + let settings = crate::test_support::tests::create_test_settings(); + let html = ""; + + let out = rewrite_creative_html(&settings, html); + + assert!( + out.contains(html), + "should leave an unparseable absolute URL untouched, including its quoting: {out}" ); + } - // Non-excluded domain should return Some - assert_eq!( - to_abs(&settings, "https://other-cdn.example.com/lib.js"), - Some("https://other-cdn.example.com/lib.js".to_owned()) + #[test] + fn unparseable_absolute_click_url_is_left_byte_identical() { + let settings = crate::test_support::tests::create_test_settings(); + let html = "x"; + + let out = rewrite_creative_html(&settings, html); + + assert!( + out.contains(html), + "should leave an unparseable click URL untouched, including its quoting: {out}" ); - assert_eq!( - to_abs(&settings, "//other-cdn.example.com/lib.js"), - Some("https://other-cdn.example.com/lib.js".to_owned()), - "should normalize a non-excluded protocol-relative URL" + assert!( + !out.contains("data-tsclick"), + "should not mark an unparseable link for the click guard: {out}" ); } #[test] - fn to_abs_respects_wildcard_domains() { + fn exclude_domains_match_case_insensitively_in_the_rewrite_pass() { let mut settings = crate::test_support::tests::create_test_settings(); - settings.rewrite.exclude_domains = vec!["*.cloudflare.com".to_owned()]; + settings + .rewrite + .exclude_domains + .extend(["Landing.Example.com", "CDN.example.com"].map(str::to_owned)); + let html = r#"x"#; - // Should exclude base domain - assert_eq!(to_abs(&settings, "https://cloudflare.com/cdn.js"), None); + let out = rewrite_creative_html(&settings, html); - // Should exclude subdomain - assert_eq!( - to_abs(&settings, "https://cdnjs.cloudflare.com/lib.js"), - None - ); - assert_eq!( - to_abs(&settings, "//cloudflare.com/cdn.js"), - None, - "should exclude a protocol-relative wildcard base domain" - ); - assert_eq!( - to_abs(&settings, "//cdnjs.cloudflare.com/lib.js"), - None, - "should exclude a protocol-relative wildcard subdomain" + assert!( + out.contains(r#""#), + "should leave a link raw when its host matches a mixed-case entry: {out}" ); - - // Should not exclude different domain - assert_eq!( - to_abs(&settings, "https://notcloudflare.com/lib.js"), - Some("https://notcloudflare.com/lib.js".to_owned()) + assert!( + out.contains(r#""#), + "should leave an asset raw when its host matches a mixed-case entry: {out}" ); } diff --git a/crates/trusted-server-core/src/proxy.rs b/crates/trusted-server-core/src/proxy.rs index 5d659cdbf..1b9139753 100644 --- a/crates/trusted-server-core/src/proxy.rs +++ b/crates/trusted-server-core/src/proxy.rs @@ -1670,37 +1670,31 @@ pub async fn handle_first_party_proxy_sign( }; let trimmed = payload.url.trim(); - let abs = if trimmed.starts_with("//") { - format!("{}:{}", request_scheme, trimmed) + let protocol_relative; + // A protocol-relative target inherits the signing request's scheme before + // normalization, so `//` and absolute input reach the same policy check. + let candidate = if trimmed.starts_with("//") { + protocol_relative = format!("{request_scheme}:{trimmed}"); + protocol_relative.as_str() } else { - crate::creative::to_abs(settings, trimmed).ok_or_else(|| { - Report::new(TrustedServerError::Proxy { - message: "unsupported url".to_string(), - }) - })? + trimmed }; - - if settings.rewrite.is_excluded(&abs) { - return Err(Report::new(TrustedServerError::Proxy { + let target = crate::creative::normalize_creative_url(candidate).ok_or_else(|| { + Report::new(TrustedServerError::Proxy { message: "unsupported url".to_string(), - })); - } - - let parsed = url::Url::parse(&abs).change_context(TrustedServerError::Proxy { - message: "invalid url".to_string(), + }) })?; - let scheme = parsed.scheme(); - if scheme != "http" && scheme != "https" { - return Err(Report::new(TrustedServerError::Proxy { - message: "unsupported scheme".to_string(), - })); - } - - let host = parsed.host_str().ok_or_else(|| { + let host = target.host_str().ok_or_else(|| { Report::new(TrustedServerError::Proxy { message: "missing host".to_string(), }) })?; + if !settings.rewrite.should_proxy_asset(host) { + log::debug!("sign request for `{host}` declined by rewrite policy"); + return Err(Report::new(TrustedServerError::Proxy { + message: "unsupported url".to_string(), + })); + } if !is_host_permitted(&settings.proxy.allowed_domains, host) { log::warn!( "sign request for `{}` blocked: host not in proxy.allowed_domains", @@ -1720,10 +1714,10 @@ pub async fn handle_first_party_proxy_sign( .to_string(); let extras = vec![(String::from("tsexp"), tsexp)]; - let mut base = parsed.clone(); + let mut base = target.clone(); base.set_query(None); base.set_fragment(None); - let proxied = crate::creative::build_proxy_url_with_extras(settings, &abs, &extras); + let proxied = crate::creative::build_proxy_url_with_extras(settings, target.as_str(), &extras); let resp = ProxySignResp { href: proxied, @@ -2695,20 +2689,58 @@ mod tests { let mut settings = create_test_settings(); settings.rewrite.exclude_domains = vec!["cdn.example".to_owned()]; - for url in ["https://cdn.example/asset.js", "//cdn.example/asset.js"] { - let body = serde_json::json!({ "url": url }); - let req = - build_http_post_json_request("https://edge.example/first-party/sign", &body); - let err: Report = - handle_first_party_proxy_sign(&settings, &noop_services(), req) - .await - .expect_err("should reject excluded URL"); + for method in [&Method::GET, &Method::POST] { + for url in [ + "https://cdn.example/asset.js", + "//cdn.example/asset.js", + "https://CDN.Example/asset.js", + ] { + let req = build_proxy_sign_request( + method, + "https://edge.example/first-party/sign", + url, + ); + let err: Report = + handle_first_party_proxy_sign(&settings, &noop_services(), req) + .await + .expect_err("should reject excluded URL"); + + assert_eq!( + err.current_context().status_code(), + StatusCode::BAD_GATEWAY, + "{} should reject excluded URL `{url}` as unsupported", + method.as_str() + ); + } + } + }); + } - assert_eq!( - err.current_context().status_code(), - StatusCode::BAD_GATEWAY, - "should reject excluded URL `{url}` as unsupported" - ); + #[test] + fn proxy_sign_rejects_excluded_urls_case_insensitively() { + futures::executor::block_on(async { + let mut settings = create_test_settings(); + settings.rewrite.exclude_domains = vec!["CDN.Example".to_owned()]; + + for method in [&Method::GET, &Method::POST] { + for url in ["https://cdn.example/asset.js", "//cdn.example/asset.js"] { + let req = build_proxy_sign_request( + method, + "https://edge.example/first-party/sign", + url, + ); + let err: Report = + handle_first_party_proxy_sign(&settings, &noop_services(), req) + .await + .expect_err("should reject a host excluded by a mixed-case entry"); + + assert_eq!( + err.current_context().status_code(), + StatusCode::BAD_GATEWAY, + "{} should reject `{url}` excluded by a mixed-case entry as unsupported", + method.as_str() + ); + } } }); } @@ -3623,6 +3655,84 @@ mod tests { ); } + #[test] + fn proxied_html_click_wrapping_follows_explicit_rewrite_clicks() { + let html = r#"Excluded"#; + // (rewrite_creatives, rewrite_clicks, expect wrapped clicks) + let cases = [ + (true, None, true), + (false, None, true), + (true, Some(true), true), + (true, Some(false), false), + (false, Some(false), false), + (false, Some(true), true), + ]; + + for (rewrite_creatives, rewrite_clicks, expect_clicks) in cases { + let mut settings = create_test_settings(); + settings.auction.rewrite_creatives = rewrite_creatives; + settings.auction.rewrite_clicks = rewrite_clicks; + settings.rewrite.exclude_domains = vec!["excluded.example.com".to_owned()]; + let label = + format!("rewrite_creatives={rewrite_creatives} rewrite_clicks={rewrite_clicks:?}"); + let req = build_http_request(Method::GET, "https://edge.example.com/first-party/proxy"); + let mut response = build_http_response(StatusCode::OK, EdgeBody::from(html)); + response.headers_mut().insert( + header::CONTENT_TYPE, + HeaderValue::from_static("text/html; charset=utf-8"), + ); + + let body = response_body_string( + finalize( + &settings, + &req, + "https://cdn.example.com/creative.html", + response, + ) + .expect("should finalize proxied HTML"), + ); + + assert!( + body.contains("/first-party/proxy?tsurl="), + "{label}: proxied HTML always proxies assets: {body}" + ); + // A wrapped landing link carries the click URL in href and data-tsclick. + assert_eq!( + body.matches("/first-party/click?tsurl=").count(), + if expect_clicks { 2 } else { 0 }, + "{label}: click wrapping in proxied HTML: {body}" + ); + assert_eq!( + body.matches("data-tsclick").count(), + usize::from(expect_clicks), + "{label}: data-tsclick in proxied HTML: {body}" + ); + if !expect_clicks { + let landing_href = body + .split("Excluded"#), + "{label}: excluded link always stays raw: {body}" + ); + assert!( + !body.contains(": {body}" + ); + assert!( + body.contains("/static/tsjs="), + "{label}: proxied HTML always receives the runtime: {body}" + ); + } + } + #[test] fn html_response_rewrite_preserves_non_standard_port() { // Verify that HTML rewriting preserves non-standard ports in sub-resource URLs. diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 61bc7efe5..254118f6f 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -22668,6 +22668,50 @@ mod tests { ); } + #[test] + fn build_bid_map_keeps_inline_anchors_raw_when_clicks_are_off() { + let mut settings = test_settings(); + settings.auction.rewrite_creatives = true; + settings.auction.rewrite_clicks = Some(false); + settings.publisher.domain = "example.com".to_string(); + + let mut winning_bids = HashMap::new(); + let mut bid = make_bid( + "atf_sidebar_ad", + 1.50, + "examplessp", + "abc123", + "https://ssp.example.com/win", + "https://ssp.example.com/bill", + ); + bid.creative = Some( + "" + .to_string(), + ); + winning_bids.insert("atf_sidebar_ad".to_string(), bid); + + let map = build_bid_map(&winning_bids, PriceGranularity::Dense, &settings, "", false); + let adm = map + .get("atf_sidebar_ad") + .and_then(|v| v.as_object()) + .and_then(|o| o.get("adm")) + .and_then(|v| v.as_str()) + .expect("should include a rewritten adm"); + + assert!( + adm.contains("https://example.com/first-party/proxy?tsurl="), + "should still proxy the image with an absolute URL, got: {adm}" + ); + assert!( + adm.contains("href=\"https://landing.example.com/page\""), + "should leave the landing link raw, got: {adm}" + ); + assert!( + !adm.contains("data-tsclick") && !adm.contains("/first-party/click"), + "should not wrap the landing link, got: {adm}" + ); + } + #[test] fn build_bid_map_uses_request_origin_for_inline_urls() { // The inline adm's absolute first-party URLs must resolve against the diff --git a/crates/trusted-server-core/src/settings.rs b/crates/trusted-server-core/src/settings.rs index bf4ff7999..0d3cad1c9 100644 --- a/crates/trusted-server-core/src/settings.rs +++ b/crates/trusted-server-core/src/settings.rs @@ -23,6 +23,7 @@ use crate::creative_opportunities::CreativeOpportunitiesConfig; use crate::error::TrustedServerError; use crate::host_header::validate_host_header_override_value; use crate::platform::PlatformImageOptimizerRegion; +use crate::proxy::is_host_allowed; use crate::redacted::Redacted; #[cfg(test)] @@ -643,30 +644,48 @@ pub struct Rewrite { } impl Rewrite { - /// Checks if a URL should be excluded from rewriting based on domain matching - #[allow(dead_code)] + /// Returns `true` when an asset URL on `host` should be rewritten to + /// `/first-party/proxy`. + /// + /// The host must not match [`Self::exclude_domains`]. Matching is + /// case-insensitive; see [`is_host_allowed`] for the pattern rules. #[must_use] - pub fn is_excluded(&self, url: &str) -> bool { - // Parse URL to extract host - let Ok(parsed) = url::Url::parse(url) else { - return false; - }; + pub fn should_proxy_asset(&self, host: &str) -> bool { + !self.is_excluded_host(host) + } - let host = parsed.host_str().unwrap_or(""); + /// Returns `true` when a click-through URL on `host` should be wrapped in + /// `/first-party/click`. + /// + /// Only [`Self::exclude_domains`] applies to click-through links. + #[must_use] + pub fn should_wrap_click(&self, host: &str) -> bool { + !self.is_excluded_host(host) + } - // Check exact domain matches (with wildcard support) - for domain in &self.exclude_domains { - if let Some(suffix) = domain.strip_prefix("*.") { - // Wildcard: *.example.com matches both example.com and sub.example.com - if host == suffix || host.ends_with(&format!(".{}", suffix)) { - return true; - } - } else if host == domain { - return true; - } - } + fn is_excluded_host(&self, host: &str) -> bool { + self.exclude_domains + .iter() + .any(|pattern| is_host_allowed(host, pattern)) + } - false + /// Trims and lowercases host patterns in place. + /// + /// Empty and bare `*` entries in `exclude_domains` are dropped with a + /// warning: neither can match a host, so dropping them changes no behavior. + fn normalize(&mut self) { + let before = self.exclude_domains.len(); + self.exclude_domains = self + .exclude_domains + .iter() + .map(|pattern| pattern.trim().to_ascii_lowercase()) + .filter(|pattern| !pattern.is_empty() && pattern != "*") + .collect(); + if self.exclude_domains.len() < before { + log::warn!( + "rewrite.exclude_domains: removed empty or bare \"*\" entries, which never match a host" + ); + } } } @@ -2988,6 +3007,7 @@ impl Settings { pub(crate) fn normalize_deserialized(&mut self) { self.cache.normalize(); self.proxy.normalize(); + self.rewrite.normalize(); self.image_optimizer.normalize(); self.debug.auction_html_comment_options.normalize(); self.tinybird.normalize(); @@ -6520,29 +6540,81 @@ source_domain = "partner.example.com" } #[test] - fn test_rewrite_is_excluded() { - let rewrite = Rewrite { - exclude_domains: vec!["cdn.example.com".to_string(), "*.example2.com".to_string()], - }; + fn rewrite_policy_matches_exclude_patterns_case_insensitively() { + let mut rewrite = Rewrite::default(); + rewrite + .exclude_domains + .extend(["cdn.example.com", "*.example.org"].map(str::to_owned)); - // Exact domain match - assert!(rewrite.is_excluded("http://cdn.example.com/image.png")); + for (host, expected) in [ + ("cdn.example.com", false), + ("CDN.EXAMPLE.COM", false), + ("example.org", false), + ("a.b.example.org", false), + ("evil-example.org", true), + ("sub.cdn.example.com", true), + ("other.example.com", true), + ] { + assert_eq!( + rewrite.should_proxy_asset(host), + expected, + "should_proxy_asset(`{host}`) should be {expected}" + ); + assert_eq!( + rewrite.should_wrap_click(host), + expected, + "should_wrap_click(`{host}`) should be {expected}" + ); + } + } - // Wildcard match - base domain - assert!(rewrite.is_excluded("https://example2.com/cdn.js")); - // Wildcard match - subdomains - assert!(rewrite.is_excluded("https://cdnjs.example2.com/lib.js")); - assert!(rewrite.is_excluded("https://sub.domain.example2.com/asset.js")); + #[test] + fn rewrite_normalize_trims_lowercases_and_drops_inert_exclude_entries() { + let mut rewrite = Rewrite::default(); + rewrite + .exclude_domains + .extend([" CDN.Example.com ", "", "*", "*.Example.ORG"].map(str::to_owned)); - // Should NOT match - assert!(!rewrite.is_excluded("https://other.example.com/asset.js")); - assert!(!rewrite.is_excluded("https://sub.cdn.example.com/asset.js")); - assert!(!rewrite.is_excluded("https://example2.com.fake.com/asset.js")); - assert!(!rewrite.is_excluded("https://notexample.com/asset.js")); + rewrite.normalize(); - // Invalid URLs should not crash and should return false - assert!(!rewrite.is_excluded("not a url")); - assert!(!rewrite.is_excluded("")); + assert_eq!( + rewrite.exclude_domains, + vec!["cdn.example.com".to_owned(), "*.example.org".to_owned()], + "should trim, lowercase, and drop empty and bare `*` entries" + ); + } + + #[test] + fn rewrite_exclude_domains_match_mixed_case_entries_from_toml() { + let toml_str = crate_test_settings_str() + + r#" + [rewrite] + exclude_domains = ["CDN.Example.com"] + "#; + + let settings = Settings::from_toml(&toml_str).expect("should parse valid TOML"); + + assert!( + !settings.rewrite.should_proxy_asset("cdn.example.com"), + "should exclude a host whose config entry was written in mixed case" + ); + } + + #[test] + fn settings_load_normalizes_rewrite_exclude_domains() { + let toml_str = crate_test_settings_str() + + r#" + [rewrite] + exclude_domains = [" CDN.Example.com ", "*", ""] + "#; + + let settings = Settings::from_toml(&toml_str).expect("should parse valid TOML"); + + assert_eq!( + settings.rewrite.exclude_domains, + vec!["cdn.example.com".to_owned()], + "should trim, lowercase, and drop inert entries when settings load" + ); } #[test] @@ -6563,6 +6635,32 @@ source_domain = "partner.example.com" !settings.auction.sanitize_creatives, "creative sanitization is opt-in when the setting is omitted" ); + assert_eq!( + settings.auction.rewrite_clicks, None, + "click rewriting stays unset when the setting is omitted" + ); + } + + #[test] + fn test_auction_rewrite_clicks_accepts_explicit_values() { + for value in [true, false] { + let toml_str = crate_test_settings_str() + + &format!( + r#" + [auction] + enabled = true + rewrite_clicks = {value} + "# + ); + + let settings = Settings::from_toml(&toml_str).expect("should parse valid TOML"); + + assert_eq!( + settings.auction.rewrite_clicks, + Some(value), + "should parse explicit rewrite_clicks = {value}" + ); + } } #[test] diff --git a/docs/guide/api-reference.md b/docs/guide/api-reference.md index e444ec147..bc95ff9ac 100644 --- a/docs/guide/api-reference.md +++ b/docs/guide/api-reference.md @@ -168,8 +168,9 @@ curl -i "https://edge.example.com/_ts/clear-tester" Browser and programmatic auction endpoint. It accepts the Trusted Server ad-unit request shape and returns an OpenRTB response. Creative markup follows the -independent `[auction].sanitize_creatives` and `[auction].rewrite_creatives` -settings; sanitization is opt-in and rewriting is enabled by default. +independent `[auction].sanitize_creatives`, `[auction].rewrite_creatives` and +`[auction].rewrite_clicks` settings; sanitization is opt-in, asset rewriting is +enabled by default, and click wrapping follows `rewrite_creatives` unless set. Configured provider IDs appear in response metadata and provider responses. Consumers that previously matched the literal provider name `prebid` must use diff --git a/docs/guide/auction-orchestration.md b/docs/guide/auction-orchestration.md index edbc38471..46347c80e 100644 --- a/docs/guide/auction-orchestration.md +++ b/docs/guide/auction-orchestration.md @@ -161,7 +161,7 @@ sequenceDiagram Note right of Client: Fragment-bound nonce and one-time acknowledgement
No allow-same-origin on the outer frame else Ordinary creative Client->>Client: Inject winning creative
Render iframe
Load creative resources - Note right of Client: Default: first-party proxy/click URLs
rewrite_creatives=false: accepted external URLs remain direct + Note right of Client: Default: first-party proxy/click URLs
rewrite_creatives=false: asset URLs direct; clicks follow rewrite_clicks end deactivate Client end @@ -195,7 +195,7 @@ AuctionOrchestrator.run_auction() Convert OrchestrationResult → OpenRTB 2.x Response │ ├─[sanitize_creatives=true] Strip executable markup - ├─[rewrite_creatives=true] Rewrite URLs and inject creative TSJS + ├─[rewrite_creatives or rewrite_clicks] Rewrite assets and/or links, inject creative TSJS ├─ Add ext.orchestrator metadata └─ Set consent and optional EID response headers ``` @@ -638,8 +638,11 @@ HTML rewriter (`lol_html`) that converts eligible external resource and click URLs to signed first-party paths, adds `data-tsclick`, rewrites inline CSS `url(...)` values, removes bidder-supplied `` elements, and injects the unified creative TSJS runtime exactly once, whether or not the bidder supplied a -`` element. In every mode, a creative -larger than the 1 MiB per-creative cap is rejected and its `adm` is dropped. +`` element. `rewrite_clicks` (unset by default, following +`rewrite_creatives`) controls click-through link wrapping separately; see +[Creative Processing](/guide/creative-processing#assets-and-clicks). In every +mode, a creative larger than the 1 MiB per-creative cap is rejected and its +`adm` is dropped. ```toml [auction] diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index f7220d58c..908239d2b 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -1927,15 +1927,16 @@ provider or bidder route. ### `[auction]` -| Field | Type | Default | Description | -| ---------------------- | ------- | ------------------ | -------------------------------------------------------------- | -| `enabled` | Boolean | `false` | Enable the auction orchestrator | -| `sanitize_creatives` | Boolean | `false` | Strip executable markup from winning-bid `adm` before delivery | -| `rewrite_creatives` | Boolean | `true` | Rewrite winning-bid `adm` through first-party endpoints | -| `timeout_ms` | Integer | `2000` | Logical auction budget in milliseconds | -| `mediator` | String | `None` | Optional separate `adserver_mock` mediator | -| `creative_store` | String | `"creative_store"` | Deprecated; creatives are delivered inline | -| `allowed_context_keys` | Array | `[]` | Request context keys admitted into the auction | +| Field | Type | Default | Description | +| ---------------------- | ------- | ------------------ | ------------------------------------------------------------------------------ | +| `enabled` | Boolean | `false` | Enable the auction orchestrator | +| `sanitize_creatives` | Boolean | `false` | Strip executable markup from winning-bid `adm` before delivery | +| `rewrite_creatives` | Boolean | `true` | Rewrite winning-bid asset URLs through first-party endpoints | +| `rewrite_clicks` | Boolean | unset | Wrap creative links in `/first-party/click`; unset follows `rewrite_creatives` | +| `timeout_ms` | Integer | `2000` | Logical auction budget in milliseconds | +| `mediator` | String | `None` | Optional separate `adserver_mock` mediator | +| `creative_store` | String | `"creative_store"` | Deprecated; creatives are delivered inline | +| `allowed_context_keys` | Array | `[]` | Request context keys admitted into the auction | Creative markup delivered by `POST /auction` and the publisher SSAT/page-bids path is processed by two independent passes. With `sanitize_creatives = true` @@ -1943,23 +1944,29 @@ path is processed by two independent passes. With `sanitize_creatives = true` and event handlers) is stripped together with its inner content. This blanks script-based creatives, so enable it only when creatives render in a context that shares the publisher's origin. With `rewrite_creatives = true` (the -default), eligible absolute or protocol-relative resource and click URLs not -excluded by rewrite configuration are converted to signed first-party -endpoints, and any bidder-supplied `` element is removed. The -`POST /auction` path emits root-relative endpoints and injects the creative TSJS -runtime exactly once, whether or not the bidder supplied a ``, since bare +default), eligible absolute or protocol-relative asset URLs not excluded by +rewrite configuration are converted to signed `/first-party/proxy` endpoints. +`rewrite_clicks` controls click-through links (``, ``) +separately: when it is unset, it follows `rewrite_creatives`, so existing +configs keep their current behavior. Any bidder-supplied `` element is +removed whenever either setting is on. The `POST /auction` path emits +root-relative endpoints and, when either setting is on, injects the creative +TSJS runtime exactly once, whether or not the bidder supplied a ``, since bare fragments are the common `adm` shape. The foreign-origin SSAT renderer emits -absolute endpoints and does not inject that bundle. With both disabled, `adm` -ships exactly as the bidder returned it, except that a creative larger than the +absolute endpoints and does not inject that bundle. With all three disabled, +`adm` ships exactly as the bidder returned it, except that a creative larger than the 1 MiB per-creative cap is rejected in every mode and its `adm` is dropped. -Accepted external URLs are not host allowlisted by the sanitizer. Neither -setting affects HTML or CSS fetched through `/first-party/proxy`. See +Accepted external URLs are not host allowlisted by the sanitizer. +`rewrite_creatives` and `sanitize_creatives` do not affect HTML or CSS fetched +through `/first-party/proxy`. An explicitly set `rewrite_clicks` does apply to +links in proxied HTML; unset, proxied HTML keeps wrapping links. See [Creative Processing](/guide/creative-processing#auction-rewrite-control). ::: warning Existing configs, upgrade sequencing, and rollback Default values are omitted from stored JSON; non-default values -(`sanitize_creatives = true`, `rewrite_creatives = false`) are serialized, and -older `AuctionConfig` schemas reject unknown fields. +(`sanitize_creatives = true`, `rewrite_creatives = false`, and any explicit +`rewrite_clicks`) are serialized, and older `AuctionConfig` schemas reject +unknown fields. **Upgrading:** binaries that predate `sanitize_creatives` reject a blob that carries it, so in a rolling deployment upgrade the binary **first**, then push @@ -1973,14 +1980,20 @@ on new code, while an explicit `true` fails startup on old code. **Rolling back:** before reverting to a binary that does not know a field, remove that field's non-default value (and any environment override), run `ts config validate`, push the resulting default-compatible blob, and only then -roll back the binary. +roll back the binary. Older binaries tie click wrapping to `rewrite_creatives` +in both directions: rolling back from `rewrite_creatives = true` with +`rewrite_clicks = false` turns click wrapping back on, and rolling back from +`rewrite_creatives = false` with `rewrite_clicks = true` turns it off. **Environment overlays:** The pinned EdgeZero loader cannot create missing TOML leaves. Existing configs must add **both** leaves under `[auction]` (`rewrite_creatives` and `sanitize_creatives`) before `TRUSTED_SERVER__AUCTION__REWRITE_CREATIVES` / `TRUSTED_SERVER__AUCTION__SANITIZE_CREATIVES` can take effect. An override for a -missing leaf is silently ignored. +missing leaf is silently ignored. `rewrite_clicks` is unset by default and so +cannot appear as a TOML leaf until you set it; add `rewrite_clicks = true` or +`false` under `[auction]` before relying on +`TRUSTED_SERVER__AUCTION__REWRITE_CLICKS`. ::: ### Provider map diff --git a/docs/guide/creative-processing.md b/docs/guide/creative-processing.md index 10dcb62d5..823ae1260 100644 --- a/docs/guide/creative-processing.md +++ b/docs/guide/creative-processing.md @@ -47,8 +47,8 @@ publisher's first-party domain. This provides: The creative rewriters are invoked by independent delivery paths: -1. **Auction `adm`**: Winning-bid HTML returned by `POST /auction` is optionally sanitized (`[auction].sanitize_creatives`, opt-in, default `false`) and rewritten (`[auction].rewrite_creatives`, default `true`). -2. **First-party proxy**: Non-streaming `text/html` and `text/css` responses fetched through `/first-party/proxy` are rewritten independently of the auction setting. +1. **Auction `adm`**: Winning-bid HTML returned by `POST /auction` is optionally sanitized (`[auction].sanitize_creatives`, opt-in, default `false`) and rewritten (`[auction].rewrite_creatives`, default `true`). Click-through links are wrapped according to `[auction].rewrite_clicks`, which follows `rewrite_creatives` when unset. +2. **First-party proxy**: Non-streaming `text/html` and `text/css` responses fetched through `/first-party/proxy` are rewritten independently of the auction setting. An explicitly set `[auction].rewrite_clicks` also applies to links in proxied HTML. 3. **Integration processing**: Publisher HTML integrations use their own registration and configuration controls. ::: info Streaming Mode @@ -57,26 +57,28 @@ When `with_streaming()` is enabled in `ProxyRequestConfig`, proxied HTML/CSS pro ## Auction Rewrite Control -Two independent auction settings control the processing applied to winning-bid -`adm` returned by `POST /auction` and delivered through the publisher -SSAT/page-bids path. Sanitization is opt-in (default `false`); rewriting is -enabled by default: +Three auction settings control the processing applied to winning-bid `adm` +returned by `POST /auction` and delivered through the publisher SSAT/page-bids +path. Sanitization is opt-in (default `false`) and asset rewriting is enabled +by default. Click wrapping is controlled by `rewrite_clicks`, which follows +`rewrite_creatives` when unset; see [Assets and clicks](#assets-and-clicks). ```toml [auction] sanitize_creatives = false rewrite_creatives = true +# rewrite_clicks unset: follows rewrite_creatives ``` Regardless of mode, a creative larger than the 1 MiB per-creative cap is rejected and its `adm` is dropped. -| `sanitize_creatives` | `rewrite_creatives` | Auction winning-bid `adm` behavior | -| -------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `false` (default) | `false` | Deliver the creative exactly as the bidder returned it (subject to the size cap). | -| `true` | `false` | Strip executable markup (`script`/`object`/`embed`/`form`, event handlers) with its inner content, then deliver without rewriting. Sanitizer-accepted external resource, click, and inline CSS URLs remain direct. | -| `false` | `true` (default) | Rewrite eligible resource/CSS and click URLs in the raw bidder markup to signed first-party endpoints, removing any bidder `` element. Executable markup is preserved. | -| `true` | `true` | Sanitize first, then rewrite. `POST /auction` emits root-relative endpoints and injects creative TSJS exactly once, whether or not the bidder supplied a ``; SSAT/page-bids emits absolute endpoints for its foreign-origin renderer and does not inject the bundle. | +| `sanitize_creatives` | `rewrite_creatives` | Auction winning-bid `adm` behavior | +| -------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `false` (default) | `false` | Asset URLs stay direct. With `rewrite_clicks` unset or `false`, deliver the creative exactly as the bidder returned it (subject to the size cap); with `rewrite_clicks = true`, wrap links as described in [Assets and clicks](#assets-and-clicks). | +| `true` | `false` | Strip executable markup (`script`/`object`/`embed`/`form`, event handlers) with its inner content, then deliver without asset rewriting. Sanitizer-accepted external resource and inline CSS URLs remain direct; links follow `rewrite_clicks` and stay direct when it is unset or `false`. | +| `false` | `true` (default) | Rewrite eligible resource/CSS URLs in the raw bidder markup to signed first-party endpoints, removing any bidder `` element. Links are wrapped unless `rewrite_clicks = false`. Executable markup is preserved. | +| `true` | `true` | Sanitize first, then rewrite. `POST /auction` emits root-relative endpoints and injects creative TSJS exactly once, whether or not the bidder supplied a ``; SSAT/page-bids emits absolute endpoints for its foreign-origin renderer and does not inject the bundle. | ::: warning Sanitization blanks script-based creatives Sanitization removes `script`/`object`/`embed`/`form` and similar elements @@ -90,6 +92,39 @@ accepted HTTP(S) URLs may cause the browser to contact external creative hosts directly. ::: +### Assets and clicks + +`rewrite_creatives` governs asset URLs. `rewrite_clicks` governs click-through +links and follows `rewrite_creatives` when unset: + +```toml +[auction] +rewrite_creatives = true +# Unset: follows rewrite_creatives. Set false to keep landing links direct. +# rewrite_clicks = false +``` + +| Assets (`rewrite_creatives`) | Clicks (`rewrite_clicks`) | Asset URLs | Links | `` | TSJS on `POST /auction` | +| ---------------------------- | ------------------------- | -------------------- | ------------------------------------- | -------- | ----------------------- | +| `true` | `true` or unset | `/first-party/proxy` | `/first-party/click` + `data-tsclick` | removed | injected | +| `true` | `false` | `/first-party/proxy` | direct | removed | injected | +| `false` | `true` | direct | `/first-party/click` + `data-tsclick` | removed | injected | +| `false` | `false` or unset | direct | direct | kept | not injected | + +SSAT/page-bids follows the same table with absolute URLs and never injects TSJS. +`exclude_domains` applies to both assets and links. + +The client-side `tsCreativeConfig.clickGuard` flag is separate. `rewrite_clicks` +decides whether the server emits signed click URLs; `clickGuard` decides whether +the creative runtime re-signs them after creative script changes their query +parameters. With `rewrite_clicks` off there is nothing for the guard to act on. + +With `rewrite_creatives = false` and `rewrite_clicks = true`, `POST /auction` +still injects TSJS. A creative that turns on `tsCreativeConfig.renderGuard` can +therefore still send assets inserted by its own script through +`/first-party/sign` and `/first-party/proxy`, even though the server left the +markup's asset URLs direct. + ::: info Runtime protections inside the sandboxed creative iframe Creatives rendered by Trusted Server's own path run in a sandboxed iframe **without** `allow-same-origin`, i.e. an opaque origin. The injected creative @@ -307,15 +342,18 @@ If the iframe content itself contains HTML, it will be processed recursively. Ea **Rewrite Mode**: Uses `/first-party/click` for direct redirects +**Controlled by**: `[auction].rewrite_clicks` (follows `rewrite_creatives` when unset) + **Example**: ```html -Buy Now +Buy Now Buy Now ``` diff --git a/trusted-server.example.toml b/trusted-server.example.toml index e841515ae..d4789570c 100644 --- a/trusted-server.example.toml +++ b/trusted-server.example.toml @@ -261,18 +261,29 @@ proxy = "enabled" # Keep disabled until provider endpoints, routes, and profile values below are # replaced with deployment-specific settings. enabled = false -# Rewrite winning-bid creative HTML to first-party endpoints (default true). Set -# false to skip proxy/click-URL conversion and creative TSJS injection. -# Sanitization is controlled separately by `sanitize_creatives` below. Restore -# true before rolling back to an older binary that rejects unknown fields. +# Rewrite winning-bid creative asset URLs (images, scripts, styles, media, +# iframes, CSS url()) to first-party /first-party/proxy endpoints (default +# true). Set false to leave asset URLs direct. Bidder removal and +# creative TSJS injection run when this or rewrite_clicks is on. Sanitization +# is controlled separately by `sanitize_creatives` below. Restore true before +# rolling back to an older binary that rejects unknown fields. rewrite_creatives = true +# Wrap creative click-through links (, ) in signed +# /first-party/click redirects. Unset (the default) follows rewrite_creatives +# for auction creatives and keeps wrapping links in HTML fetched through +# /first-party/proxy; a set value applies to both. Leave it commented out to +# keep that behavior: any explicit value is stored in the pushed config, and +# binaries older than this setting reject it. Uncomment it before relying on +# TRUSTED_SERVER__AUCTION__REWRITE_CLICKS, which cannot create a missing leaf. +# rewrite_clicks = true # Strip executable markup (script/object/embed/form/...) from winning-bid adm, # removing those elements together with their inner content. Defaults to false -# (executable markup preserved). Note that with `rewrite_creatives = true` the -# adm is still not untouched: eligible URLs are rewritten, bidder `` -# elements are removed, and the creative TSJS runtime is injected. Enable it -# whenever creatives can render in a context that shares the publisher origin -# (its primary defence there); leave it off when creatives render in a +# (executable markup preserved). Note that with `rewrite_creatives` or +# `rewrite_clicks` on, the adm is still not untouched: eligible asset URLs +# and/or links are rewritten, bidder `` elements are removed, and the +# creative TSJS runtime is injected. Enable it whenever creatives can render +# in a context that shares the publisher origin (its primary defence there); +# leave it off when creatives render in a # foreign-origin frame (e.g. the Prebid Universal Creative inside the ad # server's iframe), since it removes script-based creatives entirely and would # blank slots on a script-heavy demand stack.