diff --git a/CHANGELOG.md b/CHANGELOG.md index 4c3b6c1058..93b4dc6465 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -95,6 +95,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- A device module may answer the confidence that a person is looking at the page, `DeviceSignals::human_confidence`, which nothing reads yet, which is not stored in the identity graph with the other signals, and which stays `Unknown` for every module that does not score the request, and may tell a bidder what the device is through `DeviceModule::advertising_attributes`, which reaches the bid request's device object as its type, make, model, operating system and screen. That call carries the permissions resolved for the request, a backend the module registers during it waits no longer than the auction's timeout, and core drops a device type outside the AdCOM 1.0 device types list, a screen size that is not positive and a blank string. A module that resolves nothing adds no device object. - A prefix module, selected by `[prefix] module`, gives the start of every name the browser holds for a publisher and of every key the publisher's entries take in a shared store. Core's own module gives `ts` and `ts.` and runs when nothing is selected, so a default deployment's names are unchanged. The page's bundle tag tells the client script the public prefix as `data-ts-public-prefix`, and a template stored under one prefix is never served under another. - `[shared.]`, one table per vendor holding what every module of that vendor shares, read by the vendor's crates through `Settings::shared_config` and by nothing in core. A builder marked with `IntegrationBuilder::with_shared_settings` runs when its vendor's table is present and is selected by no section, a table no builder reads is refused when the registry is built, a table whose name is not lower case letters, digits, `_` or `-` is refused when the settings load, a refusal of a module nothing supplies names the absent table, and a secret in the table is declared, resolved and checked at push to name a key as a module's own are. A shared table, or a module's own, that cannot be read or fails its validation is refused naming the setting at fault and never its value. - `robots_txt`, core's own middleware for the file `[robots-txt]` describes, named in a `[[fetch]]` entry for `text/plain` on `/robots.txt`. Its preparation makes the file from the selected modules through their holding, and its action replaces the origin's file with it. The file is made in place of any other answer the origin gives to a `GET` or `HEAD`, such as a `404`, a page, an error or a redirect. The origin is asked for its whole file, without a crawler's conditional or range headers, and the middleware works on `/robots.txt` alone. A module that cannot answer with nothing held refuses the request with `503` and `Retry-After` before any document is processed. A section with no such entry, or with an entry naming the middleware on another path or media type, is refused when the settings load. diff --git a/crates/ad-server/mock/src/lib.rs b/crates/ad-server/mock/src/lib.rs index 0edb977eb2..07bd08e92f 100644 --- a/crates/ad-server/mock/src/lib.rs +++ b/crates/ad-server/mock/src/lib.rs @@ -662,6 +662,7 @@ mod tests { user_agent: Some("Mozilla/5.0".to_string()), ip: Some("192.168.1.1".to_string()), geo: None, + attributes: None, }), site: None, context: HashMap::new(), diff --git a/crates/auction/prebid-server/src/lib.rs b/crates/auction/prebid-server/src/lib.rs index 581b457ae2..7505cb71a5 100644 --- a/crates/auction/prebid-server/src/lib.rs +++ b/crates/auction/prebid-server/src/lib.rs @@ -3182,6 +3182,7 @@ assume_single_jurisdiction = true user_agent: Some("test-agent".to_string()), ip: Some("203.0.113.42".to_string()), geo: None, + attributes: None, }); let settings = make_settings(); let request = build_test_request(); @@ -3362,6 +3363,7 @@ assume_single_jurisdiction = true region: None, asn: None, }), + attributes: None, }); let settings = make_settings(); @@ -3456,6 +3458,7 @@ assume_single_jurisdiction = true region: Some("NY".to_string()), asn: None, }), + attributes: None, }); let settings = make_settings(); @@ -3495,6 +3498,7 @@ assume_single_jurisdiction = true region: Some("NY".to_string()), asn: None, }), + attributes: None, }); let settings = make_settings(); @@ -3792,6 +3796,7 @@ assume_single_jurisdiction = true user_agent: Some("TestAgent".to_string()), ip: None, geo: None, + attributes: None, }); let settings = make_settings(); @@ -3820,6 +3825,7 @@ assume_single_jurisdiction = true user_agent: Some("TestAgent".to_string()), ip: None, geo: None, + attributes: None, }); let settings = make_settings(); @@ -3853,6 +3859,7 @@ assume_single_jurisdiction = true user_agent: Some("TestAgent".to_string()), ip: None, geo: None, + attributes: None, }); let settings = make_settings(); @@ -3929,6 +3936,7 @@ assume_single_jurisdiction = true region: Some("NY".to_string()), asn: None, }), + attributes: None, }); let settings = make_settings(); @@ -4713,6 +4721,7 @@ assume_single_jurisdiction = true user_agent: Some("test-agent".to_string()), ip: None, geo: None, + attributes: None, }), site: None, context: HashMap::new(), diff --git a/crates/testing/seam-probe/src/lib.rs b/crates/testing/seam-probe/src/lib.rs index 0071950ce6..5c5af69369 100644 --- a/crates/testing/seam-probe/src/lib.rs +++ b/crates/testing/seam-probe/src/lib.rs @@ -36,7 +36,9 @@ use trusted_server_core::auction::demand::{ }; use trusted_server_core::auction::provider::{AuctionProvider, ProviderRequestOutcome}; use trusted_server_core::auction::types::{AuctionContext, AuctionRequest, AuctionResponse}; -use trusted_server_core::ec::device::{DeviceModule, DeviceSignals}; +use trusted_server_core::ec::device::{ + DeviceAttributes, DeviceModule, DeviceSignals, HumanConfidence, +}; use trusted_server_core::ec::module::{EdgeCookieModule, GeneratedEdgeCookie, ModuleCode}; use trusted_server_core::error::TrustedServerError; use trusted_server_core::evidence::RequestInfo; @@ -425,6 +427,30 @@ impl DeviceModule for SeamProbeDevice { looks_like_browser: true, ja4_class: None, h2_fp_hash: None, + // A scoring module's answer, which core's own modules never give. + human_confidence: HumanConfidence::Medium, + } + } + + async fn advertising_attributes(&self, _call: ModuleCall<'_>) -> Option { + Some(SeamProbeDevice::attributes()) + } +} + +impl SeamProbeDevice { + /// What the probe tells a bidder about the device, fixed so a test can + /// read it back. + #[must_use] + pub fn attributes() -> DeviceAttributes { + DeviceAttributes { + // A tablet, as the module gives `OpenRTB`'s own code. + device_type: Some(5), + make: Some("Probe".to_owned()), + model: Some("P-1".to_owned()), + os: Some("ProbeOS".to_owned()), + os_version: Some("1.0".to_owned()), + screen_width: Some(800), + screen_height: Some(1280), } } } diff --git a/crates/trusted-server-adapter-axum/tests/seam_probe.rs b/crates/trusted-server-adapter-axum/tests/seam_probe.rs index 488989102e..0993f549b1 100644 --- a/crates/trusted-server-adapter-axum/tests/seam_probe.rs +++ b/crates/trusted-server-adapter-axum/tests/seam_probe.rs @@ -1027,6 +1027,53 @@ async fn a_requests_services_carry_the_selected_modules_prefixes() { ); } +/// The device module `[device] module` resolves answers the confidence that +/// a person is there and what a bidder is told about the device, through the +/// call it is handed, so both reach core from a module it does not know. +#[tokio::test] +async fn the_resolved_device_module_answers_a_confidence_and_the_devices_attributes() { + let settings = settings_selecting_module( + r#" + [device] + module = "testing.seam-probe" + "#, + ); + let registry = registry_with_probe(&settings); + let module = registry + .device_module() + .expect("`[device] module = \"testing.seam-probe\"` should resolve the module's module"); + let request = ModuleRequest::new( + &axum::http::Method::GET, + "www.publisher.example", + "https", + "/", + ); + let context = ModuleContext::new(request).with_settings(&settings); + + let signals = module + .detect(context.call(module.id(), module.required_permissions())) + .await; + let attributes = module + .advertising_attributes(context.call(module.id(), module.required_permissions())) + .await; + + assert_eq!( + signals.human_confidence, + trusted_server_core::ec::device::HumanConfidence::Medium, + "the scoring module's answer should reach core unchanged" + ); + assert_eq!( + attributes, + Some(seam_probe::SeamProbeDevice::attributes()), + "the attributes the module resolved should reach core unchanged" + ); + assert_eq!( + attributes.and_then(|found| found.device_type), + Some(5), + "a tablet is OpenRTB device type 5" + ); +} + /// `[ec] module` naming a module resolves a module that, when driven /// through `generate`, produces an identifier the built-in HMAC module can /// never make. diff --git a/crates/trusted-server-core/src/auction/endpoints.rs b/crates/trusted-server-core/src/auction/endpoints.rs index 55671e4ac3..7602c0caae 100644 --- a/crates/trusted-server-core/src/auction/endpoints.rs +++ b/crates/trusted-server-core/src/auction/endpoints.rs @@ -26,7 +26,7 @@ use crate::settings::Settings; use super::AuctionOrchestrator; use super::formats::{ - convert_to_openrtb_response, convert_to_openrtb_response_with_report, + attach_device_attributes, convert_to_openrtb_response, convert_to_openrtb_response_with_report, convert_tsjs_to_auction_request, }; use super::telemetry::{ @@ -354,6 +354,18 @@ pub async fn handle_auction( geo, )?; + // Ask the selected device module what the bidder should be told about + // the device. + attach_device_attributes( + &mut auction_request, + &http_req, + ec_context.permissions(), + settings, + services, + settings.auction.timeout_ms, + ) + .await; + // Merge current-request client EIDs with KV-resolved EIDs, then apply // consent gating before attaching them to the auction request. let merged_eids = merge_auction_eids(client_eids, eids); @@ -647,20 +659,26 @@ mod tests { use crate::auction::types::{AuctionRequest, AuctionResponse}; use crate::consent::jurisdiction::Jurisdiction; use crate::consent::types::ConsentContext; + use crate::ec::device::{DeviceAttributes, DeviceModule}; use crate::ec::tests::{CANONICAL_COOKIE_VALUE, CANONICAL_KV_KEY, CanonicalizingModule}; use crate::error::IntoHttpResponse as _; use crate::openrtb::Uid; use crate::permissions::{Permission, PermissionSet, PermissionState}; use crate::platform::test_support::{ - NoopBackend, NoopConfigStore, NoopGeo, NoopHttpClient, NoopSecretStore, StubHttpClient, - noop_services, + DetectionServiceDevice, NamingBackend, NoopBackend, NoopConfigStore, NoopGeo, + NoopHttpClient, NoopSecretStore, StubHttpClient, + build_services_with_backend_and_http_client, noop_services, + }; + use crate::platform::{ + BackendNamingPolicy, ClientInfo, PlatformBackend, PlatformHttpClient, PlatformHttpRequest, + PlatformResponse, }; - use crate::platform::{ClientInfo, PlatformHttpClient, PlatformHttpRequest, PlatformResponse}; use crate::test_support::tests::{crate_test_settings_str, create_test_settings}; use base64::Engine as _; use base64::engine::general_purpose::STANDARD as BASE64; use serde_json::json; use std::sync::{Arc, Mutex}; + use std::time::Duration; #[derive(Default)] struct RecordingTelemetrySink { @@ -2030,4 +2048,262 @@ mod tests { ); }); } + + /// Records the [`AuctionRequest`] the orchestrator dispatched, then fails + /// its launch so no transport is needed. + struct RequestCapturingProvider { + captured: Arc>>, + } + + #[async_trait::async_trait(?Send)] + impl AuctionProvider for RequestCapturingProvider { + fn provider_name(&self) -> &str { + "request_capturing_provider" + } + + async fn request_bids( + &self, + request: &AuctionRequest, + _context: &AuctionContext<'_>, + ) -> Result> { + *self.captured.lock().expect("should lock captured request") = Some(request.clone()); + Err(Report::new(TrustedServerError::Auction { + message: "capture only".to_string(), + })) + } + + async fn parse_response( + &self, + _response: PlatformResponse, + _response_time_ms: u64, + ) -> Result> { + panic!("parse_response must not run when the launch fails"); + } + + fn timeout_ms(&self) -> u32 { + 100 + } + + fn backend_name(&self, _services: &RuntimeServices, _timeout_ms: u32) -> Option { + Some("capture-backend".to_string()) + } + } + + fn orchestrator_capturing( + captured: &Arc>>, + ) -> AuctionOrchestrator { + let mut orchestrator = AuctionOrchestrator::new(AuctionConfig { + enabled: true, + provider_names: vec!["request_capturing_provider".to_string()], + timeout_ms: 2000, + adserver_name: None, + ..Default::default() + }); + orchestrator.register_provider(Arc::new(RequestCapturingProvider { + captured: Arc::clone(captured), + })); + orchestrator + } + + fn one_slot_auction_request() -> Request { + Request::builder() + .method("POST") + .uri("https://test-publisher.com/auction") + .body(EdgeBody::from( + serde_json::to_vec(&json!({ + "adUnits": [ + { + "code": "div-gpt-ad-1", + "mediaTypes": { "banner": { "sizes": [[300, 250]] } } + } + ] + })) + .expect("should serialize body"), + )) + .expect("should build auction request") + } + + fn tablet() -> DeviceAttributes { + DeviceAttributes { + device_type: Some(5), + make: Some("ExampleCorp".to_owned()), + ..DeviceAttributes::default() + } + } + + fn selecting_basic_ads() -> PermissionSet { + PermissionSet::none().with(Permission::SelectBasicAds) + } + + fn captured_attributes( + captured: &Arc>>, + ) -> Option { + captured + .lock() + .expect("should lock captured request") + .as_ref() + .expect("should dispatch an auction request") + .device + .as_ref() + .and_then(|device| device.attributes.clone()) + } + + /// Runs `/auction` for `ec_context` with `module` selected, and returns + /// what reached the provider, if anything did. + async fn auction_with_device_module( + mut ec_context: EcContext, + module: &Arc, + services: RuntimeServices, + ) -> Arc>> { + let settings = create_test_settings(); + let captured = Arc::new(Mutex::new(None)); + let orchestrator = orchestrator_capturing(&captured); + let services = services.with_device_module(Arc::clone(module) as Arc); + + // The capturing provider fails its launch, so the auction errors. The + // assertions are on what reached the provider and the module. + let _ = handle_auction( + &settings, + &orchestrator, + None, + None, + &mut ec_context, + &services, + one_slot_auction_request(), + ) + .await; + captured + } + + #[tokio::test] + async fn auction_asks_the_device_module_with_the_readers_permissions() { + let module = Arc::new(DetectionServiceDevice::new( + selecting_basic_ads(), + Some(tablet()), + )); + let ec_context = make_non_regulated_ec_context(None) + .with_permissions_for_test(PermissionState::new(selecting_basic_ads())); + + let captured = auction_with_device_module(ec_context, &module, noop_services()).await; + + assert_eq!( + module.asked(), + 1, + "the auction should ask the device module" + ); + assert_eq!( + captured_attributes(&captured), + Some(tablet()), + "a module whose permission the reader gave should describe the device to the bidder" + ); + } + + #[tokio::test] + async fn auction_sends_no_attributes_a_reader_has_not_agreed_to() { + let module = Arc::new(DetectionServiceDevice::new( + selecting_basic_ads(), + Some(tablet()), + )); + let ec_context = make_non_regulated_ec_context(None); + + let captured = auction_with_device_module(ec_context, &module, noop_services()).await; + + assert_eq!( + module.asked(), + 1, + "the auction should ask the device module" + ); + assert_eq!( + captured_attributes(&captured), + None, + "a reader who did not give the module's permission should have no device described" + ); + } + + #[tokio::test] + async fn auction_does_not_ask_the_device_module_when_consent_denies_the_auction() { + let module = Arc::new(DetectionServiceDevice::new( + selecting_basic_ads(), + Some(tablet()), + )); + // An unknown jurisdiction with no TCF signal fails the auction gate. + let ec_context = EcContext::new_for_test(None, ConsentContext::default()) + .with_permissions_for_test(PermissionState::new(selecting_basic_ads())); + + let captured = auction_with_device_module(ec_context, &module, noop_services()).await; + + assert!( + captured + .lock() + .expect("should lock captured request") + .is_none(), + "a denied auction should dispatch nothing" + ); + assert_eq!( + module.asked(), + 0, + "a request on the consent-denied path should not reach the device module" + ); + } + + #[tokio::test] + async fn auction_caps_the_transport_timers_of_the_device_modules_backend() { + let backend = Arc::new(NamingBackend::new(BackendNamingPolicy::Axum)); + let services = build_services_with_backend_and_http_client( + Arc::clone(&backend) as Arc, + Arc::new(NoopHttpClient), + ); + let module = Arc::new(DetectionServiceDevice::new(PermissionSet::none(), None)); + + let _ = auction_with_device_module(make_non_regulated_ec_context(None), &module, services) + .await; + + let limit = Duration::from_millis(u64::from(create_test_settings().auction.timeout_ms)); + let specs = backend + .specs + .lock() + .expect("should lock registered backends"); + let spec = specs + .iter() + .find(|spec| spec.host == "detect.example") + .expect("the device module should register its service's backend"); + assert_eq!( + (spec.first_byte_timeout, spec.between_bytes_timeout), + (limit, limit), + "the module's service should hold the auction no longer than the auction's own timeout" + ); + } + + #[tokio::test] + async fn auction_drops_device_values_a_bidder_cannot_read() { + let module = Arc::new(DetectionServiceDevice::new( + PermissionSet::none(), + Some(DeviceAttributes { + device_type: Some(99), + make: Some(String::new()), + model: Some(" ".to_owned()), + os: Some("Android".to_owned()), + os_version: None, + screen_width: Some(0), + screen_height: Some(-1), + }), + )); + + let captured = auction_with_device_module( + make_non_regulated_ec_context(None), + &module, + noop_services(), + ) + .await; + + assert_eq!( + captured_attributes(&captured), + Some(DeviceAttributes { + os: Some("Android".to_owned()), + ..DeviceAttributes::default() + }), + "a device type outside the list, a size that is not positive and a blank string \ + should not reach the bidder" + ); + } } diff --git a/crates/trusted-server-core/src/auction/formats.rs b/crates/trusted-server-core/src/auction/formats.rs index d64caa8a8f..973a7869a7 100644 --- a/crates/trusted-server-core/src/auction/formats.rs +++ b/crates/trusted-server-core/src/auction/formats.rs @@ -10,6 +10,8 @@ use http::{HeaderValue, Request, Response, StatusCode, header}; use serde::Deserialize; use serde_json::Value as JsonValue; use std::collections::{BTreeMap, HashMap, HashSet}; +use std::sync::Arc; +use std::time::Duration; use url::Url; use uuid::Uuid; @@ -20,11 +22,15 @@ use crate::creative; use crate::ec::eids::encode_eids_header; use crate::error::TrustedServerError; use crate::geo::GeoInfo; +use crate::module_context::{ModuleContext, ModuleRequest}; use crate::openrtb::{ BidExt, BidTrustedServerExt, OpenRtbBid, OpenRtbResponse, ResponseExt, SeatBid, ToExt, to_openrtb_i32, }; -use crate::platform::RuntimeServices; +use crate::permissions::PermissionState; +use crate::platform::{ + BackendNamingPolicy, PlatformBackend, PlatformBackendSpec, PlatformError, RuntimeServices, +}; use crate::settings::Settings; use super::orchestrator::OrchestrationResult; @@ -224,6 +230,8 @@ pub fn convert_tsjs_to_auction_request( .map(str::to_string), ip: services.client_info().client_ip.map(|ip| ip.to_string()), geo, + // Filled by `attach_device_attributes`. + attributes: None, }); // Forward allowed config entries from the JS request into the context map. @@ -552,6 +560,104 @@ pub(crate) fn convert_to_openrtb_response_with_report( Ok(OpenRtbResponseConversion { response, delivery }) } +/// Fills the bid request's device attributes from the selected device module, +/// after each request that makes an outbound auction call is built. +/// +/// The module's call carries `permissions`, those resolved for the request, +/// so the module can do nothing the reader has not agreed to. A backend the +/// module registers during the call waits no longer than `timeout_ms`, the +/// auction's own timeout, for its service. Values a bidder cannot read are +/// dropped before the answer is attached. +pub async fn attach_device_attributes( + request: &mut AuctionRequest, + http_req: &http::Request, + permissions: &PermissionState, + settings: &Settings, + services: &RuntimeServices, + timeout_ms: u32, +) { + let Some(module) = services.device_module() else { + return; + }; + let client_ip = services + .client_info() + .client_ip + .map_or_else(String::new, |address| address.to_string()); + let request_info = + crate::evidence::BorrowedRequestInfo::new(&client_ip, Some(http_req.headers())); + let module_services = services.clone().with_backend(Arc::new(CappedBackend { + inner: Arc::clone(&services.backend), + limit: Duration::from_millis(u64::from(timeout_ms)), + })); + let context = ModuleContext::new(ModuleRequest::of(http_req, services.client_info())) + .with_evidence(&request_info) + .with_permissions(permissions) + .with_settings(settings) + .with_services(&module_services); + let Some(attributes) = module + .advertising_attributes(context.call(module.id(), module.required_permissions())) + .await + else { + return; + }; + let attributes = attributes.checked(module.id()); + // An empty object and an absent one say the same to a bidder, and the + // absent one is smaller. + if attributes.is_empty() { + return; + } + + request + .device + .get_or_insert(DeviceInfo { + user_agent: None, + ip: None, + geo: None, + attributes: None, + }) + .attributes = Some(attributes); +} + +/// Registers backends through `inner` with both transport timers capped at +/// `limit`. +/// +/// The timers bound the wait for the first byte and between two bytes, not +/// the whole call, so connecting and a slow trickle can still lengthen the +/// call, as they can an auction provider's. +struct CappedBackend { + inner: Arc, + limit: Duration, +} + +impl CappedBackend { + fn capped(&self, spec: &PlatformBackendSpec) -> PlatformBackendSpec { + PlatformBackendSpec { + first_byte_timeout: spec.first_byte_timeout.min(self.limit), + between_bytes_timeout: spec.between_bytes_timeout.min(self.limit), + ..spec.clone() + } + } +} + +impl PlatformBackend for CappedBackend { + fn naming_policy(&self) -> BackendNamingPolicy { + self.inner.naming_policy() + } + + fn predict_name(&self, spec: &PlatformBackendSpec) -> Result> { + self.inner.predict_name(&self.capped(spec)) + } + + fn ensure(&self, spec: &PlatformBackendSpec) -> Result> { + self.inner.ensure(&self.capped(spec)) + } + + fn canonicalize_transport_timeout_ms(&self, remaining_ms: u32, configured_ms: u32) -> u32 { + self.inner + .canonicalize_transport_timeout_ms(remaining_ms, configured_ms) + } +} + #[cfg(test)] mod tests { use super::*; @@ -588,6 +694,236 @@ mod tests { AuctionPlan::compile(config).expect("should compile plan") } + /// A device module that resolves fixed attributes, standing in for a + /// vendor that reaches a service. + #[derive(Debug)] + struct StubDeviceModule { + attributes: Option, + } + + #[async_trait::async_trait(?Send)] + impl crate::ec::device::DeviceModule for StubDeviceModule { + fn id(&self) -> &'static str { + "stub" + } + + async fn detect( + &self, + _call: crate::module_context::ModuleCall<'_>, + ) -> crate::ec::device::DeviceSignals { + crate::ec::device::DeviceSignals::unknown() + } + + async fn advertising_attributes( + &self, + _call: crate::module_context::ModuleCall<'_>, + ) -> Option { + self.attributes.clone() + } + } + + fn empty_request() -> http::Request { + http::Request::builder() + .uri("https://www.example.com/") + .body(EdgeBody::empty()) + .expect("should build the request") + } + + fn stub_attributes() -> crate::ec::device::DeviceAttributes { + crate::ec::device::DeviceAttributes { + device_type: Some(4), + make: Some("ExampleCorp".to_owned()), + model: Some("EX-1".to_owned()), + os: Some("Android".to_owned()), + os_version: Some("15.0".to_owned()), + screen_width: Some(1080), + screen_height: Some(2400), + } + } + + fn request_with_device(device: Option) -> AuctionRequest { + let mut request = make_auction_request(); + request.device = device; + request + } + + /// The auction timeout the device calls in these tests run under. + const TIMEOUT_MS: u32 = 2000; + + #[tokio::test] + async fn an_answer_with_no_value_a_bidder_can_read_adds_no_device_object() { + let services = crate::platform::test_support::noop_services_with_device_module( + std::sync::Arc::new(StubDeviceModule { + attributes: Some(crate::ec::device::DeviceAttributes { + device_type: Some(0), + make: Some(" ".to_owned()), + screen_width: Some(-1), + ..crate::ec::device::DeviceAttributes::default() + }), + }), + ); + let mut request = request_with_device(None); + + attach_device_attributes( + &mut request, + &empty_request(), + &PermissionState::default(), + &crate::test_support::tests::create_test_settings(), + &services, + TIMEOUT_MS, + ) + .await; + + assert!( + request.device.is_none(), + "an answer whose every value is dropped should add no device object" + ); + } + + #[tokio::test] + async fn the_selected_module_fills_the_device_object() { + let services = crate::platform::test_support::noop_services_with_device_module( + std::sync::Arc::new(StubDeviceModule { + attributes: Some(stub_attributes()), + }), + ); + let mut request = request_with_device(Some(DeviceInfo { + user_agent: Some("Mozilla/5.0".to_owned()), + ip: None, + geo: None, + attributes: None, + })); + + attach_device_attributes( + &mut request, + &empty_request(), + &PermissionState::default(), + &crate::test_support::tests::create_test_settings(), + &services, + TIMEOUT_MS, + ) + .await; + + assert_eq!( + request + .device + .as_ref() + .and_then(|device| device.attributes.as_ref()), + Some(&stub_attributes()), + "a resolved device must reach the request the bidder is sent" + ); + } + + #[tokio::test] + async fn a_module_that_resolves_an_empty_device_adds_no_device_object() { + let services = crate::platform::test_support::noop_services_with_device_module( + std::sync::Arc::new(StubDeviceModule { + attributes: Some(crate::ec::device::DeviceAttributes::default()), + }), + ); + let mut request = request_with_device(None); + + attach_device_attributes( + &mut request, + &empty_request(), + &PermissionState::default(), + &crate::test_support::tests::create_test_settings(), + &services, + TIMEOUT_MS, + ) + .await; + + assert!( + request.device.is_none(), + "an empty device object says nothing a bidder cannot read from its absence" + ); + } + + #[tokio::test] + async fn no_device_module_leaves_the_request_untouched() { + let services = crate::platform::test_support::noop_services(); + let mut request = request_with_device(Some(DeviceInfo { + user_agent: Some("Mozilla/5.0".to_owned()), + ip: None, + geo: None, + attributes: None, + })); + + attach_device_attributes( + &mut request, + &empty_request(), + &PermissionState::default(), + &crate::test_support::tests::create_test_settings(), + &services, + TIMEOUT_MS, + ) + .await; + + assert!( + request + .device + .as_ref() + .is_some_and(|device| device.attributes.is_none()), + "the default deployment supplies no device module and must send no attributes" + ); + } + + #[tokio::test] + async fn a_module_that_resolves_nothing_adds_no_device_object() { + let services = crate::platform::test_support::noop_services_with_device_module( + std::sync::Arc::new(StubDeviceModule { attributes: None }), + ); + let mut request = request_with_device(None); + + attach_device_attributes( + &mut request, + &empty_request(), + &PermissionState::default(), + &crate::test_support::tests::create_test_settings(), + &services, + TIMEOUT_MS, + ) + .await; + + assert!( + request.device.is_none(), + "an unresolved device must not become an empty device object, which a bidder \ + reads as a claim that the fields are absent rather than unknown" + ); + } + + #[tokio::test] + async fn resolved_attributes_create_the_device_object_when_there_was_none() { + let services = crate::platform::test_support::noop_services_with_device_module( + std::sync::Arc::new(StubDeviceModule { + attributes: Some(stub_attributes()), + }), + ); + // The publisher path leaves the device object out entirely when the + // request carried no User-Agent, so this is a real state and not a + // contrived one. + let mut request = request_with_device(None); + + attach_device_attributes( + &mut request, + &empty_request(), + &PermissionState::default(), + &crate::test_support::tests::create_test_settings(), + &services, + TIMEOUT_MS, + ) + .await; + + assert_eq!( + request + .device + .as_ref() + .and_then(|device| device.attributes.as_ref()), + Some(&stub_attributes()), + "attributes that were resolved must not be dropped for want of a device object" + ); + } + fn make_auction_request() -> AuctionRequest { AuctionRequest { id: "auction-1".to_string(), diff --git a/crates/trusted-server-core/src/auction/openrtb.rs b/crates/trusted-server-core/src/auction/openrtb.rs index 44b8087e92..fdea467979 100644 --- a/crates/trusted-server-core/src/auction/openrtb.rs +++ b/crates/trusted-server-core/src/auction/openrtb.rs @@ -405,6 +405,18 @@ fn build_common_request( }), dnt: routed.dnt(), language: language.clone(), + // Attributes the selected device module resolved, each absent + // when nothing resolved it. + devicetype: device.attributes.as_ref().and_then(|a| a.device_type), + make: device.attributes.as_ref().and_then(|a| a.make.clone()), + model: device.attributes.as_ref().and_then(|a| a.model.clone()), + os: device.attributes.as_ref().and_then(|a| a.os.clone()), + osv: device + .attributes + .as_ref() + .and_then(|a| a.os_version.clone()), + w: device.attributes.as_ref().and_then(|a| a.screen_width), + h: device.attributes.as_ref().and_then(|a| a.screen_height), ..Default::default() }) .or_else(|| { diff --git a/crates/trusted-server-core/src/auction/openrtb/tests.rs b/crates/trusted-server-core/src/auction/openrtb/tests.rs index 18eaf5118b..af47ffad9c 100644 --- a/crates/trusted-server-core/src/auction/openrtb/tests.rs +++ b/crates/trusted-server-core/src/auction/openrtb/tests.rs @@ -1149,3 +1149,79 @@ fn an_implementation_sees_each_impression_beside_the_slot_it_came_from() { ); } } + +fn request_with_device_attributes() -> crate::auction::types::AuctionRequest { + let mut request = canonical_parity_auction_request(); + request + .device + .as_mut() + .expect("the parity request carries a device") + .attributes = Some(crate::ec::device::DeviceAttributes { + device_type: Some(4), + make: Some("ExampleCorp".to_owned()), + model: Some("EX-1".to_owned()), + os: Some("Android".to_owned()), + os_version: Some("15.0".to_owned()), + screen_width: Some(1080), + screen_height: Some(2400), + }); + request +} + +/// The device object is the driver's, so attributes a device module +/// resolved reach every implementation without any of them mapping a field. +#[test] +fn the_driver_carries_resolved_device_attributes_to_every_implementation() { + for implementation in ["auction.fixture", "auction.plain-fixture"] { + let request = build_with_request( + implementation, + json!({}), + request_with_device_attributes(), + None, + ); + let device = request + .device + .as_ref() + .expect("should carry a device object"); + assert_eq!( + device.devicetype, + Some(4), + "{implementation}: a bidder prices on the device type" + ); + assert_eq!( + device.make.as_deref(), + Some("ExampleCorp"), + "{implementation}" + ); + assert_eq!(device.model.as_deref(), Some("EX-1"), "{implementation}"); + assert_eq!(device.os.as_deref(), Some("Android"), "{implementation}"); + assert_eq!(device.osv.as_deref(), Some("15.0"), "{implementation}"); + assert_eq!( + device.w, + Some(1080), + "{implementation}: screen width decides which creative fits" + ); + assert_eq!(device.h, Some(2400), "{implementation}"); + } +} + +#[test] +fn the_driver_omits_device_attributes_nothing_resolved() { + let request = build_with_request( + "auction.fixture", + json!({}), + canonical_parity_auction_request(), + None, + ); + let device = request + .device + .as_ref() + .expect("should carry a device object"); + assert_eq!( + device.devicetype, None, + "an absent field is what a bidder reads as unknown, and an invented default would \ + misprice the inventory" + ); + assert_eq!(device.make, None); + assert_eq!(device.w, None); +} diff --git a/crates/trusted-server-core/src/auction/routing.rs b/crates/trusted-server-core/src/auction/routing.rs index a8df9c1a29..594726ffb9 100644 --- a/crates/trusted-server-core/src/auction/routing.rs +++ b/crates/trusted-server-core/src/auction/routing.rs @@ -681,6 +681,7 @@ mod tests { user_agent: None, ip: None, geo: None, + attributes: None, }), site: Some(SiteInfo { domain: "publisher.example.test".to_string(), diff --git a/crates/trusted-server-core/src/auction/telemetry.rs b/crates/trusted-server-core/src/auction/telemetry.rs index 7afc6ffab1..1ab805e48d 100644 --- a/crates/trusted-server-core/src/auction/telemetry.rs +++ b/crates/trusted-server-core/src/auction/telemetry.rs @@ -1173,6 +1173,7 @@ mod tests { user_agent: Some(TEST_USER_AGENT.to_owned()), ip: None, geo: None, + attributes: None, }); let from_request = AuctionObservationContext::from_auction_request( diff --git a/crates/trusted-server-core/src/auction/test_support.rs b/crates/trusted-server-core/src/auction/test_support.rs index 411e1d268f..ee6f77ea7b 100644 --- a/crates/trusted-server-core/src/auction/test_support.rs +++ b/crates/trusted-server-core/src/auction/test_support.rs @@ -118,6 +118,7 @@ pub fn canonical_parity_auction_request() -> AuctionRequest { region: Some("CA".to_string()), asn: None, }), + attributes: None, }), site: None, context: HashMap::new(), diff --git a/crates/trusted-server-core/src/auction/types.rs b/crates/trusted-server-core/src/auction/types.rs index 7e500f263e..f8d83d7f9d 100644 --- a/crates/trusted-server-core/src/auction/types.rs +++ b/crates/trusted-server-core/src/auction/types.rs @@ -113,6 +113,12 @@ pub struct DeviceInfo { pub user_agent: Option, pub ip: Option, pub geo: Option, + /// Attributes a device module resolved, for the bid request's device + /// object. + /// + /// Absent when no module resolved any, which is the default. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub attributes: Option, } /// Site information. diff --git a/crates/trusted-server-core/src/ec/device.rs b/crates/trusted-server-core/src/ec/device.rs index 41cec34968..756a5c7aab 100644 --- a/crates/trusted-server-core/src/ec/device.rs +++ b/crates/trusted-server-core/src/ec/device.rs @@ -18,6 +18,7 @@ //! - **`known_browser`** — `true` if `ja4_class` + `h2_fp_hash` match a known //! browser pattern; `false` for known bots; `None` for unknown +use serde::{Deserialize, Serialize}; use sha2::{Digest as _, Sha256}; use super::kv_types::KvDevice; @@ -25,6 +26,109 @@ use crate::evidence::RequestInfo; use crate::module_context::ModuleCall; use crate::settings::Settings; +/// Device attributes for the bid request, as distinct from the gate signals. +/// +/// Kept apart from [`DeviceSignals`] on purpose. Those signals answer one +/// question, whether this request looks like a browser, and are stored in the +/// identity graph. These answer a different one, what a bidder is being asked +/// to price, and are stored nowhere. +/// +/// Every field is optional because every source is partial. A module that +/// resolves a device type but not a model fills what it has, and the `OpenRTB` +/// device object simply omits the rest, which is what a bidder expects. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct DeviceAttributes { + /// `OpenRTB` device type, from the device types list of `AdCOM` 1.0, + /// which `OpenRTB` 2.6 refers to: 1 mobile or tablet, 2 personal + /// computer, 3 connected television, 4 phone, 5 tablet, 6 connected + /// device, 7 set top box, 8 out-of-home device. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub device_type: Option, + /// Hardware vendor, the `make` field. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub make: Option, + /// Hardware model, the `model` field. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub model: Option, + /// Operating system name, the `os` field. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub os: Option, + /// Operating system version, the `osv` field. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub os_version: Option, + /// Screen width in pixels. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub screen_width: Option, + /// Screen height in pixels. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub screen_height: Option, +} + +/// The values of the device types list of `AdCOM` 1.0. +const DEVICE_TYPES: std::ops::RangeInclusive = 1..=8; + +impl DeviceAttributes { + /// Whether anything was resolved. + /// + /// A module that resolved nothing should not put an empty device object + /// on the request, because an empty object and an absent one mean the same + /// thing to a bidder and the absent one is smaller. + #[must_use] + pub fn is_empty(&self) -> bool { + *self == Self::default() + } + + /// The same attributes without the values a bidder cannot read, being a + /// device type outside the list, a screen size that is not positive and a + /// blank string. Each value dropped is logged with the id of `module`, the + /// module that answered it. + #[must_use] + pub(crate) fn checked(self, module: &str) -> Self { + Self { + device_type: kept_number(module, "device_type", self.device_type, |found| { + DEVICE_TYPES.contains(&found) + }), + make: kept_text(module, "make", self.make), + model: kept_text(module, "model", self.model), + os: kept_text(module, "os", self.os), + os_version: kept_text(module, "os_version", self.os_version), + screen_width: kept_number(module, "screen_width", self.screen_width, |found| found > 0), + screen_height: kept_number(module, "screen_height", self.screen_height, |found| { + found > 0 + }), + } + } +} + +/// `value` when `readable` holds for it, and otherwise nothing. +fn kept_number( + module: &str, + field: &str, + value: Option, + readable: fn(i32) -> bool, +) -> Option { + value.filter(|&found| { + let keep = readable(found); + if !keep { + log::warn!( + "device module `{module}`: omitting {field}={found}, which a bidder cannot read" + ); + } + keep + }) +} + +/// `value` unless it is blank, and otherwise nothing. +fn kept_text(module: &str, field: &str, value: Option) -> Option { + value.filter(|found| { + let keep = !found.trim().is_empty(); + if !keep { + log::warn!("device module `{module}`: omitting a blank {field}"); + } + keep + }) +} + /// Device signals derived from a single request. /// /// Computed in the Fastly adapter from raw TLS/H2/UA data, then passed to @@ -47,7 +151,44 @@ pub struct DeviceSignals { /// writes. Computed by the producing module: the built-in module uses a /// User-Agent-only heuristic, while the Fastly module strengthens it with /// the TLS and HTTP/2 signals. + /// + /// [`human_confidence`](Self::human_confidence) is the module's answer + /// about a person, which nothing reads yet. pub looks_like_browser: bool, + + /// How confident the device module is that a person is looking at the + /// page. A module that scores the request writes it, and every other + /// module leaves it `Unknown`. Nothing reads it yet, and it is not stored + /// in the identity graph with the other signals. + /// + /// Unlike [`looks_like_browser`](Self::looks_like_browser) this is not a + /// guess about the shape of a header. It is the module's answer from the + /// evidence it weighs, and a person the module cannot place answers + /// `Unknown` rather than being guessed at. + pub human_confidence: HumanConfidence, +} + +/// The confidence that a person is looking at the page. +/// +/// The four answers a scoring device module can give, with `Unknown` kept +/// apart from `Low` because a request alone cannot establish a person. +#[derive(Debug, Clone, Copy, Default, Eq, PartialEq)] +pub enum HumanConfidence { + /// High confidence that a person is viewing the page. + High, + + /// The evidence is mixed and the request cannot be placed either way. A + /// person may be viewing the page. + Medium, + + /// Almost certainly no person is directly involved in viewing the content. + Low, + + /// Not established. **This is not the same as `Low`.** A request alone + /// cannot establish a person, and treating it as `Low` would call every + /// first-time visitor a machine. + #[default] + Unknown, } impl DeviceSignals { @@ -77,6 +218,8 @@ impl DeviceSignals { h2_fp_hash: None, known_browser: None, looks_like_browser, + // Unknown, because the request alone cannot establish a person. + human_confidence: HumanConfidence::Unknown, } } @@ -109,6 +252,8 @@ impl DeviceSignals { h2_fp_hash, known_browser, looks_like_browser, + // As above: the request alone cannot establish this. + human_confidence: HumanConfidence::Unknown, } } @@ -134,8 +279,11 @@ impl DeviceSignals { /// These signals serve identity gating and bot detection, not bid enrichment. /// [`DeviceSignals`] deliberately carries only the coarse browser and bot /// classification the Edge Cookie gate needs, not a full device-detection -/// result such as make, model, OS version, or screen size. A richer device -/// model for the ad request is a separate concern. +/// result such as make, model, OS version, or screen size. Those are +/// [`DeviceAttributes`], answered by +/// [`advertising_attributes`](DeviceModule::advertising_attributes) on this +/// same trait, and the two answers stay separate types with separate +/// lifetimes. /// Uses `#[async_trait(?Send)]` for the same reason as /// [`PlatformHttpClient`](crate::platform::PlatformHttpClient): the trait /// object stays `Send + Sync` so it can be shared and run multi-threaded, @@ -170,6 +318,21 @@ pub trait DeviceModule: Send + Sync { fn required_permissions(&self) -> crate::permissions::PermissionSet { crate::permissions::PermissionSet::none() } + + /// Attributes for the bid request, when this module can resolve them. + /// + /// The call carries the permissions resolved for the request, so a module + /// whose use of the evidence needs a permission the reader has not given + /// answers `None` without using it. A backend the module registers during + /// the call has its transport timers capped at the auction's timeout, and + /// core drops a device type outside the list, a screen size that is not + /// positive and a blank string before it attaches the answer. + /// + /// The default returns nothing, so no existing module changes. A module + /// that resolves nothing returns `None` rather than an empty value. + async fn advertising_attributes(&self, _call: ModuleCall<'_>) -> Option { + None + } } /// The built-in device module, the default. @@ -794,6 +957,67 @@ mod tests { } } + /// A request alone cannot establish a person, so the signals read from + /// one say `Unknown`, which is not `Low`, and nothing calls a first-time + /// visitor a machine. + #[test] + fn a_request_alone_establishes_no_person() { + for signals in [ + DeviceSignals::derive_ua_only(CHROME_MAC_UA), + DeviceSignals::derive_ua_only("curl/8.0"), + DeviceSignals::derive(CHROME_MAC_UA, Some("t13d1516h2"), Some("abc123def456")), + DeviceSignals::derive("curl/8.0", None, None), + DeviceSignals::unknown(), + ] { + assert_eq!(signals.human_confidence, HumanConfidence::Unknown); + } + assert_eq!(HumanConfidence::default(), HumanConfidence::Unknown); + } + + #[test] + fn checked_keeps_only_the_values_a_bidder_can_read() { + for (device_type, kept) in [(0, None), (1, Some(1)), (8, Some(8)), (9, None)] { + let attributes = DeviceAttributes { + device_type: Some(device_type), + ..DeviceAttributes::default() + }; + assert_eq!( + attributes.checked("test").device_type, + kept, + "device type {device_type} should be kept only when it is in the list" + ); + } + for (size, kept) in [(-1, None), (0, None), (1, Some(1))] { + let attributes = DeviceAttributes { + screen_width: Some(size), + screen_height: Some(size), + ..DeviceAttributes::default() + }; + let checked = attributes.checked("test"); + assert_eq!( + (checked.screen_width, checked.screen_height), + (kept, kept), + "screen size {size} should be kept only when it is positive" + ); + } + for (text, kept) in [("", None), (" \t", None), ("Android", Some("Android"))] { + let attributes = DeviceAttributes { + make: Some(text.to_owned()), + model: Some(text.to_owned()), + os: Some(text.to_owned()), + os_version: Some(text.to_owned()), + ..DeviceAttributes::default() + }; + let checked = attributes.checked("test"); + let kept = kept.map(str::to_owned); + assert_eq!( + [checked.make, checked.model, checked.os, checked.os_version], + [kept.clone(), kept.clone(), kept.clone(), kept], + "{text:?} should be kept only when it is not blank" + ); + } + } + #[test] fn derive_ua_only_rejects_a_browser_ua_that_declares_a_bot() { // Newer crawlers send a full browser UA with a platform token; the bot diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index 9ce273799a..d9f53923dd 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -6118,6 +6118,9 @@ mod tests { h2_fp_hash: None, known_browser: None, looks_like_browser: false, + // Nothing here establishes whether a person is there, and + // `Unknown` is the value that says so rather than `Low`. + human_confidence: crate::ec::device::HumanConfidence::Unknown, } } @@ -6627,6 +6630,7 @@ mod tests { h2_fp_hash: None, known_browser: None, looks_like_browser: true, + human_confidence: crate::ec::device::HumanConfidence::Unknown, } } } diff --git a/crates/trusted-server-core/src/platform/test_support.rs b/crates/trusted-server-core/src/platform/test_support.rs index eb670dded3..7f39e096d8 100644 --- a/crates/trusted-server-core/src/platform/test_support.rs +++ b/crates/trusted-server-core/src/platform/test_support.rs @@ -1044,6 +1044,105 @@ pub fn build_services_with_geo(geo: Arc) -> RuntimeServices { .build() } +/// Build a [`RuntimeServices`] carrying a device module, so a test can drive +/// the device seam rather than setting the resolved attributes by hand. +/// +/// Setting the attributes directly would prove the `OpenRTB` mapping and +/// nothing about whether any module is ever asked, which is the half of the +/// chain that breaks. +pub fn noop_services_with_device_module( + device_module: Arc, +) -> RuntimeServices { + test_services_builder() + .config_store(Arc::new(NoopConfigStore)) + .secret_store(Arc::new(NoopSecretStore)) + .kv_store(Arc::new(edgezero_core::key_value_store::NoopKvStore)) + .backend(Arc::new(NoopBackend)) + .http_client(Arc::new(NoopHttpClient)) + .geo(Arc::new(NoopGeo)) + .client_info(ClientInfo::default()) + .build() + .with_device_module(device_module) +} + +/// The address of the service [`DetectionServiceDevice`] reaches. +pub const DETECTION_SERVICE_URL: &str = "https://detect.example/"; + +/// A device module standing in for one that reaches a detection service, so a +/// test can drive the call core makes to it through a real handler. +/// +/// For a reader whose permissions include every one in `needs`, it registers +/// a backend for the service with the default transport timers, through the +/// services its call carries, and answers `answer`. For any other reader it +/// answers `None` and uses nothing. It counts how often core asks it. +pub struct DetectionServiceDevice { + needs: crate::permissions::PermissionSet, + answer: Option, + asked: std::sync::atomic::AtomicUsize, +} + +impl DetectionServiceDevice { + pub fn new( + needs: crate::permissions::PermissionSet, + answer: Option, + ) -> Self { + Self { + needs, + answer, + asked: std::sync::atomic::AtomicUsize::new(0), + } + } + + /// How many times core asked the module for attributes. + pub fn asked(&self) -> usize { + self.asked.load(std::sync::atomic::Ordering::Relaxed) + } + + fn resolve( + &self, + permissions: Option<&crate::permissions::PermissionState>, + services: &RuntimeServices, + ) -> Option { + let agreed = + self.needs.is_empty() || permissions.is_some_and(|state| state.all_set(self.needs)); + if !agreed { + return None; + } + // A test reads the registration off the backend it supplied, so a + // refused one is not this module's to report. + let _ = crate::integrations::ensure_integration_backend( + services, + DETECTION_SERVICE_URL, + "detection-service", + None, + ); + self.answer.clone() + } +} + +#[async_trait::async_trait(?Send)] +impl crate::ec::device::DeviceModule for DetectionServiceDevice { + fn id(&self) -> &'static str { + "detection-service" + } + + async fn detect( + &self, + _call: crate::module_context::ModuleCall<'_>, + ) -> crate::ec::device::DeviceSignals { + crate::ec::device::DeviceSignals::unknown() + } + + async fn advertising_attributes( + &self, + call: crate::module_context::ModuleCall<'_>, + ) -> Option { + self.asked + .fetch_add(1, std::sync::atomic::Ordering::Relaxed); + call.inject(self, Self::resolve).ok().flatten() + } +} + /// Build a [`RuntimeServices`] carrying an Edge Cookie module, threaded the /// way a composition root threads the module it resolved, so a test can /// exercise the seam a vendor module reaches core through and check that the diff --git a/crates/trusted-server-core/src/platform/types.rs b/crates/trusted-server-core/src/platform/types.rs index 6ab20b3527..16fc73a608 100644 --- a/crates/trusted-server-core/src/platform/types.rs +++ b/crates/trusted-server-core/src/platform/types.rs @@ -440,6 +440,14 @@ impl RuntimeServices { } } + /// Returns a clone of this instance with backend registration replaced + /// by `backend`, for a caller that bounds what a module it calls may + /// register. + #[must_use] + pub(crate) fn with_backend(self, backend: Arc) -> Self { + Self { backend, ..self } + } + /// Returns a clone of this instance with the resolved Edge Cookie module /// replaced. /// diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 204ded6ae0..8025c8a587 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -5685,6 +5685,17 @@ pub async fn handle_publisher_request( path_label: "Server-side", }, ); + + // Fill the device object the bidder prices on. + crate::auction::formats::attach_device_attributes( + &mut auction_request, + &auction_client_request, + ec_context.permissions(), + settings, + services, + auction_timeout_ms, + ) + .await; let auction_context = AuctionContext { settings, request: &auction_client_request, @@ -6458,6 +6469,7 @@ fn apply_auction_eids_and_device( user_agent: None, ip: None, geo: None, + attributes: None, }); device.ip = client_ip; device.geo = targeting.geo.cloned(); @@ -6508,6 +6520,7 @@ pub(crate) fn build_auction_request( user_agent: Some(ua.to_string()), ip: None, geo: None, + attributes: None, }), site: Some(SiteInfo { domain: publisher_domain.to_owned(), @@ -8081,9 +8094,20 @@ pub async fn handle_page_bids( path_label: "Page-bids", }, ); + let timeout_ms = co_config .auction_timeout_ms .unwrap_or(settings.auction.timeout_ms); + // Fill the device object the bidder prices on. + crate::auction::formats::attach_device_attributes( + &mut auction_request, + &req, + ec_context.permissions(), + settings, + services, + timeout_ms, + ) + .await; let auction_context = AuctionContext { settings, request: &req, @@ -29537,9 +29561,11 @@ mod tests { use crate::auction::types::AuctionRequest; use crate::auction::{AuctionContext, AuctionOrchestrator}; use crate::creative_opportunities::{CreativeOpportunityFormat, CreativeOpportunitySlot}; + use crate::ec::device::{DeviceAttributes, DeviceModule}; use crate::ec::tests::{CANONICAL_COOKIE_VALUE, CANONICAL_KV_KEY, CanonicalizingModule}; + use crate::permissions::{Permission, PermissionSet, PermissionState}; use crate::platform::test_support::{ - NoopConfigStore, NoopGeo, NoopSecretStore, StubBackend, + DetectionServiceDevice, NoopConfigStore, NoopGeo, NoopSecretStore, StubBackend, }; use crate::platform::{ClientInfo, PlatformResponse}; use crate::test_support::tests::crate_test_settings_str; @@ -30337,5 +30363,210 @@ mod tests { assert_auction_used_the_canonical_row(&captured, &ec_context); } + + fn tablet() -> DeviceAttributes { + DeviceAttributes { + device_type: Some(5), + make: Some("ExampleCorp".to_owned()), + ..DeviceAttributes::default() + } + } + + /// A device module whose use needs limited-data ad selection. + fn device_needing_basic_ads() -> Arc { + Arc::new(DetectionServiceDevice::new( + PermissionSet::none().with(Permission::SelectBasicAds), + Some(tablet()), + )) + } + + /// The same context, with the reader also giving limited-data ad + /// selection. + fn selecting_basic_ads(ec_context: EcContext) -> EcContext { + let set = ec_context + .permissions() + .permissions() + .with(Permission::SelectBasicAds); + ec_context.with_permissions_for_test(PermissionState::new(set)) + } + + /// Runs the initial navigation for `ec_context` with `module` selected, + /// and returns the auction request it dispatched, if it dispatched one. + async fn navigation_with_device_module( + mut ec_context: EcContext, + module: &Arc, + ) -> Option { + let settings = settings_with_capturing_provider(); + let captured = Arc::new(Mutex::new(None)); + let orchestrator = orchestrator_capturing_request(&settings, &captured); + let stub = Arc::new(StubHttpClient::new()); + stub.push_response(200, b"ok".to_vec()); + let services = services_with( + Arc::clone(&stub) as Arc, + Arc::new(RecordingTelemetrySink::default()), + ) + .with_device_module(Arc::clone(module) as Arc); + let req = HttpRequest::builder() + .method(Method::GET) + .uri(format!("https://{EDGE_HOST}/2024/01/my-article/")) + .header(header::HOST, EDGE_HOST) + .header("sec-fetch-dest", "document") + .body(EdgeBody::empty()) + .expect("should build test request"); + + let registry = test_registry(&settings); + let _ = handle_publisher_request( + AppContext { + settings: &settings, + integration_registry: ®istry, + }, + &services, + None, + &mut ec_context, + AuctionDispatch { + orchestrator: &orchestrator, + slots: &article_slot(), + registry: None, + }, + req, + EdgeCacheHeader::SMaxageFallback, + ) + .await + .expect("should proxy publisher request"); + + captured + .lock() + .expect("should lock captured request") + .clone() + } + + /// Runs page-bids for `ec_context` with `module` selected, and returns + /// the auction request it dispatched, if it dispatched one. + async fn page_bids_with_device_module( + mut ec_context: EcContext, + module: &Arc, + ) -> Option { + let settings = settings_with_capturing_provider(); + let captured = Arc::new(Mutex::new(None)); + let orchestrator = orchestrator_capturing_request(&settings, &captured); + let services = services_with( + Arc::new(crate::platform::test_support::NoopHttpClient), + Arc::new(RecordingTelemetrySink::default()), + ) + .with_device_module(Arc::clone(module) as Arc); + let mut req = HttpRequest::builder() + .method(Method::GET) + .uri(format!( + "https://{EDGE_HOST}/_ts/page-bids?path=/2024/01/my-article/" + )) + .header(header::HOST, EDGE_HOST) + .body(EdgeBody::empty()) + .expect("should build test request"); + req.headers_mut().insert( + header::HeaderName::from_static("sec-fetch-site"), + HeaderValue::from_static("same-origin"), + ); + + let _ = handle_page_bids( + &settings, + &services, + None, + AuctionDispatch { + orchestrator: &orchestrator, + slots: &article_slot(), + registry: None, + }, + &mut ec_context, + req, + ) + .await + .expect("should return ok response"); + + captured + .lock() + .expect("should lock captured request") + .clone() + } + + #[tokio::test] + async fn initial_navigation_asks_the_device_module_with_the_readers_permissions() { + let module = device_needing_basic_ads(); + + let request = navigation_with_device_module( + selecting_basic_ads(consent_allowing_ec_context()), + &module, + ) + .await + .expect("should dispatch an auction request"); + + assert_eq!( + module.asked(), + 1, + "the navigation should ask the device module" + ); + assert_eq!( + request.device.and_then(|device| device.attributes), + Some(tablet()), + "a module whose permission the reader gave should describe the device to the bidder" + ); + } + + #[tokio::test] + async fn initial_navigation_does_not_ask_the_device_module_when_consent_denies_the_auction() + { + let module = device_needing_basic_ads(); + // An unknown jurisdiction with no TCF signal fails the auction gate. + let denied = EcContext::new_for_test(None, crate::consent::ConsentContext::default()); + + let request = navigation_with_device_module(selecting_basic_ads(denied), &module).await; + + assert!( + request.is_none(), + "a denied auction should dispatch nothing" + ); + assert_eq!( + module.asked(), + 0, + "a request on the consent-denied path should not reach the device module" + ); + } + + #[tokio::test] + async fn page_bids_asks_the_device_module_with_the_readers_permissions() { + let module = device_needing_basic_ads(); + + let request = page_bids_with_device_module( + selecting_basic_ads(consent_allowing_ec_context()), + &module, + ) + .await + .expect("should dispatch an auction request"); + + assert_eq!(module.asked(), 1, "page-bids should ask the device module"); + assert_eq!( + request.device.and_then(|device| device.attributes), + Some(tablet()), + "a module whose permission the reader gave should describe the device to the bidder" + ); + } + + #[tokio::test] + async fn page_bids_does_not_ask_the_device_module_when_consent_denies_the_auction() { + let module = device_needing_basic_ads(); + // An unknown jurisdiction with no TCF signal fails the auction gate. + let denied = EcContext::new_for_test(None, crate::consent::ConsentContext::default()); + + let request = page_bids_with_device_module(selecting_basic_ads(denied), &module).await; + + assert!( + request.is_none(), + "a denied auction should dispatch nothing" + ); + assert_eq!( + module.asked(), + 0, + "a request on the consent-denied path should not reach the device module" + ); + } } } diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index c5980a26b9..6a023c2100 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -844,7 +844,7 @@ These `TRUSTED_SERVER__` overrides apply where deployment tooling merges environ ## Device Configuration -Selects how a request is classified into the coarse device signals the Edge Cookie bot gate uses, mirroring the Edge Cookie module selection. These signals serve identifier gating and bot detection, not bid enrichment. +Selects how a request is classified into the coarse device signals the Edge Cookie bot gate uses, mirroring the Edge Cookie module selection. These signals serve identifier gating and bot detection. The same module may score how confident it is that a person is looking at the page, which nothing reads yet and which is not stored in the identity graph with the other signals, and may tell a bidder what the device is, which reaches the bid request's device object. Redirect middlewares that act on the score follow this set of changes. ### `[device]` diff --git a/docs/guide/integration-guide.md b/docs/guide/integration-guide.md index 04c387ec0b..639ac7e933 100644 --- a/docs/guide/integration-guide.md +++ b/docs/guide/integration-guide.md @@ -709,12 +709,13 @@ producer declares no permission is passed to every module. What a context carries depends on where the module is called. -| Module | Called through | What its context carries | -| ----------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Device | `DeviceModule::detect` | The request, the evidence, the settings and the services. No permission is resolved yet, so nothing whose use needs one | -| Permission signal | `SignalInput::call`, `valid_signal` and `tdls` | The request, the evidence, the connection, the consent, the settings and the services, and the location when its use needs no permission. The permissions are what the run produces | -| Edge Cookie | `EdgeCookieModule::generate` and `resolve_from_client` | Every row of the table above | -| Route | `IntegrationProxy::handle` | Every row of the table above except the evidence, because the route holds the request itself | +| Module | Called through | What its context carries | +| ----------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Device | `DeviceModule::detect` | The request, the evidence, the settings and the services. No permission is resolved yet, so nothing whose use needs one | +| Device | `DeviceModule::advertising_attributes` | The request, the evidence, the permissions, the settings and the services, for what a bidder is told about the device, being its type, make, model, operating system and screen. The default answers `None`, as a module does when the reader has not given a permission its use needs or it resolves nothing. A backend it registers waits no longer than the auction's timeout, and core drops a value a bidder cannot read | +| Permission signal | `SignalInput::call`, `valid_signal` and `tdls` | The request, the evidence, the connection, the consent, the settings and the services, and the location when its use needs no permission. The permissions are what the run produces | +| Edge Cookie | `EdgeCookieModule::generate` and `resolve_from_client` | Every row of the table above | +| Route | `IntegrationProxy::handle` | Every row of the table above except the evidence, because the route holds the request itself | A request preparer and a response finalizer are plain functions on the builder and are not handed the context. A preparer changes the request before