Add Lower Level (Complete) HTTP Request/Response API

[posborne]

Overview

Add low-level HTTP Request and Response wrappers for advanced use cases requiring direct control over HTTP primitives, streaming, and Fastly-specific features.

Context - What exists:

• ✅ WSGI adapter - Run Flask/Bottle apps unmodified
• ✅ requests facade - Client API for making backend calls (requests.get(), etc.)
• ❌ Low-level Request/Response API - This issue

What this enables:

• Streaming/proxying without buffering entire request/response bodies
• Access to Fastly-specific metadata (TLS fingerprints, client IP, compliance region)
• Request transformation (modify incoming request, send to backend)
• Cache control and surrogate key management
• Foundation for other SDK features (cache, security, image optimizer APIs)

When to use what:

Use Case	Use This	Not This
Run Flask app	WSGI adapter	This
Make API calls from app	requests.get()	This
Proxy/stream requests	This (Request/Response)	requests facade
Need TLS/IP metadata	This (Request.downstream)	WSGI
Cache override/surrogate keys	This	requests facade

WIT Interface

interface http-req {
  use types.{error};
  use http-types.{http-version};
  use http-resp.{response};
  use http-body.{body};
  use backend.{backend};

  resource request {
    new: static func() -> result<request, error>;
    set-cache-override: func(cache-override: cache-override) -> result<_, error>;
    get-header-names: func(max-len: u64, cursor: u32) -> result<tuple<string, option<u32>>, error>;
    get-header-value: func(name: string, max-len: u64) -> result<option<list<u8>>, error>;
    get-header-values: func(name: string, max-len: u64, cursor: u32) -> result<tuple<list<u8>, option<u32>>, error>;
    set-header-values: func(name: string, values: list<u8>) -> result<_, error>;
    get-method: func(max-len: u64) -> result<string, error>;
    set-method: func(method: string) -> result<_, error>;
    get-uri: func(max-len: u64) -> result<string, error>;
    set-uri: func(uri: string) -> result<_, error>;
    get-version: func() -> result<http-version, error>;
    set-version: func(version: http-version) -> result<_, error>;
    send: func(backend: borrow<backend>, body: body) -> result<response, error>;
  }
}

interface http-resp {
  use types.{error};
  use http-types.{http-version};
  use http-body.{body};

  resource response {
    new: static func() -> result<response, error>;
    get-status: func() -> result<u16, error>;
    set-status: func(status: u16) -> result<_, error>;
    get-version: func() -> result<http-version, error>;
    set-version: func(version: http-version) -> result<_, error>;
    get-header-names: func(max-len: u64, cursor: u32) -> result<tuple<string, option<u32>>, error>;
    get-header-value: func(name: string, max-len: u64) -> result<option<list<u8>>, error>;
    get-header-values: func(name: string, max-len: u64, cursor: u32) -> result<tuple<list<u8>, option<u32>>, error>;
    set-header-values: func(name: string, values: list<u8>) -> result<_, error>;
    send-downstream: func(body: body, streaming: bool) -> result<_, error>;
  }
}

interface http-downstream {
  use types.{ip-address, error};
  use http-req.{request};

  downstream-client-ip-addr: func(ds-request: borrow<request>) -> option<ip-address>;
  downstream-server-ip-addr: func(ds-request: borrow<request>) -> option<ip-address>;
  downstream-client-request-id: func(ds-request: borrow<request>, max-len: u64) -> result<string, error>;
  downstream-tls-cipher-openssl-name: func(ds-request: borrow<request>, max-len: u64) -> result<option<list<u8>>, error>;
  downstream-tls-protocol: func(ds-request: borrow<request>, max-len: u64) -> result<option<list<u8>>, error>;
  downstream-tls-ja3-md5: func(ds-request: borrow<request>) -> result<option<list<u8>>, error>;
  downstream-tls-ja4: func(ds-request: borrow<request>, max-len: u64) -> result<option<string>, error>;
}

WIT bindings: stubs/wit_world/imports/http_req.py, http_resp.py, http_downstream.py, http_body.py

API Design

Core types:

• Request - Wraps http_req.Request with Pythonic API (properties for method, uri, version)
• Response - Wraps http_resp.Response
• Headers - Dict-like interface for header manipulation
• Body - io.IOBase-compatible for streaming (use shutil.copyfileobj(), etc.)

Key features:

• Downstream metadata via request.downstream accessor:
  • client_ip() → IPv4Address | IPv6Address
  • tls_cipher(), tls_ja3_md5(), tls_ja4() → TLS fingerprints
  • compliance_region() → GDPR/data residency region
• Cache control: request.set_cache_override(ttl=..., surrogate_key=...)
• Backend requests: request.send(backend, body) → Response
• Streaming: Bodies are file-like objects, work with stdlib

Example - Proxying with transformation:

def handle(incoming_req, incoming_body):
    # Access metadata
    client_ip = incoming_req.downstream.client_ip()
    
    # Transform request
    incoming_req.headers['X-Forwarded-For'] = str(client_ip)
    incoming_req.set_cache_override(ttl=3600, surrogate_key='user-data')
    
    # Send to backend (streaming)
    response = incoming_req.send('origin', incoming_body)
    return response

Integration with Existing SDK

WSGI Adapter

Can wrap incoming WIT request in Request object and expose via environ['fastly.request']:

from flask import Flask, request

@app.route("/api/data")
def get_data():
    fastly_req = request.environ['fastly.request']
    client_ip = fastly_req.downstream.client_ip()
    return {"client_ip": str(client_ip)}

Requests Facade

The requests.get() / requests.post() API is for making outgoing requests (client use case). It should NOT be extended for proxying - that's what this low-level API is for.

Clear separation:

• Client pattern (outgoing): Use requests.get(url) - builds request from scratch
• Server/Proxy pattern (incoming): Use Request/Response API - transforms received request

The requests facade could use Request wrappers internally but public API stays the same.

Cross-SDK Comparison:

• Rust: Request/Response types wrapping HTTP standard types. Methods for headers, body streams, methods, URLs. Rich builder patterns. Strongly typed.
• Go: Standard *http.Request/*http.Response from stdlib with Fastly extensions via embedded fields/methods.
• JS: Standard Request/Response from Fetch API with Fastly extensions.

Recommended approach: Python should provide standard library-compatible types (similar to requests or urllib) while adding Fastly-specific extensions.

Viceroy Testing

Viceroy supports HTTP request/response handling with full metadata access in tests. The @on_viceroy decorator can provide synthetic requests with headers, bodies, and metadata.

Tests can verify:

• Request/response creation and manipulation
• Header handling (case-insensitive, multi-value)
• Body streaming and reading
• Downstream metadata access (may have defaults for TLS info in Viceroy)

HTTP operations are well-supported in Viceroy testing.

Reference

• Fastly HTTP Request Documentation