Optimize dev proxy upstream performance

[aram356]

Closes #895

Summary

• Add repeatable dev-proxy performance metrics and benchmark workloads for sequential, matched-concurrency, saturation, DNS, parser, and remote-latency scenarios.
• Add a bounded HTTP/1.1 upstream connection manager with safe streaming lease return, cancellation, stale-connection retry, connector ownership, and bounded shutdown.
• Retain evidence-backed CONNECT-head parsing and DNS caching while rejecting HTTP/2 and TCP_NODELAY experiments that did not satisfy their retention gates.
• Precompute upstream identities and preserve TLS, Host, --resolve, Basic authentication, hop-by-hop sanitation, and blind-tunnel isolation.

Final review corrections

• Count complete responses as successful independently from whether their upstream connection is reusable, including already-empty and Connection: close responses.
• Force stale-readiness recovery through a fresh reservation instead of selecting another idle sender.
• Remove the rejected HTTP/2 application-mode key state; retained TLS configurations advertise only HTTP/1.1.
• Forward response trailers before lease reconciliation and preserve every downstream trailer declaration.
• Backfill connector cancellation/unwind, early-response, browser-cancellation, truncated-body, trailer, credential-isolation, insecure-warning, forwarding-capacity, over-read, shutdown-order, and origin-key integration coverage.

Performance results

• Sequential TLS: 100 connections and handshakes reduced to 1, with a 59.7% median duration improvement in the recorded alternating runs.
• DNS churn workload: 13.6% median improvement with one miss and 99 cache hits.
• Concurrency-20 saturation: bounded to six upstream connections and handshakes.
• Remote saturation diagnostic: six pooled connections reduce connection and handshake churn by 94%; the six-connection safety cap intentionally trades 9.0% duration versus the unbounded baseline at concurrency 20.

Full measurements and experiment decisions are recorded in docs/superpowers/implementation-notes/2026-07-10-dev-proxy-performance.md.

Verification

• cargo fmt --all -- --check
• CLI cargo check --tests and Clippy with -D warnings
• ./scripts/test-cli.sh — 147 unit + 26 E2E tests
• All six ignored proxy_perf workloads with one test thread
• Fastly, Axum, Cloudflare, Spin, and 13 cross-adapter parity tests
• Native/Wasm Clippy matrices for Fastly, Axum, Cloudflare, and Spin with zero warnings
• JS build + 405 Vitest tests + JS/docs Prettier checks

[prk-Jr]

Summary

Adds a bounded, reusable HTTP/1.1 upstream connection manager (plus DNS caching, precomputed upstream identities, and a repeatable benchmark suite) to ts dev proxy. The design is careful and thoroughly tested: the manager is an actor with a bounded ordinary lane + unbounded control lane so lifecycle events (cancel / driver-closed / shutdown) can never deadlock behind a saturated acquire lane, and capacity reconciliation is airtight — every path leads back to release(id) (reservation drop → ConnectFailed, driver-guard drop → DriverClosed, shutdown connector abort → finish error → reservation drop). The AcquireTicket single-terminal-transition invariant closes the resolve/cancel race. No blocking issues found; approving.

Non-blocking

♻️ refactor

• Crypto provider rebuilt per call: NoVerifier::supported_verify_schemes calls aws_lc_rs::default_provider() (a non-trivial clone) on every invocation — cache in a OnceLock. --insecure-only path, cosmetic. (connect.rs:282)

🌱 seedling

• Non-idempotent 502 on stale reuse: the send_request retry is gated on provably-empty idempotent requests, so a POST racing an upstream idle-close returns 502 rather than replaying. This is the correct safe behavior; flagged only so it's a documented trade-off. (upstream/mod.rs:246)

⛏ nitpick

• Doc comments on new pub items: several new public items (ProxyMetrics::record_*, PooledResponseBody::new, etc.) lack doc comments. CLAUDE.md asks for one per public item; missing_docs isn't enforced here and it matches surrounding CLI style, so optional.

👍 praise

• Lifecycle & concurrency design: the can_reuse matrix + exhaustive test, DNS miss-coalescing that survives owner cancellation without poisoning the in-flight key, the trailer-frame deferral in PooledResponseBody, and the priority_shutdown_overtakes_a_saturated_ordinary_lane test are all excellent.
• Security hardening: Basic-auth refused on non-loopback binds, hop-by-hop + Connection-token stripping, Forwarded / Fastly-SSL stripped and authoritatively re-stamped, strict host validation before PAC / Host-header embedding, and redacted BasicAuth Debug.

CI Status

• fmt: PASS
• clippy (fastly/axum/cloudflare/spin native+wasm): PASS
• rust tests (fastly/axum/cloudflare/spin/CLI/parity): PASS
• js build + vitest + docs/ts prettier: PASS

[prk-Jr]

♻️ refactor — aws_lc_rs::default_provider() builds a fresh CryptoProvider (cloning every cipher-suite/signature algorithm) on each call, and rustls can invoke supported_verify_schemes() per handshake. Consider caching the scheme list once:

fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
    static SCHEMES: std::sync::OnceLock<Vec<rustls::SignatureScheme>> = std::sync::OnceLock::new();
    SCHEMES
        .get_or_init(|| {
            rustls::crypto::aws_lc_rs::default_provider()
                .signature_verification_algorithms
                .supported_schemes()
        })
        .clone()
}

Only reached in --insecure mode (loud-warned), so purely cosmetic — but on-theme for a PR about avoiding handshake cost.

[prk-Jr]

🌱 seedling — The stale-connection send_request retry is intentionally gated on replay_template.is_some() (provably-empty GET/HEAD/OPTIONS). A POST that races an upstream idle-close after ready() already passed therefore returns 502 with no replay. This is the correct safe choice — replaying a non-idempotent request could double-submit — noting it only so the trade-off is documented and deliberate rather than read as an oversight.

[prk-Jr]

👍 praise — The can_reuse matrix (response complete AND upload complete AND no close-intent AND driver alive) is exactly the right set of conditions, and counting a complete-but-unpoolable response as a success independent of reuse is the correct semantics. The exhaustive test alongside it makes the invariant hard to regress.

[ChristianPavilonis]

Summary

Approved with seven actionable follow-ups noted inline. The pooled HTTP/1 architecture and coverage are strong; the highest-priority changes are preserving request trailers end-to-end and tying connector-capacity release to actual task termination.

[ChristianPavilonis]

🔧 P1 — Preserve request-trailer semantics on the upstream leg

rewrite_headers removes both Trailer and TE through strip_hop_by_hop without regenerating them. RequestUploadBody still forwards trailer frames, but Hyper's HTTP/1 encoder only serializes fields declared by the outgoing Trailer header; otherwise it discards them. A chunked upload can therefore arrive intact except for silently lost trailer fields, and removing TE: trailers prevents compliant origins from knowing the proxy accepts response trailers.

Capture the browser's trailer declarations and TE: trailers before sanitation, then regenerate safe upstream-leg Trailer and TE/Connection: TE metadata. Please add an E2E test proving request trailers reach the origin and a compliant origin can conditionally return response trailers.

[ChristianPavilonis]

🔧 P1 — Release connector capacity only after cancellation completes

Dropping PendingConnection only calls AbortHandle::abort(), while the independently dropped Reservation immediately sends ConnectFailed; the manager can then release capacity before Tokio has actually dropped the DNS/TCP/TLS task. Replacement admission or shutdown acknowledgement can occur while the old connector remains live, violating the hard six-per-origin/64-global bound. The current test yields until abort.is_finished() before reacquiring, which masks this ordering.

Transfer reconciliation ownership to an exactly-once guard inside the connector task, and release the reservation only when that task actually exits (disarming the connector guard after successful driver registration). Add a test proving replacement admission and shutdown remain blocked until cancellation completion.

[ChristianPavilonis]

🔧 P2 — Continue handoff after a waiter loses the cancellation race

A waiter can cancel after this position check but before ticket.resolve(). The actor then removes that waiter, stores the returned connection as idle, and never checks the next eligible same-origin waiter. With no further connection event, that waiter can time out after 30 seconds while a reusable connection sits idle for 60 seconds; newer requests may also take it first.

Make handoff loop over matching waiters until delivery succeeds or none remain. If the oneshot send fails, recover the lease payload and retry the next waiter. A deterministic two-waiter cancellation-race test would protect the FIFO invariant.

[ChristianPavilonis]

🔧 P2 — Count setup and acquisition failures as failed requests

Acquisition errors here and connector errors at lines 209–212 return before record_request_failed; stale-replacement acquisition/connection has the same omission. forward_request converts these errors to visible 502 responses, but requests_failed remains zero, so the new diagnostics under-report common DNS/TCP/TLS/pool failures. Failed acquisition latency is also omitted.

Ensure every terminal UpstreamClient::send error records exactly one failure—preferably with a scoped outcome guard or centralized error boundary—and record acquisition duration on both success and failure. Add an unreachable-loopback-origin E2E assertion for 502, zero completions, and one failure.

[ChristianPavilonis]

🔧 P2 — Retain the completed lookup when all original receivers cancel

watch::Sender::send does not retain the value when there are no receivers. A new caller can subscribe to the still-present Loading entry between this failed send and cache-state replacement, then observe channel closure and return Interrupted even though the lookup completed.

Suggested change

	let _ = signal.send(Some(shared.clone()));
	signal.send_replace(Some(shared.clone()));

Please add a multithreaded regression test where every initial waiter cancels and a late subscriber arrives during completion.

[ChristianPavilonis]

🔧 P2 — Make the named remote benchmark self-validating

This test labels output http1_remote_model_30_30_25, but the 30 ms connect and TLS delays are enabled only when TS_PERF_VARIANT starts with remote_. Under the default pooled variant, the named remote workload silently runs with only the 25 ms response delay, allowing mislabeled benchmark evidence.

Require/assert remote_baseline or remote_pooled for this test, or construct typed remote options directly. Please also document the exact alternating commands used for the recorded remote results.

[ChristianPavilonis]

🔧 P2 — Include the latency values advertised by the troubleshooting guide

The debug summary prints only sample counts for DNS, TLS, HTTP handshake, request-to-headers, and CA-mint timing. It emits latency buckets only for parsing, TCP connect, pool acquisition, and queue wait. Consequently the guide's claim that this summary can distinguish DNS and TLS latency is not currently true.

Include totals or histogram buckets for every advertised latency phase, or narrow the documentation claim to the phases actually emitted.