fix(tern): bound every outbound call on the drive path

[aparajon]

Why this matters

A drive goroutine runs on an unbounded context while a separate heartbeat goroutine keeps renewing the apply lease. If one outbound call hangs on a black-holed connection (dropped packets, dead NAT entry, half-open TCP), the driver blocks forever — and because the heartbeat keeps the lease fresh, no peer ever sees a stale lease and reclaims the apply. One dead connection wedges the apply until a human intervenes.

What it does

Bounds all three outbound legs of the drive path; the lease/heartbeat design is untouched.

• Tern gRPC: client keepalive params (kept within gRPC's default server-side enforcement so embedder-attached servers don't GOAWAY the channel) plus a unary interceptor that applies a default per-RPC deadline when the caller's context has none — 60s for polling/control RPCs, 5m for RPCs that do synchronous engine or live-schema work (Apply, Plan, PlanDiff, PullSchema, Volume, Revert, SkipRevert). A caller-supplied deadline always wins. A DEADLINE_EXCEEDED on Apply is handled as an ambiguous dispatch outcome and recovered via the idempotency key.
• PlanetScale API: the SDK client is built on a timeout-bearing HTTP client (5m — deploy request creation and branch schema diffs can legitimately take minutes), installed after caller options so no option can displace the bound or the service-token auth. The raw-HTTP throttle endpoint moves off http.DefaultClient onto its own bounded client.
• MySQL connections: DSN normalization fills in connect (30s), read (30m), and write (1m) timeouts, keeping any explicit DSN values. The read timeout applies per network read, so streamed result sets are unaffected; it is deliberately long because it must outlast legitimate silent waits (advisory-lock waits, DDL statements). Spirit opens its own connections and is unaffected.
• All bounds are generous named constants: the invariant is "no outbound call is unbounded", not "calls are fast".

Before / after

driver goroutine                    heartbeat goroutine
  Progress RPC ─▶ (black hole,        renew lease ✓
                  hangs forever)      renew lease ✓ ...
  never returns ❌                    lease never stale ⛔ no peer reclaims
                                      apply wedged until a human intervenes

driver goroutine                    heartbeat goroutine
  Progress RPC ─▶ DEADLINE_EXCEEDED at the per-RPC bound ✅
  drive errors out, owner exits       heartbeat stops
                                      lease goes stale ─▶ peer reclaims and resumes ✅

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR ensures SchemaBot’s “drive path” cannot hang forever on black-holed network connections by adding explicit, generous bounds to outbound calls across Tern gRPC, PlanetScale HTTP API, and SchemaBot-managed MySQL connections, while keeping the existing lease/heartbeat design intact.

Changes:

• Tern gRPC client: add client keepalive parameters and a unary interceptor that applies per-method default RPC deadlines when the caller provides none.
• PlanetScale client: ensure all SDK and raw-HTTP throttle requests use a timeout-bounded HTTP client (and stop using http.DefaultClient).
• MySQL DSN normalization: fill in connect/read/write timeouts (without overriding non-zero DSN values) and return the normalized DSN string consistently.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pkg/tern/grpc_client.go	Adds keepalive params and a default-deadline unary interceptor for Tern unary RPCs.
pkg/tern/grpc_client_test.go	Adds tests proving default deadlines apply only when the caller provides no deadline, and validates keepalive settings.
pkg/psclient/client.go	Constructs PlanetScale SDK + throttle HTTP clients with bounded timeouts and pooled transport.
pkg/psclient/client_test.go	Adds tests validating bounded HTTP client configuration and throttle timeout behavior.
pkg/mysqlconn/mysqlconn.go	Normalizes DSNs to include connect/read/write timeouts (and returns formatted DSNs consistently).
pkg/mysqlconn/mysqlconn_test.go	Updates DSN normalization expectations and adds coverage for preserving explicit timeouts.
go.mod	Promotes github.com/hashicorp/go-cleanhttp to a direct dependency for pooled HTTP transport construction.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.