Skip to content

feat(api): CF Queue + DLQ for GitHub webhook ingestion - #337

Merged
Zach Dunn (zachdunn) merged 1 commit into
mainfrom
claude/webhook-queue-implementation-d43486
Jul 21, 2026
Merged

feat(api): CF Queue + DLQ for GitHub webhook ingestion#337
Zach Dunn (zachdunn) merged 1 commit into
mainfrom
claude/webhook-queue-implementation-d43486

Conversation

@zachdunn

Copy link
Copy Markdown
Member

Closes #287.

Why now

#287 deferred a CF Queue + DLQ because webhook work was purely self-healing KV
invalidation. That stopped being true: auto-promotion (#284 phase 3) and
comment reconciliation (#291/#332) now run fire-and-forget in ctx.waitUntil.
GitHub does not redeliver a 2xx delivery, so a failed promote on a
pull_request opened with no follow-up push is durable loss — exactly the
"revisit" trigger the issue defined.

Shape (matches releases/workers/webhooks prior art)

  • Compact events, not raw payloads. handleWebhook is split into a pure
    extractWebhookEvent (delivery → WebhookEvent { keys, promote?, reconcile? }
    or null) and processWebhookEvent (KV deletes + promote/reconcile). Only the
    compact event crosses the queue — GitHub payloads can exceed the 128 KB queue
    message cap. Gating (fork-head rejection, isReconcilableCommentEvent) stays
    in the pure extraction, so ordinary issue_comment traffic enqueues nothing.
  • Producer: the webhook route enqueues to uploads-github-webhook via the
    GITHUB_WEBHOOK_QUEUE binding after HMAC verification.
  • Consumer: new queue() export on the api worker
    (github-webhook-queue.ts). Per-message ack/retry; max_retries: 6
    uploads-github-webhook-dlq, whose consumer logs the compact event and acks.
  • Strict D1 lookup: promote/reconcile now use findRepoLinkStrict so a D1
    outage throws → retry, instead of the lenient findRepoLink reading as
    "repo not linked" and ack-dropping the event.
  • Degrade path: missing binding or a failed send() falls back to the
    pre-existing inline waitUntil path (errors swallowed + logged, 204 always) —
    tests and queueless local dev keep working unchanged.

Deploy prerequisite

Before the first deploy of this config:

wrangler queues create uploads-github-webhook
wrangler queues create uploads-github-webhook-dlq

Notes

  • No changeset on purpose: @uploads/api is a deploy-only ignored package, and
    a changeset against it poisons the release pipeline.
  • Tests: new producer/consumer/extraction suite; existing webhook,
    auto-promote, and reconcile suites pass unchanged on the inline path.
    Full run: 163 files / 2014 tests green; api typecheck clean
    (generated Env now includes GITHUB_WEBHOOK_QUEUE: Queue).

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
uploads-api 44e61ac Commit Preview URL

Branch Preview URL
Jul 21 2026, 12:45 PM

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (2)
  • coderabbit:review
  • review
🚫 Excluded labels (none allowed) (1)
  • wip

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d8d09024-d390-4a38-94ad-176208da9a54

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/webhook-queue-implementation-d43486

Comment @coderabbitai help to get the list of available commands.

@zachdunn
Zach Dunn (zachdunn) merged commit 348e0c8 into main Jul 21, 2026
4 checks passed
@zachdunn
Zach Dunn (zachdunn) deleted the claude/webhook-queue-implementation-d43486 branch July 21, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluate CF Queue + DLQ for GitHub webhook ingestion

1 participant