feat(mailbox): file inbound mail, and block the senders you mark as spam

.env.example

@@ -61,10 +61,19 @@ TASKDECK_STATUS_TOKEN=
 # unset it. A page includes contacts, API key hashes, and client webhook secrets.
 RELAY_TRANSFER_TOKEN=
 
-# Inbound mail. The drains that read these run under the `ingress` compose
-# profile and stay stopped until the queue URLs are set.
+# Inbound mail. Deployed environments start the drain unconditionally as
+# `relay-inbound-ingress`; it is this queue URL that decides whether there is
+# anything to drain, and an unset URL is a drain that waits rather than fails.
+# The local docker-compose stack has no inbound service.
 SQS_INBOUND_EMAIL_QUEUE_URL=
+# Optional. When set, a filed message is also announced on this topic. The
+# event names the stored message rather than carrying the body, so a consumer
+# reads it from Relay. When unset, inbound mail is only filed.
 INBOUND_EMAIL_EVENTS_TOPIC_ARN=
-INBOUND_EMAIL_IDEMPOTENCY_TABLE=
 INBOUND_EMAIL_ARTIFACT_PREFIX=processed/
+# Fallback routing, used only when no row in `inbound_addresses` matches.
+# `python manage.py provision_inbound_addresses` copies these into that table.
 INBOUND_EMAIL_ROUTES=
+# INBOUND_EMAIL_IDEMPOTENCY_TABLE is gone. Duplicate delivery is now suppressed
+# by a unique constraint on inbound_messages.message_id, so idempotency needs no
+# DynamoDB table and cannot half-succeed.

README.md

@@ -101,6 +101,37 @@ uv run python manage.py check
 uv run pytest
 ```
 
+## Receive mail
+
+Relay files accepted inbound mail in its own mailbox instead of publishing it
+onwards and keeping nothing. A message becomes an `inbound_messages` row:
+headers, a snippet and the SES verdicts in Postgres, bodies and attachments in
+object storage. Bodies stay in S3 because inbound mail is unbounded in size and
+retention, and the disk fill that once disqualified inbound in production was a
+host-disk problem.
+
+Create a receiving address in the console under **Configure > Receiving
+addresses**. That is the whole change: no Terraform edit, no apply, no redeploy.
+An address is a row, not a forwarding rule in another repository.
+
+**Mark as spam** on a message blocks the sender, and the block is checked
+*before* storage, so future mail from that sender is discarded rather than filed
+and filtered later. A discarded message is recorded as `blocked` with no body, so
+the block is auditable and unblocking does not resurrect something nobody read.
+Blocking the sender or the whole sending domain are separate buttons: one
+correspondent at a shared domain can be blocked alone.
+
+Until a domain's SES receipt rule points at Relay, nothing arrives. Addresses
+fall back to `INBOUND_EMAIL_ROUTES` when no row in `inbound_addresses` matches, so
+an estate that has created no addresses is not silently dark;
+`python manage.py provision_inbound_addresses` copies the environment routes into
+the table.
+
+The `inbound-email` SNS event still publishes when
+`INBOUND_EMAIL_EVENTS_TOPIC_ARN` is set, but it is smaller than it was: it names
+the stored message instead of carrying the body and every attachment. See
+[docs/worker-contracts.md](docs/worker-contracts.md).
+
 ## Deploy the sandbox
 
 Pushing `main` runs the test suite and deploys the complete release through

docs/design-system.md

@@ -8,9 +8,15 @@ We use our own lightweight CSS system. Do not add Bootstrap, Tailwind, React, a
 
 Use these references for UI elements and interaction quality:
 
+- DataOps design system (`../dataops/frontend/DESIGN_SYSTEM.md`) and the
+  Course Management Platform system it builds on: the primary visual
+  reference. GitHub Primer neutrals, restrained borders, compact row-based
+  content, content-width actions, and the CMP blue accent.
 - GitHub Primer: primary reference for product UI foundations, compact navigation, subdued surfaces, tables, forms, labels, focus states, and status treatment.
 - Shopify Polaris: secondary reference for admin workflow discipline, especially promoted filters vs advanced filters and clear page actions.
 - Resend: secondary reference for API docs, API keys, developer-facing settings, sparse examples, and transactional email vocabulary.
+- Dapier ("Operator's Ledger") is a sibling look, not the reference here; keep
+  Datamailer aligned with DataOps.
 
 Use Postmark only for product concepts, not visual styling:
 
@@ -73,20 +79,28 @@ All reusable styling must flow through tokens before page-specific CSS is added.
 
 ### Color Tokens
 
-Core surface and text tokens:
+Core surface and text tokens (values follow the DataOps/Primer palette; the
+light page canvas is white, the sidebar and secondary surfaces use `#f6f8fa`,
+and the action/link accent is CMP blue `#315f8f`):
 
 - `--dm-color-text`
+- `--dm-color-heading`
 - `--dm-color-muted`
+- `--dm-color-faint`
 - `--dm-color-border`
+- `--dm-color-border-strong`
 - `--dm-color-background`
 - `--dm-color-surface`
-- `--dm-color-surface-strong`
+- `--dm-color-surface-strong` (hover/active tone)
+- `--dm-color-accent-soft` (selected navigation and focus wash)
 - `--dm-color-focus`
 
 Action tokens:
 
-- `--dm-color-primary`
+- `--dm-color-primary` (filled controls; dark mode uses a lighter fill)
 - `--dm-color-primary-hover`
+- `--dm-color-link` (links and selected-nav text; stays readable in dark mode)
+- `--dm-color-link-hover`
 - `--dm-color-on-primary`
 
 State tokens:
@@ -101,8 +115,12 @@ State tokens:
 - `--dm-color-danger-hover`
 - `--dm-color-danger-surface`
 - `--dm-color-danger-border`
+- `--dm-color-info`
+- `--dm-color-info-surface`
+- `--dm-color-info-border`
 - `--dm-color-neutral`
 - `--dm-color-neutral-surface`
+- `--dm-color-neutral-border`
 
 Do not use raw hex values outside `:root` unless there is a documented exception.
 
@@ -124,14 +142,19 @@ Do not introduce one-off spacing values for page layout. If a repeated spacing n
 
 - `--dm-radius-sm`: controls, badges, nav items
 - `--dm-radius-md`: panels, empty states, table wrappers
+- `--dm-font-sans`: Inter (self-hosted, SIL OFL)
+- `--dm-font-mono`: IBM Plex Mono (self-hosted, SIL OFL); quantities, timings, and IDs render as data, not prose
 - `--dm-font-size-sm`: labels, help text, table headers
 - `--dm-font-size-base`: body and form controls
 - `--dm-font-size-lg`: section headings
 - `--dm-font-size-xl`: page headings
 - `--dm-control-height`: inputs and buttons
 - `--dm-content-width`: readable main-column width
+- `--dm-sidebar-width`: persistent sidebar width
 
 Letter spacing stays normal. Font sizes do not scale with viewport width.
+The shared component radius is 6px. Normal surfaces carry no shadow; shadows
+are reserved for overlays.
 
 ## Component Contract
 
@@ -176,12 +199,14 @@ Use these primitives before creating page-specific classes.
 
 ## Typography
 
-- Use a system font stack.
+- Use the self-hosted Inter stack (`--dm-font-sans`) for UI text and IBM Plex
+  Mono (`--dm-font-mono`) for quantities, timings, IDs, and code. Fonts must
+  not make third-party requests.
 - Keep letter spacing normal.
 - Do not scale font size with viewport width.
-- Page titles should be clear but not hero-sized.
-- Section headings should be compact.
-- Table and metadata text should remain readable at 14-15px.
+- Page titles are 32px semibold on desktop and 22px on mobile.
+- Section headings are compact (16px semibold).
+- Body and row text is 14px; table and metadata text may drop to 12-13px.
 - Help text should be short and muted.
 
 ## Color

docs/worker-contracts.md

@@ -43,49 +43,42 @@ Transactional and campaign work use separate queues and workers so campaign back
 
 ## `inbound-email` v1
 
-SES stores the raw MIME object under the private inbound bucket's `raw/` prefix. S3 sends an object-created notification to the `inbound-email` queue. The Datamailer worker parses the message and publishes one normalized SNS event for each configured recipient route.
+SES stores the raw MIME object under the private inbound bucket's `raw/` prefix. S3 sends an object-created notification to the `inbound-email` queue. Relay parses the message, files it in its own mailbox, and publishes one normalized SNS event naming the stored message. The published event is optional and off unless `INBOUND_EMAIL_EVENTS_TOPIC_ARN` is set.
+
+**The event is smaller than version 1 of this contract.** `body` and `attachments` are gone; the event carries `inbound_message_id` and a `raw_mime` reference instead, because the filed message is the record and a second full copy over SNS is the thing this path was built to stop doing. A consumer reading `body` or `attachments` must read them from Relay instead. See "The mailbox" below.
 
 The published event has this shape:
 
 ```json
 {
   "contract": "inbound-email",
   "version": 1,
-  "event_id": "sha256-of-message-id-and-route",
+  "event_id": "<sha256-of-message-id>",
   "event_type": "email.received",
   "occurred_at": "2026-07-12T07:30:01+00:00",
-  "route": "invoice",
+  "route": "invoice@mailer.dtcdev.click",
   "message_id": "<message-123@example.com>",
   "sender": {"header": "Billing <billing@example.com>", "addresses": ["billing@example.com"]},
-  "recipients": {
-    "to": "invoice@mailer.dtcdev.click",
-    "cc": "",
-    "addresses": ["invoice@mailer.dtcdev.click"],
-    "matched": ["invoice@mailer.dtcdev.click"]
-  },
+  "recipients": {"addresses": ["invoice@mailer.dtcdev.click"]},
   "subject": "July invoice",
-  "date": "Sun, 12 Jul 2026 09:30:00 +0200",
-  "body": {
-    "text": {"content_type": "text/plain", "value": "Attached", "size": 8},
-    "html": {"content_type": "text/html", "value": "<p>Attached</p>", "size": 15}
-  },
-  "attachments": [{
-    "filename": "invoice.pdf",
-    "content_type": "application/pdf",
-    "content_id": "",
-    "disposition": "attachment",
-    "size": 12345,
-    "s3": {"bucket": "private-inbound-bucket", "key": "processed/event-id/attachments/001-invoice.pdf"}
-  }],
+  "inbound_message_id": 41,
   "raw_mime": {"bucket": "private-inbound-bucket", "key": "raw/ses-object-key"}
 }
 ```
 
-Bodies up to `INBOUND_EMAIL_INLINE_BODY_MAX_BYTES` are included as text. Larger bodies, all attachments, and raw MIME are represented by private S3 references. Consumers need explicit read access to those object prefixes; Datamailer never publishes binary content through SNS.
+The raw MIME and the extracted body parts are private S3 references, never inline values. Relay does not publish binary content through SNS.
+
+## The mailbox
+
+A message that matches a receiving address is stored as an `inbound_messages` row: headers, a snippet and the SES verdicts in Postgres, bodies and attachments in object storage. `inbound_message_id` is that row. Bodies stay in S3 because inbound mail is unbounded in size and retention.
+
+Receiving addresses come from the `inbound_addresses` table, created in the console, and fall back to `INBOUND_EMAIL_ROUTES` when the table has no match -- so the sandbox keeps working on the environment variable and a new address needs no deploy. `python manage.py provision_inbound_addresses` copies the environment routes into the table.
+
+`blocked_senders` is checked before storage. A message from a blocked address or domain is filed as `blocked` with no body stored, is not announced over SNS, and is not a read message.
 
-Aliases are configured as exact address-to-route mappings in `INBOUND_EMAIL_ROUTES`, for example `invoice@mailer.dtcdev.click=invoice,todo@mailer.dtcdev.click=todo`. One message sent to aliases belonging to two routes produces two events. Duplicate delivery is suppressed by a DynamoDB conditional write on the SHA-256 of `Message-ID + route`. A failed SNS publish releases the claim so SQS can retry.
+Duplicate delivery is suppressed by a unique constraint on `message_id`, which is the same guarantee the previous DynamoDB conditional write gave and no longer needs a service. A message with no `Message-ID` is rejected, because a sender that omits it has nothing stable to deduplicate on. One message sent to two managed addresses produces one row, filed under the first address that matched.
 
-Raw MIME expires after `inbound_mail_retention_days` (60 days in the sandbox default). Extracted bodies and attachments expire after `inbound_email_artifact_retention_days` (14 days by default). Dapier must copy required artifacts to their system of record before expiry.
+Raw MIME expires after `inbound_mail_retention_days` (60 days in the sandbox default, 14 in Relay production). Extracted body parts expire on the same lifecycle. A message that aged out of the queue is still in the bucket. Any external consumer must copy what it needs to its own system of record before expiry.
 
 ## `transactional-email` v1
 

mailing/admin.py

@@ -3,6 +3,7 @@
 
 from mailing.models import (
     Audience,
+    BlockedSender,
     CallbackEndpoint,
     Campaign,
     CampaignRecipient,
@@ -14,6 +15,8 @@
     ContactTag,
     EmailEvent,
     EmailTemplate,
+    InboundAddress,
+    InboundMessage,
     MailchimpSync,
     MailchimpTagMapping,
     Organization,
@@ -390,3 +393,29 @@ class MailchimpSyncAdmin(admin.ModelAdmin):
         "last_error",
     )
     autocomplete_fields = ("contact", "client", "audience")
+
+
+@admin.register(InboundAddress)
+class InboundAddressAdmin(CreatedAtReadOnlyMixin, admin.ModelAdmin):
+    list_display = ("local_part", "domain", "is_active", "note", "created_at")
+    list_filter = ("is_active", "domain")
+    search_fields = ("local_part", "domain", "note")
+
+
+@admin.register(InboundMessage)
+class InboundMessageAdmin(CreatedAtReadOnlyMixin, admin.ModelAdmin):
+    list_display = ("sender_address", "subject", "recipient_address", "state", "created_at")
+    list_filter = ("state", "recipient_domain", "spam_verdict")
+    search_fields = ("subject", "snippet", "sender_address", "from_header", "recipient_address", "message_id")
+    readonly_fields = tuple(
+        field.name for field in InboundMessage._meta.fields if field.name not in {"id"}
+    )
+    autocomplete_fields = ("inbound_address",)
+
+
+@admin.register(BlockedSender)
+class BlockedSenderAdmin(CreatedAtReadOnlyMixin, admin.ModelAdmin):
+    list_display = ("value", "scope", "origin", "created_at")
+    list_filter = ("scope", "origin")
+    search_fields = ("value", "reason")
+    autocomplete_fields = ("origin_message",)

mailing/management/commands/provision_inbound_addresses.py

@@ -0,0 +1,56 @@
+"""Create receiving addresses from the environment routes.
+
+The cutover path. A domain that was already receiving mail through
+INBOUND_EMAIL_ROUTES has addresses the application does not know about, and
+without this command the first message after the receipt rule is repointed
+finds no managed address and is discarded as unmatched.
+
+Safe to run repeatedly. An address that already exists is left alone, so
+retiring an address is not undone by running this again -- which matters,
+because re-running it must not resurrect a deliberately retired address.
+
+    python manage.py provision_inbound_addresses
+    python manage.py provision_inbound_addresses --dry-run
+"""
+
+from django.conf import settings
+from django.core.management.base import BaseCommand
+
+from mailing.services.inbound_views import backfill_from_routes
+
+
+class Command(BaseCommand):
+    help = "Create a receiving address for every route in INBOUND_EMAIL_ROUTES."
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--dry-run",
+            action="store_true",
+            help="Report what would be created without writing anything.",
+        )
+
+    def handle(self, *args, **options):
+        routes = settings.INBOUND_EMAIL_ROUTES
+        if not routes:
+            self.stdout.write(
+                self.style.WARNING(
+                    "INBOUND_EMAIL_ROUTES is empty, so there is nothing to create. "
+                    "Receiving addresses are managed in the console under "
+                    "Configure > Receiving addresses."
+                )
+            )
+            return
+
+        if options["dry_run"]:
+            self.stdout.write(f"{len(routes)} route(s) configured; would create the missing ones:")
+            for address in sorted(routes):
+                self.stdout.write(f"  {address}")
+            return
+
+        created = backfill_from_routes(routes)
+        if not created:
+            self.stdout.write(f"All {len(routes)} route(s) already have a receiving address.")
+            return
+        for row in created:
+            self.stdout.write(self.style.SUCCESS(f"Created {row.address}"))
+        self.stdout.write(f"{len(created)} address(es) created from INBOUND_EMAIL_ROUTES.")