Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 9 additions & 25 deletions .github/justfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
# CI job bodies: each workflow job runs exactly one `gha::` recipe, so
# `just ci` is exactly CI and the workflow files carry no logic beyond
# environment setup (checkout, toolchains, caches). Step-level rationale
# lives on the root-justfile recipes these compose.
# CI job bodies; recipe-specific rationale lives in the root justfile.

set working-directory := '..'

Expand All @@ -24,9 +21,7 @@ _step recipe:
just {{recipe}}
fi

# Like _step, but failures are tolerated: the CI-side replacement for the
# workflow-level `continue-on-error` the collapsed steps used to carry. A
# failure surfaces as a ::warning annotation, never as a job failure.
# Like _step, but report failures as warnings rather than failing the job.
[private]
_step-tolerated recipe:
#!/usr/bin/env bash
Expand All @@ -43,16 +38,9 @@ _step-tolerated recipe:
just {{recipe}} || echo "just {{recipe}}: FAILED (tolerated in CI; see the recipe's comment)"
fi

# The required per-push/PR matrix job (ci.yml `core`).
# `version-guard-local` runs first, unconditionally — label-free/event-free
# tree checks (lockstep agreement, monotonicity, the protocol byte-identity
# tear check) that need no PR context, so a direct push to main is covered
# too (the #232 gap: `version-guard-pr` no-ops without PR_NUMBER, and #232
# only got caught because it happened to go through a `pull_request` run).
# `version-guard-pr` runs right after and costs seconds: a versioning
# mistake is cheap to hear about before a 40-minute matrix, and the recipe
# no-ops outside pull_request runs (no PR_NUMBER), so pushes and `just ci`
# are unaffected. ci.yml supplies PR_NUMBER / PR_BASE_SHA / GH_TOKEN.
# Context-free checks precede PR checks (no-op without PR_NUMBER).
# ci.yml supplies PR_NUMBER / PR_BASE_SHA / GH_TOKEN.
# Required core matrix.
core:
@just gha::_step version-guard-local
@just gha::_step version-guard-pr
Expand All @@ -76,9 +64,8 @@ core:
@just gha::_step conformance
@just gha::_step sched-seeds

# The post-merge browser job (ci.yml `browser`; gates the prerelease).
# The worker / shared-worker realm rows (issue #129) reuse each engine's
# expectation verbatim — a delta is a realm leak and fails naming the realm.
# Page and worker realms share per-engine expectations; WebKit is best-effort.
# Post-merge browser job; gates prerelease artifacts, not PR mergeability.
browser:
@just gha::_step shim
@just gha::_step corpus
Expand All @@ -99,18 +86,15 @@ browser:
@just gha::_step-tolerated "browsers-install-webkit --with-deps"
@just gha::_step-tolerated "browser-lane webkit"

# Never gates: the drivers themselves exit 0 on deviations, 2 on
# infrastructure failure.
# The findings-only canary lanes (canary.yml, x64 leg).
# Findings-only x64 canaries: deviations exit 0; infrastructure failures exit 2.
canary:
@just gha::_step shim
@just gha::_step corpus
@just gha::_step "shell-lane sm-nightly --json sm-results.json"
@just gha::_step "shell-lane jsc-trunk --json jsc-results.json"
@just gha::_step "deno-canary --json deno-canary-results.json"

# JSC trunk has no arm64 channel — the two lanes with arm64 coverage.
# The findings-only canary lanes (canary.yml, arm64 leg).
# Findings-only arm64 canaries; JSC has no arm64 channel.
canary-arm:
@just gha::_step shim
@just gha::_step corpus
Expand Down
307 changes: 119 additions & 188 deletions AGENTS.md

Large diffs are not rendered by default.

255 changes: 119 additions & 136 deletions README.md

Large diffs are not rendered by default.

229 changes: 98 additions & 131 deletions bench/boundary/README.md

Large diffs are not rendered by default.

140 changes: 63 additions & 77 deletions contracts/descriptor-ir.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Contract: CABI Descriptor IR

The **descriptor IR** is the type/options information that drives host-boundary
lift/lower — docs/architecture.md §8's "one IR, two executors." Producers: the
translator shim (inside `plan.json` `types` / `canonicalOptions` tables) and
tests. Consumers: the v1 interpreter (`runtime/src/cabi/`), the future
generated-JS executor, and the world-digest computation.
The **descriptor IR** supplies types and canonical options for host-boundary
lift/lower. The translator emits it in the plan's `types` and `canonicalOptions`
tables. The runtime interpreter (`runtime/src/cabi/`) and world-digest
computation consume it. A generated-JS executor is planned, not implemented; see
[architecture §8](../docs/architecture.md#8-performance-strategy).

The normative in-memory model is `runtime/src/cabi/types.ts`; this document
defines its meaning and its JSON wire form inside the plan. Known
wire↔memory divergences are pinned below; any other divergence is a bug.
defines its meaning and its JSON wire form inside the plan. Known wire↔memory
divergences are pinned below; any other divergence is a bug.

## Value type model

Expand All @@ -19,102 +19,88 @@ structurally). Kinds:
`f32`, `f64`, `char`, `string`
- `list` (`element`, optional fixed `length`), `record` (`fields`:
`{label, type}[]`), `tuple` (`elements`), `variant` (`cases`:
`{label, type|null}[]`), `enum` (`labels`), `option` (`type`),
`result` (`ok|null`, `err|null`), `flags` (`labels`), `map` (`key`,
`value` — despecializes to `list<record{0,1}>` per the reference)
- Handles: `own` / `borrow` (`resource`: index into the plan's
`resourceTables`)
`{label, type|null}[]`), `enum` (`labels`), `option` (`type`), `result`
(`ok|null`, `err|null`), `flags` (`labels`), `map` (`key`, `value` —
despecializes to `list<record{0,1}>` per the reference)
- Handles: `own` / `borrow` (`resource`: index into the plan's `resourceTables`)
- Async: `stream` / `future` (`element|null`), `error-context`

Specialized forms are preserved (tuple/enum/option/result/flags/map are not
pre-despecialized in the IR); `despecialize` is defined once, in the runtime,
mirroring `definitions.py`. Labels remain strings (interning is a
measured-need optimization).
mirroring `definitions.py`. Labels remain strings.

`FuncType` is `{ params: {label, type}[], results: ValType[], async?: bool }`.
Wire function declarations are
`{ kind: "func", params: {label, type}[], results: ValType[], async: bool }`.

Pinned wire↔memory divergences (the plan loader maps): wire `result.err` ↔
types.ts `result.error`; wire `FuncType.params` are labeled
`{label, type}[]` while types.ts drops names — names live on the wire and in
bindgen, not in the interpreter's hot path.
types.ts `result.error`; wire `FuncType.params` are labeled `{label, type}[]`
while types.ts drops names — names live on the wire and in bindgen, not in the
interpreter's hot path.

## Canonical options

Per lifted/lowered function, referencing plan tables by index (see
plan-format.md): `stringEncoding` (`utf8` | `utf16` | `latin1+utf16`),
`memory?`, `realloc?`, `postReturn?`, `callback?`, `async`, `cancellable`,
and the expected flat `coreType` (`{params, results}` of `i32|i64|f32|f64`).
This mirrors `wasmtime_environ::component::CanonicalOptions` minus
runtime-irrelevant fields; `data_model` is fixed to linear memory (the GC
data model is rejected by the shim).
plan-format.md): `stringEncoding` (`utf8` | `utf16` | `latin1+utf16`), nullable
`memory`, `realloc`, `postReturn`, `callback`, plus `async`, `cancellable`, and
the expected flat `coreType` (`{params, results}` of `i32|i64|f32|f64`). This
mirrors `wasmtime_environ::component::CanonicalOptions` minus runtime-irrelevant
fields; `data_model` is fixed to linear memory (the GC data model is rejected by
the shim).

## Flattening

The plan does **not** precompute flat lane lists. Executors compute flattening
from `ValType` via the shared rules in `runtime/src/cabi/flatten.ts`, which is
tested against fixtures generated from `definitions.py` (`flatten_functype`,
MAX_FLAT_PARAMS=16, MAX_FLAT_RESULTS=1, async variants with their own
limits, spill-to-memory rules). Rationale: one implementation of the trickiest
rules, differentially anchored to the executable spec; smaller plans; less
shim logic. The consistency check between computed flattening and the
options' `coreType` is an instantiate-time assertion — validated across the
whole fixture corpus. (Precomputed lanes can be added later as a pure
optimization without changing this contract's semantics.)
MAX_FLAT_PARAMS=16, MAX_FLAT_RESULTS=1, async variants with their own limits,
spill-to-memory rules). Rationale: one implementation of the trickiest rules,
differentially anchored to the executable spec; smaller plans; less shim logic.
The consistency check between computed flattening and the options' `coreType` is
an instantiate-time assertion — validated across the fixture corpus.

## Host value shapes

Host-facing value conventions are `contracts/embedder-api.md`'s territory
(implemented by the bindgen-generated layer). The raw executor boundary
mirrors `definitions.py`'s value *semantics*; its *representation* is this
implementation's to choose, and diverges deliberately where measurement
justifies it (docs/architecture.md §1 sanctions exactly this — "parity
means functional parity, not behavioral identity", and lists JS-native
host value shapes among the divergences). The shapes are: variant as
`{kind: label, value: payload}`, enum as `{kind: label, value: null}`,
option as `{kind: "none", value: null} / {kind: "some", value: v}`, result
error kind `"error"`, tuple as despecialized record. It is an internal
surface with no stability promise.

Integer lanes wrap mod 2⁶⁴ at the raw boundary, matching
definitions.py's `% 2**64`; host-side range *asserts* (host-precondition
errors, not traps) exist only on the scalar `storeInt` path. NaN handling,
lane widening/padding (i64 lanes as `bigint`, `0n` padding), and
latin1(windows-1252) details follow the decisions recorded in
`runtime/README.md` and docs/architecture.md §7.

The variant family carries its case in a `kind` property rather than as the
object's sole key — the one place the representation departs from the
reference's dicts. `definitions.py` uses a single-key mapping; that form
cost a computed-key literal here (a distinct hidden class per case label,
so every variant-reading site went megamorphic) plus an `Object.keys()`
allocation at each end to read one key. `value` is always present, `null`
for a payload-free case: omitting it to match the host layer exactly was
measured slower, because the producer site then emits two shapes instead
of one. Both findings are measured on `bench/boundary`'s compound-element
lanes; see issue #261 and the PR that landed this for the numbers and
method. The property names deliberately match `contracts/embedder-api.md`'s
host variant shape, but **the two are not interchangeable** — that
document's "Implementation strategy" enumerates every way they still
differ.
The runtime facade implements [embedder-api.md](embedder-api.md); bindgen emits
types and typed wrappers for that facade. The raw executor preserves the
reference's value semantics in these internal shapes: variant as
`{kind: label, value: payload}`, enum as `{kind: label, value: null}`, option as
`{kind: "none", value: null} / {kind: "some", value: v}`, result error kind
`"error"`, tuple as despecialized record. It is an internal surface with no
stability promise.

Core integer lanes are normalized to unsigned values of their width before
lifting; narrower component integers wrap according to `definitions.py`. i64
values use `bigint`, including `0n` padding. The public facade validates host
values before lowering. At the raw internal boundary, scalar `storeInt` asserts
range while bulk numeric stores wrap; neither is a substitute for facade
validation. NaNs are canonicalized. Latin-1 decoding uses the ISO-8859-1 byte
mapping, not WHATWG `TextDecoder`'s Windows-1252 alias. See
[architecture §7](../docs/architecture.md#7-canonical-abi-decisions).

Fixed `kind`/`value` properties avoid per-case object shapes and key
enumeration. `value` is always present, with `null` for a payloadless case.
Despite shared property names, raw and facade values are not interchangeable;
see [the adaptation table](embedder-api.md#implementation-strategy).

## Trap discipline

Lift/lower failures raise the runtime's `ComponentTrap` (not arbitrary
`Error`s), with the trap conditions of `definitions.py` (`trap_if`) as the
authority. Executors must produce the same trap/no-trap verdict for the same
inputs — this is part of the differential-testing contract.
Guest lift/lower violations raise `Trap`, following the reference's `trap_if`
conditions. Internal/precondition assertions use `AssertionError`; invalid
public host values are rejected by facade validation. These failure classes must
not be conflated with unsupported capabilities or translator errors.

## Executor contract

Interpreter (v1) and generated-JS executors consume this IR unchanged;
the differential test harness runs both over the same fixture corpus
(`runtime/tests/fixtures/`, regenerable from the Python reference). Any IR
extension must land with fixtures.
The interpreter is checked against fixtures generated from the Python reference
(`runtime/tests/fixtures/`). A future generated-JS executor must consume the
same IR and agree on values and trap conditions. Any IR extension must land with
fixtures.

## Resource-type identity

The shim emits `resource` indices into the plan's `resourceTables`; the
runtime builds identity tokens (`ResourceTypeInfo`) at plan-load time.
**Tokens must be fresh per instantiation** (the executor re-runs plan
loading per instantiate), so resource-type identity never leaks across
instances.
The shim emits `resource` indices into the plan's `resourceTables`; the runtime
builds identity tokens (`ResourceTypeInfo`) at plan-load time. **Tokens must be
fresh per instantiation** (the executor re-runs plan loading per instantiate),
so resource-type identity never leaks across instances. Within one
instantiation, concrete tables naming the same resource share a token; table
indices are aliases, not distinct types.
62 changes: 30 additions & 32 deletions contracts/digest.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,17 @@
# Contract: Canonical World Digest (`cewd:1`)

The digest is the skew-protection handshake of docs/architecture.md §9: bindings generated
from WIT embed an expected digest; the generated typed `instantiate` recomputes
it from the loaded plan and fails fast on mismatch,
before any guest code runs (contracts/embedder-api.md §"Module wiring and
instantiation" — the runtime's untyped `instantiate` names no world, so it
does not verify). A digest match must imply ABI-shape compatibility for
positional calling.
Bindings embed a digest of their expected WIT world. Their typed `instantiate`
wrapper recomputes it from the plan and rejects mismatches before running guest
code. Untyped runtime instantiation does not check an expected world. A digest
match must imply ABI-shape compatibility for positional calling. It does not
authenticate a component or prove its behavior.

Normalization version: **`cewd:1`**. The version tag is folded into the
hashed document, so any future incompatible renormalization cannot collide.
Normalization version: **`cewd:1`**. The version tag is folded into the hashed
document, so any future incompatible renormalization cannot collide.

## Reference implementations (normative, kept in lockstep)

- Rust, from `wit_parser::Resolve` + world: `crates/bindgen/src/digest.rs`
(module docs carry the full rule-by-rule spec)
- TypeScript, from a loaded plan's types/imports/exports:
`runtime/src/digest/digest.ts`

Expand All @@ -25,35 +22,36 @@ implementations plus fixtures in the same commit.

## Normalization rules (summary)

- **Order-independent**: import/export lists and nested interface-instance
item lists are sorted by name — the only order normalized away.
- **Order-independent**: import/export lists and nested interface-instance item
lists are sorted by name — the only order normalized away.
- **Order-preserved (ABI-relevant)**: record fields, tuple elements,
variant/enum case order (discriminants), flags label order (bit
positions), function parameter order, fixed-list lengths.
variant/enum case order (discriminants), flags label order (bit positions),
function parameter order, fixed-list lengths.
- **Resource identity** by qualified name, not table index.
- **Excluded**: function parameter *labels* (calling is positional; renames
are not ABI changes — two worlds differing only in param names digest
equal, by design), docs/stability gates/spans, plan `features`,
`importedResources`, table ordering, `producer` metadata.
- **Excluded**: function parameter _labels_ (calling is positional; renames are
not ABI changes — two worlds differing only in param names digest equal, by
design), docs/stability gates/spans, plan `features`, `importedResources`,
table ordering, `producer` metadata.
- **Included**: package `@version` in qualified interface names.
- Only functions and resources contribute as export/import *items*; named
- Only functions and resources contribute as export/import _items_; named
non-resource types are structural and appear where referenced.
- Hash: sha256 over a canonical JSON document prefixed `{"cewd":1,…}`;
name sorting is byte-wise/UTF-16-code-unit (equivalent for ASCII WIT
identifiers — comment pinned on both sides).
- Hash: sha256 over a canonical JSON document prefixed `{"cewd":1,…}`; name
sorting is byte-wise/UTF-16-code-unit (equivalent for ASCII WIT identifiers —
comment pinned on both sides).

## Guards (fail loudly, never guess)

`DigestError` is thrown — rather than risking a wrong-but-matching digest —
when the plan carries **imported resources**, or **≥2 named resources**
whose own/borrow table indices cannot be soundly attributed, or own/borrow
references with **zero** named resources. All three paths are test-pinned.
Lifting these requires a plan-format extension carrying an explicit
resource-alias map (open item, tracked in plan-format.md).
The current plan-side implementation throws `DigestError` for imported
resources, unresolved own/borrow references, and ambiguous multi-resource table
attribution. It maps all table references to the sole name in a single-resource
world. Multi-resource worlds use direct table-to-name matches and reject
unresolved aliases. These are limits of the digest implementation, not of
resource execution; the executor separately resolves concrete resource-table
aliases.

## Known divergence (loud, not silent)

Interface *imports* are digested nested on the Rust side and flattened on
the TS side — guaranteed handshake mismatch (safe failure direction) for
worlds with interface imports; no current fixture has any. Resolve both
sides together when the imports corpus lands.
Interface imports are nested on the Rust side and flattened on the TS side. Such
worlds can fail the digest handshake despite being executable by the untyped
runtime. Correcting this requires coordinated Rust/TS normalization and
cross-language fixtures, not bypassing the mismatch check.
Loading
Loading