Skip to content

feat(api): same-template node affinity to ready same-build base sharing - #3838

Open
AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:feat/template-affinity-placement
Open

AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:feat/template-affinity-placement

Conversation

@AdaAibaby

@AdaAibaby AdaAibaby commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Scheduling prep for same-template read-only base sharing (pmem+DAX).
Refs #3824, #3825. Stacked conceptually on the observability in #3771. Off by default.

Why

Sandboxes of one build already share a single source memfile on the host, but UFFDIO_COPY faults a private physical copy of those identical read-only base pages into each sandbox. Collapsing N copies to one (a shared read-only base, pmem+DAX) is only possible if the siblings sit on the same node — and placement spreads them today. This biases placement so they land together, and measures how dense same-template colocation actually is, which is the go/no-go data for building the sharing.

What

  • Thread build_id through SandboxResources into the best-of-K score.
  • Multiplicative discount (1 - weight*min(sameBuild,cap)/cap) on the load score: affinity only reorders candidates that already pass admission and never outweighs real load; capped so it cannot pull an unbounded pile onto one node.
  • Off by default (weight 0) -> byte-for-byte the current score. Opt in via best-of-k-template-affinity-weight / -cap.

Observability (recorded even when the weight is 0)

  • api.orchestrator.template_affinity_placements{effect=colocated|spread}
  • api.orchestrator.template_affinity_siblings histogram — in-flight same-build siblings on the chosen node, i.e. the base copies a shared base could collapse there.
  • Structured debug log per same-build placement.

Tests

  • Discount math + SameBuildInProgress (table-driven).
  • Metric recording asserted through a ManualReader over the real placeSandbox path (effect classification, siblings count, build-less creates skipped).
  • Seeded multi-template fleet simulation through the real chooseNode: weight 0.6 tightens nodes-per-build 6.9 -> 5.6 and raises colocation ~22% -> ~36% (24-seed mean). Deliberately asserts direction, not strict point-wise monotonicity (the discount interacts with best-of-K sampling).

Safety / scope

api-only; serve/fault path untouched; orchestrator/envd unchanged. Zero added cardinality and zero behaviour change until the weight is raised.

/cc @ryan-sana Looking forward to your code review.

Sandboxes of one build already share a single source memfile on the host,
but UFFDIO_COPY faults a private physical copy of those identical read-only
base pages into each sandbox. Collapsing N copies to one (pmem+DAX shared
read-only base) needs the siblings on the same node first, and placement
spreads them today.

Bias best-of-K toward nodes already running same-build sandboxes:

- Thread build_id through SandboxResources into the score.
- Multiplicative discount (1 - weight*min(sameBuild,cap)/cap) on the load
  score, so affinity only reorders candidates that already pass admission
  and never outweighs real load. Capped so it cannot pull an unbounded pile
  onto one node.
- Off by default (weight 0): zero behaviour change until an operator opts in
  via best-of-k-template-affinity-weight / -cap.

Observability (recorded whether or not the weight is on, so the baseline is
visible before sharing is built and the lift after it is dialed up):

- api.orchestrator.template_affinity_placements{effect=colocated|spread}
- api.orchestrator.template_affinity_siblings histogram: in-flight same-build
  siblings on the chosen node, i.e. the base copies a shared base could
  collapse there.
- Structured debug log per same-build placement.

Tests: discount math and SameBuildInProgress (table), metric recording via a
ManualReader through the real placeSandbox path, and a seeded multi-template
fleet simulation through the real chooseNode showing the weight tightens
colocation (nodes-per-build 6.9 -> 5.6, colocation ~22% -> ~36% at weight
0.6, 24-seed mean).

Refs e2b-dev#3824, e2b-dev#3825.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant