Skip to content

feat: mid-session model switching per thread, with TUI + desktop UI and e2e tests#67

Merged
shadaj merged 1 commit into
mainfrom
sandbox-40cde97a-42ef-4dfd-ae40-9e80f268d1fc
Jul 13, 2026
Merged

feat: mid-session model switching per thread, with TUI + desktop UI and e2e tests#67
shadaj merged 1 commit into
mainfrom
sandbox-40cde97a-42ef-4dfd-ae40-9e80f268d1fc

Conversation

@shadaj

@shadaj shadaj commented Jul 10, 2026

Copy link
Copy Markdown
Member

Implements ClientMessage::SwitchModel (previously a stub) so the model can be
changed mid-session. The switch is scoped to a single thread — no propagation
to children — and never disturbs an in-flight completion.

Daemon core

  • SessionManager::switch_model: validates the model against the catalog
    (immediate Error reply on unknown model/thread), persists the selection on
    the thread via set_thread_model, then forwards it to a live worker through
    the agent loop; if nothing is running, the persisted selection is resolved
    at the next worker boot. Returns a DaemonMessage::ModelSwitched
    confirmation for the requesting client.
  • agent_loop: new AgentMessage::SwitchModel routed to a live worker's
    dedicated model_switch_tx channel; dropped (no worker spawn) when the
    thread isn't running since the selection is already persisted.
  • thread_worker: the current model lives in a slot
    (Option<(ModelRef, Arc<dyn ModelProvider>)>). Each completion round takes
    the model out of the slot and the round future owns it (async move
    wrapper around process_batch + inner cancel forwarding via an outer
    oneshot), returning it when the round ends. Applying a switch fills the
    slot, so a mid-round switch simply drops the old model when the round
    finishes — in-flight completions always finish on the model they started
    with, and the next round uses the latest selection (switch channel is also
    drained right before each round starts). Switches are handled in biased
    select! arms both while a completion is active and while idle-waiting,
    broadcasting ModelSwitched to the thread's subscribers on apply.
    context_window updates with the switch so auto-compaction thresholds
    follow the active model. process_batch itself is untouched.
  • Protocol: DaemonMessage::ModelSwitched { thread_id, model_name, context_window, provider_id } (+ remote-prefix handling); documented
    SwitchModel semantics.
  • InMemoryConversationStore::has_thread for validating switch targets.
  • Unit tests: switch while waiting for an async tool result applies to the
    next completion; switch during an in-flight completion leaves it
    undisturbed and applies to the next round.

TUI

  • DaemonMessage::ModelSwitched updates the status-line pill
    (model/provider/context window) via a new terminal::ModelSwitched channel
    when it targets the viewed thread (the /model picker was already wired to
    send SwitchModel).
  • New e2e tests (vt100 + real daemon + mock providers):
    switch_model_mid_session (picker → switch persists while idle → next
    worker boot uses the new provider) and switch_model_while_busy (switch
    mid-stream; old model finishes, follow-up hits the new one), with screen
    snapshots.

Desktop web UI

  • The model pill (top right) now drops down on hover with the list of
    available models (provider, name, context size, active marker); clicking
    one sends SwitchModel for the currently viewed thread. The pill and a
    "Switched model to …" transcript line update from the ModelSwitched
    confirmation (no optimistic update).
  • infinity-ui protocol types: added ModelRef and ModelSwitched, fixed
    the stale SwitchModel/CreateSession client types.
  • New Playwright e2e switch_model_mid_session (two mock providers): hover
    dropdown, select, confirmation line, next request routed to the second
    provider; two new golden screenshots.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 10, 2026

Copy link
Copy Markdown

Deploying infinity with  Cloudflare Pages  Cloudflare Pages

Latest commit: ebd2980
Status: ✅  Deploy successful!
Preview URL: https://5481d7ab.infinity-dc7.pages.dev
Branch Preview URL: https://sandbox-40cde97a-42ef-4dfd-a.infinity-dc7.pages.dev

View logs

shadaj added a commit that referenced this pull request Jul 13, 2026
…nd e2e tests

Implements `ClientMessage::SwitchModel` (previously a stub) so the model can be
changed mid-session. The switch is scoped to a single thread — no propagation
to children — and never disturbs an in-flight completion.

## Daemon core

* `SessionManager::switch_model`: validates the model against the catalog
  (immediate `Error` reply on unknown model/thread), persists the selection on
  the thread via `set_thread_model`, then forwards it to a live worker through
  the agent loop; if nothing is running, the persisted selection is resolved
  at the next worker boot. Returns a `DaemonMessage::ModelSwitched`
  confirmation for the requesting client.
* `agent_loop`: new `AgentMessage::SwitchModel` routed to a live worker's
  dedicated `model_switch_tx` channel; dropped (no worker spawn) when the
  thread isn't running since the selection is already persisted.
* `thread_worker`: the current model lives in a *slot*
  (`Option<(ModelRef, Arc<dyn ModelProvider>)>`). Each completion round takes
  the model out of the slot and the round future owns it (`async move`
  wrapper around `process_batch` + inner cancel forwarding via an outer
  oneshot), returning it when the round ends. Applying a switch fills the
  slot, so a mid-round switch simply drops the old model when the round
  finishes — in-flight completions always finish on the model they started
  with, and the next round uses the latest selection (switch channel is also
  drained right before each round starts). Switches are handled in biased
  `select!` arms both while a completion is active and while idle-waiting,
  broadcasting `ModelSwitched` to the thread's subscribers on apply.
  `context_window` updates with the switch so auto-compaction thresholds
  follow the active model. `process_batch` itself is untouched.
* Protocol: `DaemonMessage::ModelSwitched { thread_id, model_name,
  context_window, provider_id }` (+ remote-prefix handling); documented
  `SwitchModel` semantics.
* `InMemoryConversationStore::has_thread` for validating switch targets.
* Unit tests: switch while waiting for an async tool result applies to the
  next completion; switch during an in-flight completion leaves it
  undisturbed and applies to the next round.

## TUI

* `DaemonMessage::ModelSwitched` updates the status-line pill
  (model/provider/context window) via a new `terminal::ModelSwitched` channel
  when it targets the viewed thread (the `/model` picker was already wired to
  send `SwitchModel`).
* New e2e tests (vt100 + real daemon + mock providers):
  `switch_model_mid_session` (picker → switch persists while idle → next
  worker boot uses the new provider) and `switch_model_while_busy` (switch
  mid-stream; old model finishes, follow-up hits the new one), with screen
  snapshots.

## Desktop web UI

* The model pill (top right) now drops down on hover with the list of
  available models (provider, name, context size, active marker); clicking
  one sends `SwitchModel` for the currently viewed thread. The pill and a
  "Switched model to …" transcript line update from the `ModelSwitched`
  confirmation (no optimistic update).
* `infinity-ui` protocol types: added `ModelRef` and `ModelSwitched`, fixed
  the stale `SwitchModel`/`CreateSession` client types.
* New Playwright e2e `switch_model_mid_session` (two mock providers): hover
  dropdown, select, confirmation line, next request routed to the second
  provider; two new golden screenshots.

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #67
@shadaj
shadaj force-pushed the sandbox-d9022bfb-d303-4377-bf02-b56fecd72e86 branch from a7f7a45 to bd11d2d Compare July 13, 2026 17:59
@shadaj
shadaj force-pushed the sandbox-40cde97a-42ef-4dfd-ae40-9e80f268d1fc branch from 1af109c to 2999758 Compare July 13, 2026 17:59
shadaj added a commit that referenced this pull request Jul 13, 2026
…nd e2e tests

(Rebased onto latest main.)

Implements `ClientMessage::SwitchModel` (previously a stub) so the model can be
changed mid-session. The switch is scoped to a single thread — no propagation
to children — and never disturbs an in-flight completion.

## Daemon core

* `SessionManager::switch_model`: validates the model against the catalog
  (immediate `Error` reply on unknown model/thread), persists the selection on
  the thread via `set_thread_model`, then forwards it to a live worker through
  the agent loop; if nothing is running, the persisted selection is resolved
  at the next worker boot. Returns a `DaemonMessage::ModelSwitched`
  confirmation for the requesting client.
* `agent_loop`: new `AgentMessage::SwitchModel` routed to a live worker's
  dedicated `model_switch_tx` channel; dropped (no worker spawn) when the
  thread isn't running since the selection is already persisted.
* `thread_worker`: the current model lives in a *slot*
  (`Option<(ModelRef, Arc<dyn ModelProvider>)>`). Each completion round takes
  the model out of the slot and the round future owns it (`async move`
  wrapper around `process_batch` + inner cancel forwarding via an outer
  oneshot), returning it when the round ends. Applying a switch fills the
  slot, so a mid-round switch simply drops the old model when the round
  finishes — in-flight completions always finish on the model they started
  with, and the next round uses the latest selection (switch channel is also
  drained right before each round starts). Switches are handled in biased
  `select!` arms both while a completion is active and while idle-waiting,
  broadcasting `ModelSwitched` to the thread's subscribers on apply.
  `context_window` updates with the switch so auto-compaction thresholds
  follow the active model. `process_batch` itself is untouched.
* Protocol: `DaemonMessage::ModelSwitched { thread_id, model_name,
  context_window, provider_id }` (+ remote-prefix handling); documented
  `SwitchModel` semantics.
* `InMemoryConversationStore::has_thread` for validating switch targets.
* Unit tests: switch while waiting for an async tool result applies to the
  next completion; switch during an in-flight completion leaves it
  undisturbed and applies to the next round.

## TUI

* `DaemonMessage::ModelSwitched` updates the status-line pill
  (model/provider/context window) via a new `terminal::ModelSwitched` channel
  when it targets the viewed thread (the `/model` picker was already wired to
  send `SwitchModel`).
* New e2e tests (vt100 + real daemon + mock providers):
  `switch_model_mid_session` (picker → switch persists while idle → next
  worker boot uses the new provider) and `switch_model_while_busy` (switch
  mid-stream; old model finishes, follow-up hits the new one), with screen
  snapshots.

## Desktop web UI

* The model pill (top right) now drops down on hover with the list of
  available models (provider, name, context size, active marker); clicking
  one sends `SwitchModel` for the currently viewed thread. The pill and a
  "Switched model to …" transcript line update from the `ModelSwitched`
  confirmation (no optimistic update).
* `infinity-ui` protocol types: added `ModelRef` and `ModelSwitched`, fixed
  the stale `SwitchModel`/`CreateSession` client types.
* New Playwright e2e `switch_model_mid_session` (two mock providers): hover
  dropdown, select, confirmation line, next request routed to the second
  provider; two new golden screenshots. Regenerated `chat-title-set.png`
  after the rebase (17 pixels of ±1 antialiasing drift at the pill edge under
  the strict `max_diff_pixels(0)` comparison).

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #67
@shadaj
shadaj force-pushed the sandbox-40cde97a-42ef-4dfd-ae40-9e80f268d1fc branch from 2999758 to 65c1c1f Compare July 13, 2026 21:20
@shadaj
shadaj changed the base branch from sandbox-d9022bfb-d303-4377-bf02-b56fecd72e86 to main July 13, 2026 21:20
…nd e2e tests

(Rebased onto latest main.)

Implements `ClientMessage::SwitchModel` (previously a stub) so the model can be
changed mid-session. The switch is scoped to a single thread — no propagation
to children — and never disturbs an in-flight completion.

## Daemon core

* `SessionManager::switch_model`: validates the model against the catalog
  (immediate `Error` reply on unknown model/thread), persists the selection on
  the thread via `set_thread_model`, then forwards it to a live worker through
  the agent loop; if nothing is running, the persisted selection is resolved
  at the next worker boot. Returns a `DaemonMessage::ModelSwitched`
  confirmation for the requesting client.
* `agent_loop`: new `AgentMessage::SwitchModel` routed to a live worker's
  dedicated `model_switch_tx` channel; dropped (no worker spawn) when the
  thread isn't running since the selection is already persisted.
* `thread_worker`: the current model lives in a *slot*
  (`Option<(ModelRef, Arc<dyn ModelProvider>)>`). Each completion round takes
  the model out of the slot and the round future owns it (`async move`
  wrapper around `process_batch` + inner cancel forwarding via an outer
  oneshot), returning it when the round ends. Applying a switch fills the
  slot, so a mid-round switch simply drops the old model when the round
  finishes — in-flight completions always finish on the model they started
  with, and the next round uses the latest selection (switch channel is also
  drained right before each round starts). Switches are handled in biased
  `select!` arms both while a completion is active and while idle-waiting,
  broadcasting `ModelSwitched` to the thread's subscribers on apply.
  `context_window` updates with the switch so auto-compaction thresholds
  follow the active model. `process_batch` itself is untouched.
* Protocol: `DaemonMessage::ModelSwitched { thread_id, model_name,
  context_window, provider_id }` (+ remote-prefix handling); documented
  `SwitchModel` semantics.
* `InMemoryConversationStore::has_thread` for validating switch targets.
* Unit tests: switch while waiting for an async tool result applies to the
  next completion; switch during an in-flight completion leaves it
  undisturbed and applies to the next round.

## TUI

* `DaemonMessage::ModelSwitched` updates the status-line pill
  (model/provider/context window) via a new `terminal::ModelSwitched` channel
  when it targets the viewed thread (the `/model` picker was already wired to
  send `SwitchModel`).
* New e2e tests (vt100 + real daemon + mock providers):
  `switch_model_mid_session` (picker → switch persists while idle → next
  worker boot uses the new provider) and `switch_model_while_busy` (switch
  mid-stream; old model finishes, follow-up hits the new one), with screen
  snapshots.

## Desktop web UI

* The model pill (top right) now drops down on hover with the list of
  available models (provider, name, context size, active marker); clicking
  one sends `SwitchModel` for the currently viewed thread. The pill and a
  "Switched model to …" transcript line update from the `ModelSwitched`
  confirmation (no optimistic update).
* `infinity-ui` protocol types: added `ModelRef` and `ModelSwitched`, fixed
  the stale `SwitchModel`/`CreateSession` client types.
* New Playwright e2e `switch_model_mid_session` (two mock providers): hover
  dropdown, select, confirmation line, next request routed to the second
  provider; two new golden screenshots. Regenerated `chat-title-set.png`
  after the rebase (17 pixels of ±1 antialiasing drift at the pill edge under
  the strict `max_diff_pixels(0)` comparison).

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #67
@shadaj
shadaj force-pushed the sandbox-40cde97a-42ef-4dfd-ae40-9e80f268d1fc branch from 65c1c1f to ebd2980 Compare July 13, 2026 21:47
@shadaj
shadaj marked this pull request as ready for review July 13, 2026 21:56
@shadaj
shadaj merged commit 1c4f71a into main Jul 13, 2026
9 checks passed
@shadaj
shadaj deleted the sandbox-40cde97a-42ef-4dfd-ae40-9e80f268d1fc branch July 13, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants