feat(webapp,sdk): in-dashboard AI agent#4018
Draft
ericallam wants to merge 28 commits into
Draft
Conversation
Range-scoped to ^18 so packages on react 19 (rsc) stay untouched. Collapses the duplicate react copies that otherwise break hook-based SDK components with an invalid hook call.
When set, the returned action runs under the given baseURL and access token (via apiClientManager.runWithConfig) instead of the ambient SDK config, so one server can start chat sessions in a different runtime environment.
Adds an "Ask the agent" side panel on every environment page, backed by a chat.agent task that runs as its own internal Trigger project with no main-database or ClickHouse access. Conversations persist to a dedicated Postgres schema via a small Drizzle store; the panel restores the last open chat and resumes streaming without replay. No tools yet: the agent answers from the model only. Tools, delegated auth, and the data lane land in follow-ups.
Adds hasDashboardAgentAccess to the flag catalog, off by default via DASHBOARD_AGENT_ENABLED. The panel renders only when the flag is on for the org or the viewer is staff, and the resource route enforces the same check so a non-flagged user cannot start sessions by calling it directly. Controllable globally and per-org from the admin flags UI.
…+ caching Drive the agent and title models from dashboard-managed prompts resolved through an Anthropic provider registry (agent claude-sonnet-4-6, title claude-haiku-4-5). Generate the chat title in the background after the first turn with the cheaper model so it never blocks the response, writing it only while the chat still has the default title. Add Anthropic prompt caching: an ephemeral breakpoint on the system block plus a rolling breakpoint on the last message.
The deploy-time prompt dedup hashed only the prompt text, so changing a code prompt's model or config was silently skipped and the previous version kept serving. Include model and config in the version-definition hash so those changes create a new version.
Open the agent inside a ResizablePanelGroup (content + handle + agent panel) using the shared Resizable primitive, so the panel drag-resizes between 320 and 720px with the standard handle and keyboard support. autosaveId persists the width across opens and reloads.
The project worker-by-tag endpoint now accepts a delegated user-actor token the same way it accepts a personal access token, so a first-party caller can list a project's deployed tasks on the user's behalf. It stays identity-scoped, with no change to what the user can already do.
A caret range resolved a newer ai build than the SDK uses, pulling a second copy of the AI SDK tool types and breaking type-checking against the chat agent's tool set. Pinning the exact version dedupes them.
The dashboard agent can now read your projects, environments, runs, and deployed tasks by calling the public API as you. Each turn mints a short-lived, read-only delegated token on the server and hands it to the agent through a same-origin proxy on the message-send path, so the token is never exposed to the browser. The response stream stays pointed directly at the realtime host. Tools: list_projects, list_environments, get_run, list_tasks.
The dashboard agent can now list recent runs in the current environment (filterable by status, task, and a time window of up to 30 days) and fetch a run's execution trace to explain why it failed, retried, or was slow.
chat.headStart now accepts an apiClient (base URL + access token), so the head-start route can create the session and trigger the agent run against a different project or environment than the warm server's ambient config. Mirrors chat.createStartSessionAction; the run callback's LLM keys are unaffected.
Pins ai (^6) and @ai-sdk/provider-utils (^4) via pnpm overrides so the AI SDK tool types resolve to one instance across the app, the SDK, and internal packages. Two provider-utils copies previously broke type-checking a shared tool set across package boundaries. Also adds @ai-sdk/anthropic to the webapp for the agent's first-turn route.
The first turn of a new dashboard agent chat now streams step 1 from the webapp while the agent run boots in parallel, cutting first-token latency. The warm route runs step 1 with schema-only tools (the tool execute fns stay in the agent, never the webapp bundle) and mints a fresh read-only delegated token server-side, injected into the run so the first turn's tool calls are authed as the user without the token ever reaching the browser. Falls back to the normal path when no Anthropic key is configured.
The dashboard agent can now read error groups as the signed-in user: list_errors lists distinct errors by fingerprint with occurrence counts and status, get_error returns the full detail for one, and list_runs can filter to the runs behind an error group. All read-only and scoped to the current environment.
…eMessages A Head Start handover hands the first turn's pending tool call to the agent as a tool-approval round whose trailing tool message must reach the model untouched for the call to execute. A prepareMessages hook that rewrote the last message (for example the recommended prompt-caching breakpoint) dropped it, so the turn failed with "tool_use ids were found without tool_result". The agent now preserves that approval tail across prepareMessages, so caching and Head Start compose cleanly.
Drive the agent through real turns offline with mockChatAgent and MockLanguageModelV3: text streaming, tool execution, the prompt-cache breakpoint, the Head Start handover resume, and the read tools failing closed without a delegated token. The agent's datastore and model are now injectable via locals so the tests need no database or provider.
…answer quality Vitest evals that exercise the agent against a real model with fixture tools: a tool-selection set scored on expected tool choices, plus an LLM-as-judge case for answer quality. Runs behind its own config so it stays out of the unit test run, and *.eval.ts files are kept out of the task index.
After each turn the agent enqueues a decoupled, idempotency-keyed task that judges the turn (grounding, whether the question was answered, intent, outcome) and flags product signal (capability gaps, docs gaps, support opportunities, feature requests), then writes one row. The judge runs out of band so it never blocks or bills the agent run. Adds the chat_turn_evals table and migration to @internal/dashboard-agent-db.
…mode When a project has a connected GitHub repo, the in-dashboard agent can now read its source to ground answers. It pulls the repo at the tracked commit onto its own filesystem and exposes read-only list, read, and grep tools, so it can explain a run or error against the actual task code, citing file and line. The webapp resolves a short-lived signed archive URL server-side and injects it per turn, so the GitHub token never reaches the agent. With no connected repo the agent stays in its usual assistant mode.
…deployed from The agent code tools now take an optional runId. When investigating a specific run, the agent reads the exact source that run version was deployed from instead of the latest commit, so the explanation matches the code that actually ran. A new endpoint maps the run to its deployment commit and resolves a signed archive URL for it server-side (the GitHub token stays on the server); the agent downloads and reads that commit. Runs with no deployed version fall back to the tracked branch head.
The dashboard agent now answers "why did this run fail?" with a structured failure card (summary, category, likely cause, confidence, evidence, impact, next steps, and action buttons) instead of plain prose. The card is the first block in a small view catalog: the agent emits a render_view tool call constrained to a fixed set of blocks, and the dashboard renders it through a component registry. Only known block types render, so the agent can never produce arbitrary markup. New blocks plug in by adding a schema member plus a registry entry.
…iew block The dashboard agent can now answer analytics questions with data and charts. Two read-only query tools let it discover the TRQL schema and run queries against the current environment, and a new "chart" block in the view catalog renders a line or bar chart. The chart block carries the TRQL query rather than rows: the panel runs it through the dashboard's existing query execution and chart components, so the chart is live and matches the Query page. Queries run as the user over the public query API (read:query), so the agent still reaches no data directly.
… questions The dashboard agent can now answer "how does Trigger.dev work?" questions (docs, concepts, features, how-tos) by asking the Trigger.dev support assistant, instead of guessing or limiting itself to the user's own data. The ask_support tool forwards the question to a service-to-service ask endpoint (the support assistant composes the answer) and returns it. It carries no user data and uses a shared secret server-side only, so nothing reaches the browser.
AgentChat's .in/append and .out SSE calls built their headers by hand and omitted x-trigger-branch, so a chat.agent deployed to a preview branch returned 401 "x-trigger-branch header required for preview env" the moment a message was appended. sessions.start already sends the branch via the API client; these two raw fetches now do too, from the same apiClientManager.branchName.
Resolve dashboard agent access in the env layout loader (global env override, then admins, then the global or per-org feature flag, default off) and pass it to the launcher. The button previously read only the per-org flag on the client, so enabling the agent globally granted server access but left the button hidden for non-admins. The launcher now matches the server check.
🦋 Changeset detectedLatest commit: 8a37eda The changes in this PR will be included in the next version bump. This PR includes changesets to release 28 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
Caution Review failedAn error occurred during the review process. Please try again later. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 13
🧹 Nitpick comments (3)
internal-packages/dashboard-agent-db/src/client.ts (1)
15-25: 🧹 Nitpick | 🔵 Trivial | 💤 Low valuePrefer
typeoverinterfacefor data shapes.
CreateDashboardAgentDbOptionsis a pure data shape (no methods), so per coding guidelines it should usetypeinstead ofinterface.♻️ Suggested change
-export interface CreateDashboardAgentDbOptions { +export type CreateDashboardAgentDbOptions = { /** * Max client-side pool size. Keep small — the agent runs in many short-lived * task containers and PlanetScale's pooler does the real connection pooling. */ max?: number; /** Idle timeout (seconds) so suspended agent runs release connections. */ idleTimeoutSeconds?: number; /** Connection timeout (seconds). */ connectTimeoutSeconds?: number; -} +};Source: Coding guidelines
internal-packages/dashboard-agent-db/src/queries.ts (1)
14-22: 🧹 Nitpick | 🔵 Trivial | 💤 Low valuePrefer
typeoverinterfacefor data shapes.
ChatListItemis a pure data shape, so per coding guidelines it should usetype.♻️ Suggested change
-export interface ChatListItem { +export type ChatListItem = { id: string; title: string; pinnedAt: Date | null; lastMessageAt: Date | null; createdAt: Date; updatedAt: Date; metadata: Record<string, unknown>; -} +};Source: Coding guidelines
.claude/skills/drizzle/SKILL.md (1)
47-60: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winKeep schema and query snippets consistent.
Line 114 orders by
chats.pinnedAt, but the schema snippet in Lines 47-60 doesn’t definepinnedAt, so the example set is internally inconsistent.Suggested fix
export const chats = dashboardAgentSchema.table( "chats", { @@ messages: jsonb("messages").$type<unknown[]>().notNull().default([]), metadata: jsonb("metadata").$type<Record<string, unknown>>().notNull().default({}), + pinnedAt: timestamp("pinned_at", { withTimezone: true }), deletedAt: timestamp("deleted_at", { withTimezone: true }), // soft delete lastMessageAt: timestamp("last_message_at", { withTimezone: true }),Also applies to: 114-114
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: a62f9d56-f2aa-4d7d-a8a9-9c957c74d2bf
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (73)
.changeset/chat-agent-preview-branch.md.changeset/chat-head-start-prepare-messages.md.changeset/chat-headstart-api-client.md.changeset/chat-start-session-api-client.md.claude/skills/drizzle/SKILL.md.gitignoreapps/webapp/app/components/dashboard-agent/AgentChart.tsxapps/webapp/app/components/dashboard-agent/DashboardAgent.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentChat.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentComposer.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentContextBanner.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentHeader.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentHistory.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentMessages.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentPanel.tsxapps/webapp/app/components/dashboard-agent/DashboardAgentSuggestedPrompts.tsxapps/webapp/app/components/dashboard-agent/RunDiagnosisCard.tsxapps/webapp/app/components/dashboard-agent/view-catalog.tsxapps/webapp/app/env.server.tsapps/webapp/app/hooks/useApiOrigin.tsapps/webapp/app/presenters/OrganizationsPresenter.server.tsapps/webapp/app/root.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam/route.tsxapps/webapp/app/routes/api.v1.projects.$projectRef.$env.repo.snapshot.tsapps/webapp/app/routes/api.v1.projects.$projectRef.$env.workers.$tagName.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.headstart.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.in.$.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.tsapps/webapp/app/routes/storybook.agent-ui/route.tsxapps/webapp/app/routes/storybook/route.tsxapps/webapp/app/services/dashboardAgent.server.tsapps/webapp/app/services/dashboardAgentDb.server.tsapps/webapp/app/v3/canAccessDashboardAgent.server.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/app/v3/services/createBackgroundWorker.server.tsapps/webapp/package.jsondocs/ai-chat/fast-starts.mdxdocs/ai-chat/prompt-caching.mdxinternal-packages/dashboard-agent-db/README.mdinternal-packages/dashboard-agent-db/drizzle.config.tsinternal-packages/dashboard-agent-db/drizzle/0000_magenta_lilandra.sqlinternal-packages/dashboard-agent-db/drizzle/0001_slimy_living_tribunal.sqlinternal-packages/dashboard-agent-db/drizzle/meta/0000_snapshot.jsoninternal-packages/dashboard-agent-db/drizzle/meta/0001_snapshot.jsoninternal-packages/dashboard-agent-db/drizzle/meta/_journal.jsoninternal-packages/dashboard-agent-db/package.jsoninternal-packages/dashboard-agent-db/src/client.tsinternal-packages/dashboard-agent-db/src/index.tsinternal-packages/dashboard-agent-db/src/queries.tsinternal-packages/dashboard-agent-db/src/schema.tsinternal-packages/dashboard-agent-db/tsconfig.jsoninternal-packages/dashboard-agent/.gitignoreinternal-packages/dashboard-agent/README.mdinternal-packages/dashboard-agent/eval-setup.tsinternal-packages/dashboard-agent/package.jsoninternal-packages/dashboard-agent/src/dashboard-agent.eval.tsinternal-packages/dashboard-agent/src/dashboard-agent.test.tsinternal-packages/dashboard-agent/src/dashboard-agent.tsinternal-packages/dashboard-agent/src/eval-turn.tsinternal-packages/dashboard-agent/src/index.tsinternal-packages/dashboard-agent/src/prompts.tsinternal-packages/dashboard-agent/src/repo-tools.test.tsinternal-packages/dashboard-agent/src/repo-tools.tsinternal-packages/dashboard-agent/src/tool-schemas.tsinternal-packages/dashboard-agent/src/tools.tsinternal-packages/dashboard-agent/trigger.config.tsinternal-packages/dashboard-agent/tsconfig.jsoninternal-packages/dashboard-agent/vitest.config.tsinternal-packages/dashboard-agent/vitest.eval.config.tspackage.jsonpackages/trigger-sdk/src/v3/ai.tspackages/trigger-sdk/src/v3/chat-client.tspackages/trigger-sdk/src/v3/chat-server.ts
Self-hosted containers now apply the trigger_dashboard_agent schema migrations on startup, alongside the Prisma and ClickHouse migrations, via a small runtime migrator (drizzle-orm over the committed SQL, no drizzle-kit in the image). A new SKIP_DASHBOARD_AGENT_MIGRATIONS flag lets deployments that apply migrations out of band opt out.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: fdda1616-3516-4bcb-b09c-c0d2588397f2
📒 Files selected for processing (3)
docker/scripts/entrypoint.shinternal-packages/dashboard-agent-db/migrate.mjsinternal-packages/dashboard-agent-db/package.json
🚧 Files skipped from review as they are similar to previous changes (1)
- internal-packages/dashboard-agent-db/package.json
📜 Review details
⏰ Context from checks skipped due to timeout. (38)
- GitHub Check: internal / 🧪 Unit Tests: Internal (10, 12)
- GitHub Check: sdk-compat / Node.js 20.20 (ubuntu-latest)
- GitHub Check: internal / 🧪 Unit Tests: Internal (7, 12)
- GitHub Check: internal / 🧪 Unit Tests: Internal (1, 12)
- GitHub Check: internal / 🧪 Unit Tests: Internal (12, 12)
- GitHub Check: internal / 🧪 Unit Tests: Internal (9, 12)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (9, 10)
- GitHub Check: internal / 🧪 Unit Tests: Internal (3, 12)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (6, 10)
- GitHub Check: internal / 🧪 Unit Tests: Internal (5, 12)
- GitHub Check: internal / 🧪 Unit Tests: Internal (11, 12)
- GitHub Check: internal / 🧪 Unit Tests: Internal (2, 12)
- GitHub Check: internal / 🧪 Unit Tests: Internal (8, 12)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (3, 10)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (10, 10)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (8, 10)
- GitHub Check: internal / 🧪 Unit Tests: Internal (4, 12)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (1, 10)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (5, 10)
- GitHub Check: internal / 🧪 Unit Tests: Internal (6, 12)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (7, 10)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (4, 10)
- GitHub Check: sdk-compat / Node.js 22.12 (ubuntu-latest)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: webapp / 🧪 Unit Tests: Webapp (2, 10)
- GitHub Check: sdk-compat / Cloudflare Workers
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: e2e-webapp / 🧪 E2E Tests: Webapp
- GitHub Check: sdk-compat / Bun Runtime
- GitHub Check: packages / 🧪 Unit Tests: Packages (3, 3)
- GitHub Check: packages / 🧪 Unit Tests: Packages (1, 3)
- GitHub Check: packages / 🧪 Unit Tests: Packages (2, 3)
- GitHub Check: sdk-compat / Deno Runtime
- GitHub Check: 🛡️ E2E Auth Tests (full)
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (1)
internal-packages/dashboard-agent-db/migrate.mjs (1)
1-57: LGTM!
Comment on lines
+16
to
+23
| if [ "$SKIP_DASHBOARD_AGENT_MIGRATIONS" != "1" ]; then | ||
| echo "Running dashboard agent migrations" | ||
| pnpm --filter @internal/dashboard-agent-db db:migrate:deploy | ||
| echo "Dashboard agent migrations done" | ||
| else | ||
| echo "SKIP_DASHBOARD_AGENT_MIGRATIONS=1, skipping dashboard agent migrations." | ||
| fi | ||
|
|
Contributor
There was a problem hiding this comment.
Wait for the dashboard-agent DB endpoint before running its migrations.
If DASHBOARD_AGENT_DATABASE_URL points to a different host than the one checked at Line 4, this block can race DB readiness and fail container startup.
Suggested fix
+if [ -n "$DASHBOARD_AGENT_DATABASE_HOST" ]; then
+ scripts/wait-for-it.sh "${DASHBOARD_AGENT_DATABASE_HOST}" -- echo "dashboard-agent database is up"
+fi
+
if [ "$SKIP_DASHBOARD_AGENT_MIGRATIONS" != "1" ]; then
echo "Running dashboard agent migrations"
pnpm --filter `@internal/dashboard-agent-db` db:migrate:deploy
echo "Dashboard agent migrations done"
else
echo "SKIP_DASHBOARD_AGENT_MIGRATIONS=1, skipping dashboard agent migrations."
fiThe dashboard agent now creates chat sessions server-side: the server generates the chat id and owns the chat record (bound to the user and org), and only issues a session token for a chat the requesting user owns. A new chat opens in a draft composer; the first message creates the session via chat.startHeadStart, so the client never chooses the id. Also hardens the repo-read tools with collision-resistant workspace keys and a realpath guard against symlink escapes.
ericallam
force-pushed
the
feat/dashboard-agent-scaffold
branch
from
0d6821e to
8a37eda
Compare
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an in-dashboard AI agent: a chat panel, reachable from any environment
page, that answers questions about your runs, errors, tasks, and analytics,
diagnoses why a run failed, charts your data, reads your connected repo's
source, and answers product and how-to questions. It is gated behind the
hasDashboardAgentAccessfeature flag (global or per-org, default off), sothis PR ships disabled: the launcher is hidden unless the flag is enabled.
Design
The agent runs as a standalone
chat.agentTrigger task in its own internalpackage, with no access to the webapp database, Prisma, or ClickHouse. It reads
the user's data over the public API, acting as the user via a short-lived
delegated user-actor token minted server-side each turn (never in the browser),
building on #3997. The
error and analytics tools use #4005
and the TRQL query API.
The first turn of a new chat streams from a warm webapp route (Head Start) while
the durable agent boots in parallel. Structured answers (a run-failure diagnosis
card, a live chart) render through a small typed view catalog rather than
arbitrary markup. A knowledge lane forwards product and how-to questions to the
support assistant.
Conversation history lives in a separate Drizzle-backed store on its own
Postgres schema, kept as a display read-model so it can never corrupt the
agent's model context.
The SDK changes add an
apiClientoption tochat.createStartSessionActionandchat.headStart, and keep the Head Start tool-approval tail intact across acustom
prepareMessageshook so prompt caching and Head Start compose.