diff --git a/workshop/22-error-handling-and-resilience.md b/workshop/22-error-handling-and-resilience.md index a83a9422..dbeaa246 100644 --- a/workshop/22-error-handling-and-resilience.md +++ b/workshop/22-error-handling-and-resilience.md @@ -17,24 +17,15 @@ Learn the most common ways agentic workflows fail in production and apply three ### Understand common failure modes -Agentic workflows can fail for several reasons: +Agentic workflows most often fail in one of four ways: empty data, tool errors, timeouts, and prompt drift. This step applies three mitigations — a defensive brief, `timeout-minutes`, and a fallback safe-output — that address all four. -| Failure type | Example | Effect | -|---|---|---| -| **Empty data** | No open issues to summarise | Agent produces a vague or empty report | -| **Tool error** | GitHub API rate-limit hit mid-run | Agent stops mid-task without writing output | -| **Timeout** | Complex reasoning takes too long | Workflow job is cancelled by Actions | -| **Prompt drift** | Instructions are ambiguous | Agent takes an unexpected code path | - -Recognising these patterns helps you write instructions that stay on track. - -The diagram below shows how these failure modes map to the three mitigations covered in this step. - - - - - Four failure modes — prompt drift, timeout, tool error, and empty data — each mapped to one of three mitigations: defensive brief, timeout-minutes, and fallback safe-output, which together produce a reliably running workflow - +> [!TIP] +>
+> Optional Side Quest: Want a worked example of each failure mode and practice matching failures to fixes before you dive in? +> +> Work through [Side Quest: Recognizing Common Agentic Workflow Failure Modes](side-quest-22-01-failure-modes.md), then come back here. +> +>
### Apply all three changes with the skill diff --git a/workshop/README.md b/workshop/README.md index fa87d919..83fe2472 100644 --- a/workshop/README.md +++ b/workshop/README.md @@ -106,6 +106,7 @@ A hands-on workshop that takes you from zero to a fully automated, AI-powered wo - [Self-Hosted Runner Infrastructure Deep Dive](side-quest-24-01-runner-infrastructure.md) — enterprise infrastructure primer covering ephemeral and JIT runners, proxy configuration, and network isolation for air-gapped environments; branches from [Step 24](24-self-hosted-runners.md). - [Project Future AI Credit Costs with `gh aw forecast`](side-quest-26-01-forecast-costs.md) — full walkthrough of `gh aw forecast`: reading P10/P50/P90 output, using `--period week` and `--days 7`, forecasting all workflows, and deriving a `max-daily-ai-credits` value from the P90 figure; branches from [Step 26](26-manage-costs-and-budgets.md). - [Skill Injection Strategies — Hint, Fusion, and Inline](side-quest-29-01-skill-injection-strategies.md) — decision table, code examples, and a practice exercise for the hint, fusion, and inline strategies for wiring a `SKILL.md` into a workflow prompt; branches from [Step 29](29-skills-and-domain-knowledge.md). +- [Recognizing Common Agentic Workflow Failure Modes](side-quest-22-01-failure-modes.md) — worked examples of empty data, tool error, timeout, and prompt drift failures with a match-the-fix practice exercise; branches from [Step 22](22-error-handling-and-resilience.md). ## Getting Started diff --git a/workshop/side-quest-22-01-failure-modes.md b/workshop/side-quest-22-01-failure-modes.md new file mode 100644 index 00000000..87b610fb --- /dev/null +++ b/workshop/side-quest-22-01-failure-modes.md @@ -0,0 +1,66 @@ + + +# Side Quest: Recognizing Common Agentic Workflow Failure Modes + +> _Before you can fix a broken run, you need a name for what went wrong — this primer gives you four._ + +## :dart: What You'll Do + +You'll learn the four most common ways agentic workflows fail in production, see a worked example of each, and practice matching a failure type to its effect. By the end, you'll be able to look at a run log and name the failure mode in one word before you start debugging. + +## :clipboard: Before You Start + +- You have a working scheduled workflow (see [Refine, Test, and Improve Your Workflow](09-agentic-editing.md)). +- You're starting (or have already started) [Make Your Workflows Resilient to Failure](22-error-handling-and-resilience.md), which uses this vocabulary. + +## Steps + +### The four failure modes + +Agentic workflows can fail for several reasons: + +| Failure type | Example | Effect | +|---|---|---| +| **Empty data** | No open issues to summarise | Agent produces a vague or empty report | +| **Tool error** | GitHub API rate-limit hit mid-run | Agent stops mid-task without writing output | +| **Timeout** | Complex reasoning takes too long | Workflow job is cancelled by Actions | +| **Prompt drift** | Instructions are ambiguous | Agent takes an unexpected code path | + +Recognising these patterns helps you write instructions that stay on track — most workflow bugs are one of these four things, not something exotic. + +The diagram below shows how these failure modes map to three mitigations: a defensive brief, a `timeout-minutes` setting, and a fallback [safe-output](https://github.github.com/gh-aw/reference/safe-outputs/). + + + + + Four failure modes — prompt drift, timeout, tool error, and empty data — each mapped to one of three mitigations: defensive brief, timeout-minutes, and fallback safe-output, which together produce a reliably running workflow + + +### Practice: match the failure to the mitigation + +Before checking your answer, decide which mitigation (defensive brief, `timeout-minutes`, or fallback safe-output) best addresses each scenario: + +1. A run consistently takes 18 minutes to finish reasoning about a large diff, and Actions cancels it. +2. A run finishes cleanly but never calls a safe-output tool because the repository had no activity that day. +3. A run's summary is technically correct but ignores the instruction to flag blockers, because the brief never defined what a "blocker" is. + +
+Reveal the answers + +1. **Timeout.** Set `timeout-minutes` to a value that gives the agent headroom, or reduce the size of the input it reasons over. +2. **Empty data.** Add a defensive brief instruction that tells the agent to write a "no activity" report — and always call the safe output — even when nothing changed. +3. **Prompt drift.** The brief was ambiguous about what counts as a blocker. Tighten the instruction with a concrete definition or example. + +
+ +### Watch for this pattern in your own runs + +Open a recent run of your own workflow in the **Actions** tab and skim the log. Ask yourself: does anything here match one of the four failure types above, even if the run technically succeeded? A run can "succeed" (green checkmark) and still exhibit prompt drift or produce empty-data output. + +## :white_check_mark: Checkpoint + +- [ ] You can name the four common agentic workflow failure modes without looking at the table +- [ ] You matched each of the three practice scenarios to its correct mitigation +- [ ] You reviewed one of your own workflow runs and identified whether any failure mode applied + +**Return to the main adventure:** [Make Your Workflows Resilient to Failure](22-error-handling-and-resilience.md)