Fix Prompt Optimization Ollama sandbox access#247
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot review workflow following playwright.md guidance |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Reviewed against |
|
@copilot review the playwright instructions in /agentic-workflows and fix this issue again. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Reworked the workflow to use a digest-pinned declarative Ollama service, as the sandbox guidance prescribes for host services. The agent now reaches only the dynamically published service port; model pull/load requests have explicit retry and API-error checks. Latest commit: d8c2588. |
|
@copilot resolve the merge conflicts on this branch. |
There was a problem hiding this comment.
🟡 Changes recommended
The workflow change introduces reliability/operational concerns (no explicit service readiness wait; models will be re-downloaded every run) and the implementation currently contradicts the PR description’s runner-hosted Ollama approach.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates the Prompt Optimization agentic workflow to make the Ollama evaluation endpoint reachable from the gh-aw sandboxed agent container, addressing the infrastructure failure described in #242.
Changes:
- Switches Ollama provisioning to a GitHub Actions job service and maps port
11434. - Updates the prompt instructions and eval URLs to use
http://host.docker.internal:11434/v1from inside the sandbox. - Regenerates
prompt-optimization.lock.ymlto reflect the updated workflow body/hash and the new service/port wiring.
File summaries
| File | Description |
|---|---|
.github/workflows/prompt-optimization.md |
Adds an Ollama service, updates model-loading logic and eval endpoints to be sandbox-reachable. |
.github/workflows/prompt-optimization.lock.yml |
Recompiled lockfile reflecting the service container setup and updated compiled command-line flags/hashes. |
Review details
Suppressed comments (1)
.github/workflows/prompt-optimization.md:65
- The workflow now starts pulling models immediately, but there’s no explicit wait for the Ollama service to become ready (the previous /api/version readiness loop was removed). With a job service container this can be flaky if Ollama takes longer than a few retries to start accepting connections; add a short readiness wait before the pull loop.
run: |
set -e
for model in \
hf.co/bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M \
hf.co/unsloth/SmolLM2-360M-Instruct-GGUF:Q4_K_M; do
- Files reviewed: 2/2 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| - name: Load and verify evaluation models | ||
| run: | | ||
| set -e | ||
| for model in \ | ||
| hf.co/bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M \ |
| services: | ||
| ollama: | ||
| image: ollama/ollama:0.33.2@sha256:020e4134285e2ef4d8fd801234176de3b4faadc992a3eb06c8e66a2f9d4c4ba2 | ||
| ports: | ||
| - 11434:11434 |
The Prompt Optimization agent ran in an isolated container and could not reach the Ollama daemon started on the Actions runner. It fell back to a partial local installation with no models or inference backend.
Runner service exposure
Sandbox evaluation endpoint
host.docker.internalfor desktop and iOS proxy evaluation requests.prompt-optimization.lock.ymland its source-body hash.