Skip to content

Fix Prompt Optimization Ollama sandbox access - #247

Closed
pelikhan with Copilot wants to merge 11 commits into
mainfrom
copilot/fix-prompt-optimization-result
Closed

Fix Prompt Optimization Ollama sandbox access#247
pelikhan with Copilot wants to merge 11 commits into
mainfrom
copilot/fix-prompt-optimization-result

Conversation

Copilot AI commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

The Prompt Optimization agent ran in an isolated container and could not reach the Ollama daemon started on the Actions runner. It fell back to a partial local installation with no models or inference backend.

  • Runner service exposure

    • Bind Ollama to all interfaces so the sandbox can reach the runner-hosted daemon.
  • Sandbox evaluation endpoint

    • Use host.docker.internal for desktop and iOS proxy evaluation requests.
    • Preserve the existing restricted host-port allowlist.
OLLAMA_HOST=0.0.0.0:11434 ollama serve
--eval-url http://host.docker.internal:11434/v1
--ios-eval-url http://host.docker.internal:11434/v1
  • Generated workflow
    • Synchronize prompt-optimization.lock.yml and its source-body hash.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix incomplete result reported in prompt optimization Fix Prompt Optimization Ollama sandbox access Sep 1, 2026
Copilot AI requested a review from pelikhan September 1, 2026 15:33
@pelikhan

pelikhan commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

@copilot review workflow following playwright.md guidance

Copilot AI and others added 2 commits September 1, 2026 16:03
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

@copilot review workflow following playwright.md guidance

Reviewed against playwright.md. Its no-0.0.0.0 rule applies to servers started inside the agent sandbox; this is a runner-hosted Ollama service, which must be exposed through the existing restricted host-port path. I also fixed the setup action's pre-started loopback daemon conflict in 81b38b8 and b342ed7.

@pelikhan

pelikhan commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot review the playwright instructions in /agentic-workflows and fix this issue again.

Copilot AI and others added 7 commits September 5, 2026 00:53
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

@copilot review the playwright instructions in /agentic-workflows and fix this issue again.

Reworked the workflow to use a digest-pinned declarative Ollama service, as the sandbox guidance prescribes for host services. The agent now reaches only the dynamically published service port; model pull/load requests have explicit retry and API-error checks. Latest commit: d8c2588.

@pelikhan
pelikhan marked this pull request as ready for review September 5, 2026 02:23
Copilot AI lite review requested due to automatic review settings September 5, 2026 02:23
@pelikhan

pelikhan commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot resolve the merge conflicts on this branch.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow change introduces reliability/operational concerns (no explicit service readiness wait; models will be re-downloaded every run) and the implementation currently contradicts the PR description’s runner-hosted Ollama approach.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates the Prompt Optimization agentic workflow to make the Ollama evaluation endpoint reachable from the gh-aw sandboxed agent container, addressing the infrastructure failure described in #242.

Changes:

  • Switches Ollama provisioning to a GitHub Actions job service and maps port 11434.
  • Updates the prompt instructions and eval URLs to use http://host.docker.internal:11434/v1 from inside the sandbox.
  • Regenerates prompt-optimization.lock.yml to reflect the updated workflow body/hash and the new service/port wiring.
File summaries
File Description
.github/workflows/prompt-optimization.md Adds an Ollama service, updates model-loading logic and eval endpoints to be sandbox-reachable.
.github/workflows/prompt-optimization.lock.yml Recompiled lockfile reflecting the service container setup and updated compiled command-line flags/hashes.
Review details

Suppressed comments (1)

.github/workflows/prompt-optimization.md:65

  • The workflow now starts pulling models immediately, but there’s no explicit wait for the Ollama service to become ready (the previous /api/version readiness loop was removed). With a job service container this can be flaky if Ollama takes longer than a few retries to start accepting connections; add a short readiness wait before the pull loop.
    run: |
      set -e
      for model in \
        hf.co/bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M \
        hf.co/unsloth/SmolLM2-360M-Instruct-GGUF:Q4_K_M; do
  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 60 to 64
- name: Load and verify evaluation models
run: |
set -e
for model in \
hf.co/bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M \
Comment on lines +43 to +47
services:
ollama:
image: ollama/ollama:0.33.2@sha256:020e4134285e2ef4d8fd801234176de3b4faadc992a3eb06c8e66a2f9d4c4ba2
ports:
- 11434:11434
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Prompt Optimization reported incomplete result

3 participants