Skip to content

NGLASS-2330 feat: Add optional async workflow submission timeout - #897

Open
afonsobspinto wants to merge 2 commits into
developfrom
codex/NGLASS-2330-workflow-timeout
Open

afonsobspinto wants to merge 2 commits into
developfrom
codex/NGLASS-2330-workflow-timeout

Conversation

@afonsobspinto

@afonsobspinto afonsobspinto commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Problem

Neuroglass's skeleton ingestion admission holds a study-row lock while submitting its Argo Workflow. CloudHarness's asynchronous execute() has no HTTP submission timeout, so a stalled Argo response can leave the API request and row lock waiting indefinitely. MNP currently bypasses execute() and constructs the SDK request directly to bound that wait.

Change

Add an optional keyword-only request_timeout to argo_service.submit_workflow(). AsyncOperation.execute(**kwargs) and ContainerizedOperation.submit(**kwargs) forward submission options to that service:

workflow = operation.execute(request_timeout=(2, 10))

The value is forwarded to the existing Argo SDK as its HTTP timeout, accepting seconds or a (connect, read) tuple. Existing calls without the keyword retain their behaviour and return the same Workflow wrapper. Synchronous operations' existing completion-wait timeout is unchanged. The submission HTTP client is closed on success and failure; exceptions propagate and no automatic resubmission is added.

This allows MNP #1466 to use CloudHarness's normal execution path while retaining exact-name reconciliation of uncertain submissions. A timeout does not imply that Argo rejected the Workflow.

Validation

  • CI on the generic keyword forwarding update (0f7c4bf): 72 common-library tests, 10 model tests, 281 deployment CLI tests, and 8 Argo component tests passed. Lint and CodeQL also passed.
  • 54 CloudHarness SDK/workflow tests passed, covering legacy calls, no-argument submit overrides, numeric/tuple timeouts, unchanged return values, one POST and client cleanup on errors.
  • Repository lint-check.sh passed.
  • 53 MNP ingestion workflow/progress/result/source tests passed against this branch, including a real stalled local HTTP server through execute(): one CREATE times out and is reconciled without waiting for the server or submitting again.
  • No live cluster changes or full-import rerun.

@afonsobspinto
afonsobspinto marked this pull request as ready for review November 9, 2026 19:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant