temporalio: add build-temporalio.yml for riscv64 wheels - #477
Merged
Conversation
Mirrors the build_binaries job of temporalio/sdk-python's release-publish.yml, narrowed to the manylinux riscv64 leg. The bridge is a PyO3 `abi3-py310` cdylib over the sdk-core Rust workspace, so upstream's single cp310 build produces the one cp310-abi3 wheel that covers every non-free-threaded CPython >= 3.10; upstream's [tool.cibuildwheel.linux] before-all (openssl-devel + rustup) and environment are inherited unchanged. Two riscv64 overrides. Upstream's before-build installs protoc-wheel-0, which publishes neither a riscv64 wheel nor an sdist; the manylinux_2_39_riscv64 image is Rocky 10 with CRB already enabled, so protoc comes from protobuf-compiler/protobuf-devel instead. That protoc is 3.19.6, which unlike protoc >= 22 does not carry the well-known types inside the binary, hence PROTOC_INCLUDE=/usr/include (prost-build 0.14 passes it through as an -I). Verified by building temporalio-protos against it in quay.io/pypa/manylinux_2_39_aarch64: the 95 .proto files are proto2/proto3 with no editions, so 3.19 compiles the whole descriptor set. Testing cannot follow upstream's `pytest -k test_workflow_hello`, which needs a local Temporal dev server: temporal.download answers HTTP 400 for arch=riscv64, so no ephemeral server binary exists. The test command instead runs the 401 tests of upstream's suite that need no server -- including tests/bridge and the in-process Replayer tests, which exercise sdk-core directly -- with the server-dependent tests in the mixed modules deselected by node id. TEMPORAL_INTEGRATION_TEST=1 turns on upstream's own conftest assertion that `temporalio` is imported from the installed wheel, and the test command is prefixed with an assertion that the bridge module resolves to the .so. protobuf is held below 7 (upstream's uv.lock has 6.33.6) because tests/nexus/test_temporal_system_nexus.py's round-trip helper eagerly evaluates `getattr(field, "label")`, removed from the upb FieldDescriptor in protobuf 7 -- reproduced on macOS/arm64, so arch-independent. PIP_ONLY_BINARY=:all: keeps cryptography and pydantic-core resolving to our registry's riscv64 wheels (pip resolves cryptography 45.0.4 cp39-abi3, pydantic 2.13.4/pydantic-core 2.46.4 for cp310) rather than source-building them. Validated end to end with cibuildwheel 4.2.0 on the equivalent manylinux_2_39_aarch64 image: one temporalio-1.32.0-cp310-abi3-manylinux_2_39_aarch64.whl (14.2 MB, against upstream's 13.9 MB on PyPI), abi3audit --strict clean (baseline 3.10, computed 3.10), 401 tests passing in-container. The wheel carries LICENSE (MIT) in dist-info/licenses plus sdk-core's and the vendored protos' LICENSE files, and maturin emits a CycloneDX SBOM of the whole statically linked Rust dependency graph, so the linked crates are covered. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The first riscv64 run built the wheel fine (100 minutes) but the test phase came back 228 passed / 173 errors, every error `RuntimeError: Failed starting Temporal dev server: Unsupported arch: riscv64` raised from the `env`/`client` fixtures, and every one of them in tests/nexus. The macOS dry run had missed them because there the fixture happily downloads and runs a real dev server, so "no server needed" was never actually tested. Reproduce it properly instead of by inspection: install the wheel into a venv in a docker volume, then run the suite in a second `docker run --network none` against `quay.io/pypa/manylinux_2_39_aarch64`. That reproduced the CI numbers exactly (228 passed, 8 deselected, 173 errors) and enumerates the offending modules. Fifteen of the twenty-one tests/nexus modules contain server-dependent tests, so tests/nexus is replaced by the six that do not, plus test_temporal_system_nexus.py with its two server tests deselected by node id. 219 tests, offline-verified green. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
.github/workflows/build-temporalio.yml, mirroring thebuild_binariesjob of temporalio/sdk-python'srelease-publish.ymlnarrowed to the manylinux riscv64 leg.temporaliois a PyO3abi3-py310cdylib (temporalio/bridge) over the sdk-core Rust workspace, so upstream's singlecp310cibuildwheel build produces the onecp310-abi3wheel that covers every non-free-threaded CPython >= 3.10, and upstream's[tool.cibuildwheel.linux]before-all(openssl-devel + rustup) andenvironmentare inherited unchanged.riscv64 overrides
protoc. Upstream's
before-buildscript installsprotoc-wheel-0, which publishes neither a riscv64 wheel nor an sdist, so it is replaced withyum install -y protobuf-compiler protobuf-devel: themanylinux_2_39_riscv64image is Rocky 10 with the CRB repo already enabled, and CRB carries protoc. That protoc is 3.19.6, which — unlike protoc >= 22 — does not carry the well-known types inside the binary, hencePROTOC_INCLUDE=/usr/include(prost-build 0.14 forwards it as an extra-I). Verified by buildingtemporalio-protosagainst it inquay.io/pypa/manylinux_2_39_aarch64: sdk-core's 95.protofiles are proto2/proto3 with no editions, so 3.19 compiles the whole descriptor set cleanly.Testing. Upstream tests its release wheels with
pytest -s tests/worker/test_workflow.py -k test_workflow_hello, which needs a local Temporal dev server — and there is no riscv64 build of it:temporal.downloadanswers HTTP 400 forarch=riscv64, and the bridge raisesRuntimeError: Failed starting Temporal dev server: Unsupported arch: riscv64. The test command instead runs the 219 tests of upstream's own suite that need no server, includingtests/bridgeand the in-processReplayertests, which drive sdk-core directly; the six server-dependent tests insidetests/worker/test_replayer.pyand two insidetests/nexus/test_temporal_system_nexus.pyare dropped by node id, and the fifteentests/nexusmodules that are entirely server-dependent are left out.TEMPORAL_INTEGRATION_TEST=1turns on upstream's ownconftest.pyassertion thattemporaliois imported from the installed wheel rather than a source tree, and the command is prefixed with an assertion thattemporalio.bridge.temporal_sdk_bridge.__file__ends in.so.Test dependency pins.
protobuf<7matches upstream'suv.lock(6.33.6):tests/nexus/test_temporal_system_nexus.py's proto round-trip helper eagerly evaluatesgetattr(field, "label"), which the upbFieldDescriptordropped in protobuf 7 — reproduced on macOS/arm64, so it is toolchain drift rather than anything riscv64-specific.PIP_ONLY_BINARY=:all:(test phase only) keepscryptographyandpydantic-coreresolving to our registry's riscv64 wheels instead of source-building; apip download --platform manylinux_2_39_riscv64 --python-version 310dry run resolves cryptography 45.0.4 (cp39-abi3), pydantic 2.13.4 with pydantic-core 2.46.4, and protobuf 6.33.6, all as wheels.Local validation
Ran the workflow end to end with cibuildwheel 4.2.0 as
--only cp310-manylinux_aarch64against the equivalentquay.io/pypa/manylinux_2_39_aarch64image (same Rocky 10 family, so it exercises the identicalbefore-build, environment,test-sourcesand test command): onetemporalio-1.32.0-cp310-abi3-manylinux_2_39_aarch64.whlat 14.2 MB (upstream ships 13.9 MB on PyPI for aarch64),abi3audit --strictclean (baseline 3.10, computed 3.10), and 401 tests passing inside the container. The wheel is built on cp310, the oldest interpreter its abi3 tag claims, as upstream does.The "needs no server" half of the test selection is verified the same way rather than by inspection: the wheel is installed into a venv in a docker volume and the suite is then run in a second
docker run --network noneagainst the same image, so a fixture that would silently download and start a real dev server on a networked host fails there exactly as it does on riscv64. The first riscv64 run is what exposed the gap — it built the wheel cleanly in 100 minutes and then reported 228 passed / 173 errors, all of themUnsupported arch: riscv64out oftests/nexus— and the offline aarch64 run reproduces those numbers exactly (228 passed, 8 deselected, 173 errors), which is how the module list was derived. The same 401 tests were also run on macOS/arm64 against upstream's released 1.32.0 wheel on both CPython 3.10 and 3.12, using the exact dependency versions the riscv64 resolution picks.Licensing
The wheel ships
LICENSE(MIT) indist-info/licenses/, plus sdk-core'sLICENSE.txtand the vendored api/api-cloud protos'LICENSEfiles, and maturin emits a CycloneDX SBOM (dist-info/sboms/temporal-sdk-bridge.cyclonedx.json) covering the whole statically linked Rust dependency graph — same as upstream's own wheels, so nothing riscv64-specific is missing.