wit-bindgen 0.50#1365
Merged
Merged
Conversation
This commit switches the way tests are built such that all binaries produced are "reactors" or those without a `main` function. This ensures that all imports/exports are fully described with WIT in this repository as opposed to implicitly relying on `wasi:cli/run`. This additionally means that testing in this repository no longer relies on toolchain conventions for the `wasi:cli/run` world making testing a bit more self-contained here, especially in the context of async. This notably came up on #1447 which should make some of the tests there easier to write and more idiomatic.
This is on its way out and shouldn't be used.
* reboot `wit-bindgen-go` This re-introduces Go support, with a new implementation written from scratch. While the previous incarnation targeted TinyGo and reused the C generator for lifting and lowering, this one targets "big" Go and handles lifting and lowering itself. In addition, it supports idiomatic, goroutine-based concurrency on top of the component model async ABI, including streams and futures. This implementation is also distinct from the [go-modules](https://github.com/bytecodealliance/go-modules) project, which has a number of limitations (e.g. no async support, not safe for use with "big" Go) and is no longer being actively maintained. Note that the async support currently requires [a small patch](dicej/go@a1c8322) to the Go `runtime` library. I plan to work with the upstream project to make that patch unecessary in the future. Components that don't use async features should work with stock, unpatched Go. One of the tricky parts about lowering values and passing pointers to e.g. `{stream,future}.{read,write}` is that we must tell the Go garbage collector to pin the pointers (i.e. mark the pointee as both immovable and uncollectable) for as long as the host has access to them, but then unpin them promptly afterward to avoid leaks. We do this using `runtime.Pinner` instances, most of which are locally scoped and easy to reason about. However, we must use a couple of globally-scoped pinners as well: one for `cabi_realloc` allocations by the host when it lowers parameters (which we unpin after lifting those parameters) and another for returning values from sync-lifted exports (which we unpin in post-return functions). There are a couple of known missing features which I'll open GitHub issues for: 1. Resource handles are not being restored for unwritten items in the case of an incomplete stream or future write. 2. Importing and/or exporting multiple versions of the same package will cause name clashes. In addition, I plan to expand the test coverage beyond the basics covered in this commit. Signed-off-by: Joel Dice <joel.dice@fermyon.com> add ping-pong Go tests Signed-off-by: Joel Dice <joel.dice@fermyon.com> address review feedback Signed-off-by: Joel Dice <joel.dice@fermyon.com> add wit-bindgen-go to ci/publish.rs Signed-off-by: Joel Dice <joel.dice@fermyon.com> enable Go in test CI matrix I'm assuming the default version of Go on the GitHub worker image will be sufficient; if not, we can install a different one explicitly. I'm not enabling the async tests for Go yet, since that will require publishing and installing a build of Go with [this patch](dicej/go@a1c8322); I'll need to do some homework to find out the best way to do that. Signed-off-by: Joel Dice <joel.dice@fermyon.com> fix codegen tests Signed-off-by: Joel Dice <joel.dice@fermyon.com> update Go runtime tests to use `run` export Signed-off-by: Joel Dice <joel.dice@fermyon.com> run cargo fmt Signed-off-by: Joel Dice <joel.dice@fermyon.com> remove `publish = false` from crates/go/Cargo.toml Signed-off-by: Joel Dice <joel.dice@fermyon.com> temporarily skip async codegen tests for Go Signed-off-by: Joel Dice <joel.dice@fermyon.com> add `async = true` config directive to `async-trait-function.wit` Signed-off-by: Joel Dice <joel.dice@fermyon.com> indicate that `async-trait-function.wit-no-std` works for Rust Signed-off-by: Joel Dice <joel.dice@fermyon.com> add error contexts to `Go::compile` Signed-off-by: Joel Dice <joel.dice@fermyon.com> add `actions/setup-go` to workflow Signed-off-by: Joel Dice <joel.dice@fermyon.com> add finalizers where appropriate; tweak `StreamReader` API Signed-off-by: Joel Dice <joel.dice@fermyon.com> update workspace to 2024 edition Signed-off-by: Joel Dice <joel.dice@fermyon.com> * remove indirection in `wit_types.Option` Signed-off-by: Joel Dice <joel.dice@fermyon.com> * add crates/go/README.md Signed-off-by: Joel Dice <joel.dice@fermyon.com> --------- Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Signed-off-by: Andrew Steurer <94206073+asteurer@users.noreply.github.com>
* add `string_dup_n` to C generator * c: add `string_dup_n` runtime tests and docs
Also, I've updated the Go runtime patch to provide a private symbol (`wasiOnIdle`) rather than a public one (`WasiOnIdle`) and access it using a `//go:linkname` directive. Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* run `gofmt` on generated code if available Signed-off-by: Joel Dice <joel.dice@fermyon.com> * add option to toggle gofmt Signed-off-by: Joel Dice <joel.dice@fermyon.com> --------- Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Signed-off-by: Andrew Steurer <94206073+asteurer@users.noreply.github.com>
* chore: fmt & improve code * test: add more tests * test: add resource test * chore: store ffi files separately * chore: move files * refactor: fix warnings * chore: simplify function bindgen * refactor: inline ffi * chore: separate derive_opts * chore: cleanup * refactor: extract pkg resolver * refactor: separate cases for type and type constructor * refactor: simplify InterfaceGenerator::finish * refactor: simplify code snippet structures * refactor: extract async related code * chore: fmt
Any type that implements `From<Vec<T>>`, `Into<Vec<T>>`, `Deref<Target=Vec<T>>`, and `Clone` can be used for a WIT type `list<T>` via the `with` option. The `Deref` and `Clone` impls are only required when the type is used as part of a non-canonical type.
Apparently github actions networking is really just that unreliable...
* Add new functions to generate rust bindings in `build.rs` Add `RustWasm::generate_to_out_dir` that parses the `wit` folder, selects a world, generates bindings and places them in a file in the `OUT_DIR`. `generate_to_out_dir_modify` allows modifying the generated file. * Use world name as output path in `generate_to_out_dir`
* support separation of imported interfaces to libraries * oh, the test name has to be unique across folders
We now keep track of any resource/stream/future handles we've lowered while writing to a stream or future and restore any that were unwritten so they can be used (e.g. possibly written) again. While I was working on this, clippy pointed out that `std::io::pipe` (which I started using in a previous commit) was added in Rust 1.87, so I've bumped the MSRV to match. Also, the test case I added to cover this revealed another bug, which I've fixed here: we weren't generating valid code for async functions which return tuples. Fixes #1458 Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* remove `CallbackCode::Poll` As of WebAssembly/component-model#578, that code no longer exists. Instead, we can emulate it via a combination of `CallbackCode::Yield` and `waitable-set.poll`. I'm working on a PR to make the corresponding changes to Wasmtime, and needed this to make the tests pass. * update C and Go generators to eliminate CALLBACK_CODE_POLL See my earlier commit updating the Rust generator for details. * fix test comment typos
[automatically-tag-and-release-this-commit] Co-authored-by: Auto Release Process <auto-release-process@users.noreply.github.com>
Merge upstream wit-bindgen 0.50.0 into the vendored bindgen subtree. - Bump wit-bindgen-core 0.49 -> 0.50 (wit-parser stays 0.243); no API churn. Non-carries (not applicable to wRPC, dropped from the merge): - the rebooted Go generator (bytecodealliance/wit-bindgen#1447, a brand-new `crates/go/` of ~4k lines plus Go runtime sources). Upstream removed its original Go backend at 0.41 and rewrote a completely different one here; wRPC's `wit-bindgen-go` is the pre-removal fork and continues independently, so the rebooted `crates/go` is dropped (and added to the merge DROP filter going forward). All subsequent Go-only upstream changes likewise do not apply. - bytecodealliance/wit-bindgen#1442 (custom Rust types for `list`) is implemented in the canonical-ABI `bindgen.rs`; wRPC already supports remapping named `list` type aliases through its per-type `with` remapping (bytecodealliance/wit-bindgen#1173, exercised by the `aliased_lists` test). - bytecodealliance/wit-bindgen#1444 (`build.rs` binding-generation helpers) changes `Opts::build()` to return a concrete generator, which is incompatible with wRPC's CLI dispatching Rust and Go generators polymorphically via `Box<dyn WorldGenerator>`; wRPC's primary path is the `generate!` macro. rust codegen (243) and go codegen (81) tests pass; `cargo clippy --workspace` and `cargo doc --workspace` are clean; wasmtime and all examples build. Assisted-by: claude:claude-opus-4-8 Upstream diff: bytecodealliance/wit-bindgen@v0.49.0...v0.50.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge upstream wit-bindgen 0.50.0 into the vendored bindgen subtree.
Non-carries (not applicable to wRPC, dropped from the merge):
wit-bindgen-gowit-bindgen#1447, a brand-newcrates/go/of ~4k lines plusGo runtime sources). Upstream removed its original Go backend at 0.41 and
rewrote a completely different one here; wRPC's
wit-bindgen-gois thepre-removal fork and continues independently, so the rebooted
crates/goisdropped (and added to the merge DROP filter going forward). All subsequent
Go-only upstream changes likewise do not apply.
listwit-bindgen#1442 (custom Rust types forlist) is implemented in the canonical-ABIbindgen.rs; wRPC already supports remapping namedlisttype aliases throughits per-type
withremapping (feature:withindividual type remapping support for Rust wit-bindgen#1173, exercised by thealiased_liststest).build.rswit-bindgen#1444 (build.rsbinding-generation helpers) changesOpts::build()to returna concrete generator, which is incompatible with wRPC's CLI dispatching Rust
and Go generators polymorphically via
Box<dyn WorldGenerator>; wRPC's primarypath is the
generate!macro.rust codegen (243) and go codegen (81) tests pass;
cargo clippy --workspaceand
cargo doc --workspaceare clean; wasmtime and all examples build.Assisted-by: claude:claude-opus-4-8
Upstream diff: bytecodealliance/wit-bindgen@v0.49.0...v0.50.0