Update wasmtime (v24.0.0) #406

keithmattix · 2024-08-09T22:40:51Z

Builds on top of #404 and #402

martijneken · 2024-08-11T17:42:24Z

Ran CI. Looks like:

bazel/dependencies.bzl needs formatting
diffs in cargo vendor:
- run bazelisk run //bazel/cargo/wasmsign:crates_vendor -- --repin
- run bazelisk run //bazel/cargo/wasmtime:crates_vendor -- --repin
- commit above changes
CI unhappy (rust panics) with --define engine=v8 and --define engine=wasmtime (might be fixed by above)

bazel/external/rules_rust.patch

bazel/dependencies.bzl

martijneken · 2024-08-12T18:31:00Z

Looks like buildifier and wasmtime remain. Wasmtime build error:

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld.lld: error: undefined symbol: wasm_module_new
>>> referenced by wasmtime.cc
>>>               wasmtime.o:(proxy_wasm::wasmtime::Wasmtime::load(std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::unordered_map<unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<unsigned int>, std::__1::equal_to<unsigned int>, std::__1::allocator<std::__1::pair<unsigned int const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&)) in archive bazel-out/k8-fastbuild/bin/libwasmtime_lib.a

keithmattix · 2024-08-12T18:47:51Z

Yep iterating on it locally. I suspect The wasmtime build failure is related to the fact that wasmtime no longer vendors WASM-c-api so I'm figuring out how to make the new pathing with Bazel

PiotrSikora · 2024-08-12T19:51:42Z

Yep iterating on it locally. I suspect The wasmtime build failure is related to the fact that wasmtime no longer vendors WASM-c-api so I'm figuring out how to make the new pathing with Bazel

wasm-c-api is still vendored in Wasmtime (wheras previously it was imported as git submodule).

bazel/cargo/wasmtime/Cargo.toml

compile_commands.json

.vscode/settings.json

keithmattix · 2024-08-13T01:28:40Z

Looks like CI is pretty much passing except for buildifier (I have the fix staged locally) and this random windows failure. I wonder if it's related to #368 (comment)

keithmattix · 2024-08-13T18:37:56Z

Getting closer....I tried to repro the Windows issue on a local machine but no dice (rust-lld isn't happy for whatever reason). If anyone has a clue what's happening please let me know. I should also mention that Windows isn't actively supported in Envoy anymore; not sure what proxy-wasm's stance is on Windows

keithmattix · 2024-08-14T14:23:02Z

Looks like rules_rust no longer aims to support Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms. The current Windows failure seems to be a weird interaction with the Windows linker and path escaping: all of the file paths be linked use a combination of backslashes and escaped forward slashes

bazel-out/x64_windows-opt-exec-2B5CBBC6/bin/external/cu__wasmtime-versioned-export-macros-23.0.1\wasmtime_versioned_export_macros-1891060849.wasmtime_versioned_export_macros.92cc13fa569db95f-cgu.[0-9].rcgu.o

but the final error has all forward slashes

note: LINK : fatal error LNK1181: cannot open input file 'bazel-out\x64_windows-opt-exec-2B5CBBC6\bin\external\cu__wasmtime-versioned-export-macros-23.0.1\wasmtime_versioned_export_macros-1891060849.wasmtime_versioned_export_macros.92cc13fa569db95f-cgu.0.rcgu.o'

@PiotrSikora @martijneken @mpwarres can you advise on the support goals for Wasmtime windows in this repo? With Envoy and rules_rust turning down CI, does it make sense to do the same here?

martijneken · 2024-08-14T19:01:05Z

I don't feel strongly about supporting Wasmtime for Windows, since our dependencies don't support it. Could we keep NullVM on Windows (no Rust, passing) while ditching Wasmtime on Windows (depends on Rust)? @PiotrSikora WDYT

PiotrSikora · 2024-08-14T21:20:55Z

Could we keep NullVM on Windows (no Rust, passing) while ditching Wasmtime on Windows (depends on Rust)?

There is a big gap here - all Wasm engines other than Wasmtime. Notably, Wasmtime on Windows is the only build on the CI executing real WasmVM code paths (because it is the fastest to compile), so it would be helpful to add tests using another Wasm engine on Windows, before removing Wasmtime on Windows... assuming that you want to support it at all.

cc @shukitchan for ATS, which relies on proxy-wasm-cpp-host.

keithmattix · 2024-08-14T22:43:25Z

If any other proxy implementations have windows expertise, I'd appreciate the help/guidance!

PiotrSikora

Nitpicking as usual, but could this be split into rules_rust update followed by Wasmtime update?

.gitignore

keithmattix · 2024-08-16T12:48:14Z

Nitpicking as usual, but could this be split into rules_rust update followed by Wasmtime update?

I can give it a try

keithmattix · 2024-08-16T13:36:16Z

#410 opened. @mpwarres @PiotrSikora @martijneken can one of you kick off CI?

PiotrSikora · 2024-08-21T09:09:16Z

Could you rebase this on top of main branch and update Wasmtime to v24.0.0? Thanks!

This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com>

Signed-off-by: Martijn Stevenson <mstevenson@google.com>

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

keithmattix · 2024-08-21T14:38:14Z

CI looks mostly good after the rebase + update. @PiotrSikora @mpwarres @martijneken what's the final verdict on Windows support?

bazel/external/rules_rust.patch

PiotrSikora · 2024-08-22T17:59:56Z

@PiotrSikora @mpwarres @martijneken what's the final verdict on Windows support?

I think it would be useful to have something working, even if it's not Wasmtime, but I don't have a horse in this race, so the final decision is up to @mpwarres and @martijneken who are the current maintainers.

mpwarres · 2024-08-22T18:12:28Z

CI looks mostly good after the rebase + update. @PiotrSikora @mpwarres @martijneken what's the final verdict on Windows support?

My individual opinion is that the same considerations that led Bazel and Envoy to drop support for Windows CI (insufficient Windows expertise + lack of resources to track down Windows issues) apply here as well, plus the way that we build Wasmtime depends on rules_rust which no longer supports Windows, so we'd be on shaky ground. I'm in favor of dropping the Wasmtime Windows CI action, but keeping NullVm as a safeguard against total bitrot on Windows, leaving the option of reintroducing support in the future should circumstances change. @martijneken WDYT?

keithmattix · 2024-08-23T15:45:32Z

Looks like WAMR on windows still passes in CI (confirmed in #413). Can I get some reviews on that and I'll rebase once that merges

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

bazel/cargo/wasmtime/Cargo.toml

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

PiotrSikora

Thanks!

bazel/cargo/wasmtime/Cargo.toml

keithmattix · 2024-08-30T16:17:05Z

@mpwarres @martijneken is this in a good enough state to merge?

martijneken

Removed the wasmtime-windows CI action as required. Should be good to merge now.

bazel/external/wasm-c-api.BUILD

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fix proxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixes proxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

keithmattix requested review from PiotrSikora, martijneken and mpwarres as code owners August 9, 2024 22:40

PiotrSikora reviewed Aug 11, 2024

View reviewed changes

bazel/external/rules_rust.patch Show resolved Hide resolved

bazel/dependencies.bzl Outdated Show resolved Hide resolved

keithmattix force-pushed the update-wasmtime branch from a55a2d8 to 4478994 Compare August 12, 2024 02:11

PiotrSikora reviewed Aug 12, 2024

View reviewed changes

bazel/cargo/wasmtime/Cargo.toml Outdated Show resolved Hide resolved

PiotrSikora reviewed Aug 12, 2024

View reviewed changes

compile_commands.json Outdated Show resolved Hide resolved

PiotrSikora reviewed Aug 12, 2024

View reviewed changes

.vscode/settings.json Outdated Show resolved Hide resolved

keithmattix requested a review from PiotrSikora August 12, 2024 21:12

martijneken mentioned this pull request Aug 13, 2024

deps: Bump proxy_wasm_rust_sdk -> 0.2.2 envoyproxy/envoy#35323

Merged

PiotrSikora reviewed Aug 16, 2024

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

keithmattix mentioned this pull request Aug 16, 2024

Update rules_rust to v0.42.1 (with Rust v1.77.2). #410

Merged

martijneken and others added 5 commits August 21, 2024 13:26

Formatting fix

4291110

Signed-off-by: Martijn Stevenson <mstevenson@google.com>

Update wasmtime, Rust, and rules_rust

7a1c8a5

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Fixup

ddd0659

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Add other features from cargo

983dc6c

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

keithmattix added 4 commits August 21, 2024 13:30

Get tests passing locally

07b4096

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Buildifier format

53a4a5e

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Don't copy include.h so we don't accidentally clobber in the future

0d5bb6e

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Bump wasmtime to 24.0.0

334f0b3

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

keithmattix force-pushed the update-wasmtime branch from 58b260d to 334f0b3 Compare August 21, 2024 13:55

Remove .vscode from gitignore

9bfa856

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

keithmattix changed the title ~~Update wasmtime and Rust~~ Update wasmtime (v24.0.0) Aug 21, 2024

PiotrSikora reviewed Aug 22, 2024

View reviewed changes

bazel/external/rules_rust.patch Show resolved Hide resolved

PiotrSikora mentioned this pull request Aug 23, 2024

Remove Windows from CI #413

Closed

keithmattix added 2 commits August 23, 2024 16:30

Remove wasmtime on Windows

09adbd5

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Add back blank space

0d260ac

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

PiotrSikora reviewed Aug 27, 2024

View reviewed changes

bazel/cargo/wasmtime/Cargo.toml Show resolved Hide resolved

bazel/cargo/wasmtime/Cargo.toml Outdated Show resolved Hide resolved

keithmattix added 2 commits August 27, 2024 14:07

Change rust version and fix dependencies

e5e77c8

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Add back env_logger

2a32d25

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

PiotrSikora approved these changes Aug 27, 2024

View reviewed changes

bazel/cargo/wasmtime/Cargo.toml Show resolved Hide resolved

martijneken approved these changes Sep 4, 2024

View reviewed changes

martijneken merged commit 21a5b08 into proxy-wasm:main Sep 4, 2024

PiotrSikora reviewed Sep 26, 2024

View reviewed changes

bazel/external/wasm-c-api.BUILD Show resolved Hide resolved

keithmattix deleted the update-wasmtime branch September 26, 2024 17:41

martijneken mentioned this pull request Jan 6, 2025

wasmtime: update to v13.0.0. #368

Closed

Update wasmtime (v24.0.0) #406

Update wasmtime (v24.0.0) #406

Uh oh!

Conversation

keithmattix commented Aug 9, 2024

Uh oh!

martijneken commented Aug 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

martijneken commented Aug 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

keithmattix commented Aug 12, 2024

Uh oh!

PiotrSikora commented Aug 12, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

keithmattix commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

keithmattix commented Aug 13, 2024

Uh oh!

keithmattix commented Aug 14, 2024

Uh oh!

martijneken commented Aug 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PiotrSikora commented Aug 14, 2024

Uh oh!

keithmattix commented Aug 14, 2024

Uh oh!

PiotrSikora left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

keithmattix commented Aug 16, 2024

Uh oh!

keithmattix commented Aug 16, 2024

Uh oh!

PiotrSikora commented Aug 21, 2024

Uh oh!

keithmattix commented Aug 21, 2024

Uh oh!

Uh oh!

PiotrSikora commented Aug 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mpwarres commented Aug 22, 2024

Uh oh!

keithmattix commented Aug 23, 2024

Uh oh!

Uh oh!

Uh oh!

PiotrSikora left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

keithmattix commented Aug 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

martijneken left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

martijneken commented Aug 11, 2024 •

edited

Loading

martijneken commented Aug 12, 2024 •

edited

Loading

keithmattix commented Aug 13, 2024 •

edited

Loading

martijneken commented Aug 14, 2024 •

edited

Loading

PiotrSikora commented Aug 22, 2024 •

edited

Loading

keithmattix commented Aug 30, 2024 •

edited

Loading