feat(p3): implement `wasi:http` #11440

rvolosatovs · 2025-08-15T17:53:14Z

This PR builds on top of #11221 and will contain the full implementation of wasi:http proposal with refcounting (no child resources). This will be made ready-for-review once all currently ignored tests pass

4819aad is currently the only commit here worth looking at

alexcrichton · 2025-08-15T18:47:28Z

crates/wasi-http/src/p3/host/mod.rs

+fn get_fields<'a>(
+    table: &'a ResourceTable,
+    fields: &Resource<Fields>,
+) -> wasmtime::Result<&'a Fields> {
+    table
+        .get(&fields)
+        .context("failed to get fields from table")
+}
+
+fn get_fields_mut<'a>(
+    table: &'a mut ResourceTable,
+    fields: &Resource<Fields>,
+) -> wasmtime::Result<&'a mut Fields> {
+    table
+        .get_mut(&fields)
+        .context("failed to get fields from table")
+}


One suggestion perhaps for many of these methods is to have a generic extension trait which supports an arbitrary T for Resource<T> which could help deduplicate these methods?

crates/wasi-http/src/p3/host/types.rs

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

rvolosatovs · 2025-09-05T17:55:55Z

I'm done for the week and will continue on Monday, but this is pretty close to having all test cases passing.

@alexcrichton @dicej feel free to make the necessary changes in the PR to make the rest of the test cases work and get this over the line if there's a desire to still finish wasi:http this week.
Alternatively, merging this PR as-is and addressing the rest in follow-ups would work as well.

If we're okay with waiting until next week, this should be complete by Monday evening

Try to keep fields private and use public ctors instead.

It's still semi-failing, but defer that to bytecodealliance#11631

alexcrichton

Ok I've gone over this and it looks good to me, thanks again @rvolosatovs!

I've left a bunch of comments below which I'd like to resolve at some point, but doesn't need to block. I'm going to flag this for merge and I'll file follow-ups.

crates/wasi-http/tests/all/p3/mod.rs

alexcrichton · 2025-09-05T21:54:37Z

crates/wasi-http/src/p3/response.rs

+    pub fn into_http<T: WasiHttpView + 'static>(
+        self,
+        store: impl AsContextMut<Data = T>,
+        fut: impl Future<Output = Result<(), ErrorCode>> + Send + 'static,


Could you comment this function to the purpose of fut?

alexcrichton · 2025-09-05T21:54:53Z

crates/wasi-http/src/p3/response.rs

+        body: impl Into<BoxBody<Bytes, ErrorCode>>,
+    ) -> (
+        Self,
+        impl Future<Output = Result<(), ErrorCode>> + Send + 'static,


Could you comment function as to what this future is?

alexcrichton · 2025-09-05T21:55:17Z

crates/wasi-http/src/p3/request.rs

+        req: http::Request<T>,
+    ) -> (
+        Self,
+        impl Future<Output = Result<(), ErrorCode>> + Send + 'static,


Like response below, could you comment this returned future here and with new?

alexcrichton · 2025-09-05T22:14:50Z

crates/wasi-http/src/p3/body.rs

+    }
+}
+
+pub(crate) struct IncomingResponseBody {


Mind adding a small doc block here explaining what this is doing? The code is pretty self-explanatory but I still think it'd be good to have an intro

crates/wasi-http/src/p3/body.rs

alexcrichton · 2025-09-05T22:17:11Z

crates/wasi-http/src/p3/body.rs

+    fn poll_frame(
+        mut self: Pin<&mut Self>,
+        cx: &mut Context<'_>,
+    ) -> Poll<Option<Result<http_body::Frame<Self::Data>, Self::Error>>> {
+        if let Some(contents_rx) = self.contents_rx.as_mut() {
+            while let Some(buf) = ready!(contents_rx.poll_recv(cx)) {
+                return Poll::Ready(Some(Ok(http_body::Frame::data(buf))));
+            }
+            self.contents_rx = None;
+        }
+
+        let Some(trailers_rx) = self.trailers_rx.as_mut() else {
+            return Poll::Ready(None);
+        };
+
+        let res = ready!(Pin::new(trailers_rx).poll(cx));
+        self.trailers_rx = None;
+        match res {
+            Ok(Ok(Some(trailers))) => Poll::Ready(Some(Ok(http_body::Frame::trailers(
+                Arc::unwrap_or_clone(trailers),
+            )))),
+            Ok(Ok(None)) => Poll::Ready(None),
+            Ok(Err(err)) => Poll::Ready(Some(Err(err))),
+            Err(..) => Poll::Ready(None),
+        }
+    }


Mind leaving some comments here (and in is_end_stream) about the state machine being managed here? Doesn't need to be too wordy but having headings to grasp on I think would be helpful

crates/wasi-http/src/p3/host/types.rs

alexcrichton · 2025-09-05T22:22:23Z

crates/wasi-http/src/p3/host/types.rs

+    fn poll_produce<'a>(
+        mut self: Pin<&mut Self>,
+        cx: &mut Context<'_>,
+        mut store: StoreContextMut<'a, D>,
+        mut dst: Destination<'a, Self::Item, Self::Buffer>,
+        finish: bool,
+    ) -> Poll<wasmtime::Result<StreamResult>> {


Mind leaving some small comments in this function body to help navigate what the state machine is doing? (e.g. a ocmment explaining the signficance of each arm here and there)

crates/test-programs/src/bin/p3_http_outbound_request_content_length.rs

* feat(p3-http): begin implementation Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): implement incoming HTTP Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): switch to new API Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): begin outgoing implementation Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * test(p3-http): test invalid header Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * fix(p3-http): await I/O if future receiver dropped Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * Re-internalize some fields Try to keep fields private and use public ctors instead. * Uncomment `p3_http_outbound_request_content_length` test It's still semi-failing, but defer that to bytecodealliance#11631 * Get tests passing CI again --------- Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> Co-authored-by: Alex Crichton <alex@alexcrichton.com>

* wasip3: Update std{out,err} implementation (#11618) * wasip3: Update std{out,err} implementation Match the WASIp2 stdio implementation and "lie" about stdio actually being async. Instead of using Tokio's primitives use the `std::io` primitives which will block the current program while the output is produced. This matches WASIp2 semantics and we always have the option of making this more async in the future (or even have it as a runtime flag). This avoids the need to call `poll_flush` in stdio unconditionally in the `wasi:cli` implementation which didn't feel quite right. * One fewer run for Windows It's got a shorter command line length limit * wasip3: Refactor file in/out streams (#11619) * Deduplicate write/append streams * Deduplicate poll-the-task and spawn-the-task-then-poll * Explicitly ignore the `finish` flag * Support store-access in `bindgen!` generated imports (#11628) * Support store-access in `bindgen!` generated imports This commit adds support to accessing the store in `bindgen!`-generated import functions in traits. Since the inception of `bindgen!` this has never been possible and access to the store requires manually working with `Linker`, for example. This is not easy to do because it requires surgically editing code or working around what bindings generation parts you do want. The implementation here is a small step away from what component-model-async has already implemented for async functions. Effectively it's a small extension of the `*WithStore` traits to also have synchronous functions with `Access` parameters instead of `async` functions with an `Accessor` parameter. This is something we're going to want for the WASIp3 implementation where I've noticed some resource destructors are going to want access to the store to close out streams and such and this'll provide the bindings necessary for that. Closes #11590 * Update test expectations * feat(p3): implement `wasi:http` (#11440) * feat(p3-http): begin implementation Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): implement incoming HTTP Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): switch to new API Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): begin outgoing implementation Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * test(p3-http): test invalid header Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * fix(p3-http): await I/O if future receiver dropped Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * Re-internalize some fields Try to keep fields private and use public ctors instead. * Uncomment `p3_http_outbound_request_content_length` test It's still semi-failing, but defer that to #11631 * Get tests passing CI again --------- Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> Co-authored-by: Alex Crichton <alex@alexcrichton.com> * Tune new "much stdout" test for runtime (#11635) On Pulley platforms this test is taking 40+ minutes in CI. Locally it took 2 minutes to complete. After this change it should still roughly test the same thing but takes less than 100ms to complete locally. --------- Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> Co-authored-by: Roman Volosatovs <rvolosatovs@users.noreply.github.com>

* feat(p3-http): begin implementation Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): implement incoming HTTP Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): switch to new API Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(p3-http): begin outgoing implementation Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * test(p3-http): test invalid header Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * fix(p3-http): await I/O if future receiver dropped Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * Re-internalize some fields Try to keep fields private and use public ctors instead. * Uncomment `p3_http_outbound_request_content_length` test It's still semi-failing, but defer that to bytecodealliance#11631 * Get tests passing CI again --------- Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> Co-authored-by: Alex Crichton <alex@alexcrichton.com>

rvolosatovs force-pushed the feat/wasip3-http-impl branch from 480558d to 4819aad Compare August 15, 2025 18:08

alexcrichton reviewed Aug 15, 2025

View reviewed changes

rvolosatovs force-pushed the feat/wasip3-http-impl branch 4 times, most recently from a68602e to 07e7597 Compare August 19, 2025 15:04

rvolosatovs mentioned this pull request Aug 28, 2025

Revamp component model stream/future host API (again) #11515

Merged

rvolosatovs force-pushed the feat/wasip3-http-impl branch 2 times, most recently from 41d8d1a to 19f51fc Compare September 5, 2025 13:55

github-actions bot added the wasi Issues pertaining to WASI label Sep 5, 2025

alexcrichton added this to Ship WASIp3 Sep 5, 2025

alexcrichton moved this to In progress in Ship WASIp3 Sep 5, 2025

rvolosatovs force-pushed the feat/wasip3-http-impl branch from 19f51fc to 1916d2a Compare September 5, 2025 17:25

rvolosatovs added 2 commits September 5, 2025 19:27

feat(p3-http): begin implementation

eaee3a1

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

feat(p3-http): implement incoming HTTP

127d1e7

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

rvolosatovs force-pushed the feat/wasip3-http-impl branch from 1916d2a to 3f80110 Compare September 5, 2025 17:27

feat(p3-http): switch to new API

66d87ce

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

rvolosatovs force-pushed the feat/wasip3-http-impl branch from 3f80110 to d26886d Compare September 5, 2025 17:31

feat(p3-http): begin outgoing implementation

d7b5ed6

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

rvolosatovs force-pushed the feat/wasip3-http-impl branch from d26886d to d7b5ed6 Compare September 5, 2025 17:31

rvolosatovs added 2 commits September 5, 2025 19:42

test(p3-http): test invalid header

0b34085

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

fix(p3-http): await I/O if future receiver dropped

db4652c

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

Re-internalize some fields

989769c

Try to keep fields private and use public ctors instead.

alexcrichton mentioned this pull request Sep 5, 2025

wasip3: HTTP: content-length handling needs filling out #11631

Closed

Uncomment p3_http_outbound_request_content_length test

f2ff970

It's still semi-failing, but defer that to bytecodealliance#11631

alexcrichton approved these changes Sep 5, 2025

View reviewed changes

alexcrichton marked this pull request as ready for review September 5, 2025 22:24

alexcrichton requested review from a team as code owners September 5, 2025 22:24

alexcrichton requested a review from a team as a code owner September 5, 2025 22:24

alexcrichton requested review from dicej and removed request for a team September 5, 2025 22:24

alexcrichton enabled auto-merge September 5, 2025 22:25

alexcrichton mentioned this pull request Sep 5, 2025

wasip3: HTTP Apply follow-up feedback from review #11632

Closed

alexcrichton added this pull request to the merge queue Sep 5, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 5, 2025

rvolosatovs commented Sep 6, 2025

View reviewed changes

crates/test-programs/src/bin/p3_http_outbound_request_content_length.rs Outdated Show resolved Hide resolved

Get tests passing CI again

22a5159

alexcrichton enabled auto-merge September 7, 2025 21:43

alexcrichton added this pull request to the merge queue Sep 7, 2025

Merged via the queue into bytecodealliance:main with commit 5674e4b Sep 7, 2025
86 of 88 checks passed

github-project-automation bot moved this from In progress to Done in Ship WASIp3 Sep 7, 2025

rvolosatovs deleted the feat/wasip3-http-impl branch September 8, 2025 08:46

rvolosatovs mentioned this pull request Sep 8, 2025

p3-http: finish wasi:http@0.3 implementation #11636

Merged

rvolosatovs mentioned this pull request Sep 9, 2025

p3-http: implementation follow-up #11649

Merged

rvolosatovs self-assigned this Sep 9, 2025

rvolosatovs mentioned this pull request Oct 17, 2025

Add Request::into_http #11843

Open

Uh oh!

feat(p3): implement wasi:http #11440

feat(p3): implement wasi:http #11440

Uh oh!

Conversation

rvolosatovs commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcrichton Aug 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rvolosatovs commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alexcrichton Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

alexcrichton Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

alexcrichton Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

alexcrichton Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alexcrichton Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alexcrichton Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat(p3): implement `wasi:http` #11440

feat(p3): implement `wasi:http` #11440

rvolosatovs commented Aug 15, 2025 •

edited

Loading

rvolosatovs commented Sep 5, 2025 •

edited

Loading