wasip3::http_compat: update content-length, empty body, and error handling

[lann]

• BREAKING: Major changes to wasip3::http_compat error handling, including:

  • Renamed Error to BodyError
  • Added ConversionError
  • Changed many result error types

• Backfill missing content-length for requests in certain situations, which can improve compatibility with some servers.

• Skip content stream creation and spawn when creating wasi Requests / Responses with known-empty bodies.

[lann]

@fibonacci1729 @dicej I need to test a bit more but would appreciate a sanity check in parallel.

[pchickey]

After dealing with this in a number of different wasi-http embeddings, I advocate that we let the wasi-http implementation determine whether its appropriate to add a content-length, or use a transfer encoding, especially because content-length doesn't actually have meaning in http2 and http3. bytecodealliance/wstd#137

I would interpret the RFC here as the wasi-http implementation is the user agent, not the guest code - the guest code is the "user". In particular, content-length is forbidden in request headers in implementations backed by whatwg fetch https://fetch.spec.whatwg.org/#forbidden-request-header. So, this could potentially fail because the header is not-permitted, not just because it might be immutable.

[lann]

I advocate that we let the wasi-http implementation determine whether its appropriate to add a content-length

Depending on whether "wasi-http implementation" refers to the body producer or consumer here:

• Producer: This http_compat feature / module is effectively a (minimal) wasi-http implementation; it exists because it is quite difficult to build an efficient mapping from http/http-body crate types to wasi:http types.

• Consumer: The information from http_body::Body::size_hint / ::is_end_stream is lost when converting to wasi types; by the time the consumer sees it there is only a stream<u8> so it cannot choose to add content-length.

content-length doesn't actually have meaning in http2 and http3.

I don't think this is quite right; it is true that HTTP/2 (and 3 I assume) can send bodies without content-length or transfer-encoding, but content-length is still useful for e.g. early rejection of overly-large bodies. I think the RFC guidance is intentional here.

That said, the primary reason for this PR is indeed HTTP/1.1 request bodies where content-length is effectively mandatory in some scenarios.

So, this could potentially fail because the header is not-permitted, not just because it might be immutable.

Ah yeah that's what I had in mind, I just forgot the distinction between not-permitted and immutable here. 👍