Skip to content

Let clients request gzipped responses from the external API#10341

Open
david-crespo wants to merge 1 commit intomainfrom
nexus-gzip
Open

Let clients request gzipped responses from the external API#10341
david-crespo wants to merge 1 commit intomainfrom
nexus-gzip

Conversation

@david-crespo
Copy link
Copy Markdown
Contributor

@david-crespo david-crespo commented Apr 29, 2026

Will update with examples of how much smaller the responses are and more detail about what I've tested locally and how we should test this on real systems. In my most recent testing we get about 80% compression on bigger responses. JSON is extremely repetitive.

Changes

Flip CompressionConfig::NoneCompressionConfig::Gzip on dropshot_external in sled-agent/src/services.rs. Internal and lockstep are left as-is for now. oxidecomputer/dropshot#1448 in dropshot 0.17 does all the real work:

  • streaming gzip via async-compression
  • content-type allowlist (application/json, text/*, application/xml, ndjson, +json/+xml)
  • exclusions for HEAD, 1xx (so the serial-console websocket upgrade is safe), 204, 304, 206, Content-Range, already-encoded responses, text/event-stream, and bodies under 512 bytes
  • RFC-compliant Accept-Encoding + q-value parsing
  • opt-out via a NoCompression extension marker
  • Vary: Accept-Encoding on every compressible response whether or not it was actually compressed

The console asset logic in nexus/src/external_api/console_api.rs already serves precompressed .gz files with Content-Encoding: gzip set, and dropshot's should_compress_response skips anything with that header, which avoids double-encoding.

HandlerTaskMode::Detached,
log_headers: vec![],
compression: dropshot::CompressionConfig::None,
compression: dropshot::CompressionConfig::Gzip,
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this is the one that matters for production.

# concurrently.
bind_address = "127.0.0.1:0"
default_request_body_max_bytes = 1048576
compression = "gzip"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this turns it on for all tests. Not sure we want to do that, but for the most part it doesn't matter because I don't think requests in tests ask for gzipped responses by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant