Skip to content

fix: handle non-string request bodies in error-logging round-trip dump - #1497

Open
mittalpk wants to merge 1 commit into
databricks:mainfrom
mittalpk:fix/round-trip-logger-non-string-request-body
Open

fix: handle non-string request bodies in error-logging round-trip dump#1497
mittalpk wants to merge 1 commit into
databricks:mainfrom
mittalpk:fix/round-trip-logger-non-string-request-body

Conversation

@mittalpk

Copy link
Copy Markdown

Summary

RoundTrip.generate() (databricks/sdk/logger/round_trip_logger.py) passes request.body directly to _redacted_dump(), which assumes a str. This crashes in two ways, both inside the "unable to parse response" debug-log path itself — so the crash masks the actual API error behind an unrelated Python traceback:

  1. Streamed upload (a file-like object passed as data= to requests, e.g. dbutils.files.upload from a local file): requests leaves request.body as the original file object rather than reading it. _redacted_dump's len(body) call then raises TypeError: object of type '...' has no len().
  2. Non-UTF8 bytes body: once JSON parsing fails, the fallback path calls body.encode(...) on an already-bytes value, raising AttributeError: 'bytes' object has no attribute 'encode'.

Fixes #1489 (the streamed-upload crash); the non-UTF8-bytes crash is the same root cause (an assumption that request.body is always str) so it's fixed and tested alongside it rather than left for a follow-up.

Fix

Added _request_body_str() to normalize request.body into a string before it reaches _redacted_dump:

  • str passes through unchanged.
  • bytes/bytearray decode with errors="replace" (matching the existing pattern already used for response.content a few lines below).
  • Anything else (a stream/file-like object) is represented with a fixed "[stream body]" placeholder rather than read — reading it here would consume the stream (which may already be at an unknown position after the actual send) with no guarantee its content is UTF-8 text.

Test plan

  • Added two cases to tests/test_errors.py::test_get_api_error covering both crash scenarios (streamed body via io.BytesIO, non-UTF8 bytes body), extending the existing fake_raw_response helper with an optional request_body parameter that's passed through to requests.Request(..., data=...).prepare() — this reproduces exactly how requests represents each body type in production, rather than hand-constructing a fake object.
  • Both new tests verified to fail against pre-fix code (git stash the fix in round_trip_logger.py, rerun — one raises TypeError, the other AttributeError, matching the two reported/found crash shapes) and pass post-fix.
  • pytest tests/test_errors.py: 52 passed.
  • pytest tests/ (excluding tests/integration): 2109 passed; the only failures (test_open_ai_mixin.py, 7 tests) are pre-existing in this environment due to the optional openai/langchain packages not being installed, unrelated to this change — confirmed by running that file alone and seeing the same ModuleNotFoundError on unmodified main.
  • ruff check / ruff format --check: clean on both changed files.

RoundTrip.generate() passed request.body directly to _redacted_dump(),
which assumes a str. For a streamed upload (a file-like object passed
as data=), body is left as the original file object by requests, and
_redacted_dump's len(body) call raised TypeError: object of type
'...' has no len(). For a non-UTF8 bytes body, the JSONDecodeError
fallback path called body.encode() on already-decoded bytes, raising
AttributeError: 'bytes' object has no attribute 'encode'.

Both crashes happened inside the "unable to parse response" debug-log
path itself, masking the actual API error behind an unrelated
Python traceback.

Added _request_body_str() to normalize body to a string before it
reaches _redacted_dump: str passes through, bytes/bytearray decode
with errors="replace", and any other (stream/file-like) object is
represented with a fixed "[stream body]" placeholder rather than
read, since reading it here would consume the stream (which may
already be at an unknown position after the actual send) with no
guarantee its content is UTF-8 text.

Fixes databricks#1489

Signed-off-by: Praveen Mittal <pkmittal28@gmail.com>
@github-actions

Copy link
Copy Markdown

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-py

Inputs:

  • PR number: 1497
  • Commit SHA: 2aa9554e0abc2a7d08f59a5a178954cd7735ecd9

Checks will be approved automatically on success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ISSUE] get_api_error fails with TypeError: object of type '_io.BufferedReader' has no len()¨

1 participant