Skip to content

Fix SELECT on tables with bytea columns hanging in process worker#171

Open
AlexZeitler wants to merge 1 commit intoMaxteabag:mainfrom
AlexZeitler:fix/bytea-memoryview-pickle-hang
Open

Fix SELECT on tables with bytea columns hanging in process worker#171
AlexZeitler wants to merge 1 commit intoMaxteabag:mainfrom
AlexZeitler:fix/bytea-memoryview-pickle-hang

Conversation

@AlexZeitler
Copy link

Summary

  • psycopg2 returns memoryview objects for bytea columns which cannot be pickled through multiprocessing.Pipe
  • The worker's send() silently swallowed the serialization error, leaving the client waiting forever (spinner)
  • Convert memoryview to bytes in CursorBasedAdapter.execute_query() before rows are returned, so they can be safely pickled
  • Added unit tests for the sanitization logic

Test plan

  • Unit tests for _sanitize_cell and _sanitize_row pass
  • Connect to PostgreSQL 17
  • Create table with bytea column and insert rows (see setup SQL in SELECT on table with bytea column hangs when executed as single statement #161)
  • Run SELECT * FROM bytea_repro; as single statement — should return 3 rows immediately
  • Run SELECT id, name FROM bytea_repro; — should still work
  • Run both statements together — should still work

Fixes #161

psycopg2 returns memoryview objects for bytea columns which cannot be
pickled through multiprocessing.Pipe. The worker's send() silently
swallowed the serialization error, leaving the client waiting forever.

Convert memoryview to bytes in CursorBasedAdapter.execute_query() before
rows are returned, so they can be safely pickled.

Fixes Maxteabag#161
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SELECT on table with bytea column hangs when executed as single statement

1 participant