Fix transaction wrapping for sqlite-utils 4.0 compatibility - #2871
Open
bunlongheng wants to merge 1 commit into
Open
Fix transaction wrapping for sqlite-utils 4.0 compatibility#2871bunlongheng wants to merge 1 commit into
bunlongheng wants to merge 1 commit into
Conversation
…mpat
sqlite-utils 4.0 checks conn.in_transaction at call-time; if False it
opens and immediately commits its own transaction, bypassing Datasette's
outer BEGIN IMMEDIATE. The old 'with conn: conn.execute("BEGIN IMMEDIATE")'
pattern was fragile - Python's context manager only calls commit/rollback
on exit, it does not guarantee in_transaction is True during fn().
Replace both the non-threaded path and the write-thread path with an
explicit try/except: execute("BEGIN IMMEDIATE") before fn(), then
conn.commit() on success and conn.rollback() on failure. Python's
conn.commit()/conn.rollback() methods are idempotent no-ops when no
transaction is active, so write functions that manage their own inner
'with conn:' block (e.g. alter_table) continue to work correctly.
Closes simonw#2831
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
sqlite-utils 4.0 changed transaction behaviour: every write (
insert_all,upsert_all,execute, etc.) now checksconn.in_transactionat call-time. If that flag isFalse, sqlite-utils opens and immediately commits its own transaction, bypassing Datasette's outerBEGIN IMMEDIATE. A subsequent failure leaves the committed rows behind with no rollback path.The old Datasette pattern:
relies on Python's
sqlite3.Connectioncontext manager callingconn.commit()on exit. This is fragile: it does not guarantee thatconn.in_transactionisTrueduringfn(), and if an innerwith conn:inside the write function already committed, the outerwith conn:exit becomes a no-op commit on a closed transaction.Fix
Replace both the non-threaded path (
database.py:352) and the write-thread path (database.py:491) with an explicittry/except:Key properties:
conn.in_transactionisTruefromBEGIN IMMEDIATEuntilcommit()/rollback(), so sqlite-utils 4.0 correctly detects the outer transaction and usesSAVEPOINTinstead of auto-commitconn.commit()/conn.rollback()are idempotent no-ops when no transaction is active, so write functions that manage their own innerwith conn:(e.g.alter_table) continue to workTesting
test_api_write.pytests passtest_internals_database.pytests passCloses #2831