Skip to content

Migration column-extraction regexes require the literal COLUMN keyword, but SQLite doesn't #8368

Description

@JSONbored

Context

src/db/migration-column-extraction.ts's extractSchemaEvents uses three regexes to detect
ADD/DROP/RENAME COLUMN statements for the CI-gating collision-detection system
(detectColumnCollisions, backing the db:migrations:check script per this file's own header
comment):

const renameColumnMatch = /\bALTER\s+TABLE\s+(\w+)\s+RENAME\s+COLUMN\s+(\w+)\s+TO\s+(\w+)/i.exec(statement);
const dropColumnMatch = /\bALTER\s+TABLE\s+(\w+)\s+DROP\s+COLUMN\s+(\w+)/i.exec(statement);
const addColumnMatch = /\bALTER\s+TABLE\s+(\w+)\s+ADD\s+COLUMN\s+(\w+)/i.exec(statement);

All three require the literal keyword COLUMN. SQLite's actual ALTER TABLE grammar makes COLUMN
optional for all three forms — ALTER TABLE t ADD x INTEGER, ALTER TABLE t DROP x, and
ALTER TABLE t RENAME x TO y are all valid SQLite statements that omit it. Verified directly against
real sqlite3:

$ sqlite3 :memory: "CREATE TABLE t(id INTEGER); ALTER TABLE t ADD x INTEGER;"
$ sqlite3 :memory: "CREATE TABLE t(id INTEGER, x INTEGER); ALTER TABLE t DROP x;"
$ sqlite3 :memory: "CREATE TABLE t(id INTEGER, x INTEGER); ALTER TABLE t RENAME x TO y;"

All three succeed and mutate the schema as expected.

Because none of the three regexes tolerate the absent COLUMN keyword, a migration written with the
terser (equally valid) syntax produces zero SchemaEvents for that statement — completely
invisible to detectColumnCollisions, silently defeating the CI gate this file exists to power. The
current migration corpus doesn't use the terser form (verified by grep across migrations/**/*.sql),
so this hasn't caused a real miss yet, but it's a live gap in a CI-gating parser that a future
migration author could hit at any time with no warning.

Requirements

  • Update all three regexes (renameColumnMatch, dropColumnMatch, addColumnMatch) so COLUMN is
    optional: change \s+COLUMN\s+ to \s+(?:COLUMN\s+)? in each (adjust exact whitespace handling as
    needed so both the with-COLUMN and without-COLUMN forms still capture the correct column
    name(s) — verify against real SQLite output for both forms, not just one).
  • Do not change any other parsing logic in extractSchemaEvents or the collision-detection logic
    downstream in detectColumnCollisions — this issue is scoped to making these three regexes tolerate
    the optional keyword only.
  • Do not add support for any other SQLite ALTER TABLE variant not already handled (e.g. multi-column
    operations) — out of scope for this issue.

Deliverables

  • All three regexes in src/db/migration-column-extraction.ts correctly extract SchemaEvents
    from both the COLUMN-present and COLUMN-absent forms of ADD/DROP/RENAME.
  • test/unit/migration-column-extraction.test.ts gains new test cases for all three statement
    types using the terser (no COLUMN keyword) syntax, alongside the existing ...COLUMN... tests.

Test Coverage Requirements

This repo's Codecov patch gate is 99%+ of changed lines and branches (src/** is covered). Every
regex change must be covered by both the existing COLUMN-present tests (already passing, must
continue to pass) and new COLUMN-absent tests for all three statement types.

Expected Outcome

A migration using SQLite's terser ALTER TABLE t ADD x INTEGER / DROP x / RENAME x TO y syntax
(without the COLUMN keyword) is correctly detected by extractSchemaEvents and participates in
collision detection exactly like the COLUMN-present form does today.

Links & Resources

  • src/db/migration-column-extraction.ts — the three regexes to fix (extractSchemaEvents)
  • test/unit/migration-column-extraction.test.ts — existing tests to extend
  • scripts/check-migrations.ts — the CI script this parser backs

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions