Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 97 additions & 14 deletions database/spanner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,33 +14,116 @@ as described in [README.md#database-urls](../../README.md#database-urls)
| Param | WithInstance Config | Description |
| ----- | ------------------- | ----------- |
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
| `x-clean-statements` | `CleanStatements` | Whether to parse and clean DDL statements before running migration towards Spanner (Required for comments and multiple statements) |
| `x-clean-statements` | `CleanStatements` | **Deprecated.** This parameter is ignored. Statements are now always automatically parsed and comments are stripped. |
| `url` | `DatabaseName` | The full path to the Spanner database resource. If provided as part of `Config` it must not contain a scheme or query string to match the format `projects/{projectId}/instances/{instanceId}/databases/{databaseName}`|
| `projectId` || The Google Cloud Platform project id
| `instanceId` || The id of the instance running Spanner
| `databaseName` || The name of the Spanner database
| `projectId` || The Google Cloud Platform project id |
| `instanceId` || The id of the instance running Spanner |
| `databaseName` || The name of the Spanner database |

> **Note:** Google Cloud Spanner migrations can take a considerable amount of
> time. The migrations provided as part of the example take about 6 minutes to
> **Note:** Google Cloud Spanner DDL migrations can take a considerable amount of
> time. The migrations provided as part of the example take about 6 minutes to
> run on a small instance.
>
> ```log
> 1481574547/u create_users_table (21.354507597s)
> 1496539702/u add_city_to_users (41.647359754s)
> 1496601752/u add_index_on_user_emails (2m12.155787369s)
> 1496602638/u create_books_table (2m30.77299181s)
> ```

## DDL with comments
## Supported Statement Types

At the moment the GCP Spanner backed does not seem to allow for comments (See https://issuetracker.google.com/issues/159730604)
so in order to be able to use migration with DDL containing comments `x-clean-statements` is required
The Spanner driver supports three types of statements in migration files:

## Multiple statements
### DDL (Data Definition Language)

In order to be able to use more than 1 DDL statement in the same migration file, the file has to be parsed and therefore the `x-clean-statements` flag is required
Schema modification statements like `CREATE TABLE`, `ALTER TABLE`, `DROP TABLE`, `CREATE INDEX`, etc.
These are executed using Spanner's `UpdateDatabaseDdl` API.

```sql
CREATE TABLE Users (
UserId INT64 NOT NULL,
Name STRING(100)
) PRIMARY KEY(UserId);
```

### DML (Data Manipulation Language) - INSERT

`INSERT` statements are executed within a read-write transaction, allowing multiple inserts to be atomic.

```sql
INSERT INTO Users (UserId, Name) VALUES (1, 'Alice');
INSERT INTO Users (UserId, Name) VALUES (2, 'Bob');
```

### Partitioned DML - UPDATE and DELETE

`UPDATE` and `DELETE` statements are executed using Spanner's `PartitionedUpdate` API,
which is optimized for large-scale data modifications.

```sql
UPDATE Users SET Name = 'Updated' WHERE UserId = 1;
```

```sql
DELETE FROM Users WHERE UserId = 1;
```

### Statement Type Restrictions

**Important:** Each migration file must contain only one type of statement. You cannot mix:

- DDL with DML
- INSERT with UPDATE/DELETE

For example, the following migration file will fail:

```sql
-- This will fail: mixing INSERT and UPDATE
INSERT INTO Users (UserId, Name) VALUES (1, 'Alice');
UPDATE Users SET Name = 'Bob' WHERE UserId = 1;
```

## Comments in Migrations

Migration files can contain SQL comments. The driver automatically parses and strips comments
before execution since Spanner's `UpdateDatabaseDdl` API does not support comments.

Supported comment styles:

- Single-line comments: `-- comment`
- Multi-line comments: `/* comment */`

```sql
-- This migration creates the users table
/*
* Author: migrate
* Description: Initial schema setup
*/
CREATE TABLE Users (
UserId INT64 NOT NULL, -- primary key
Name STRING(100)
) PRIMARY KEY(UserId);
```

## Multiple Statements

Multiple statements of the same type can be included in a single migration file,
separated by semicolons:

```sql
CREATE TABLE Users (
UserId INT64 NOT NULL
) PRIMARY KEY(UserId);

CREATE INDEX UsersByName ON Users(Name);
```

## Testing

To unit test the `spanner` driver, `SPANNER_DATABASE` needs to be set. You'll
need to sign-up to Google Cloud Platform (GCP) and have a running Spanner
instance since it is not possible to run Google Spanner outside GCP.
The Spanner driver can be tested using the Spanner emulator provided by the
`cloud.google.com/go/spanner/spannertest` package. The unit tests use this
emulator and do not require a real Spanner instance.

For integration testing against a real Spanner instance, set the `SPANNER_DATABASE`
environment variable to your database's full resource path.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
DELETE FROM Users WHERE UserId IN (1, 2);
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
INSERT INTO Users (UserId, Name, Email) VALUES (1, 'Alice', 'alice@example.com');
INSERT INTO Users (UserId, Name, Email) VALUES (2, 'Bob', 'bob@example.com');
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
UPDATE Users SET Email = 'alice@example.com' WHERE UserId = 1;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
UPDATE Users SET Email = 'alice.updated@example.com' WHERE UserId = 1;
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
-- Rollback: Remove status column from Users table
ALTER TABLE Users DROP COLUMN Status;
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
-- Migration: Add status column to Users table
-- Author: migrate
-- Description: This migration adds a status column to track user account state

/*
* The Status column will store the current state of the user account.
* Valid values: 'active', 'inactive', 'suspended'
*/
ALTER TABLE Users ADD COLUMN Status STRING(20); -- default will be NULL
Loading