Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"plugins": [
{
"name": "kbagent",
"version": "0.74.0",
"version": "0.75.0",
"source": "./plugins/kbagent",
"description": "AI-friendly interface to Keboola Connection projects — explore configs, jobs, lineage, call MCP tools, manage dev branches, and debug SQL in workspaces",
"category": "development"
Expand Down
8 changes: 8 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,14 @@ kbagent storage delete-column --project NAME --table-id ID --column COL [--colum
kbagent storage delete-bucket --project NAME --bucket-id ID [--bucket-id ...] [--force] [--dry-run] [--yes] [--branch ID]
kbagent storage swap-tables --project NAME --table-id ID --target-table-id ID --branch ID [--dry-run] [--yes]
kbagent storage clone-table --project NAME --table-id ID --branch ID [--dry-run]
kbagent storage snapshot-create --project NAME --table-id ID [--description D] [--branch ID]
kbagent storage snapshots --project NAME --table-id ID [--limit N] [--branch ID]
kbagent storage snapshot-detail --project NAME --snapshot-id ID
kbagent storage snapshot-delete --project NAME --snapshot-id ID [--snapshot-id ...] [--dry-run] [--yes]
kbagent storage table-from-snapshot --project NAME --snapshot-id ID --bucket-id ID --name NAME [--branch ID] [--dry-run]
# Snapshots (0.75.0+, #512): point-in-time table backup (data+columns+PK) and restore as a NEW table.
# table-from-snapshot goes through the classic tables-async endpoint (NOT tables-definition), --name is
# REQUIRED (API rejects empty), no overwrite -- restore under a new name, then swap/delete yourself.
kbagent storage describe-bucket --project NAME --bucket-id ID [--text STR | --file PATH | --stdin] [--branch ID]
kbagent storage describe-table --project NAME --table-id ID [--text STR | --file PATH | --stdin] [--branch ID]
kbagent storage describe-column --project NAME --table-id ID --column NAME=DESC [--column ...] [--branch ID]
Expand Down
2 changes: 1 addition & 1 deletion plugins/kbagent/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "kbagent",
"version": "0.74.0",
"version": "0.75.0",
"description": "AI-friendly interface to Keboola Connection projects — explore configs, jobs, lineage, call MCP tools, manage dev branches, and debug SQL in workspaces",
"author": {
"name": "Keboola",
Expand Down
6 changes: 6 additions & 0 deletions plugins/kbagent/agents/keboola-expert.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ a critical failure.
| Promote typed rebuild back into the original name | `kbagent storage swap-tables --project P --table-id in.c-foo.data --target-table-id in.c-foo.data_change_log --branch <ID> --yes` (0.28.0+) -- async storage job (`tableSwap`); client polls to completion. Service refuses without a branch; any branch incl. prod | -- | renaming or deleting + re-uploading (loses history; downstream configs need to be rewritten) |
| Repartition / recluster a populated BigQuery table | `kbagent storage create-table --project P --bucket-id in.c-main --name events_repart --source-table-id in.c-main.events --time-partitioning-type DAY --time-partitioning-field created_at --clustering-field tenant_id --primary-key id` (0.66.0+, BigQuery only) to copy the data into the new layout, then `kbagent storage swap-tables --table-id in.c-main.events --target-table-id in.c-main.events_repart --branch <ID> --yes` to flip it into place. `--source-table-id` derives the schema from the source so `--column` is forbidden (mutually exclusive); a non-BigQuery project fails fast (pre-flight backend check, exit 2). Range partitioning instead: `--range-partitioning-field/-start/-end/-interval` (all four; bounds are strings; mutually exclusive with time partitioning) | -- | raw `POST /v2/storage/buckets/.../tables-definition` with a `source` object, then manual swap; or a `CREATE TABLE ... AS SELECT` in a workspace (drops NOT NULL + primary key) |
| Re-seed a table without losing its schema / PK / dependents | `kbagent storage truncate-table --project P --table-id in.c-foo.data [--branch ID] [--dry-run] [--yes]` (0.32.0+) -- DELETE `/tables/{id}/rows?allowTruncate=1`; endpoint is uniformly async on every branch (returns a queued `tableRowsDelete` job; client polls via `_wait_for_storage_job`). Do NOT pass `async=true` -- the API rejects it. Batch via repeated `--table-id`. Returns `{truncated[], failed[], dry_run, project_alias}` with `truncated[]` entries carrying `{table_id, rows_before, rows_after, branch_id}`. Permission class: `destructive` | `tool call delete_table_rows` if the upstream MCP exposes it | drop + recreate the table (loses descriptions, PK, sharing edges, and breaks every downstream config reference); deleting rows via raw SQL in a workspace (bypasses the Storage API audit trail) |
| Point-in-time backup of a table before a risky change | `kbagent storage snapshot-create --project P --table-id in.c-foo.data [--description D]` (0.75.0+, #512) -- async `tableSnapshotCreate` job; receipt carries `snapshot_id`. List with `storage snapshots --table-id ...`, inspect with `snapshot-detail --snapshot-id ID` (embeds the source table object) | -- | exporting to CSV as a "backup" (loses column types + PK); relying on the destructive command's `--dry-run` alone |
| Restore a snapshot as a NEW table | `kbagent storage table-from-snapshot --project P --snapshot-id ID --bucket-id in.c-foo --name restored [--dry-run]` (0.75.0+, #512) -- classic `tables-async` endpoint; restores data + columns + PK. `--name` REQUIRED (API rejects empty; PHP-client "defaults to snapshot name" docblock is stale). No overwrite: restore under a new name, verify, then `swap-tables`/`delete-table` to promote | -- | trying `create-table --snapshot-id` (not a thing -- `tables-definition` does not accept snapshots); restoring straight onto the production name (duplicate-name error) |
| Debug a failed job | `kbagent job detail --project P --job-id J --json` + `kbagent job run ... --log-tail-lines 200` | `kbagent workspace from-transformation` for SQL repro | "I think the issue is..." without reading logs |
| Ad-hoc SQL / row-count / type audit | `kbagent workspace create` + `kbagent workspace load` + `kbagent workspace query --sql "..."` (0.59.0+: results come back inline+fast but **capped at `--limit`, default 500** -- check `statements[].truncated`/`total_rows`, use `COUNT(*)` for counts, `--full` for the complete set) | `kbagent workspace from-transformation` for existing transform debugging; `workspace list --qs-compatible` (0.42.0+, #304) for data-app reuse | trusting a default `SELECT *` as the full result (it is truncated at 500); querying Storage via raw Snowflake credentials outside the workspace abstraction |
| Export a FILTERED or INCREMENTAL slice of a table (no workspace) | `kbagent storage download-table --project P --table-id in.c-foo.data --where-column status --where-value active [--where-operator eq\|neq] [--changed-since "-2 days"] [--changed-until WHEN]` (0.62.0+) -- server-side row filter + import-time window on the credential-only export path | `kbagent workspace query` with a `WHERE` clause when you need real SQL (needs a workspace) | downloading the whole table then filtering locally |
Expand Down Expand Up @@ -252,6 +254,10 @@ read it when a trigger fires. Each `(X.Y.Z+)` tag is the version floor.
"bucket not found" until you clone the prod table branch-local first.
- **`truncate-table` is row-only** (0.32.0+): schema / PK / dependents survive;
uniformly async-via-job on every branch; do NOT pass `async=true`.
- **Snapshot restore never overwrites** (0.75.0+, #512): `table-from-snapshot`
needs a REQUIRED `--name` (API rejects empty) and fails on an existing table
name -- restore under a new name, verify, then `swap-tables`. `snapshot-delete`
only forecloses restores; the source table is untouched.
- **`bucket-detail` is dialect-aware** (0.25.3+): read `sql_dialect` + per-table
`sql_path` (already correctly quoted) -- don't branch on the backend yourself.
- **`set-default-bucket`** (0.26.0+) writes `storage.output.default_bucket`
Expand Down
25 changes: 16 additions & 9 deletions plugins/kbagent/skills/kbagent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,20 @@
name: kbagent
description: >
Use when working with Keboola Connection projects via the kbagent CLI.
Covers: exploring and searching component configurations, job history and
job runs, cross-project data lineage, Keboola MCP tools, development
branches, SQL debugging in temporary workspaces, GitOps sync of configs as
local files (pull/push/diff/clone), bucket sharing and linking, encrypting
secrets, Storage tables and files, data apps (create/deploy/logs/secrets),
conditional flows and schedules, project members and invitations, feature
Covers: exploring and searching configurations, job history and runs,
cross-project data lineage, Keboola MCP tools, dev branches, SQL debugging in workspaces, GitOps sync of configs
(pull/push/diff/clone), bucket sharing and linking, encrypting
secrets, Storage tables, files, and snapshots (backup + restore as a new
table), data apps (create/deploy/logs/secrets),
flows and schedules, members and invitations, feature
flags, OTLP data streams, scoped Storage tokens, the semantic layer
(models, datasets, metrics, constraints, reference data), and the Keboola
Developer Portal. Triggers: kbagent, Keboola, keboola config, keboola job,
(models, metrics, constraints, reference data), and the Developer
Portal. Triggers: kbagent, Keboola, keboola config, keboola job,
keboola lineage, keboola sync, gitops, dev branch, workspace SQL, data app,
streamlit deploy, semantic layer, sl, dev-portal, data stream, OTLP,
scoped token, bucket sharing, encrypt secrets, feature flag, flow schedule,
invite member, SQL transformation edit, sync action, keboola docs.
invite member, SQL transformation edit, sync action, keboola docs,
table snapshot, snapshot restore.
---

# kbagent -- Keboola Agent CLI
Expand Down Expand Up @@ -168,6 +169,11 @@ When working inside a git repository or project directory, run `kbagent init` (o
| Delete one or more Storage Files | `kbagent storage file-delete --project PROJECT --file-id FILE-ID` |
| Load a Storage File into a table | `kbagent storage load-file --project PROJECT --file-id FILE-ID --table-id TABLE-ID` |
| Export a table to a Storage File | `kbagent storage unload-table --project PROJECT --table-id TABLE-ID` |
| List snapshots of a table | `kbagent storage snapshots --project PROJECT --table-id TABLE-ID` |
| Create a snapshot of a table (data + columns + primary key) | `kbagent storage snapshot-create --project PROJECT --table-id TABLE-ID` |
| Show one snapshot's detail (source table, creation time, description) | `kbagent storage snapshot-detail --project PROJECT --snapshot-id SNAPSHOT-ID` |
| Delete one or more table snapshots (the source tables are untouched) | `kbagent storage snapshot-delete --project PROJECT --snapshot-id SNAPSHOT-ID` |
| Create a NEW table from an existing snapshot (snapshot restore) | `kbagent storage table-from-snapshot --project PROJECT --snapshot-id SNAPSHOT-ID --bucket-id BUCKET-ID --name NAME` |
| List Data Streams sources in a project | `kbagent stream list --project PROJECT` |
| Create an OTLP (or HTTP) source and return its endpoint | `kbagent stream create-source --project PROJECT --name NAME` |
| Show a source's endpoints, protocol, and destination tables | `kbagent stream detail [SOURCE-ID] --project PROJECT` |
Expand Down Expand Up @@ -377,6 +383,7 @@ For detailed response parsing rules and common pitfalls, see [gotchas](reference
| **Agent Tasks via REST** (`kbagent http <verb> /agents...` from inside scheduled subprocesses; SSE streaming) | [agent-tasks-rest-workflow](references/agent-tasks-rest-workflow.md) |
| **Data apps** (create / deploy / start / stop / password / delete; the §9 redeploy contract) | [data-app-workflow](references/data-app-workflow.md) |
| Storage Files (upload, download, tags, load/unload) | [storage-files-workflow](references/storage-files-workflow.md) |
| **Table snapshots** (point-in-time backup; restore as a NEW table; `--name` required, no overwrite) | [snapshot-workflow](references/snapshot-workflow.md) |
| **Python library** (`from keboola_agent_cli import Client` -- in-process query + Storage Files, no CLI/daemon/config-dir) | [library-workflow](references/library-workflow.md) |
| **Data Streams (OTLP / OpenTelemetry)** (create/inspect OTLP source, masked secret-in-URL, OTEL_EXPORTER_OTLP_ENDPOINT) | [stream-workflow](references/stream-workflow.md) |
| **Storage column types** (native types, NOT NULL, DEFAULT, branch materialize) | [storage-types-workflow](references/storage-types-workflow.md) |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,11 @@ Requires a **super-admin** Manage API token (same kind as `org setup`). Same def
- `storage delete-bucket --project NAME --bucket-id ID [--bucket-id ...] [--force] [--dry-run] [--yes] [--branch ID]` -- delete buckets (branch-aware)
- `storage swap-tables --project NAME --table-id ID --target-table-id ID --branch ID [--dry-run] [--yes]` (since v0.28.0) -- swap two storage tables in any branch, including the default/production branch (POST `/tables/{id}/swap`). Both tables exchange physical positions; aliases are NOT transferred (they keep pointing at the same physical position and therefore expose the OTHER table's data after the swap). Service refuses without a branch (active branch via `branch use` works too). Use to flip a typed rebuild ("data_change_log") into the original name ("data") without touching downstream config references
- `storage clone-table --project NAME --table-id ID --branch ID [--dry-run]` (since v0.52.0) -- pull (clone) a production table into a dev branch (POST `/tables/{id}/pull`, operationName `devBranchTablePull`). On `storage-branches` projects a dev branch reads prod tables transparently until the first write, so an in-branch schema mutation (`swap-tables`, dropping a column) fails with a misleading "bucket not found" until the table is materialized branch-local; `clone-table` does that. One-way (default -> branch). Service refuses without a branch (active branch via `branch use` works too). Permission class `write`
- `storage snapshot-create --project NAME --table-id ID [--description D] [--branch ID]` (since v0.75.0) -- create a point-in-time snapshot of a table (data + columns + primary key; async `tableSnapshotCreate` job, client polls to completion). The receipt carries `snapshot_id` -- restores are addressed by it. Permission class `write`
- `storage snapshots --project NAME --table-id ID [--limit N] [--branch ID]` (since v0.75.0) -- list a table's snapshots (`id`, `createdTime`, `description`, `creatorToken`). Read command: production endpoint by default
- `storage snapshot-detail --project NAME --snapshot-id ID` (since v0.75.0) -- one snapshot's detail. Snapshot IDs are global (not table-scoped); the detail embeds the source `table` object (id, columns, primaryKey), so it traces a bare snapshot ID back to its origin table
- `storage table-from-snapshot --project NAME --snapshot-id ID --bucket-id ID --name NAME [--branch ID] [--dry-run]` (since v0.75.0) -- create a NEW table from an existing snapshot (issue #512; restore). Restores data, columns, and primary key into `--bucket-id` under `--name` via the classic `tables-async` endpoint (NOT `tables-definition` -- that is why it is not a `create-table` flag). `--name` is REQUIRED (API rejects empty). No overwrite: restoring onto an existing table name fails -- restore under a new name, verify, then `swap-tables` or `delete-table` yourself. Permission class `write`
- `storage snapshot-delete --project NAME --snapshot-id ID [--snapshot-id ...] [--dry-run] [--yes]` (since v0.75.0) -- delete snapshots (destructive: forecloses restores; source tables untouched). Batch-tolerant, exit 1 when any ID failed
- `storage describe-bucket --project NAME --bucket-id ID [--text STR | --file PATH | --stdin] [--branch ID]` -- set a bucket description (stored as `KBC.description` in bucket metadata, upsert). Provide exactly one of `--text`, `--file`, `--stdin`. Read back via `storage bucket-detail`
- `storage describe-table --project NAME --table-id ID [--text STR | --file PATH | --stdin] [--branch ID]` -- set a table description (stored as `KBC.description` in table metadata, upsert). Provide exactly one of `--text`, `--file`, `--stdin`. Read back via `storage table-detail`
- `storage describe-column --project NAME --table-id ID --column NAME=DESCRIPTION [--column ...] [--branch ID]` -- set one or more column descriptions. Stored as `KBC.column.{name}.description` keys in the table's metadata (Keboola has no user-writable column-metadata endpoint). Read back in `storage table-detail` under `column_details[].description`
Expand Down
23 changes: 23 additions & 0 deletions plugins/kbagent/skills/kbagent/references/gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -3026,3 +3026,26 @@ Four related sync-engine behaviors landed together (issues #466 / #467 / #472 /
detected ... Local check only -- run 'kbagent sync diff'": `status` never
contacts the API, so it cannot see remote drift; treating it as a
local-vs-production audit was the #466 trap.

## Table snapshots: restore is `tables-async`, `--name` is required, no overwrite (since v0.75.0)

`kbagent storage table-from-snapshot` (issue #512) creates a NEW table from an
existing snapshot. Three traps, all verified live (us-east4.gcp, 2026-07-22):

- **`--name` is REQUIRED.** The API rejects an omitted or empty name with
`Table create option "name" is required and cannot be empty` -- the reference
PHP client's "defaults to the snapshotted table's name" docblock is stale.
- **No overwrite semantics.** Restoring onto an existing table name fails with
a duplicate-name error. The safe pattern: restore under a new name, verify
the data, then `storage swap-tables` (or `delete-table` + re-restore) to
promote it. The destination bucket must already exist.
- **It is not a `create-table` flag.** Restore goes through the classic
`tables-async` import endpoint; `create-table` uses `tables-definition`,
which does not accept `snapshotId` -- hence the dedicated command.

Lifecycle: `snapshot-create` (write; receipt carries `snapshot_id`),
`snapshots` (read, per table), `snapshot-detail` (read; global snapshot ID ->
embeds the source `table` object), `snapshot-delete` (destructive: forecloses
restores, source tables untouched; batch-tolerant, exit 1 on any failure).
Snapshot create and restore are async storage jobs -- the CLI polls to
completion, so the receipt's `table.rowsCount` is authoritative.
Loading