Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"plugins": [
{
"name": "kbagent",
"version": "0.78.0",
"version": "0.79.0",
"source": "./plugins/kbagent",
"description": "AI-friendly interface to Keboola Connection projects — explore configs, jobs, lineage, call MCP tools, manage dev branches, and debug SQL in workspaces",
"category": "development"
Expand Down
5 changes: 5 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -664,6 +664,11 @@ kbagent update [--beta]
# source build; falls back to git+ when absent). Env `KBAGENT_UPDATE_TIMEOUT` (integer
# seconds, default 300) raises the self-update subprocess timeout for the slow git+
# fallback on WSL. Bootstrap install: `curl -LsSf .../main/install.sh | sh`.
# Since 0.79.0 a STANDALONE (PyInstaller) binary (choco/winget/brew/apt/dnf/zip) refuses
# the kbagent self-update stage and reports that channel's own command instead -- a uv/pip
# reinstall would install a SECOND, unrelated kbagent that shadows it on PATH. `version
# --json` gains additive `install_channel` + `upgrade_hint`; `upgrade_command` is empty for
# a hand-unpacked archive. The keboola-mcp-server stage still runs (separate distribution).
# Since 0.76.2 self-update completes discovery first, updates MCP before the terminal
# exact-version full kbagent reinstall, then immediately re-executes; failures print a
# copy-paste recovery command.
Expand Down
2 changes: 1 addition & 1 deletion plugins/kbagent/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "kbagent",
"version": "0.78.0",
"version": "0.79.0",
"description": "AI-friendly interface to Keboola Connection projects — explore configs, jobs, lineage, call MCP tools, manage dev branches, and debug SQL in workspaces",
"author": {
"name": "Keboola",
Expand Down
8 changes: 8 additions & 0 deletions plugins/kbagent/agents/keboola-expert.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ a critical failure.
`"Cannot proceed safely on kbagent <version>. Missing: <commands>.
Ask user to run kbagent update, then re-invoke me."` Do not attempt
the task with workarounds that use MCP strip-bug-prone tools.
**Standalone binaries do not take `kbagent update`** -- if
`kbagent --json version` carries `kbagent.install_channel`, quote its
`upgrade_command` (or `upgrade_hint` when that is empty) instead (0.79.0+).

7. **ALWAYS USE `--json`**. Every `kbagent` invocation MUST have
`--json` as the first flag after `kbagent`. This makes output
Expand Down Expand Up @@ -169,6 +172,11 @@ One-line triggers only. Full prose, exact error strings, issue numbers, and
API quirks live in [`gotchas.md`](../skills/kbagent/references/gotchas.md) --
read it when a trigger fires. Each `(X.Y.Z+)` tag is the version floor.

**Upgrading kbagent itself**
- `install_channel` in `kbagent --json version` => native binary; `kbagent
update` REFUSES by design. Quote `upgrade_command` (choco/winget/brew/apt/dnf);
it is `""` for `archive`/`system`, then quote `upgrade_hint`. (0.79.0+)

**Flow / config edits**

- **Conditional flows only (since 0.57.0)**: `flow` targets `keboola.flow`;
Expand Down
5 changes: 5 additions & 0 deletions plugins/kbagent/skills/kbagent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -399,6 +399,11 @@ For detailed response parsing rules and common pitfalls, see [gotchas](reference
| SQL migration (input mapping removal) | [sql-migration-workflow](references/sql-migration-workflow.md) |
| **Semantic layer (metastore)** -- models, metrics, datasets, constraints, glossary; validate / export / diff / promote / build / token | [semantic-layer-workflow](references/semantic-layer-workflow.md) |
| **Developer Portal** (identity CRUD, list/get apps, create/patch/upload-icon/publish/deprecate; TTY-confirm on writes) | [dev-portal-workflow](references/dev-portal-workflow.md) |
| **Config metadata** (list/get/set/delete arbitrary key-value metadata on a configuration) | [config-metadata-workflow](references/config-metadata-workflow.md) |
| **Storage descriptions** (describe bucket / table / column, batch from YAML) | [storage-describe-workflow](references/storage-describe-workflow.md) |
| **Deep column-level lineage** (`lineage build --ai`, column graph, ER + HTML output) | [lineage-deep-workflow](references/lineage-deep-workflow.md) |
| **Session permissions firewall** (`--deny-writes` / `--deny-destructive`, persisted policies, `permissions check`) | [permissions-workflow](references/permissions-workflow.md) |
| **Kai** (project-aware AI Q&A: ping / preflight / ask / chat / history) | [kai-workflow](references/kai-workflow.md) |
| Response parsing gotchas | [gotchas](references/gotchas.md) |

## First-time setup
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ All commands support `--json` for structured output. Multi-project flags (`--pro
## Setup & Info
- `init [--from-global] [--project ALIAS ...]` -- create local `.kbagent/` workspace in current directory; `--project ALIAS` (repeatable) copies only the named project(s) from the global config and implies `--from-global`
- `doctor [--fix]` -- health check for CLI config and MCP server
- `version [--beta]` -- show version info and dependency update status. `--beta` (since v0.42.0) reports the latest pre-release (beta / rc) instead of the latest stable. Env override: `KBAGENT_INCLUDE_PRERELEASE=1`
- `update [--beta]` -- self-update to latest version. `--beta` (since v0.42.0) opts into pre-release versions (PEP 440 betas / rc, e.g. `0.43.0b1`). Default behaviour: GitHub's `/releases/latest` endpoint filters prereleases server-side, so the startup auto-update hook never silently lands on a beta. Resolver-level opt-in (`--prerelease=allow` for uv, `--pre` for pip) is added automatically when `--beta` is set
- `version [--beta]` -- show version info and dependency update status. On a standalone binary the payload carries additive `kbagent.install_channel` + `kbagent.upgrade_hint` keys and `upgrade_command` holds the channel's command (empty for a hand-unpacked archive) (since v0.79.0). `--beta` (since v0.42.0) reports the latest pre-release (beta / rc) instead of the latest stable. Env override: `KBAGENT_INCLUDE_PRERELEASE=1`
- `update [--beta]` -- self-update to latest version. `--beta` (since v0.42.0) opts into pre-release versions (PEP 440 betas / rc, e.g. `0.43.0b1`). Default behaviour: GitHub's `/releases/latest` endpoint filters prereleases server-side, so the startup auto-update hook never silently lands on a beta. Resolver-level opt-in (`--prerelease=allow` for uv, `--pre` for pip) is added automatically when `--beta` is set. **Standalone (PyInstaller) binaries refuse the self-update** and report their own channel's command instead -- a uv/pip reinstall would install a second, unrelated kbagent rather than upgrade the packaged one (since v0.79.0)
- `changelog [--limit N] [--full]` -- show recent changelog (default: last 5 versions, one-line summary per version; `--full` / `-v` expands every note). After auto-update, "What's new" is printed automatically (summarised). Manual trigger: `KBAGENT_UPDATED_FROM=0.17.0 kbagent version`
- `context` -- print full CLI reference for AI agents

Expand Down Expand Up @@ -368,7 +368,7 @@ CLI parity for the `/agents` REST surface. Reads/writes `<config_dir>/agents.jso
## Utility
- `init [--from-global] [--project ALIAS ...]` -- create local `.kbagent/` workspace (per-directory isolation); `--project ALIAS` (repeatable) copies only the named project(s) and implies `--from-global`
- `doctor [--fix]` -- health checks; `--fix` auto-installs MCP server binary. Includes a `sync_secrets` check (since 0.55.0): when run inside a sync working tree (`.keboola/manifest.json`), warns if any in-sync config holds plaintext `#`-secrets (#378); `skip` outside a sync tree
- `version` -- show version and check for MCP server updates
- `version` -- show version and check for MCP server updates; on a standalone binary it advertises the native channel's upgrade command instead of `kbagent update` (v0.79.0+)
- `context` -- full usage instructions for AI agents

## Global Flags
Expand Down
70 changes: 70 additions & 0 deletions plugins/kbagent/skills/kbagent/references/gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -3112,3 +3112,73 @@ embeds the source `table` object), `snapshot-delete` (destructive: forecloses
restores, source tables untouched; batch-tolerant, exit 1 on any failure).
Snapshot create and restore are async storage jobs -- the CLI polls to
completion, so the receipt's `table.rowsCount` is authoritative.

## The standalone binary never self-updates; recommend its own channel (since v0.79.0)

kbagent ships through two channels: a **Python distribution** (`uv tool install`
/ `pip`) and a **self-contained PyInstaller binary** with no Python runtime,
delivered by Chocolatey, WinGet, Homebrew, apt, dnf, or a signed zip. They are
upgraded in completely different ways, and the wrong advice is actively harmful.

- **Never tell a native-binary user to run `uv tool install` (or `pip install`,
or `kbagent update`).** That does not upgrade the running binary at all -- it
creates a SECOND, unrelated kbagent in the uv tool directory, which usually
precedes the package manager's directory on `PATH`. The user silently starts
running a different install than the one `choco` / `brew` / `apt` tracks,
while the packaged binary stays stale. With no Python on the machine it just
fails.
- **kbagent detects this itself now.** A frozen build is identified via
`sys.frozen` / `sys._MEIPASS`, and the channel from the binary's own path.
The startup auto-update hook prints a notification instead of reinstalling
(throttled to once per version-cache TTL), and `kbagent update` refuses with
the right command rather than running uv. This includes the deferred Windows
helper from v0.78.0 -- the guard sits ahead of the `should_defer()` branch, so
a frozen binary is never scheduled for an install it cannot receive either.
- **Read the channel out of `kbagent version --json`.** A frozen build carries
two additive keys: `kbagent.install_channel` (`chocolatey` / `winget` /
`homebrew` / `debian` / `rpm` / `system` / `archive`) and
`kbagent.upgrade_hint` (always a human sentence). Both are **absent** on
uv/pip installs -- so `install_channel` present is the reliable "this is a
native binary" signal. `upgrade_command` stays runnable-or-empty: for
`archive` and `system` there is no single correct command, so it is `""` and
the sentence lives in `upgrade_hint`. **Never shell out to `upgrade_hint`**,
and check `upgrade_command` is non-empty before running it.
Do not infer it from the version string; a frozen binary reports a perfectly
normal version (PyInstaller bundles the dist metadata, so there is no
`0.0.0-dev` tell).
- **Upgrade commands by channel:** `choco upgrade keboola-cli2` ·
`winget upgrade Keboola.KeboolaCLI2` · `brew upgrade keboola-cli2` ·
`sudo apt-get install --only-upgrade keboola-cli2` ·
`sudo dnf upgrade keboola-cli2` · hand-unpacked archive -> re-download from
the GitHub release page. Note the **package** is `keboola-cli2` while the
**binary** is `kbagent`; the PyPI distribution is a third name, `keboola-cli`.
- **keboola-mcp-server still auto-updates on a frozen build**, by design: it is
a separate Python distribution the binary only spawns as a subprocess. If the
user has no Python tooling, install-method detection returns `none` and the
stage does nothing.

## `--json` is written as UTF-8, independent of the console codepage (since v0.78.0)

`--json` output no longer goes through the terminal's text encoder. It is
written straight to `sys.stdout.buffer` as UTF-8, so the bytes you parse never
depend on the active console codepage.

- **What this fixes (#546).** On a default Czech / Polish / Hungarian Windows
console (cp1250), any non-ASCII character in the payload raised
`UnicodeEncodeError` and killed the command -- an arrow in a flow name was
enough to make `kbagent --json flow list` unusable. The crash hit the two
pydantic paths (`model_dump_json` emits raw non-ASCII); `json.dumps` paths
escaped to `\uXXXX` under its `ensure_ascii` default and so survived.
- **Covers all machine output**, not just the success envelope: the error
envelope, the `kbagent --json agent run --stream` NDJSON event lines (which
use `ensure_ascii=False` on purpose so event text stays readable), and
`kbagent http`'s JSON printer.
- **Always decode as UTF-8.** Do not decode with `locale.getpreferredencoding()`
on Windows -- that is cp1250 and will mangle or fail on the same characters
the fix was written for.
- **Windows line endings changed for JSON only**: machine output now ends `LF`,
not `CRLF`, because the binary buffer does no newline translation. Human
(Rich) output is unaffected. No JSON/NDJSON parser cares, but a test that
string-compares raw stdout bytes on Windows might.
- **Captured or replaced streams fall back to the plain text write** (there is
no binary buffer to bypass), so in-process test harnesses behave as before.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "keboola-cli"
version = "0.78.0"
version = "0.79.0"
description = "AI-friendly CLI for managing Keboola projects"
readme = "README.md"
requires-python = ">=3.12"
Expand Down
77 changes: 76 additions & 1 deletion src/keboola_agent_cli/auto_update.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
VERSION_CACHE_FILENAME,
VERSION_CHECK_TIMEOUT,
)
from .frozen_dist import FrozenDistribution, detect_frozen_distribution, is_frozen_build
from .services.version_service import (
MCP_PACKAGE_NAME,
MCP_UV_PRERELEASE_FLAG,
Expand Down Expand Up @@ -179,7 +180,21 @@ def _is_dev_install() -> bool:
Returns True if:
- __version__ is '0.0.0-dev' (PackageNotFoundError fallback), or
- The package was installed in editable mode (PEP 660 direct_url.json).

A frozen (PyInstaller) build is NEVER a dev tree, whatever its bundled
metadata claims -- and the claim is routinely wrong. The release workflow
freezes from a ``uv run`` sync, which installs the project editable, and
``--collect-all keboola_agent_cli`` copies the whole ``.dist-info`` into the
binary *including* ``direct_url.json`` with ``"editable": true``. Every
shipped binary therefore looked like a developer checkout here, which
silently disabled the entire startup hook -- including the frozen-build
notification that is supposed to replace the self-update for exactly those
users (see :mod:`keboola_agent_cli.frozen_dist`). The check must come first:
the bundled marker describes the BUILD MACHINE, not the machine running it.
"""
if is_frozen_build():
return False

if __version__ == "0.0.0-dev":
return True

Expand Down Expand Up @@ -565,6 +580,31 @@ def _schedule_deferred_update(plan: KbagentUpdatePlan) -> None:
)


def _notify_frozen_update_available(
distribution: FrozenDistribution, latest_version: str | None
) -> None:
"""Report a new release to a native-binary user instead of self-updating.

Replaces Stage 1 for frozen (PyInstaller) builds. Neither the inline
reinstall nor the deferred Windows helper can upgrade a Chocolatey /
Homebrew / apt / dnf install -- both would create an unrelated second copy
that shadows the real binary on PATH (see
:mod:`keboola_agent_cli.frozen_dist`). So we only tell the user, naming the
command their own channel actually accepts.

Silent when already current or when the latest version is unknown (offline,
or the re-exec guard suppressed the fetch): a version banner with nothing
actionable behind it is noise.
"""
if _is_up_to_date(__version__, latest_version) is not False:
return
sys.stderr.write(
f"kbagent v{__version__} -> v{latest_version} available. Self-update is "
f"disabled for the standalone binary ({distribution.channel.value}); "
f"{distribution.upgrade_hint}\n"
)


def _prepare_auto_kbagent_plan(latest_version: str | None) -> KbagentUpdatePlan:
"""Adapt the shared plan to the startup comparison seam used by tests."""
prepared = prepare_kbagent_update_plan(latest_version)
Expand Down Expand Up @@ -594,6 +634,19 @@ def maybe_auto_update() -> None:
No re-exec is needed: the MCP server is spawned by ``tool call``
commands and the next spawn picks up the new version.

**Frozen (PyInstaller) builds replace Stage 1 with a notification.** A
native binary from Chocolatey / WinGet / Homebrew / apt / dnf is not a
uv-managed tool environment, so neither the inline reinstall nor the
deferred Windows helper applies -- both would install an unrelated second
copy instead of upgrading the running one (full rationale in
:mod:`keboola_agent_cli.frozen_dist`). **Stage 2 still runs there**, and
that is deliberate: ``keboola-mcp-server`` is a *separate* Python
distribution that a frozen kbagent only ever spawns as a subprocess, so
upgrading it neither touches nor depends on the frozen binary. A pure
binary user with no Python at all is unaffected either way --
:func:`_detect_mcp_install_method` returns ``"none"`` and the stage
short-circuits without installing anything.

Critical invariant: **the re-exec'd process (KBAGENT_SKIP_UPDATE=1)
skips ONLY Stage 1**. Stage 2 always runs, so a kbagent self-upgrade
on startup leaves the user with both kbagent AND MCP refreshed in
Expand Down Expand Up @@ -636,6 +689,12 @@ def maybe_auto_update() -> None:
cached_kbagent = cache.get("latest_version") if cache else None
cached_mcp = cache.get("mcp_latest_version") if cache else None
skip_kbagent_stage = _should_skip_kbagent_stage()
# A frozen (PyInstaller) binary is upgraded by the package manager that
# placed it, never by us. Detected BEFORE planning so the wheel-URL HEAD
# probe inside prepare_kbagent_update_plan is skipped as well -- that
# would be a wasted network round-trip on every single startup, for a
# command this process is never allowed to run.
frozen_dist = detect_frozen_distribution()
use_cached_kbagent = cache_is_fresh and isinstance(cached_kbagent, str)
use_cached_mcp = cache_is_fresh and isinstance(cached_mcp, str)
latest_version = (
Expand All @@ -659,7 +718,7 @@ def maybe_auto_update() -> None:
mcp_plan = prepare_mcp_update_plan(mcp_latest)
kbagent_plan = (
_prepare_auto_kbagent_plan(latest_version)
if not skip_kbagent_stage
if not skip_kbagent_stage and frozen_dist is None
else KbagentUpdatePlan(__version__, latest_version, True, None, None)
)

Expand All @@ -685,6 +744,22 @@ def maybe_auto_update() -> None:
mcp_install_method=mcp_plan.install_method,
)

# Frozen builds: Stage 1 becomes a notification. Deliberately placed
# AFTER the MCP stage and the cache write -- both stay fully active (see
# this function's docstring for why MCP is still updated), and letting
# the TTL tick is what throttles the banner below. Placed BEFORE the
# `should_defer()` branch further down, so the deferred Windows helper
# is never scheduled for a binary it cannot install over either.
if frozen_dist is not None:
# Only on a run that actually refreshed the cache, i.e. at most once
# per AUTO_UPDATE_CHECK_INTERVAL. Unlike the normal path this banner
# cannot resolve itself by re-exec'ing, so without throttling it
# would print on every kbagent invocation until the user upgrades --
# pure noise in any script that shells out to kbagent in a loop.
if not use_cached_kbagent:
_notify_frozen_update_available(frozen_dist, latest_version)
return
Comment thread
padak marked this conversation as resolved.

if kbagent_plan.up_to_date is not False:
return
if kbagent_plan.command is None:
Expand Down
Loading