Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,10 @@ src/keboola_agent_cli/
# LAYER 3 -- HTTP clients (all inherit BaseHttpClient in http_base.py:
# shared 429/5xx retry + exponential backoff)
http_base.py # BaseHttpClient - shared retry/backoff + common HTTP infra
client.py # Storage API + Queue API (X-StorageApi-Token)
client/ # Storage API + Queue API package (X-StorageApi-Token);
# split by endpoint family (storage_tables/storage_files/configs/
# queue/tokens/branches/stream/query/workspaces/misc + _core/_transfer),
# composed into one KeboolaClient via mixins (#520)
manage_client.py # Manage API (X-KBC-ManageApiToken)
ai_client.py # AI Service API (component schemas, Kai)
data_science_client.py # Data Science API (data apps)
Expand Down Expand Up @@ -112,19 +115,19 @@ tests/ # ~137 files; mirror the layers (one test_<module>.py pe
## Architecture: 3-Layer Design

```
CLI Commands (commands/) --> Services (services/) --> API Client (client.py, manage_client.py)
CLI Commands (commands/) --> Services (services/) --> API Client (client/, manage_client.py)
Typer, output Business logic HTTP, endpoints
```

- API changes: modify only the relevant LAYER 3 client (`client.py`, `manage_client.py`, ...)
- API changes: modify only the relevant LAYER 3 client (`client/` package, `manage_client.py`, ...)
- Business logic changes: modify only `services/`
- UI changes: modify only `commands/`

### HTTP Clients

Seven clients, all inheriting `BaseHttpClient` (`http_base.py`) which provides shared retry/backoff logic (429/5xx, exponential backoff, 3 retries) and common HTTP infrastructure:

- **KeboolaClient** (`client.py`): Storage API + Queue API, auth via `X-StorageApi-Token`
- **KeboolaClient** (`client/` package): Storage API + Queue API, auth via `X-StorageApi-Token`
- **ManageClient** (`manage_client.py`): Manage API, auth via `X-KBC-ManageApiToken`
- **AiServiceClient** (`ai_client.py`): AI Service API (component schemas, Kai), URL derived as `ai.{stack_suffix}`
- **DataScienceClient** (`data_science_client.py`): Data Science API (data apps)
Expand Down
12 changes: 6 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,15 @@ Read this **before** writing code. It will save review rounds.
### 3-Layer architecture -- respect the boundaries

```
CLI Commands (commands/) --> Services (services/) --> API Client (client.py, manage_client.py)
CLI Commands (commands/) --> Services (services/) --> API Client (client/, manage_client.py)
Typer, output Business logic HTTP, endpoints
```

| Layer | What goes here | What does NOT go here |
|-------|---------------|----------------------|
| **Commands** (`commands/`) | Typer option parsing, `OutputFormatter` calls, error-to-exit-code mapping | Business logic, HTTP calls, data transformation |
| **Services** (`services/`) | Orchestration, validation, data normalization, parallel execution | Typer imports, output formatting, raw HTTP |
| **Clients** (`client.py`, etc.) | HTTP requests, URL construction, response parsing, retry logic | Business decisions, output formatting |
| **Clients** (`client/` package, `manage_client.py`, etc.) | HTTP requests, URL construction, response parsing, retry logic | Business decisions, output formatting |

When adding a new feature, you will almost always touch all three layers.
If you find yourself importing `typer` in a service or calling `httpx` in a command, stop -- you're in the wrong layer.
Expand Down Expand Up @@ -188,12 +188,12 @@ Hard ceiling per file:
|-------|--------------|--------------|
| `commands/*.py` | 800 LOC | 1200 LOC |
| `services/*.py` | 1000 LOC | 1500 LOC |
| `client.py` / `manage_client.py` | 1500 LOC | 2000 LOC |
| `client/*.py` (per module) / `manage_client.py` | 1500 LOC | 2000 LOC |

When a file crosses the **soft** ceiling, the next PR that adds material to it should split first. When a file crosses the **hard** ceiling, splitting is required before merging more functionality into it.

How to split:
- `client.py` mixing multiple Keboola subsystems (Storage, Queue, Sandboxes, Manage proxy, AI, encryption, ...) → split by **endpoint family**, e.g. `client/storage.py`, `client/queue.py`, `client/sandboxes.py`. Keep `BaseHttpClient` shared.
- A client mixing multiple Keboola subsystems (Storage, Queue, Sandboxes, ...) → split by **endpoint family** into a package, e.g. `client/storage_tables.py`, `client/queue.py`, `client/configs.py`, composed into one class via mixins. Keep `BaseHttpClient` shared. (This is exactly what `client.py` -> the `client/` package was in #520.)
- A service crossing the ceiling almost always mixes orchestration with parsing/transformation → extract pure helpers into a sibling `_helpers.py` or `_transformers.py`.

This is a guideline driven by review feedback (kbagent 0.31.0: `client.py` ≈3000 LOC, `storage_service.py` ≈2180 LOC, `sync_service.py` ≈2765 LOC); the soft ceilings exist so the situation does not get worse before it gets better.
Expand Down Expand Up @@ -269,7 +269,7 @@ Many Storage API operations offer both sync and async variants. Sync endpoints a
simpler but have lower limits (e.g., file size caps, timeouts). Always use the async
variant for production code unless there is a specific reason not to.

Use `_wait_for_storage_job()` from `client.py` for polling -- it already handles
Use `_wait_for_storage_job()` from the client (`client/_core.py`) for polling -- it already handles
intervals, backoff, timeout, and error extraction.

### Graceful resource creation (UX principle)
Expand Down Expand Up @@ -325,7 +325,7 @@ When adding a new command (e.g., `kbagent storage create-foo`), you must update

### Code changes

- [ ] **Client method** in `client.py` (or `manage_client.py`) -- HTTP layer
- [ ] **Client method** in the relevant `client/*.py` mixin (or `manage_client.py`) -- HTTP layer
- [ ] **Service method** in `services/` -- business logic, validation, orchestration
- [ ] **Command function** in `commands/` -- Typer options, formatter, error handling
- [ ] **Permission registration** in `permissions.py` (`OPERATION_REGISTRY` dict)
Expand Down
2 changes: 1 addition & 1 deletion docs/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ kbagent is a strict 3-layer codebase (see [CONTRIBUTING.md](../CONTRIBUTING.md#3

```
CLI command ─┐
REST route ─┼─► Service (services/*.py) ─► KeboolaClient (client.py) ─► HTTP
REST route ─┼─► Service (services/*.py) ─► KeboolaClient (client/ pkg) ─► HTTP
SDK facade ─┘ business logic endpoints, retry
(lib.py)
```
Expand Down
2 changes: 1 addition & 1 deletion plugins/kbagent/agents/kbagent-pr-reviewer.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ be flagged. Of the newly-added ones:
- Any tracked file (not in `.gitignore`) containing a real-looking token
(`9d{3,5}-\d{6,8}-[A-Za-z0-9]{32,}`)? BLOCKING + warn the author to
rotate the credential.
- Any new `httpx` call to `*.keboola.com` URL outside `client.py` /
- Any new `httpx` call to `*.keboola.com` URL outside the `client/` package /
`manage_client.py` / `ai_client.py`? BLOCKING (3-layer + bypasses retry).

---
Expand Down
Loading