Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 21 additions & 15 deletions libs/k8s-client/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,18 +47,27 @@ docker compose run --rm dev-k8s-client bash -c "vendor/bin/phpunit --filter test

### Three-Layer Structure

1. **ClientFacadeFactory** - Creates configured client instances
- `GenericClientFacadeFactory` - For explicit cluster credentials
- `InClusterClientFacadeFactory` - For Pods running inside K8s (uses service account)
- `AutoDetectClientFacadeFactory` - Auto-detects environment
- `EnvVariablesClientFacadeFactory` - Loads config from environment variables
1. **ClientFactory** - Two separate concerns, both under `Keboola\K8sClient\ClientFactory\`:
- Client factories (`KubernetesApiClientFactory` implementations) resolve credentials and produce a single
configured `KubernetesApiClient`:
- `StaticKubernetesApiClientFactory` - For explicit cluster credentials
- `InClusterKubernetesApiClientFactory` - For Pods running inside K8s (uses service account)
- `EnvVariablesKubernetesApiClientFactory` - Loads config from environment variables
- `AutoDetectKubernetesApiClientFactory` - Tries env variables first, falls back to in-cluster
- `KubernetesApiClientFacadeFactory` - Universal factory that assembles a `KubernetesApiClientFacade` from any
already-configured `KubernetesApiClient` (regardless of which client factory produced it), via `create()`
- `ClientConfigurator` / `Token\{TokenInterface,StaticToken,InClusterToken}` - shared low-level helpers used by
the client factories to configure the underlying `kubernetes/php-client` `Client` singleton

2. **KubernetesApiClientFacade** - High-level facade providing:
- Type-safe resource operations (`createModels`, `deleteModels`, `mergePatch`, etc.)
- Convenience methods for multiple resources at once
- Waiting operations (`waitWhileExists`)
- Resource listing with pagination (`listMatching`)
- Access to specific API clients via getters
- `client(string $modelClass)` - generic accessor resolving any registered resource type (core or extra)
- `$extraClients` constructor param - lets consumers register their own CRD API clients (e.g. custom
Keboola CRDs) without the library needing to own the model/BaseApi/typed-client classes for them

3. **ApiClient Wrappers** - Namespace/cluster-scoped API wrappers
- Wrap `kubernetes/php-client` API classes
Expand All @@ -76,18 +85,16 @@ docker compose run --rm dev-k8s-client bash -c "vendor/bin/phpunit --filter test
- `PodsApiClient` - Pods (includes log streaming)
- `SecretsApiClient` - Secrets
- `ServicesApiClient` - Services
- `AppsApiClient` - Custom App CRD (Keboola-specific)
- `AppRunsApiClient` - Custom AppRun CRD (Keboola-specific)

**Cluster-scoped:**
- `PersistentVolumesApiClient` - PVs

### Custom Resources (CRDs)

The library includes custom Keboola CRDs for App and AppRun resources:
- Models: `src/Model/Io/Keboola/Apps/V1/`
- Used for billing and cost tracking
- CRD definitions must be installed on K8s clusters for functional tests (see README.md)
The library does not own any custom Keboola CRD (model/BaseApi/typed-client) classes itself. Consumers that
need a custom CRD (e.g. the `App`/`AppRun` CRDs used by sandboxes-service for billing) implement their own
model/BaseApi/typed-client classes and register the typed client via `KubernetesApiClientFacade`'s
`$extraClients` constructor param; it is then accessed through `client(SomeModel::class)`.

## Implementing New API Support

Expand All @@ -104,15 +111,14 @@ To add support for a new Kubernetes API:
- Add resource class to `$resourceTypeClientMap` array
- Update type annotations for generic methods (`createModels`, `deleteModels`, etc.)

3. **Update factories** in `ClientFacadeFactory/`:
- `GenericClientFacadeFactory` - Instantiate new API client and inject into facade
3. **Update `KubernetesApiClientFacadeFactory::create()`** in `ClientFactory/`:
- Instantiate new API client and inject into facade

## Code Quality Standards

### PHPStan Configuration
- Level: `max`
- Custom ignoreErrors for external library issues:
- `src/BaseApi/*` - Guzzle and kubernetes-runtime return type mismatches
- Custom ignoreErrors: `missingType.iterableValue` (broad, library-wide)
- Stub file: `tests/stubs/K8s.stub` for external type definitions

### PHP_CodeSniffer
Expand Down
96 changes: 63 additions & 33 deletions libs/k8s-client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,31 @@ it in many ways:
* high-level operations like `create` multiple resources at once, `waitWhileExists` to wait while given resource exists etc.

## Usage
To create a client, use one of provided client factories:
* `GenericClientFacadeFactory` if you have cluster credentials
* `InClusterClientFacadeFactory` if you run inside a Pod which has access to K8S API
To create a client, first pick a `Keboola\K8sClient\ClientFactory\KubernetesApiClientFactory` implementation that
matches how you obtain credentials, then use it together with the universal `KubernetesApiClientFacadeFactory` to
build the high-level facade:
* `StaticKubernetesApiClientFactory` if you have explicit cluster credentials
* `InClusterKubernetesApiClientFactory` if you run inside a Pod which has access to K8S API
* `EnvVariablesKubernetesApiClientFactory` if credentials are provided via `K8S_HOST`/`K8S_TOKEN`/`K8S_CA_CERT_PATH`/`K8S_NAMESPACE` env variables
* `AutoDetectKubernetesApiClientFactory` to try env variables first, falling back to in-cluster credentials

```php
<?php

use Keboola\K8sClient\ClientFacadeFactory\GenericClientFacadeFactory;
use Keboola\K8sClient\ClientFactory\StaticKubernetesApiClientFactory;
use Keboola\K8sClient\KubernetesApiClientFacade;
use Kubernetes\Model\Io\K8s\Api\Core\V1\Container;
use Kubernetes\Model\Io\K8s\Api\Core\V1\Pod;

$clientFactory = new GenericClientFacadeFactory($retryProxy, $logger);
$client = $clientFactory->createClusterClient(
$clientFactory = new StaticKubernetesApiClientFactory(
$retryProxy,
'https://api.k8s-cluster.example.com',
'secret-token',
'var/k8s/caCert.pem',
'default'
'default',
);
$apiClient = $clientFactory->createApiClient();
$client = KubernetesApiClientFacade::create($apiClient, $logger);

$pod = new Pod([
'metadata' => [
Expand Down Expand Up @@ -68,6 +75,54 @@ $client->deleteModels([
]);
```

## Custom resources (CRDs)
The facade also serves resources it doesn't ship. Implement an API client for your CRD by extending
`Keboola\K8sClient\ApiClient\BaseNamespaceApiClient` (or `BaseClusterApiClient`), then register it via the
`$extraClients` map — keyed by the CRD model class — when building the facade:

```php
$facade = KubernetesApiClientFacade::create($apiClient, $logger, [
My\Crd\Model::class => new My\Crd\ApiClient($apiClient),
]);

$facade->client(My\Crd\Model::class)->get('my-resource'); // typed access
$facade->mergePatch($myCrdModel); // generic methods route via the map too
```

## Symfony integration
The library ships no bundle, so wire the pieces as services. Minimal setup with auto-detected credentials
(env vars in dev, in-cluster ServiceAccount token in prod) using the shared-client + `create()` split:

```yaml
services:
# credential strategy — AutoDetect needs its two sub-factories defined (RetryProxy is not autowirable)
Keboola\K8sClient\ClientFactory\EnvVariablesKubernetesApiClientFactory:
arguments:
$retryProxy: !service { class: Retry\RetryProxy }
Keboola\K8sClient\ClientFactory\InClusterKubernetesApiClientFactory:
arguments:
$retryProxy: !service { class: Retry\RetryProxy }
Keboola\K8sClient\ClientFactory\AutoDetectKubernetesApiClientFactory: ~

# the shared low-level client
app.k8s.api_client:
class: Keboola\K8sClient\KubernetesApiClient
factory: ['@Keboola\K8sClient\ClientFactory\AutoDetectKubernetesApiClientFactory', 'createApiClient']
arguments:
$namespace: '%env(K8S_NAMESPACE)%'

# the high-level facade (static factory called as a class-string); register CRD clients via $extraClients
Keboola\K8sClient\KubernetesApiClientFacade:
factory: ['Keboola\K8sClient\KubernetesApiClientFacade', 'create']
arguments:
$apiClient: '@app.k8s.api_client'
$logger: '@logger'
$extraClients: {} # e.g. 'App\Crd\Model': '@App\Crd\ApiClient'
```

For a single credential source you can skip `AutoDetect` and point the client at
`StaticKubernetesApiClientFactory` (explicit values) or `InClusterKubernetesApiClientFactory` directly.

## Development
Prerequisites:
* configured `az` and `aws` CLI tools (run `az login` and `aws configure --profile keboola-dev-platform-services`)
Expand All @@ -90,39 +145,14 @@ docker compose run --rm dev composer install
docker compose run --rm dev composer ci
```

### Installing AppRun CRD on Dev/CI Clusters
The functional tests require App and AppRun Custom Resource Definitions (CRDs) to be installed on the Kubernetes cluster. For manually maintained CI clusters, install CRDs manually on **all three clusters** (GCP, AWS, Azure):

```bash
# Download App and AppRun CRD definitions from keboola-operator repository:
# - https://github.com/keboola/keboola-operator/blob/canary-operator/config/crd/bases/apps.keboola.com_apps.yaml
# - https://github.com/keboola/keboola-operator/blob/canary-operator/config/crd/bases/apps.keboola.com_appruns.yaml
# Save files locally as apps.keboola.com_appruns.yaml

# Install on GCP CI cluster
kubectl --context="gke_kbc-ci-platform-services_us-central1_gcp-ci-ps" apply -f apps.keboola.com_apps.yaml
kubectl --context="gke_kbc-ci-platform-services_us-central1_gcp-ci-ps" apply -f apps.keboola.com_appruns.yaml

# Install on AWS CI cluster
kubectl --context="arn:aws:eks:eu-central-1:480319613404:cluster/ci-ps-eu-central-1" apply -f apps.keboola.com_apps.yaml
kubectl --context="arn:aws:eks:eu-central-1:480319613404:cluster/ci-ps-eu-central-1" apply -f apps.keboola.com_appruns.yaml

# Install on Azure CI cluster
kubectl --context="sandboxes-ci-2021-aks" apply -f apps.keboola.com_apps.yaml
kubectl --context="sandboxes-ci-2021-aks" apply -f apps.keboola.com_appruns.yaml
```

This is a one-time setup per cluster. The CRD defines the schema for AppRun resources used for billing and cost tracking.


## Implementing new API
Only few K8S APIs we needed are implement so far. To implement new API, do following:
* create API client wrapper in `Keboola\K8sClient\ApiClient`
* this is a wrapper around `kubernetes/php-client` API class, takes care of handling results
* add the wrapper to `KubernetesApiClientFacade`
* inject the `kubernetes/php-client` client through constructor
* add support for the new resource to methods signatures
* update `GenericClientFacadeFactory` to provide new API class to `KubernetesApiClientFacade`
* update `KubernetesApiClientFacade::create()` to provide the new API class to the facade

## License

Expand Down
8 changes: 0 additions & 8 deletions libs/k8s-client/phpstan.neon
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,5 @@ parameters:
- tests/stubs/K8s.stub
ignoreErrors:
- identifier: missingType.iterableValue
-
# BaseApi: $this->client->request() returns mixed (Guzzle Client), but we know it returns ResponseInterface
message: '#Parameter \#1 \$response of method KubernetesRuntime\\AbstractAPI::parseResponse\(\) expects Psr\\Http\\Message\\ResponseInterface, mixed given\.#'
path: src/BaseApi/*
-
# BaseApi: parseResponse() returns mixed (external library), but actual types are defined by method return types
message: '#Method .+ should return .+ but returns mixed\.#'
path: src/BaseApi/*
includes:
- vendor/phpstan/phpstan-phpunit/extension.neon
1 change: 1 addition & 0 deletions libs/k8s-client/src/ApiClient/ApiClientInterface.php
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ public function get(string $name, array $queries = []): AbstractModel;

/**
* @param TItem $model
* @return TItem
*/
public function create(AbstractModel $model, array $queries = []): AbstractModel;

Expand Down
26 changes: 0 additions & 26 deletions libs/k8s-client/src/ApiClient/AppRunsApiClient.php

This file was deleted.

26 changes: 0 additions & 26 deletions libs/k8s-client/src/ApiClient/AppsApiClient.php

This file was deleted.

Loading
Loading