-
Notifications
You must be signed in to change notification settings - Fork 23
HYPERFLEET-1186 - fix: align documentation with codebase and fix pageSize default #211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
5243d53
d2d1e75
bdb9bed
83ff866
bcf3896
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,77 +1,25 @@ | ||
| # HyperFleet API | ||
|
|
||
| HyperFleet API - Simple REST API for cluster lifecycle management. Provides CRUD operations for clusters and status sub-resources. Pure data layer with PostgreSQL integration - no business logic or event creation. Stateless design enables horizontal scaling. | ||
|
|
||
| ## Architecture | ||
|
|
||
| ### Technology Stack | ||
|
|
||
| - **Language**: Go 1.24+ | ||
| - **API Definition**: OpenAPI 3.0 | ||
| - **Code Generation**: oapi-codegen | ||
| - **Database**: PostgreSQL with GORM ORM | ||
| - **Container Runtime**: Podman | ||
| - **Testing**: Gomega + Resty | ||
|
|
||
| ### Core Features | ||
|
|
||
| * OpenAPI 3.0 specification | ||
| * Automated Go code generation from OpenAPI | ||
| * Cluster and NodePool lifecycle management | ||
| * Adapter-based status reporting with Kubernetes-style conditions | ||
| * Pagination and search capabilities | ||
| * Complete integration test coverage | ||
| * Database migrations with GORM | ||
| * Embedded OpenAPI specification using `//go:embed` | ||
|
|
||
| ### Project Structure | ||
|
|
||
| ```text | ||
| hyperfleet-api/ | ||
| ├── cmd/hyperfleet-api/ # Application entry point | ||
| ├── pkg/ | ||
| │ ├── api/ # API models and handlers | ||
| │ ├── dao/ # Data access layer | ||
| │ ├── db/ # Database setup and migrations | ||
| │ ├── handlers/ # HTTP request handlers | ||
| │ └── services/ # Business logic | ||
| ├── openapi/ # Generated artifacts from hyperfleet-api-spec module | ||
| ├── test/ # Integration tests and factories | ||
| ├── docs/ # Detailed documentation | ||
| └── Makefile # Build automation | ||
| ``` | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ### Prerequisites | ||
| The HyperFleet API is the data storage and status aggregation layer of the HyperFleet platform. It exposes a REST API for CRUD operations on customizable entities (e.g. Cluster, NodePool) backed by PostgreSQL. | ||
|
|
||
| - **Go 1.24+**, **Podman**, **PostgreSQL 13+**, **Make** | ||
| The API is the source of truth for desired state of resources that live in remote clusters. It persists resource specs, increments generation on spec changes, and aggregates adapter-reported conditions into Kubernetes-style status. | ||
|
|
||
| See [PREREQUISITES.md](PREREQUISITES.md) for installation instructions. | ||
| It does not reconcile infrastructure or publish events itself. For that it collaborates with other HyperFleet components: | ||
|
|
||
| ### Installation | ||
| * **[Sentinel](https://github.com/openshift-hyperfleet/hyperfleet-sentinel)** component polls the API for unreconciled resources and publishes a message for reconciliation actions | ||
| * **[Adapter](https://github.com/openshift-hyperfleet/hyperfleet-adapter)** component listens to events, performs actions needed to reconcile a resource and reports the status to the API. | ||
|
|
||
| ```bash | ||
| # 1. Generate OpenAPI code and mocks | ||
| make generate-all | ||
|
|
||
| # 2. Install dependencies | ||
| go mod download | ||
| Stateless design enables horizontal scaling. Adapters fetch full resource state from the API after receiving minimal CloudEvents (anemic events pattern). | ||
|
|
||
| # 3. Build binary | ||
| make build | ||
| ## Getting Started | ||
|
|
||
| # 4. Setup database | ||
| make db/setup | ||
| ### Deploying to Kubernetes | ||
|
|
||
| # 5. Run migrations | ||
| ./bin/hyperfleet-api migrate | ||
| For Helm-based deployment to staging, production, or partner environments, see the **[Deployment Guide](docs/deployment.md)** — covers container images, Helm values, external databases, schema validation, monitoring, and production checklists. | ||
|
|
||
| # 6. Start service (no auth) | ||
| make run-no-auth | ||
| ``` | ||
| ### Local Development | ||
|
|
||
| **Note**: Generated code is not tracked in git. You must run `make generate-all` after cloning. | ||
| For setting up a local development environment, see the **[Development Guide](docs/development.md)** — covers prerequisites, code generation, mock generation, database setup, running tests, pre-commit hooks, and development workflows. | ||
|
|
||
| ### Accessing the API | ||
|
|
||
|
|
@@ -97,7 +45,8 @@ Kubernetes clusters with provider-specific configurations, labels, and adapter-b | |
|
|
||
| **Main endpoints:** | ||
| - `GET/POST /api/hyperfleet/v1/clusters` | ||
| - `GET /api/hyperfleet/v1/clusters/{id}` | ||
| - `GET/PATCH/DELETE /api/hyperfleet/v1/clusters/{id}` | ||
| - `POST /api/hyperfleet/v1/clusters/{id}/force-delete` | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Check if force-delete handler enforces stricter authz than regular DELETE
rg -n -C5 'force-delete|ForceDelete' --type=go pkg/handlers/ | head -80Repository: openshift-hyperfleet/hyperfleet-api Length of output: 4993 🏁 Script executed: #!/bin/bash
set -euo pipefail
# 1) Locate routes/endpoints mentioning force-delete and where authz/middleware is wired.
rg -n --hidden --no-ignore-vcs "force-delete" . || true
rg -n --hidden --no-ignore-vcs "ForceDelete" . || true
# 2) Inspect surrounding handler code for ForceDelete and any normal DELETE methods.
echo "---- cluster.go around ForceDelete and any Delete handlers ----"
sed -n '130,240p' pkg/handlers/cluster.go || true
echo "---- cluster_nodepools.go around ForceDelete and any Delete handlers ----"
sed -n '1,260p' pkg/handlers/cluster_nodepools.go || true
# 3) Inspect framework handler for handleForceDelete / authz-related hooks.
sed -n '1,240p' pkg/handlers/framework.go || true
# 4) README snippet around line ~122 (as referenced by review comment)
ls -la || true
sed -n '90,150p' README.md || trueRepository: openshift-hyperfleet/hyperfleet-api Length of output: 30129 🏁 Script executed: #!/bin/bash
set -euo pipefail
# 1) Inspect plugin/router wiring around force-delete handlers.
sed -n '1,140p' plugins/clusters/plugin.go
# 2) Find router/middleware/auth wiring keywords.
rg -n --hidden --no-ignore-vcs "middleware|auth|RBAC|Role|scope|Authorize|authorization|Permission|entitle|claims|user|principal" pkg cmd plugins charts configs serve . || true
# 3) Check how handlers get context: ownership/user extracted somewhere.
rg -n --hidden --no-ignore-vcs "GetByIDAndOwner|owner|tenant|project|account|userID|principal|subject" pkg | head -200 || true
# 4) Inspect cluster/node_pool service ForceDelete implementations for any explicit authz checks.
sed -n '450,660p' pkg/services/cluster.go
sed -n '220,380p' pkg/services/node_pool.go
# 5) Inspect regular DELETE handlers to compare logic surface.
# cluster DELETE handler
sed -n '120,220p' pkg/handlers/cluster.go
# nodepool DELETE handler
sed -n '80,200p' pkg/handlers/cluster_nodepools.goRepository: openshift-hyperfleet/hyperfleet-api Length of output: 50392 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "---- pkg/services/cluster.go: ForceDelete + any SoftDelete/Delete around it ----"
rg -n "func \\(s \\*sqlClusterService\\) (ForceDelete|SoftDelete|Delete)" pkg/services/cluster.go
sed -n '430,620p' pkg/services/cluster.go
echo "---- pkg/services/node_pool.go: ForceDelete + any SoftDelete/Delete around it ----"
rg -n "func \\(s \\*sqlNodePoolService\\) (ForceDelete|SoftDelete|Delete)" pkg/services/node_pool.go
sed -n '180,360p' pkg/services/node_pool.go
echo "---- pkg/dao: ForceDelete-related queries in cluster/node_pool ----"
rg -n "ForceDelete" pkg/dao || true
rg -n "SoftDelete|Soft delete|deleted_time" pkg/dao/cluster*.go pkg/dao/*node*pool*.go || true
echo "---- handler delete endpoints (for comparison): pkg/handlers/cluster.go SoftDelete/DELETE semantics ----"
rg -n "func \\(h ClusterHandler\\) (SoftDelete|Delete|ForceDelete)" pkg/handlers/cluster.go
sed -n '120,220p' pkg/handlers/cluster.go
echo "---- handler delete endpoints (for comparison): pkg/handlers/cluster_nodepools.go SoftDelete/ForceDelete semantics ----"
rg -n "func \\(h ClusterNodePoolsHandler\\) (SoftDelete|ForceDelete)" pkg/handlers/cluster_nodepools.go
sed -n '80,200p' pkg/handlers/cluster_nodepools.go
echo "---- cmd/hyperfleet-api/server: middleware/router auth wiring (targeted) ----"
rg -n "Use\\(|middleware|auth\\.|authz|RBAC|authorize|permission|JWT|Identity" cmd/hyperfleet-api/server cmd/hyperfleet-api/environments cmd/hyperfleet-api/*.go || trueRepository: openshift-hyperfleet/hyperfleet-api Length of output: 22615 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Cluster service: SoftDelete implementation details
echo "---- pkg/services/cluster.go: SoftDelete (and related delete methods) ----"
sed -n '120,260p' pkg/services/cluster.go
sed -n '260,420p' pkg/services/cluster.go
sed -n '420,560p' pkg/services/cluster.go
# Node pool service: see SoftDelete/ Delete / ForceDelete call patterns re owner checks
echo "---- pkg/services/node_pool.go: SoftDelete and Delete ----"
sed -n '150,280p' pkg/services/node_pool.go
sed -n '280,360p' pkg/services/node_pool.go
# DAO cluster: see whether GetForUpdate or SoftDelete uses caller/owner scoping
echo "---- pkg/dao/cluster*.go: search for GetForUpdate / SoftDelete ownership ----"
rg -n "type .*ClusterDao|GetForUpdate|SoftDelete|DeletedTime|ByOwner|Owner" pkg/dao --glob '*cluster*' || true
rg -n "GetForUpdate\\(|SoftDelete\\(|Delete\\(" pkg/dao --glob '*cluster*' || true
# Show cluster dao implementation for GetForUpdate and any owner-filtered queries
# (we don't know filename; use fd-like search with rg results)
echo "---- showing matching cluster dao files with GetForUpdate ----"
rg -n "GetForUpdate\\(" pkg/dao --glob '*cluster*' --files-with-matches || true
# Print relevant sections from all files that define GetForUpdate in cluster dao (limited)
for f in $(rg -n "func \\(.*\\) GetForUpdate\\(" pkg/dao --glob '*cluster*' --files-with-matches); do
echo "==== $f ===="
sed -n '1,220p' "$f"
sed -n '220,420p' "$f"
done || true
# Node pool DAO: GetByIDAndOwner scoping
echo "---- pkg/dao/node*pool*.go: GetByIDAndOwner ----"
rg -n "GetByIDAndOwner\\(" pkg/dao --glob '*node*pool*' || true
for f in $(rg -n "GetByIDAndOwner\\(" pkg/dao --glob '*node*pool*' --files-with-matches || true); do
echo "==== $f ===="
sed -n '1,260p' "$f"
done
# Handlers/tests: look for authz expectations for force-delete
echo "---- pkg/handlers/cluster_test.go: search for Unauthorized/Forbidden around ForceDelete ----"
rg -n "ForceDelete|force-delete|401|403|Unauthorized|Forbidden" pkg/handlers/cluster_test.go pkg/handlers/cluster_nodepools_test.go | head -200 || true
echo "---- pkg/handlers/cluster_nodepools_test.go: search for authz around ForceDelete ----"
rg -n "Unauthorized|Forbidden|401|403" pkg/handlers/cluster_nodepools_test.go pkg/handlers/cluster_test.go | head -200 || true
echo "---- integration tests: search for force-delete authz expectations ----"
rg -n "force-delete.*(401|403|Unauthorized|Forbidden)" test/integration -S --glob '*force*' || true
rg -n "Unauthorized|Forbidden|401|403" test/integration/clusters_test.go test/integration/node_pools_test.go | head -200 || trueRepository: openshift-hyperfleet/hyperfleet-api Length of output: 38188 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "---- README.md: search for force-delete auth/RBAC/runbook text ----"
rg -n "force-delete|ForceDelete|RBAC|role|permission|authorize|authorization|runbook|safety|admin" README.md docs/api-resources.md docs/runbook.md docs/database.md || true
echo "---- README.md around line ~100-150 (includes force-delete endpoint list) ----"
sed -n '90,160p' README.md || true
echo "---- docs/api-resources.md: force-delete cluster section ----"
rg -n "POST /api/hyperfleet/v1/clusters/\\{.*\\}/force-delete|force-delete cluster|Force Delete Cluster|Force Delete" docs/api-resources.md || true
# show likely relevant sections by line ranges
sed -n '340,420p' docs/api-resources.md || true
echo "---- docs/api-resources.md: force-delete nodepool section ----"
sed -n '520,600p' docs/api-resources.md || true
echo "---- cmd/hyperfleet-api/server/routes.go: full to see if authz is wired ----"
sed -n '1,200p' cmd/hyperfleet-api/server/routes.go || true
sed -n '200,420p' cmd/hyperfleet-api/server/routes.go || true
echo "---- search cmd/hyperfleet-api/server for authorize/authz/RBAC names ----"
rg -n --no-ignore-vcs "authorize|authorization|authz|RBAC|role|permission|Scope" cmd/hyperfleet-api/server pkg/middleware pkg/auth || trueRepository: openshift-hyperfleet/hyperfleet-api Length of output: 16418 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "---- Find Cluster model struct in pkg/api ----"
rg -n "type Cluster struct" pkg/api || true
# show likely file
f=$(rg -n "type Cluster struct" pkg/api --files-with-matches | head -n 1 || true)
echo "cluster struct file: $f"
if [ -n "${f:-}" ]; then
sed -n '1,220p' "$f"
fi
echo "---- Search for owner_id / OwnerID in Cluster struct or fields ----"
rg -n "OwnerID|owner_id|owner" "$f" || true
echo "---- Confirm cluster handler SoftDelete (regular DELETE) calls service without ownership scoping ----"
sed -n '130,220p' pkg/handlers/cluster.go
echo "---- docs/api-resources.md force-delete cluster section for any RBAC/access control text ----"
# narrow by lines around "Force Delete Cluster"
rg -n "### Force Delete Cluster|Force Delete Cluster|force-delete cluster" docs/api-resources.md
# print a chunk around those lines
sed -n '350,410p' docs/api-resources.md
echo "---- docs/api-resources.md force-delete nodepool section for RBAC/access control text ----"
sed -n '540,590p' docs/api-resources.mdRepository: openshift-hyperfleet/hyperfleet-api Length of output: 7568 Enforce authorization for force-delete endpoints (CWE-285).
🤖 Prompt for AI Agents
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We're just listing the endpoints in the README.md. I think we can talk about authorization verification in a later section, with a reminder in api-resources for example, pointing to the authentication.md ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
That said, the documentation concern is separate from the underlying code issue (CWE-285): Would you like me to open a follow-up GitHub issue to track the missing authorization enforcement in the force-delete handlers? |
||
| - `GET/PUT /api/hyperfleet/v1/clusters/{id}/statuses` | ||
|
|
||
| ### NodePools | ||
|
|
@@ -107,10 +56,19 @@ Groups of compute nodes within clusters. | |
| **Main endpoints:** | ||
| - `GET /api/hyperfleet/v1/nodepools` | ||
| - `GET/POST /api/hyperfleet/v1/clusters/{cluster_id}/nodepools` | ||
| - `GET /api/hyperfleet/v1/clusters/{cluster_id}/nodepools/{nodepool_id}` | ||
| - `GET/PATCH/DELETE /api/hyperfleet/v1/clusters/{cluster_id}/nodepools/{nodepool_id}` | ||
| - `POST /api/hyperfleet/v1/clusters/{cluster_id}/nodepools/{nodepool_id}/force-delete` | ||
| - `GET/PUT /api/hyperfleet/v1/clusters/{cluster_id}/nodepools/{nodepool_id}/statuses` | ||
|
|
||
| Both resources support pagination, label-based search, and adapter status reporting. See [docs/api-resources.md](docs/api-resources.md) for complete API documentation. | ||
| ### Generic Resources | ||
|
|
||
| The API also supports generic resource types registered via the plugin system. Currently available: | ||
|
|
||
| - **WifConfigs** — `GET/POST /api/hyperfleet/v1/wifconfigs`, `GET/PATCH/DELETE .../wifconfigs/{id}` | ||
| - **Channels** — `GET/POST /api/hyperfleet/v1/channels`, `GET/PATCH/DELETE .../channels/{id}` | ||
| - **Versions** — `GET/POST /api/hyperfleet/v1/channels/{parent_id}/versions`, `GET/PATCH/DELETE .../versions/{id}` (child of Channel) | ||
|
|
||
| All resources support pagination, label-based search, and spec validation. Clusters and NodePools additionally support adapter status reporting. See [docs/api-resources.md](docs/api-resources.md) for complete API documentation. | ||
|
|
||
| ## Example Usage | ||
|
|
||
|
|
@@ -131,36 +89,6 @@ curl -G http://localhost:8000/api/hyperfleet/v1/clusters \ | |
|
|
||
| See [docs/search.md](docs/search.md) for search and filtering documentation. | ||
|
|
||
| ## Development | ||
|
|
||
| ### Common Commands | ||
|
|
||
| ```bash | ||
| make build # Build binary to bin/ | ||
| make run-no-auth # Run without authentication | ||
| make test # Run unit tests | ||
| make test-integration # Run integration tests | ||
| make generate # Generate OpenAPI models | ||
| make generate-mocks # Generate test mocks | ||
| make generate-all # Generate OpenAPI models and mocks | ||
| make db/setup # Create PostgreSQL container | ||
| make image # Build container image | ||
| ``` | ||
|
|
||
| See [docs/development.md](docs/development.md) for detailed workflows. | ||
|
|
||
| ### CLI Subcommands | ||
|
|
||
| ```bash | ||
| ./bin/hyperfleet-api serve # Start the HTTP server | ||
| ./bin/hyperfleet-api migrate # Run database migrations | ||
| ./bin/hyperfleet-api version # Print version, commit, and build date | ||
| ``` | ||
|
|
||
| ### Pre-commit Hooks | ||
|
|
||
| This project uses [pre-commit](https://pre-commit.io/) for code quality checks. See [docs/development.md](docs/development.md#pre-commit-hooks-optional) for setup instructions. | ||
|
|
||
| ## Documentation | ||
|
|
||
| ### Core Documentation | ||
|
|
@@ -169,6 +97,7 @@ This project uses [pre-commit](https://pre-commit.io/) for code quality checks. | |
| - **[Development Guide](docs/development.md)** - Local setup, testing, code generation, and workflows | ||
| - **[Database](docs/database.md)** - Schema, migrations, and data model | ||
| - **[Deployment](docs/deployment.md)** - Container images, Kubernetes deployment, and configuration | ||
| - **[Configuration](docs/config.md)** - Complete configuration reference (database, server, caller identity, adapters) | ||
| - **[Authentication](docs/authentication.md)** - Development and production auth | ||
| - **[Logging](docs/logging.md)** - Structured logging, OpenTelemetry integration, and data masking | ||
| - **[Validation Schema](openapi/README.md#validation-schema)** - How to supply a custom OpenAPI schema for runtime `spec` field validation | ||
|
|
@@ -183,4 +112,4 @@ This project uses [pre-commit](https://pre-commit.io/) for code quality checks. | |
|
|
||
| ## License | ||
|
|
||
| [License information to be added] | ||
| This project is licensed under the Apache License 2.0. See [LICENSE](LICENSE) for details. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I personally would go for a simpler README
e.g. the Architecture section is a list of features, with different abstraction/detail level.
Also the endpoints list could be stripped away since we have a doc for that
Regarding the definition of the API, now that I read it again I find it to be misleading...
I am thinking on something like:
The HyperFleet API is the data storage and status aggregation layer of the HyperFleet platform. It exposes a REST API for CRUD operations on customizable entities (e.g. Cluster, NodePool) backed by PostgreSQL.
The API is the source of truth for desired state of resources that live in remote clusters. It persists resource specs, increments
generationon spec changes, and aggregates adapter-reported conditions into Kubernetes-style status.It does not reconcile infrastructure or publish events itself. For that it collaborates with other HyperFleet components:
Stateless design enables horizontal scaling. Adapters fetch full resource state from the API after receiving minimal CloudEvents (anemic events pattern).