feat(fleetcli): protobuf-driven Fleet command-line client#467
feat(fleetcli): protobuf-driven Fleet command-line client#467krisztiankurucz wants to merge 18 commits into
Conversation
github-actions
Bot
added
documentation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Introduces a protobuf-driven generator for fleetcli, adds handwritten runtime support (auth/session, pairing streaming, firmware HTTP lifecycle), and expands automated validation via unit + e2e tests.
Changes:
- Add
server/tools/generate-fleet-cligenerator (templates, overrides, report) and wire it intojust gen. - Implement/extend
server/cmd/fleetcliruntime (auth resolution, generated command runtime, pairing + firmware command groups). - Add extensive unit tests and e2e coverage for the CLI workflows.
Reviewed changes
Copilot reviewed 24 out of 36 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| server/tools/generate-fleet-cli/templates/fleet-group.gotmpl | Template for generated per-group command files. |
| server/tools/generate-fleet-cli/templates/fleet-commands.gotmpl | Template for generated command registry file. |
| server/tools/generate-fleet-cli/overrides.json | Declarative mapping of services/methods to CLI groups and naming/auth overrides. |
| server/tools/generate-fleet-cli/main.go | Core generator: loads descriptor set + overrides, analyzes requests, renders Go sources and a coverage report. |
| server/tools/generate-fleet-cli/main_test.go | Unit tests for goPackageInfo behavior. |
| server/go.mod | Adds direct deps used by fleetcli output/formatting and cli framework. |
| server/go.sum | Records checksums for new dependencies. |
| server/e2e/plugin_integration_test.go | Updates integration test auth flow (cookie session -> API key) and modernizes selector/telemetry calls. |
| server/e2e/curtailment_e2e_test.go | Switches proto-sim discovery host to container-reachable hostname. |
| server/e2e/fleetcli_e2e_test.go | New e2e test for core CLI workflow (onboarding -> discover -> pair -> miners -> performance). |
| server/e2e/fleetcli_firmware_e2e_test.go | New e2e test covering firmware lifecycle via CLI. |
| server/e2e/fleetcli_subcommands_e2e_test.go | New broad e2e test exercising many CLI subcommands and asserting leaf-command coverage. |
| server/docs/fleet-cli-generation.md | Documents generator architecture, inputs, statuses, and workflow. |
| server/cmd/fleetcli/main.go | Root CLI + manual command groups (auth/apikey/performance/firmware) and shared helpers. |
| server/cmd/fleetcli/client.go | CLI HTTP/Connect client, auth/session handling, URL normalization, secure-cookie loopback support. |
| server/cmd/fleetcli/generated_runtime.go | Shared runtime invoked by generated command expressions (auth dispatch + selector helpers). |
| server/cmd/fleetcli/pairing.go | Handwritten pairing group (streaming discover + selector-based pair). |
| server/cmd/fleetcli/firmware_client.go | Firmware REST client for config/check/upload/list/delete endpoints. |
| server/cmd/fleetcli/firmware.go | Handwritten firmware command group and upload orchestration + progress reporting. |
| server/cmd/fleetcli/manual.go | Aggregates handwritten commands for registry composition. |
| server/cmd/fleetcli/main_test.go | Unit tests for auth flag/env resolution + enum normalization + regression cases. |
| server/cmd/fleetcli/client_test.go | Unit tests for base URL normalization and loopback secure-cookie semantics. |
| server/cmd/fleetcli/pairing_test.go | Unit tests for discover/pair request builders and selector/credential behaviors. |
| server/cmd/fleetcli/firmware_test.go | Unit tests for firmware config/check/upload modes, chunking, validation, and error shaping. |
| justfile | Adds gen-fleet-cli / build-fleet-cli and runs generator as part of gen. |
| bin/scripts/fleetcli | Convenience script that builds and execs the local CLI binary. |
| bin/hermit.hcl | Adds scripts directory to PATH for hermit-managed environments. |
Files not reviewed (9)
- server/cmd/fleetcli/cmd_commands.go: Generated file
- server/cmd/fleetcli/cmd_groups.go: Generated file
- server/cmd/fleetcli/cmd_minercommand.go: Generated file
- server/cmd/fleetcli/cmd_miners.go: Generated file
- server/cmd/fleetcli/cmd_networkinfo.go: Generated file
- server/cmd/fleetcli/cmd_onboarding.go: Generated file
- server/cmd/fleetcli/cmd_pools.go: Generated file
- server/cmd/fleetcli/cmd_racks.go: Generated file
- server/cmd/fleetcli/cmd_schedule.go: Generated file
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
…ing unchanged files Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
…atch Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
…nputs through Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
…e build output Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
Signed-off-by: Krisztian Kurucz <kkurucz@squareup.com>
krisztiankurucz
force-pushed
the
kkurucz/port-fleet-client-api
branch
from
c202bcd to
d2f92ad
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c202bcd920
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6f78fad713
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2a7aed9bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 05605e994a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ankitgoswami
left a comment
ankitgoswami
left a comment
There was a problem hiding this comment.
I’m very supportive of having a CLI. I think it would be useful for operators, scripting, debugging, and workflows where the UI is too heavy. My hesitation is mostly about timing and where the CLI should attach.
RFC 0001 moves plugin orchestration, miner I/O, discovery, pairing, and command execution toward fleet nodes. Given that, I think the long-term CLI probably wants to be designed around the fleet-node boundary rather than around today’s transitional server/combined-mode API shape.
The main upside of making the CLI run through fleet nodes is that it would match the architecture we’re already moving toward: commands execute near the miners, LAN-only devices stay behind the customer network, local/degraded-mode workflows become possible, and the CLI semantics line up with fleet-node ownership instead of cloud reachability. It also gives us a cleaner story for multi-site and air-gapped setups.
The trade-off is that it likely means waiting for more of the fleet-node surface to settle, and the first version may be less broad. There’s also more complexity around node selection, enrollment state, offline behavior, and explaining which operations are server-wide versus scoped to a specific node. But I think that complexity is inherent in the product direction anyway, so baking it into the CLI from the start seems better than shipping a large CLI now and reshaping it soon after.
This fleetcli tool is a dependency for some additional testrack / rig reservation tooling I’m working on, so I still need a CLI surface for higher-level server / control-plane actions. That said, I agree with the concern about not baking in today’s transitional server-centric model for miner-local operations. For this first version, I’m happy to narrow the scope to control-plane workflows only, and remove or defer miner I/O, discovery, pairing, and command execution while the fleet-node command surface solidifies. Concretely, I’d keep things like auth/API key management, onboarding, groups/racks metadata, saved pool/config metadata, and read-only server state where appropriate. I’d defer pairing, minercommand, LAN/network discovery, and any command paths that imply direct miner execution or node-involved operations. Longer term, once the fleet-node API is settled, we can bring those operational commands back with explicit node/site scoping instead of treating the server API as the permanent CLI boundary. Does that make sense? |
|
yea, that makes sense to me. another thing to note here is that fleetnodes already have a CLI and will essentially be the way to communicate with a cloud hosted version of fleet. I'll review this once you have updated it. |
Summary
fleetcliis a new command-line client for the Fleet API. Most of its surface is generated from the same protobuf definitions the server uses — each RPC service becomes a command group with typed flags — so the CLI tracks the API automatically instead of drifting behind a hand-maintained wrapper. A thin handwritten runtime layers in what codegen can't express: transport, auth, and the streaming / long-running flows.How it works
Generated layer.
server/tools/generate-fleet-clireads a descriptor set (buf build --as-file-descriptor-set) plusoverrides.jsonand emits onecmd_<group>.goper service —miners,racks,groups,pools,schedule,networkinfo,onboarding,minercommand— wired together incmd_commands.go. Generated commands delegate request construction and execution to a shared runtime, keeping the emitted code small. The generator only rewrites a file when its contents actually change, so routine regen keeps file mtimes stable and never trips the docker-compose watch into restartingfleet-api.Handwritten runtime.
main.go/client.goown the global flags (--server,--api-key,--username,--password,--insecure— all env-backed), the HTTP JSON-RPC transport, and auth dispatch. Flows that don't fit the generated mold are hand-built:auth(session login),apikey(key lifecycle),pairing(streaming discovery),performance(telemetry), andfirmware(file upload/lifecycle over the server's plain HTTP endpoints).Tooling.
just gen-fleet-clibuilds the descriptor set and runs the generator (also folded intojust gen);just build-fleet-clicompiles to.cache/so the binary lives outsideserver/and a CLI rebuild never restartsfleet-api. Abin/scripts/fleetcliwrapper plus a Hermit entry make it runnable in-repo. Full architecture inserver/docs/fleet-cli-generation.md.Test plan
just lint— clean (buf, eslint, golangci-lint).cd server && go test ./cmd/fleetcli/... ./tools/generate-fleet-cli/...— pass.just gen-fleet-cliis idempotent against the committedcmd_*.go— generated code is in sync.cd server && go test -tags=e2e ./e2e -run TestFleetCLI— covers the command workflow, firmware upload/file lifecycle, subcommands, and leaf-command coverage.Manual testing:
E2E tests:
Co-authored-by: Caleb Johnston calebjohnston@block.xyz