feat: gRPC mock server (third mock engine)#51
Merged
Conversation
Design for `agctl mock grpc` — a gRPC mock engine alongside HTTP/Kafka, serving all four call types via a descriptor-driven generic servicer, with auto-served Health + Reflection. Reuses the existing stub pipeline and the gRPC client's descriptor/JSON↔protobuf machinery (extracted into a shared proto kernel). Co-Authored-By: Claude <noreply@anthropic.com>
Task-by-task plan: shared proto kernel, public status helper, config models, jq/capture walkers, pure dispatch core, MockGrpcServer (construction + servicer + health/reflection), engine/daemon/commands/ discovery extensions, integration tests, docs sync. Co-Authored-By: Claude <noreply@anthropic.com>
…ors.py Move the proto-only helpers (descriptor resolution, service/method lookup, JSON<->protobuf translation) out of GrpcClient into a new shared kernel module so the upcoming gRPC mock server (Tasks 6/7) reuses one source of truth. Pure behavior-preserving refactor: - GrpcClient.resolve_descriptors/find_method/call_type_of/_msg_class/ _serialize/_deserialize now delegate to grpc_descriptors.* (reflection path stays in grpc_client.py — client-only). - GrpcClient.call_type_of remains accessible as a delegating staticmethod. - Lazy-import discipline enforced: the kernel never imports grpc / grpc_tools / google.protobuf at module top; missing libraries surface as ConfigError pointing at pip install 'agctl[grpc]'. Adds tests/fixtures/mock_grpc/echo.proto (echo.EchoService with all four call types) and tests/unit/conftest.py with a session-scoped pool fixture shared by this and later tasks (6/7/12). All 33 existing GrpcClient tests still pass; full unit suite (1235 tests) passes. Co-Authored-By: Claude <noreply@anthropic.com>
Factor the duplicated name-or-code gRPC status parsing out of validate_grpc_assertion_args and evaluate_grpc_assertions into one public parse_grpc_status(status) -> (code, name) helper. Pure dedupe; behavior preserved (case-sensitive name lookup, digit-string coercion, int-code lookup). The helper's own error message drops the CLI --status prefix so it is reusable beyond the CLI (config model in Task 3, mock server in Task 5); validate_grpc_assertion_args keeps its CLI-prefixed message via try/except ... from None. Helper does not import grpc. Co-Authored-By: Claude <noreply@anthropic.com>
Add typed Pydantic v2 config models for mocks.grpc, parallel to the existing mocks.http/mocks.kafka models, with offline structural validation only (response-shape-vs-call-type deferred to Task 6): - GrpcMatch: body subset + jq predicate (mirrors HttpMatch) - GrpcResponseMessage: single streaming-response element (message + delay_ms ge=0) - GrpcResponse: exactly-one-of message/messages (model_validator); status validated via parse_grpc_status and stored verbatim for render-time (code,name) resolution - GrpcStub: service/method/match/capture/response/delay_ms - GrpcMockConfig: listen (parse_listen validator), descriptors, reflection, health, concurrency_cap (ge=1), stubs - MocksConfig.grpc wired (forward ref + model_rebuild, since GrpcMockConfig depends on GrpcDescriptorSource from the gRPC section) Co-Authored-By: Claude <noreply@anthropic.com>
…stubs Co-Authored-By: Claude <noreply@anthropic.com>
Transport-agnostic dispatch brain for the gRPC mock server. Module-level
functions + GrpcDispatchOutcome dataclass only (no class yet — Task 6/7 add
MockGrpcServer). NO `import grpc` anywhere in the file: the dispatch core
is unit-testable without grpcio installed (verified by a runtime probe and
a source-level test).
Exports:
- build_envelope(service, method, metadata, *, message=None, messages=None)
Shapes the per-call-type §8.1 envelope: {service, method, metadata
(lowercased), message} for unary/server_stream/bidi; {..., messages,
count} for client_stream.
- GrpcDispatchOutcome dataclass: matched, stub_name, messages, status,
missing_captures. Unmatched outcome clears all other fields.
- dispatch_grpc(stubs, envelope, call_type, *, emit_capture_missing)
First-match-wins over stubs; match.body (json_subset) AND match.jq
(jq_bool) AND-ed; omitted match matches unconditionally. client_stream
skips match.body (subset over aggregated list is ill-defined). On match:
resolve_captures -> emit_capture_missing per missing -> render_typed per
response message (server_stream: each messages[*]; others: response.message)
-> parse_grpc_status for terminal status. Unmatched -> matched=False.
Signature deviation from brief: stubs is dict[str, GrpcStub] (insertion-
ordered), not list[GrpcStub]. GrpcStub (Task 3) has no .name field — its
identity is the dict key in mocks.grpc.stubs.<name>, exactly like HttpStub.
The dict form carries that name through dispatch without modifying Task 3's
model. Matches MockHTTPServer's stubs: dict[str, HttpStub] precedent.
Reuses json_subset/jq_bool/parse_grpc_status (assertions.py),
resolve_captures (mock/capture.py), render_typed (resolution.py) unchanged.
26 new tests in tests/unit/test_mock_grpc_dispatch.py cover all 10 brief
scenarios (a-j) plus extras (default/int/digit-string status, empty stub
map, omitted match, client_stream body-skip, bidi, dataclass shape,
grpcio-free source probe). 7 tests are jq-gated (per-test
pytest.importorskip("jq")) — they skip cleanly when jq is missing; the
other 19 run without jq. Full unit suite: 1315 passed (was 1289; +26).
Co-Authored-By: Claude <noreply@anthropic.com>
…docstring
Fix 1 (Important): rewrite test_grpc_server_module_does_not_import_grpc to
AST-parse agctl/mock/grpc_server.py and inspect only top-level body nodes
for banned imports (grpc, grpcio, grpc_tools, google, google.protobuf).
The prior substring grep ('import grpc' / 'from grpc' / 'import grpcio')
was a tripwire: a lazy 'import grpc' inside a function body contains the
same substring and would have failed Task 6's first commit even though
the no-top-level-import invariant holds. AST inspection of tree.body only
flags true module-top imports; nested lazy imports (the Task 6/7 contract)
no longer trip the test. Verified: a temporary 'import grpc' placed inside
build_envelope left the test green; change reverted.
Fix 2 (Minor): correct build_envelope docstring — the code checks 'messages'
first, so 'messages' wins (not 'message'). Wording only, no behavior change.
Co-Authored-By: Claude <noreply@anthropic.com>
…d table Task 6 of the gRPC mock server. MockGrpcServer.__init__ resolves the descriptor pool ONCE (via the shared kernel, or an injected pool for tests), validates each stub's service/method + response-shape-vs-call-type against the derived descriptor, and precomputes stubs_by_method / method_meta. Construction failures raise ConfigError (exit 2) before any bind. grpc-runtime wiring (generic servicer, Health, Reflection, lifecycle) remains Task 7 — _server stays None and no grpc import is added. stubs_by_method is the dict-of-dicts form (OVERRIDE of the brief's Produces): dict[tuple[str,str], dict[str, GrpcStub]] — the inner ordered name->stub dict carries stub names for Task 5's dispatch_grpc(stubs: dict[str, GrpcStub], ...), which needs them for emit_capture_missing. Co-Authored-By: Claude <noreply@anthropic.com>
…ifecycle Wire the grpc runtime into MockGrpcServer (Task 7 of 13). Adds: - _build_server(): builds grpc.Server with one grpc.method_handlers_generic_handler per configured service; each method's RpcMethodHandler is wired with the kernel's serialize/deserialize + a behavior callable that routes through dispatch_grpc. - Four behavior handlers (_handle_unary/_handle_server_stream/ _handle_client_stream/_handle_bidi) covering match/unmatched/non-OK status/per-message delay_ms, emitting grpc.hit/grpc.unmatched/grpc.error events via the engine's emit_event channel. - Auto-served Health (config.health) marking every service + overall "" as SERVING; auto-served Reflection (config.reflection) via reflection.enable_server_reflection, passing the descriptor pool so file_containing_symbol responses are populated. - Lifecycle: start() (ephemeral-port recompute, EADDRINUSE -> ConfigError), serve_forever(stop_event) polling every 200ms, idempotent shutdown(2s grace), actual_listen() returning the bound host:port. All grpc/grpc_health/grpc_reflection imports live inside method bodies so the module stays grpcio-free at module top (AST invariant preserved). Missing grpc extra surfaces as ConfigError pointing at 'pip install agctl[grpc]'. 18 new TDD tests (RED -> GREEN) cover all four call types, Health/Reflection on/off, port-in-use, missing-extra, and lifecycle. 1355/1355 unit tests pass. Co-Authored-By: Claude <noreply@anthropic.com>
…ts/summary)
Extend MockEngine to construct/run/shut down a MockGrpcServer as a third
engine alongside HTTP and Kafka:
- __init__ gains run_grpc/grpc_listen/grpc_server_factory (default-safe so
pre-Task-10 new_mock_engine call sites still work). The default factory
lazy-imports MockGrpcServer inside the closure body so the engine module
stays grpcio-free at import time.
- start() Step 2b: resolve listen (--grpc-listen overrides mocks.grpc.listen
via model_copy), construct via factory, server.start() binds (EADDRINUSE
already converted to ConfigError by MockGrpcServer). Assigns to
self._grpc_server only after successful bind so the failure path doesn't
leave a partially-constructed server.
- run(): daemon thread running serve_forever(self._stop) alongside the
HTTP/reactor threads.
- shutdown(): guarded server.shutdown() + thread join.
- emit_event: grpc.hit tallies; grpc.unmatched/grpc.error tally AND set
_runtime_error unconditionally (both are fatal per DESIGN §8.3) so run()
exits 1 and the summary surfaces the counts.
- _emit_started_line: grpc block {listen, stubs, services, reflection, health}.
- _emit_summary_line: grpc_hits/grpc_unmatched/grpc_errors under the snapshot
lock alongside the existing counters.
Top-level Config.grpc.descriptors fallback is threaded in Task 10 (the
engine only sees MocksConfig); for now top_level_descriptors=None and
MockGrpcServer surfaces an empty-descriptors ConfigError if mocks.grpc.
descriptors is also None.
Unit-tested via a FakeGrpcServer injected through grpc_server_factory —
the engine test suite stays grpcio-free.
Co-Authored-By: Claude <noreply@anthropic.com>
Task 9: extend the pure daemon functions for gRPC-only and multi-engine Co-Authored-By: Claude <noreply@anthropic.com>
test_resolve_target_listen_matches_http_listen previously set the pidfile's legacy listen AND http_listen to the same value as the lookup, so the match could succeed via the legacy listen field even if the http_listen branch were deleted. Set legacy listen to a different address (127.0.0.1:19090) so the match can only succeed through http_listen. Verified via temporary mutation: removing mock.http_listen from the membership tuple made the test fail; restoring it made it pass again. Co-Authored-By: Claude <noreply@anthropic.com>
…ult grpc block
Wire the gRPC mock server into the `mock` CLI per Task 10:
- `_resolve_engines` returns (run_http, run_kafka, run_grpc); new `--only grpc`
branch guards mocks.grpc.stubs; existing http/kafka branches gain trailing
False; default resolution picks up mocks.grpc when stubs are configured.
- `new_mock_engine` threads run_grpc / grpc_listen / grpc_server_factory /
top_level_descriptors to MockEngine (defaults preserve pre-Task-10 callers).
- `mock run` + `mock start` Click: --only choice now [http, kafka, grpc];
new --grpc-listen option (literal, mirrors --http-listen).
- `mock run` resolves grpc_listen and threads cfg.grpc.descriptors as the
top-level fallback (Task 8 obligation).
- `_mock_start_core` resolves grpc_listen; keys grpc-only pidfiles under
mock-grpc-<port>.{pid,log} via engine="grpc" (Task 9 plumbing); daemon
argv forwards --grpc-listen and --only grpc; already-running pre-check
branches on http→grpc→kafka.
- Pidfile JSON now persists http_listen and grpc_listen (Task 9 obligation)
so mock status/stop selection by either listen works end-to-end.
- mock start result adds a `grpc` block ({listen, stubs, services, reflection,
health}) when run_grpc; HTTP listen/stubs stay as HTTP values.
- MockEngine gains top_level_descriptors param and forwards it to the
grpc_server_factory call (was hardcoded None).
20 new TDD tests (RED→GREEN); full unit suite 1398 passing.
Co-Authored-By: Claude <noreply@anthropic.com>
Surfaces gRPC mock stubs in agctl discover, mirroring the existing
mock-http-stubs category: a new mock-grpc-stubs entry in _VALID_CATEGORIES,
a grpc_mock_stubs Level-0 count, and Level-1/Level-2/search branches that
read cfg.mocks.grpc.stubs via the _mock_grpc_stubs accessor.
- _mock_grpc_stubs(cfg) / _mock_grpc_listen(cfg) accessors (None-safe)
- _grpc_mock_params(stub): capture keys + {placeholder} scan of response
message(s), folded into params per Task 11 brief
- _mock_grpc_example: ready-to-use grpcurl hint (gRPC analogue of the curl
example for mock-http-stubs), with wildcard/IPv6 normalization
- _item_core: match (if set) via model_dump; missing name -> TemplateNotFound
- Fixture gains 2 gRPC mock stubs (echo-ok, echo-status)
Co-Authored-By: Claude <noreply@anthropic.com>
…ealth, reflection)
Task 12 of 13. Spins a real in-process grpcio mock server via MockEngine
(run_grpc=True, default factory lazy-imports the real MockGrpcServer) and
drives all four gRPC call types + Health + Reflection against it through the
`agctl grpc call` / `agctl grpc healthcheck` CLI path via CliRunner.
Self-contained: gated ONLY on the grpc extra (module-level
pytest.importorskip on grpc/grpc_tools/grpc_health/grpc_reflection). No
AGCTL_TEST_LIVE, no Docker — the gRPC mock is in-process server + client.
Verified 9/9 PASS with grpc installed (0.16s); clean skip when grpc is
missing (simulated). Full unit suite (1406) still green.
Scenarios covered:
- unary stub with {msg} capture -> templated response
- status: NOT_FOUND stub -> NOT_FOUND result (ok:true, D6)
- server-stream -> 3 messages in authored order
- client-stream -> one aggregated response templated on {count}
- bidi -> one response per request, in order
- grpc healthcheck --target mock -> SERVING
- reflection-only target (no client descriptors) -> resolves via reflection
- body-match stub + non-matching request -> UNIMPLEMENTED + grpc.unmatched
- engine shutdown summary -> grpc_hits/grpc_unmatched/grpc_errors counters
Co-Authored-By: Claude <noreply@anthropic.com>
Closeout for the gRPC mock feature: ship a commented `mocks:` example in the
packaged sample config (loaded by `agctl config init`) so users can discover
the mock server. The gRPC sub-block shows listen, reflection, health, a
commented descriptors proto/descriptor_set pair, and two stubs covering unary
`response.message` and server-stream `response.messages`, with match/capture
(capture rooted at `.message.`, the gRPC envelope root) and a gRPC status.
The http/kafka sub-blocks were missing from the sample too (the earlier
mock-server feature never updated it), so all three are added together for
coherence, matching DESIGN §2.1 and tests/fixtures/agctl.yaml. The whole block
is commented out → the sample still parses with `mocks: null` and validates
with no environment set.
Mirrored the identical block into README.md's "Complete, copy-paste-ready
config" to satisfy the test_sample_matches_readle_block drift guard
(byte-identical, verified).
Verified: unit 1431 passed / 20 skipped; integration test_mock_grpc 9 passed;
config validate/show on sample green; mock run --help lists --grpc-listen and
--only {http,kafka,grpc}; discover reports grpc_mock_stubs.
Co-Authored-By: Claude <noreply@anthropic.com>
DOCUMENTS-ONLY. DESIGN (WHAT/WHY): mocks.grpc schema (§2.1), mock run/start --grpc-listen/--only grpc + grpc events + result blocks (§3.6), config validate scope (§3.7), mock-grpc-stubs discover category (§3.9), grpc event vocabulary (§4.2), gRPC-mock v1 limitations (§10). ARCHITECTURE (HOW): module map incl. mock/grpc_server.py + clients/grpc_descriptors.py (§3), grpc stub request-lifecycle trace (§4), grpc events + fail-loudly (§6/§7), shared proto kernel + new MockGrpcServer subsection (§8), packaging/tests/deltas/limitations (§11/§12/§14/§15). Consumer skills (skills/agctl, skills/agctl-config + reference/mocks): --only grpc/--grpc-listen, mock-grpc-stubs discover, grpc fatal events in the post-mock grep, mocks.grpc authoring guide. Co-Authored-By: Claude <noreply@anthropic.com>
1. GrpcResponse docstring: correct the inverted message/messages↔call-type mapping (message = unary/client_stream/bidi, messages = server_stream). 2. _grpc_mock_params docstring: same correction (was backwards). 3. mock run --help smoke test: pin --grpc-listen alongside the other flags. 4. MockGrpcServer.start(): explicitly stop(grace=0) the built grpc.Server on bind failure before resetting _server, instead of relying on GC. 5. MockGrpcServer._check_response_shape: reject server_stream stubs with an empty response.messages list (fail-loud on surprising authoring); new unit test covers the empty-list case. All 1407 unit tests + 9 gRPC integration tests pass; docstrings now match the runtime mapping. Co-Authored-By: Claude <noreply@anthropic.com>
Final-gate fresh-eyes review of PR #51 found 5 issues the prior waves missed: 1. (Important) grpc_descriptors.deserialize: MessageToDict lost snake_case field names (user_id -> userId) and default scalars (n=0 dropped), silently breaking match.body and capture.from. Pass preserving_proto_field_name=True + always_print_fields_with_no_presence=True. 2. (Minor) parse_grpc_status: Unicode digit-strings like "²" satisfy isdigit() but int("²") raises ValueError, escaping as an uncaught traceback on the CLI path. Guard the coercion with isascii() so Unicode digits fall through to the documented ConfigError. 3. (Important) MockEngine.run: drain the gRPC server (shutdown -> grace) BEFORE the exit-code snapshot, mirroring the kafka join-before-snapshot pattern. Without it, an in-flight grpc.error during shutdown set counters the summary reflected but the exit code did not -> false-green exit 0. 4. (Important) grpc_server handlers: implement stub.response.metadata (documented but previously a silent no-op) across all 4 call types via send_initial_metadata (once-per-RPC, before first response) + set_trailing_metadata (before abort/return/stream-end). Unmatched path sends nothing. 5. (Minor) mock start: a grpc-only daemon's startup-error detail.listen pointed at the HTTP default (0.0.0.0:18080); now resolves to the gRPC address when run_grpc and not run_http. Tests: +14 (echo.proto gains UserService/UserRequest for snake_case; Fix 1 covers match.body + capture.from + zero-int; Fix 4 adds TestResponseMetadata across unary/non-OK/unmatched/server_stream/bidi via real grpcio). Full unit suite 1421 passed; gRPC integration 9 passed. Co-Authored-By: Claude <noreply@anthropic.com>
The new class-based `mock start` gRPC tests invoke the managed-daemon command, which is POSIX-only (`_require_posix_daemon` rejects win32). The pre-existing `mock start` tests carry a per-function `@pytest.mark.skipif(os.name == "nt", ...)`, but TestMockStartGrpc is the first class-based mock-start suite and was missing the gate — so on Windows CI all 6 tests hit the platform ConfigError (one masquerading as KeyError:'listen' because the gate's exit-2 detail lacks `listen`). Add the same class-level skipif the module-level suites already use. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a gRPC mock server as a third mock engine alongside HTTP and Kafka. The SUT's gRPC client can now point at a fake server (
mocks.grpc), parallel tomocks.http— closing the false-green surface for runbooks that depend on downstream gRPC services.grpc.Server, dynamic dispatch through a sharedDescriptorPool).grpc healthcheck+ reflection-based clients work out of the box).json_subset/jq_bool/resolve_captures/render_typed) and the gRPC client's proto machinery (extracted into a shared kernelagctl/clients/grpc_descriptors.py). Same match → capture → render → status model as HTTP/Kafka stubs.grpcextra →ConfigErrorpointing atpip install 'agctl[grpc]'.What changed
agctl/clients/grpc_descriptors.py(shared proto kernel),agctl/mock/grpc_server.py(dispatch core +MockGrpcServer).config/models.py(mocks.grpcmodels),mock/{engine,daemon,jq_precompile,capture_validate}.py,commands/{mock_commands,discover_commands}.py,assertions.py(publicparse_grpc_status),clients/grpc_client.py(delegates to the kernel — behavior-preserving),data/sample-config.yaml.mock run/startgain--grpc-listen+--only grpc;mock start/stop/statuscarry agrpcblock; new eventsgrpc.hit/grpc.unmatched/grpc.error(the latter two fatal); newdiscovercategorymock-grpc-stubs.Config is additive (version stays
"3", no migration).Design & process
docs/superpowers/specs/active/2026-07-17-grpc-mock-server-design.mddocs/superpowers/plans/active/2026-07-17-grpc-mock-server.mdTest status
tests/integration/test_mock_grpc.py9/9 green — drives a real in-processgrpc.Serverviaagctl grpc call/healthcheckfor all four call types + Health + reflection-only resolution + status simulation + summary counters. Self-contained (no Docker); runs whenever thegrpcextra is installed.v1 limitations (documented in DESIGN §10 / ARCHITECTURE §15)
Plaintext only (no TLS on the mock); bidi is request/response pairing (no stateful/server-push); client-stream aggregates at close; no mid-stream abort; descriptors required (reflection is served but cannot bootstrap the mock); grpcio
SO_REUSEPORTmeans two grpc servers can bind the same port silently.🤖 Generated with Claude Code