Beginner workbench engine implementation #3204
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michaeljguarino
force-pushed
the
more-workbench-work
branch
from
e5a80bc to
d7e1342
Compare
Greptile OverviewGreptile SummaryThis PR introduces an initial “workbench engine” execution loop and a GenStage polling pipeline that picks up pending/stale Key integration points include the new Confidence Score: 2/5
|
| Filename | Overview |
|---|---|
| assets/src/generated/graphql.ts | Regenerates GraphQL TS types to reflect workbench-related schema changes. |
| charts/console-rapid/charts/controller-0.0.168.tgz | Updates vendored controller Helm chart artifact. |
| charts/console-rapid/charts/kas-0.3.1.tgz | Updates vendored kas Helm chart artifact. |
| charts/console/charts/controller-0.0.168.tgz | Updates vendored controller Helm chart artifact. |
| charts/console/charts/kas-0.3.1.tgz | Updates vendored kas Helm chart artifact. |
| charts/controller/crds/deployments.plural.sh_sentinels.yaml | Adjusts Sentinel CRD manifest (non-code YAML change). |
| go/client/models_gen.go | Regenerates Go client models to match updated API/CRDs. |
| go/controller/api/v1alpha1/sentinel_types.go | Small update to Sentinel API types to match CRD changes. |
| go/controller/config/crd/bases/deployments.plural.sh_sentinels.yaml | Updates generated CRD base for Sentinels. |
| go/controller/docs/api.md | Regenerates controller API docs for Sentinel changes. |
| lib/console/ai/chat.ex | Small adjustments to AI chat module to support new workbench behavior/tools. |
| lib/console/ai/fixer/base.ex | Minor update to fixer base to accommodate new tool invocation behavior. |
| lib/console/ai/provider.ex | Provider changes to support passing tool lists for workbench flows. |
| lib/console/ai/tool.ex | Extends Tool helpers to accept module or struct tools (name/validate/implement) for workbench integration. |
| lib/console/ai/tools/workbench/base.ex | Adds base macro for workbench tools using Ecto embedded schemas. |
| lib/console/ai/tools/workbench/complete.ex | Adds workbench_complete tool embedded schema for job completion. |
| lib/console/ai/tools/workbench/fech_notes.ex | Adds fetch_workbench_notes tool but defines name/1 + implement/1, creating interface mismatch with MemoryEngine expectations. |
| lib/console/ai/tools/workbench/http.ex | Adds HTTP integration tool for workbench with JSON schema validation and HTTPoison request execution. |
| lib/console/ai/tools/workbench/notes.ex | Adds workbench_notes tool for recording status/todos; returns struct for engine reducer. |
| lib/console/ai/tools/workbench/plan.ex | Adds plan tool schema/prompt plumbing for initial workbench planning step. |
| lib/console/ai/tools/workbench/result.ex | Adds subagent result tool schema for returning structured subagent outputs. |
| lib/console/ai/tools/workbench/skill.ex | Adds workbench_skill tool for retrieving a single skill definition. |
| lib/console/ai/tools/workbench/skills.ex | Adds workbench_skills tool but defines name/1 + implement/2, incompatible with MemoryEngine tool-calling API. |
| lib/console/ai/tools/workbench/subagent.ex | Adds workbench_subagent tool schema for requesting a specific subagent action. |
| lib/console/ai/tools/workbench/subagents.ex | Adds workbench_subagents tool schema for listing available subagents. |
| lib/console/ai/tools/workbench/summarize_component.ex | Adds tool for summarizing a service component as part of workbench/integration workflows. |
| lib/console/ai/workbench/engine.ex | Introduces main workbench orchestration loop, but currently passes tools in a shape incompatible with MemoryEngine.call_tools. |
| lib/console/ai/workbench/environment.ex | Adds workbench environment builder (skills/tools/subagents) for engine execution. |
| lib/console/ai/workbench/message.ex | Adds message conversion helpers for rendering activities into chat memory messages. |
| lib/console/ai/workbench/skills.ex | Adds skill-loading utilities for workbenches, feeding the engine environment. |
| lib/console/ai/workbench/subagents/base.ex | Adds base behavior/helpers for workbench subagents. |
| lib/console/ai/workbench/subagents/infrastructure.ex | Adds infrastructure subagent implementation and prompt wiring. |
| lib/console/ai/workbench/subagents/integration.ex | Adds integration subagent implementation and prompt wiring. |
| lib/console/ai/workbench/subagents/plan.ex | Adds plan subagent implementation used as the initial workbench step. |
| lib/console/deployments/git/cache.ex | Minor change in git cache support for workbench/agent flows. |
| lib/console/deployments/workbenches.ex | Adds fail_job/2 helper to persist job failures with completed_at and error message. |
| lib/console/graphql/deployments/workbench.ex | GraphQL deployment workbench resolver updates to expose new job fields/behavior. |
| lib/console/pipelines/ai/workbench/pipeline.ex | Adds workbench pipeline consumer but currently crashes by piping {:ok, engine} into Engine.run/1. |
| lib/console/pipelines/ai/workbench/producer.ex | Adds polling producer for pending/stale workbench jobs using WorkbenchJob.pollable/1. |
| lib/console/pipelines/supervisor.ex | Wires new AI.Workbench producer/pipeline into the pipeline supervisor. |
| lib/console/schema/agent_run.ex | Exposes todo_changeset/2 publicly so it can be reused by workbench result/todo embeds. |
| lib/console/schema/service_component.ex | Minor schema tweak to support workbench component summarization workflows. |
| lib/console/schema/workbench.ex | Minor workbench schema adjustments for engine/job execution metadata. |
| lib/console/schema/workbench_job.ex | Adds error field and pollable/1 query to select pending or stale running jobs. |
| lib/console/schema/workbench_job_result.ex | Switches embedded todos to AgentRun.Todo and uses AgentRun.todo_changeset/2 for validation. |
| lib/console/schema/workbench_tool.ex | Adjusts workbench tool schema to support HTTP/tooling configuration for engine. |
| lib/console/schema/workbench_tool_association.ex | Minor update to tool association schema to support workbench tool selection. |
| priv/prompts/workbench/infrastructure.md.eex | Adds infrastructure subagent prompt template. |
| priv/prompts/workbench/integration.md.eex | Adds integration subagent prompt template. |
| priv/prompts/workbench/job.md.eex | Adds system prompt template used by workbench engine memory loop. |
| priv/prompts/workbench/message.txt.eex | Adds message formatting prompt/template for workbench memory history. |
| priv/prompts/workbench/plan.md | Adds baseline plan prompt content for workbench planning. |
| priv/repo/migrations/20260211012856_add_workbench_job_error.exs | Adds DB migration to persist workbench job error messages. |
| priv/tools/empty.json | Adds an empty JSON schema used for no-argument tools. |
| priv/tools/workbench/complete.json | Adds JSON schema for the workbench_complete tool. |
| priv/tools/workbench/notes.json | Adds JSON schema for the workbench_notes tool. |
| priv/tools/workbench/plan.json | Adds JSON schema for the workbench plan tool. |
| priv/tools/workbench/read_skill.json | Adds JSON schema for reading a skill definition tool. |
| priv/tools/workbench/subagent.json | Adds JSON schema for subagent invocation tool. |
| priv/tools/workbench/subagent_result.json | Adds JSON schema for subagent result tool. |
| schema/schema.graphql | Updates GraphQL schema to expose new/updated workbench/job fields and types. |
| test/console/deployments/git/agent_test.exs | Adjusts git agent tests to match updated schemas/tooling changes. |
| test/console/deployments/workbenches_test.exs | Adds/updates unit tests for workbench job polling/failure behavior. |
Sequence Diagram
sequenceDiagram
participant Producer as AI.Workbench.Producer
participant Pipeline as AI.Workbench.Pipeline
participant Engine as AI.Workbench.Engine
participant SAPlan as SA.Plan
participant Memory as Chat.MemoryEngine
participant Tools as Workbench Tools
participant Workbenches as Deployments.Workbenches
Producer->>Producer: poll(demand)
Producer->>Pipeline: emit WorkbenchJob
Pipeline->>Engine: new(job)
Engine->>Engine: preload job/user/workbench
Engine->>SAPlan: run(job, env)
SAPlan-->>Engine: {:ok, job}
loop Iteration
Engine->>Memory: reduce(messages, reducer)
Memory->>Tools: call_tools(tool_calls)
Tools-->>Memory: tool responses
Memory-->>Engine: %Complete{} or actions
alt Complete
Engine->>Workbenches: complete_job(conclusion, job)
else Actions
Engine->>Workbenches: create_job_activity
Engine->>Tools: run subagents / notes
Engine->>Workbenches: update_job_activity / update_job_status
end
end
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
michaeljguarino
force-pushed
the
more-workbench-work
branch
from
d7e1342 to
3322c51
Compare
Member
Author
Greptile OverviewGreptile SummaryThis PR implements the initial workbench engine for orchestrating AI subagents. The engine follows a plan-execute loop pattern where it first creates a plan, then iteratively delegates work to specialized subagents (infrastructure, integration, coding) that report results back. Key changes:
Critical issues found:
Confidence Score: 2/5
|
| Filename | Overview |
|---|---|
| lib/console/ai/workbench/engine.ex | Core workbench orchestrator with critical tool interface mismatch that will cause runtime failures |
| lib/console/pipelines/ai/workbench/pipeline.ex | Pipeline entry point with tuple-piping bug that will raise FunctionClauseError |
| lib/console/ai/workbench/environment.ex | Environment setup with typo in subagent categorization (:obsevability instead of :observability) |
| lib/console/ai/tools/workbench/complete.ex | Missing implement/1 callback required by Tool behavior, will fail when called |
| lib/console/ai/workbench/subagents/plan.ex | Plan subagent implementation, same tool interface issue as engine.ex |
Sequence Diagram
sequenceDiagram
participant Producer as Workbench Producer
participant Pipeline as Workbench Pipeline
participant Engine as Workbench Engine
participant PlanAgent as Plan Subagent
participant MemEngine as Memory Engine
participant InfraAgent as Infrastructure Subagent
participant IntegAgent as Integration Subagent
participant DB as Database
Producer->>DB: Poll for pending/stale jobs
DB-->>Producer: WorkbenchJob(s)
Producer->>Pipeline: handle_event(job)
Pipeline->>Engine: new(job)
Engine->>DB: Preload job, user, workbench
Engine->>Engine: Load skills & tools
Engine-->>Pipeline: {:ok, engine}
Pipeline->>Engine: run(engine)
Engine->>PlanAgent: run(job, environment)
PlanAgent->>MemEngine: Create plan with tools
MemEngine-->>PlanAgent: Plan with todos
PlanAgent->>DB: Update job with plan
PlanAgent-->>Engine: {:ok, job}
loop Execution Loop (max 200 iterations)
Engine->>MemEngine: Execute with subagent tools
MemEngine-->>Engine: List of actions (Subagent, Notes, Complete)
alt Subagent Action
par Concurrent Subagent Execution
Engine->>DB: Create activity
Engine->>InfraAgent: run(activity, job, env)
InfraAgent->>MemEngine: Execute with infra tools
MemEngine-->>InfraAgent: Result
InfraAgent-->>Engine: Activity result
Engine->>DB: Update activity
and
Engine->>IntegAgent: run(activity, job, env)
IntegAgent->>MemEngine: Execute with integration tools
MemEngine-->>IntegAgent: Result
IntegAgent-->>Engine: Activity result
end
end
alt Notes Action
Engine->>DB: Update job status
end
alt Complete Action
Engine->>DB: Mark job successful
Engine-->>Pipeline: Job complete
end
end
michaeljguarino
force-pushed
the
more-workbench-work
branch
4 times, most recently
from
dc41eb9 to
4295307
Compare
Member
Author
Greptile OverviewGreptile SummaryThis PR implements the initial workbench engine for orchestrating AI-powered task execution through multiple specialized subagents. Key Changes:
Critical Issues Found:
Architecture: Confidence Score: 1/5
|
| Filename | Overview |
|---|---|
| lib/console/ai/workbench/engine.ex | Core orchestration engine for workbench execution. Critical tool interface mismatch between struct instances and module-based expectations will cause runtime failures. |
| lib/console/ai/workbench/environment.ex | Environment management for workbench execution context. Contains spelling error in atom :obsevability that could cause bugs. |
| lib/console/ai/tools/workbench/skills.ex | Skills listing tool. Implements arity-1 callbacks (name/1, implement/3) incompatible with MemoryEngine expectations. |
| lib/console/ai/tools/workbench/skill.ex | Single skill retrieval tool. Uses arity-1/3 interface (name/1, implement/3) that won't work with MemoryEngine. |
| lib/console/ai/tools/workbench/subagent.ex | Subagent invocation tool. Uses arity-1/2 interface incompatible with module-based expectations from MemoryEngine. |
| lib/console/ai/workbench/subagents/plan.ex | Plan subagent for initial job planning. Mixes struct instances and modules in tools list similar to main engine. |
| lib/console/ai/workbench/subagents/infrastructure.ex | Infrastructure subagent for service/stack/k8s operations. Has same tool interface mixing issue. |
| lib/console/ai/workbench/subagents/coding.ex | Coding subagent with agent run polling. Mixes tool interfaces; polling logic appears sound with timeout and jitter. |
Sequence Diagram
sequenceDiagram
participant Pipeline
participant Engine
participant Plan
participant MemoryEngine
participant Provider
participant Subagents
Pipeline->>Engine: new(job)
Engine->>Engine: Load environment & skills
Engine-->>Pipeline: {:ok, engine}
Pipeline->>Engine: run(engine)
Engine->>Plan: run(job, environment)
Plan->>MemoryEngine: new(tools) + reduce(messages)
MemoryEngine->>Provider: completion(messages, tools)
Provider-->>MemoryEngine: plan tool call
MemoryEngine-->>Plan: %Plan{todos: [...]}
Plan-->>Engine: {:ok, job}
Engine->>Engine: loop(engine)
Engine->>MemoryEngine: new(tools) + reduce(messages)
MemoryEngine->>Provider: completion(messages, tools)
Provider-->>MemoryEngine: subagent tool calls
MemoryEngine-->>Engine: [%Subagent{...}, ...]
Engine->>Subagents: spawn_activities (parallel)
Subagents->>MemoryEngine: run subagent with tools
MemoryEngine->>Provider: completion(subagent messages)
Provider-->>MemoryEngine: result tool call
MemoryEngine-->>Subagents: %Result{output: "..."}
Subagents-->>Engine: activity results
Engine->>Engine: loop(next iteration)
Engine->>MemoryEngine: new(tools) + reduce(messages)
MemoryEngine->>Provider: completion(messages, tools)
Provider-->>MemoryEngine: complete tool call
MemoryEngine-->>Engine: %Complete{conclusion: "..."}
Engine-->>Pipeline: {:ok, completed_job}
jsladerman
approved these changes
Feb 12, 2026
michaeljguarino
force-pushed
the
more-workbench-work
branch
7 times, most recently
from
b16d706 to
7540149
Compare
This should be the initial version for implementing the workbench engine, it supports: 1. The core loop for calling all subagents associated with a workbench 2. Basic plan + infrastructure + integration agent implementations 3. pipeline setup to poll and execute pending or stale workbenches (which supports durable retry in event of failure) 4. a bunch of tools needed to make everything happy
michaeljguarino
force-pushed
the
more-workbench-work
branch
from
7540149 to
6d1ea3a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This should be the initial version for implementing the workbench engine, it supports:
Test Plan
unit tests
Checklist
Plural Flow: console