-
Notifications
You must be signed in to change notification settings - Fork 0
feat: adb_vision - clean ADB + VLM game automation stack #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
24f572e
39e2936
e2d399e
fc15be0
4a70bd1
f8b5927
c384249
94a1b25
4be43dc
8de8f85
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,2 +1,15 @@ | ||||||||||||
| { | ||||||||||||
| } | ||||||||||||
| "mcpServers": { | ||||||||||||
| "adb-vision": { | ||||||||||||
| "command": "uv", | ||||||||||||
| "args": [ | ||||||||||||
| "run", | ||||||||||||
| "--directory", | ||||||||||||
| "adb_vision", | ||||||||||||
| "server.py", | ||||||||||||
| "--serial", | ||||||||||||
| "127.0.0.1:21513" | ||||||||||||
| ] | ||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔥 The Roast: Changing the port from 21503 to 21513? I see we're playing 'guess the MEmu instance' today. The AGENTS.md specifically documents 🩹 The Fix: Add a comment explaining why 21513 is the correct port for this setup, or revert to 21503 if this was unintentional. Future-you will thank present-you when debugging connection failures. 📏 Severity: suggestion |
||||||||||||
| } | ||||||||||||
| } | ||||||||||||
| } | ||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔥 The Roast: Missing newline at end of file. It's 2026 and we're still having this conversation? Your editor is gaslighting you into believing this file is complete when git diff is side-eyeing you hard. 🩹 The Fix:
Suggested change
📏 Severity: nitpick There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔥 The Roast: Missing newline at end of file. It's 2026 and we're still doing this? POSIX tools everywhere are shedding tears. This isn't just pedantry — 🩹 The Fix: Add a newline at the end of the file. In VS Code: cursor to end, press Enter once, save. 📏 Severity: nitpick |
||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,170 @@ | ||
| --- | ||
| name: 'alas-separation' | ||
| description: 'Separate agent orchestrator from ALAS-with-Dashboard while preserving state engine' | ||
| agent: 'agent' | ||
| model: 'Claude Sonnet 4' | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔥 The Roast: 🩹 The Fix: Use 📏 Severity: warning |
||
| tools: ['todos', 'runSubagent', 'codebase', 'askQuestions', 'readFile', 'search', 'editFiles'] | ||
| --- | ||
|
|
||
| # ALAS Separation: Orchestrator Independence | ||
|
|
||
| **Goal:** Separate the agent orchestrator from ALAS-with-Dashboard into a clean, standalone architecture while preserving the critical state engine work. | ||
|
|
||
| **Pre-Flight Check:** | ||
| - Verify git remotes: `origin` → Coldaine/alas, `upstream` → Zuosizhu/Alas-with-Dashboard | ||
| - Confirm MEmu setup: Admin-at-startup already solved, ADB serial 127.0.0.1:21513 | ||
|
|
||
| --- | ||
|
|
||
| ## Phase 1: Discovery #codebase #runSubagent | ||
|
|
||
| Spawn parallel subagents to find all existing documentation and plans: | ||
|
|
||
| **Subagent A - Repository Documentation:** | ||
| ``` | ||
| Read and summarize all planning documents in: | ||
| - docs/plans/ | ||
| - docs/ROADMAP.md | ||
| - docs/NORTH_STAR.md | ||
| - docs/ARCHITECTURE.md | ||
| - docs/state_machine/ | ||
|
|
||
| Report: What separation plans exist? What was already decided? What's missing? | ||
| ``` | ||
|
|
||
| **Subagent B - Fork Investigation:** | ||
| ``` | ||
| Check for related repos or branches with standalone agent work: | ||
| - Search for azurlane-agent references | ||
| - Check branch history for separation experiments | ||
| - Look for MCP server work outside ALAS imports | ||
|
|
||
| Report: What work exists elsewhere that should be integrated? | ||
| ``` | ||
|
|
||
| **Subagent C - State Machine Audit:** | ||
| ``` | ||
| Document the current state engine: | ||
| - module/ui/page.py (Page.all_pages, 43 pages, 98 transitions) | ||
| - module/state_machine.py | ||
| - How state is currently exposed via MCP | ||
|
|
||
| Report: What must be preserved? How tightly coupled to ALAS internals? | ||
| ``` | ||
|
|
||
| **Consolidate findings** into `.kilo-prompt/discovered_state.md` | ||
|
|
||
| --- | ||
|
|
||
| ## Phase 2: User Alignment #askQuestions | ||
|
|
||
| Interview the user on these key points: | ||
|
|
||
| 1. **Git Remotes:** Confirm origin/upstream configuration is correct | ||
| 2. **State Engine Priority:** Which state machine features are absolutely critical? | ||
| 3. **Entry Point:** Start with behavioral catalog OR MCP boundary refactor? | ||
| 4. **MaaMCP:** Evaluate adopting MaaFramework/MaaMCP as foundation? | ||
| 5. **Branch Strategy:** Which branch to checkout? (pr/all-changes-no-secret? new feature?) | ||
| 6. **Timeline:** When to flip from manual pilot (coding agent) to autonomous? | ||
|
|
||
| Record answers in `.kilo-prompt/user_alignment_answers.md` | ||
|
|
||
| --- | ||
|
|
||
| ## Phase 3: Implementation Analysis #runSubagent | ||
|
|
||
| Once Phase 2 is complete, spawn parallel subagents: | ||
|
|
||
| **Subagent D - Git/Branch Analysis:** | ||
| ``` | ||
| Analyze current git state: | ||
| - List all local and remote branches | ||
| - Identify cleanest separation starting point | ||
| - Document exact checkout command needed | ||
|
|
||
| DO NOT execute git commands - document only. | ||
| ``` | ||
|
|
||
| **Subagent E - MCP Tool Dependency Map:** | ||
| ``` | ||
| Analyze agent_orchestrator/alas_mcp_server.py: | ||
| - Which tools import ALAS internals? (sys.path hacks) | ||
| - Which tools are already standalone? (ADB-only) | ||
| - Minimal set needed for basic operation? | ||
|
|
||
| Report: Refactoring order - which tools first? | ||
| ``` | ||
|
|
||
| **Subagent F - State Engine Extraction Plan:** | ||
| ``` | ||
| Plan extracting the 43-page state machine: | ||
| - Can Page.all_pages be exposed via MCP without importing? | ||
| - Which transition logic must be replicated standalone? | ||
| - API contract between orchestrator and state queries? | ||
|
|
||
| Report: Extraction strategy with minimal ALAS dependency. | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Phase 4: Write Implementation Plan #todos | ||
|
|
||
| Create comprehensive plan: | ||
| - Location: `.kilo-prompt/implementation_plan.md` | ||
| - Include: Branch to checkout, step-by-step order, MCP refactoring sequence, state engine approach, testing strategy | ||
|
|
||
| Add all phases to `#todos` tracker. | ||
|
|
||
| --- | ||
|
|
||
| ## Key Principles | ||
|
|
||
| ### Dual-Use Constraint (Non-Negotiable) | ||
| `gui.py --run PatrickCustom` must work forever. Bot development is additive only. | ||
|
|
||
| ### Extract, Don't Wrap | ||
| ALAS is reference/spec only. Build standalone tools with no `from module.*` imports. | ||
|
|
||
| ### Two-Tier Operation | ||
| - **Tier 1 (hot path):** Deterministic tools with strict contract `{success, data, error, observed_state, expected_state}` | ||
| - **Tier 2 (recovery only):** VLM vision when deterministic tools fail | ||
|
|
||
| ### Single-Harness Principle | ||
| Same MCP tool surface for dev (Claude Code manual piloting) and prod (autonomous Gemini). | ||
|
|
||
| ### MEmu Environment | ||
| - Multiple Instance Manager runs with admin at system startup | ||
| - ADB serial: 127.0.0.1:21513 (instance index 1) | ||
| - Screenshot: DroidCast (bypasses OpenGL issues) | ||
|
|
||
| ### State Engine Preservation | ||
| 43-page state machine with 98 transitions is valuable extracted knowledge: | ||
| - Expose via MCP tools (`alas.get_current_state`, `alas.goto`) | ||
| - Do not import ALAS modules in orchestrator | ||
| - Document transitions as behavioral catalog | ||
|
|
||
| --- | ||
|
|
||
| ## Success Criteria | ||
|
|
||
| - [ ] Phase 1: All documentation discovered and consolidated | ||
| - [ ] Phase 2: User interview completed with answers recorded | ||
| - [ ] Phase 3: Implementation entry point identified | ||
| - [ ] Phase 4: Comprehensive plan written and added to #todos | ||
| - [ ] Git remotes verified correct | ||
| - [ ] State engine extraction strategy documented | ||
| - [ ] MCP tool refactoring sequence determined | ||
|
|
||
| --- | ||
|
|
||
| ## Execution Flow | ||
|
|
||
| ``` | ||
| 1. READ docs (parallel subagents) → discovered_state.md | ||
| 2. INTERVIEW user → user_alignment_answers.md | ||
| 3. ANALYZE entry point (parallel subagents) → subagent_reports/ | ||
| 4. WRITE plan → implementation_plan.md | ||
| 5. TRACK in #todos | ||
| ``` | ||
|
|
||
| **Begin with Phase 1 when this prompt is executed.** | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,10 +5,10 @@ | |
| "args": [ | ||
| "run", | ||
| "--directory", | ||
| "agent_orchestrator", | ||
| "alas_mcp_server.py", | ||
| "--config", | ||
| "alas" | ||
| "adb_vision", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔥 The Roast: The MCP server name at line 3 is still 🩹 The Fix: Rename the server key from 📏 Severity: suggestion |
||
| "server.py", | ||
| "--serial", | ||
| "127.0.0.1:21513" | ||
| ], | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔥 The Roast: Same port switcharoo here — 21503 became 21513 like it's no big deal. The AGENTS.md clearly states 🩹 The Fix: Align with documented environment or update AGENTS.md to explain the port change rationale. 📏 Severity: critical There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔥 The Roast: Same port switcharoo here — 21503 became 21513. But wait, there's more! The VS Code MCP server entry is still named 🩹 The Fix: Either rename the server to 📏 Severity: suggestion |
||
| "env": { | ||
| "PYTHONIOENCODING": "utf-8" | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔥 The Roast: Changing the port from 21503 to 21513? I see we're playing 'guess which MEmu instance' today. According to AGENTS.md, the documented environment uses
127.0.0.1:21503for MEmuPlayer. This change breaks the 'Known local environment' documented in your own setup guide.🩹 The Fix: Either document why port 21513 is needed, make this configurable via environment variable, or keep the documented default. Don't leave developers playing port roulette.
📏 Severity: critical