[Feedback] Please clarify the closed-source status of automata-agent (or consider open-sourcing it)

[beautifulcn]

Hi there, thanks for building ui-automata. The underlying declarative RPA engine and the Windows UIA integration are incredibly solid.
However, I am writing to provide some feedback regarding the developer experience, specifically concerning the automata-agent MCP server.
The Context & The Pain Point
Recently, I spent a massive amount of time and effort trying to optimize ui-automata for local, smaller-context LLMs (like Llama-3 and Qwen). The current architecture, which pre-loads huge files like AGENT.md and CLAUDE.md, causes severe Context Bloat and crashes local LLMs.
My goal was to refactor the project to support dynamic lazy-loading (RAG style) for workflows and schema interfaces. I designed the optimization plan and started digging into the Rust source code (registry.rs, runner.rs, ui-workflow.rs, etc.) to find the MCP tool registration endpoints (like desktop, --stdio, etc.) to implement my new tools.
After exhaustive searching and debugging, I finally realized through external documentation that automata-agent is a closed-source component and is NOT included in this repository.
Why this is problematic:
The repository structure and installation scripts (install-windows.ps1) heavily intertwine the open-source library with the closed-source automata-agent. For developers looking to contribute, extend the MCP tools, or adapt the agent for non-Claude models, it creates a massive "wild goose chase". We invest significant energy into understanding and improving the codebase, only to hit a hard wall when realizing the core MCP bridge is black-boxed.
Actionable Suggestions:
Immediate fix - Explicit Documentation: Please state clearly and prominently in the main README.md that automata-agent is a closed-source, pre-compiled binary and not part of the open-source repository. This will save future developers hours of futile source-code hunting.
Ideal fix - Open-Source the Agent: Since the agent is the crucial bridge connecting the LLM community to your excellent RPA engine, keeping it closed-source severely limits community contributions. If it were open-sourced, developers like me could have already submitted PRs to optimize it for local LLMs, add new MCP features, and expand its ecosystem. Please consider open-sourcing it if it doesn't conflict with your core business model.
Thank you for your hard work on the core engine. I hope this feedback helps improve the transparency and community experience of the project.

[m13v]

the closed-source point is fair but the bottleneck is upstream of where you were patching. AGENT.md preloading does crash local llms, sure, but the actual token bomb in UIA-based RPA is the element tree itself, not the workflow definitions. a dense SAP GUI screen or a complex EHR form can blow past 20k tokens before any agent prose enters the picture. lazy-loading workflows fixes maybe 10% of the context pressure. the real fix is filtered AX snapshots scoped to the focused subtree, returning element_index handles so the model never re-reads the raw tree. that's the layer worth open-sourcing if local-llm viability is the actual goal.

[AndreyPolovodov]

Guys, let me share my experience with you. I forked this project and started adding some missing features (for my use case). I also asked about the agent status here: #1

I finally realized that MCP is not necessary at all. I banned some MCP tools and forced the coding agent to use only console tools (which are available as source code and I have patched for my purpose).

And @m13v is absolutely right - it's critical to use a filtered UIA tree. I force the coding agent to always use selectors and avoid getting the full tree dump.

Hope this helps.

BTW, it's a good project. I'm happy I found it, but it required some tuning to be usable in real life (at least for my case).

[beautifulcn]

@AndreyPolovodov I have had the same experience as you, but the key issue is that I spent a significant amount of time and effort optimizing ui-automata to better support local, small-context LLMs (such as Llama-3 and Qwen). The current architecture preloads large files like AGENT.md and CLAUDE.md, leading to severe context bloat and ultimately causing local LLMs to crash.
My goal was to refactor the project to support dynamic lazy loading (RAG-style) for workflows and schema interfaces. I designed an optimization plan and began diving into the Rust source code (registry.rs, runner.rs, ui-workflow.rs, etc.) to find the registration endpoints for MCP tools (such as desktop, --stdio, etc.) in order to implement my new tools.
After exhaustive searching and debugging, I finally discovered through external documentation that automata-agent is a closed-source component and is not included in this repository. Consequently, all these modifications and optimizations were a waste of effort because the registration endpoints could not be found. If this project had clearly and prominently stated in the main README.md file that "automata-agent is a closed-source, pre-compiled binary and is not included in the open-source codebase," this waste of time and energy could have been avoided.
@m13v You have no idea what the root cause of the problem is!

[AndreyPolovodov]

@beautifulcn finally I dropped mcp using at all and switched to console utils only and my own skill. still use ui-workflow script engine and element-tree tools. and that helps. possibly you could do the same. Or your own mcp also is an option, I think...

[beautifulcn]

@AndreyPolovodov Thank you for the suggestion; you are quite right, and I am planning to adopt that very strategy. We should move away from relying solely on automation tools based on coordinates or full-path scraping. Instead, we should draw inspiration from the PAD UI control selector concept: subtract during the generation phase and intelligently add during the matching phase. By eliminating redundant hierarchies, we ensure "simplicity," and by leveraging attribute weighting and scope constraints, we achieve "efficiency."