[feat] LLM Based Template Logic Executor

[devanshi00]

Overview

This PR introduces an LLM-based generic logic executor for Accord Project templates. It enables execution of templates even when no logic.ts is provided, using a reasoning-based fallback powered by an LLM.

The goal is to explore a flexible, extensible alternative to traditional TypeScript-based logic while maintaining full backward compatibility.

• LLM Executor

  • Acts as a fallback or replacement for logic.ts
  • Processes:
    • contract text
    • Concerto models
    • current state
    • incoming request
  • Produces:
    • updated state
    • response object
    • emitted events

• Execution Modes

  • disabled – default behavior (TypeScript only)
  • fallback – use LLM only if logic.ts is absent/fails
  • force – always use LLM executor

• Integration

  • Integrated into TemplateArchiveProcessor
  • Seamlessly fits into existing execution pipeline

• Normalization Layer

  • Ensures outputs conform to Accord runtime expectations:
    • timestamps
    • identifiers
    • ordering

• Backward Compatibility

  • Existing templates with logic.ts remain unaffected

• Logging

  • Clear indication of execution path:
    • TypeScript vs LLM

PR Review

@DianaLease @dselman

[mttrbrts]

Thanks for this contribution! Before we can review, please resolve the merge conflicts with the base branch and fix the DCO sign-off.

git fetch origin main
git rebase origin/main --signoff
# Resolve conflicts
git push --force-with-lease

Once the conflicts are resolved and commits are signed off, we'll proceed with the review.

---

This comment was generated by AI on behalf of @mttrbrts.

[devanshi00]

@mttrbrts @DianaLease @dselman added the llm executor with json schema for anthropic and openai provider. Also added a stateful and stateless templates results to check the feasibility and correctness of the llm executor.

[mttrbrts]

Note: This review was generated with the assistance of Claude (AI). The comments reflect the maintainer's views but were drafted and posted by an AI code review tool.

---

Hi @devanshi00 👋 — really appreciate the effort here! An LLM-based fallback executor is a genuinely exciting direction for Accord Project, and the core design (three modes, schema-enforced structured outputs, provider abstraction) is solid. Thanks for tackling this!

Before we can merge, there are a few things we'd like to work through together — some are blockers, and some are things we'd love to discuss to find the best path to land this incrementally. I've left inline comments on the specifics, but the high-level themes are:

Blockers for any merge:

• The PR's head branch appears to be main, which means it likely has unresolved merge conflicts with upstream. Could you rebase onto the current main and resolve those before we do a deeper review?
• The project README has been fully replaced with LLM Executor-specific content. The existing README is the public face of this project — the LLM executor is a great addition, not a replacement. Let's keep the original content and add a new section (or a docs/llm-executor.md) for this feature.

Things to discuss for incremental merging:

• The PR is currently 233 files / ~15k lines, which makes it very hard to review safely and increases the risk of merge conflicts going forward. We'd love to find a way to land the core feature (src/llm/ + the TemplateArchiveProcessor integration) first, and follow up with examples in a separate PR.
• The four LLM SDK packages (openai, @anthropic-ai/sdk, @google/genai, @mistralai/mistralai) are currently in dependencies, which means every downstream consumer of template-engine installs them even if they never use the LLM executor. We should chat about moving these to optionalDependencies or peerDependencies so the package stays lightweight for users who don't need this feature.
• The package version jumped from 2.7.1 → 3.0.1 and also upgrades several @accordproject/* packages to new major versions. Were those dependency upgrades intentional as part of this PR? If so, it might be cleaner to land them separately so the diff is easier to follow.

None of this should block the conversation — please don't be discouraged! The core idea is great and we want to get it merged. Happy to jump on a call if it would help work through the path forward. 🙏

[mttrbrts]

Bumping to 3.0.1 is quite a jump from 2.7.1! A major semver bump tells downstream users there's a breaking change. Since the TemplateArchiveProcessor constructor change is backward-compatible (the new llmConfig param is optional), we'd normally keep this as a minor bump (2.8.0). Could you clarify whether the major bump is intentional? If it's tied to the @accordproject/* dep upgrades, it might be cleaner to land those in a separate PR so each change is easy to reason about independently.

[mttrbrts]

These four LLM SDK packages are large and have their own transitive dependency trees. Adding them to dependencies means every consumer of template-engine — including those who only use the existing TemplateMark drafting pipeline and never touch the LLM executor — will install all of them unconditionally.

Two options worth discussing:

1. optionalDependencies — npm installs them by default but won't fail if they're absent, and bundlers can tree-shake them. Requires dynamic import() in Reasoners.ts with a helpful error if the package isn't found.
2. peerDependencies — the caller installs only what they need. Requires a bit more documentation but keeps the package lean.

What do you think? Option 1 is probably the least disruptive for the PR as it stands, and we could tighten it further in a follow-up.

[mttrbrts]

These console.log statements will fire on every trigger() and init() call, even for users who haven't configured an LLM at all (the disabled default). That's going to be noisy for existing consumers who upgrade.

Since LLMExecutorConfig already has a verbose flag, these could be gated on this.llmConfig?.verbose — or moved entirely into LLMExecutor where the verbose flag is already used consistently. The TemplateArchiveProcessor itself probably shouldn't log anything unconditionally.

[mttrbrts]

Module-level mutable globals are a bit fragile here — if someone constructs two LLMExecutor instances for different templates (each with their own schema.json), the second setSchemaPath() call silently invalidates the cache for the first. It also makes the API slightly surprising: the caller has to remember to call setSchemaPath() before constructing the executor, with no enforcement at the type level.

One alternative worth considering: pass the schema path (or the resolved schema object) directly in LLMExecutorConfig:

export interface LLMExecutorConfig {
  mode: LLMMode;
  provider: LLMProviderConfig;
  schemaPath?: string;   // or: schema?: { definitions: Record<string, any> }
  verbose?: boolean;
}

That way each executor is fully self-contained and the contract is clear at construction time. Happy to discuss — there may be a reason for the current approach (e.g. Anthropic grammar cache requiring the same schema object reference) that I'm missing!

[mttrbrts]

dangerouslyAllowBrowser: true suppresses a security warning from the OpenAI SDK that exists for good reason — sending API keys from a browser exposes them to users via DevTools. If browser support isn't a goal for the LLM executor right now, this flag shouldn't be needed at all (the executor would only run server-side). If browser support is intentional, it needs a proxy layer so the key never reaches the client. Worth flagging so we make a deliberate decision here rather than inheriting it silently!

[mttrbrts]

Exporting setSchemaPath from the package root makes it part of the public API, which means we'd have to support it as a breaking change if we ever want to remove it. If we move schemaPath into LLMExecutorConfig (as suggested in the comment on LLMExecutor.ts), this export goes away entirely and the public API surface stays cleaner. Worth discussing as part of the same design question!