Mixture of Models (MoM)

One OpenAI-compatible endpoint, many models. MoM routes every chat completion through an internal proposer → refuter → synthesizer pipeline so several models propose, critique, and reconcile each answer.

Mixture of Models (MoM) is a Python/FastAPI service that exposes a standard /v1/chat/completions API and, behind it, runs every request through an internal multi-model pipeline. Any OpenAI-compatible client works unchanged — including native function/tool calling and streaming. Upstream providers are pluggable: OpenRouter or a deterministic zero-cost mock.

The user-facing contract is intentionally boring:

OPENAI_BASE_URL=http://127.0.0.1:8000/v1
OPENAI_API_KEY=mom_...
MODEL=mom-chat

Clients call /v1/chat/completions the same way they would call a normal chat model. The MoM machinery is internal to mom-chat. The endpoint supports normal text responses and OpenAI-style function tool calls.

The internal engine is not naive fusion. It runs one fixed architecture:

OpenAI-compatible request
  -> MoM engine
  -> internal proposer/refuter/synthesizer passes
  -> OpenAI-compatible response

Bearer API-key enforcement is available through MOM_API_KEYS so OpenAI-compatible clients can be wired up early.

Features

• Drop-in OpenAI compatibility — works with any client that speaks /v1/chat/completions; just change the base URL and model.
• Multi-model architecture — proposer, refuter, and synthesizer roles, each mappable to a different model.
• Native tool calling — returns OpenAI-style tool_calls with finish_reason: "tool_calls", plus streaming (SSE).
• Pluggable upstreams — OpenRouter or a zero-cost deterministic mock for offline development and CI.
• Optional bearer auth — gate the public surface with MOM_API_KEYS.
• No keys required to start — the mock provider runs the full test suite and local server with zero configuration.

Project Layout

• mom/api/: FastAPI app, OpenAI-compatible schemas, and response builders.
• mom/core/: MoM engine, internal architecture, claim, refutation, synthesis, tool-call parsing, and telemetry structures.
• mom/providers/: Pluggable provider interface, deterministic mock provider, and OpenAI-compatible provider.
• tests/: API compatibility and claim lifecycle tests.

Run Locally

Install in editable mode with development dependencies:

python -m pip install -e ".[dev]"

Run tests:

pytest

Start the dev server (uses the zero-cost mock provider with no keys set):

uvicorn mom.api.server:app --reload

Prompt it from another terminal:

scripts/prompt.sh "Explain what this API does in one paragraph."

Check the API directly:

curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/v1/models

To require bearer API keys in local/server mode, set MOM_API_KEYS to a comma-separated list:

MOM_API_KEYS=mom_dev_key uvicorn mom.api.server:app --reload

Then clients must send:

Authorization: Bearer mom_dev_key

Upstream Selection

The upstream provider is selected with MOM_UPSTREAM.

MOM_UPSTREAM=mock        # zero-cost deterministic local provider
MOM_UPSTREAM=openrouter  # direct OpenRouter API provider

If MOM_UPSTREAM is not set but OPENROUTER_API_KEY is present, the API uses OpenRouter. Otherwise it uses the deterministic mock provider.

OpenRouter Upstream

Use this for direct API testing and production-like integration.

Copy .env.example to .env, set OPENROUTER_API_KEY, then start the dev server:

cp .env.example .env
# edit .env to add your OPENROUTER_API_KEY
scripts/dev-openrouter.sh

scripts/dev-openrouter.sh loads .env and applies sensible model defaults. To run uvicorn directly with an explicit configuration instead:

OPENROUTER_API_KEY=or_...
MOM_PROPOSER_MODELS=~openai/gpt-latest,~anthropic/claude-sonnet-latest
MOM_REFUTER_MODEL=~minimax/minimax-m3
MOM_SYNTHESIZER_MODEL=~openai/gpt-latest
uvicorn mom.api.server:app --reload

Optional OpenRouter settings:

OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
OPENROUTER_HTTP_REFERER=https://github.com/luckeyfaraday/MoM-engine
OPENROUTER_APP_TITLE="Mixture of Models"
MOM_PROVIDER_TIMEOUT_SECONDS=60

API Goal

The default response from /v1/chat/completions looks like a minimal OpenAI chat completion so existing clients can integrate with the service. When a client supplies tools and the internal synthesizer decides a tool is needed, the API returns assistant tool_calls with finish_reason: "tool_calls". The client executes the tool and sends the result back as a normal tool message on the next request.

FAQ

What is a Mixture of Models? Instead of returning one model's answer, MoM asks several models to propose candidate answers, has a model critique (refute) them, and has a synthesizer model produce a single reconciled reply — all behind a normal OpenAI chat endpoint.

How is this different from a router or load balancer? A router picks one model per request. MoM combines multiple models per request through a fixed proposer/refuter/synthesizer architecture.

How does this relate to other systems for combining AI models? Combining multiple AI models to produce better answers is an active area — for example Sakana AI's Fugu, a Japanese model that takes a similar approach of blending several models together. MoM does this as inference-time fusion: it combines the outputs of several models per request (think "OpenRouter fusion" when they are served via OpenRouter), keeping each model separate and reconciling their answers live rather than merging weights into one offline model.

Why "loops"? The proposer → refuter → synthesizer passes form a short refinement loop over a single request, so each answer is critiqued before it's returned rather than emitted in one shot.

Do I need API keys to try it? No. With no environment variables set, MoM uses a deterministic, zero-cost mock provider, so the server and tests run offline.

Which clients work with it? Anything OpenAI-compatible — the OpenAI SDKs, LangChain, curl, etc. Point them at /v1 and use model mom-chat.

Is it production-ready? It's early-stage (alpha). The OpenRouter upstream is the path for production-like integration.

Contributing

Contributions are welcome — see CONTRIBUTING.md and the Code of Conduct. For security reports, see SECURITY.md.

License

Released under the MIT License.

---

_{Keywords: mixture of models · MoM · combining AI models · OpenRouter fusion · multi-model LLM API · OpenAI-compatible API · inference-time model fusion · model ensembling · LLM synthesis · proposer / refuter / synthesizer · agentic loops · Sakana Fugu · FastAPI LLM gateway · tool calling.}