Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/tinyagents-drift-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ site is crate-backed (the migration's scaffold → flip → delete pattern).
| Site | Crate-native builder | Status |
| --- | --- | --- |
| Managed OpenHuman backend (common path: chat turns, memory/learning/meeting summaries) | `factory::make_openhuman_backend_model` → `OpenHumanBackendModel` (dynamic JWT + `thread_id` + billing envelope bridged onto crate `OpenAiModel`) | **CUT OVER** — `create_chat_model_with_model_id` routes it (commit `7e98c1b39`); test-provider override still wins. |
| Generic OpenAI-compat cloud slug | `crate_openai::make_crate_openai_chat_model` | **BUILDER READY** — not yet the factory default; blocked on responses-fallback + codex-oauth parity (see below). |
| Wire-equivalent BYOK cloud slug (Anthropic / None / plain-Bearer, no codex-oauth, no `/v1/responses`) | `factory::try_create_cloud_slug_chat_model` → `crate_openai::make_crate_openai_chat_model` | **CUT OVER (conservative subset)** — `create_chat_model` routes these crate-native after the managed + local short-circuits. Resolution is shared via `resolve_cloud_slug` (the legacy `make_cloud_provider_by_slug` was refactored onto it, so eligible slugs resolve **identically**; only the wire client differs). The same `enforce_local_only_inference` + `verify_session_active` gate runs first. Covers the common non-OpenAI BYOK providers (DeepSeek, Groq, Mistral, xAI, …) via the crate wire the managed backend already proves. |
| `openai` / codex + custom-proxy cloud slugs | stay `Provider` path | **PHASE 3** — `openai` keeps the `/v1/responses` fallback + codex-oauth (query-param auth, account/originator headers, user-agent, responses-API-primary); custom slugs may proxy `/v1/responses`. `try_create_cloud_slug_chat_model` returns `None` for these. Land with the crate `/responses` port. |
| Local runtimes (Ollama/LM Studio/MLX/OMLX/local-openai) | `factory::try_create_local_runtime_chat_model` → `crate_openai::make_crate_local_runtime_chat_model` (native tools + vision forced off; `num_ctx` baked as `{"options":{"num_ctx":N}}`) | **CUT OVER** — `create_chat_model_with_model_id` routes local runtimes crate-native (after the managed short-circuit). The flip **re-runs the same gate** the `Provider` path applies (`enforce_local_only_inference` + `verify_session_active`), so it cannot bypass privacy mode or the session requirement. Temperature rides the per-call `ModelRequest` (parity with managed). **Loopback error handling defers to upstream:** an upstream merge (`b709a993…`/`04ffc029…`) replaced the earlier `..._offline_trips_halt_guard` test with `cron_agent_job_short_loopback_send_error_stays_retryable` — i.e. an offline local provider now **stays retryable** (it may be transiently starting up). So the transient cron `{e:#}` cause-chain surfacing + the `is_non_retryable` loopback fast-fail were reverted, and the host stays pinned at `7c6e81a` (before [tinyagents#50](https://github.com/tinyhumansai/tinyagents/pull/50) `error_source_chain`) so the crate-native local error does not surface the `connection refused` errno the classifier would trip on. #50 remains a good crate improvement but is deliberately **not consumed** here to keep loopback retryable. |
| Bespoke (managed backend, `claude_code`, `claude_agent_sdk`, `openai_codex`) | stay host `ChatModel` impls | **HOST-OWNED** — subprocess / `/v1/responses` / query-param auth have no crate equivalent; never route through `crate_openai`. |

Expand Down
163 changes: 158 additions & 5 deletions src/openhuman/inference/provider/factory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -964,6 +964,13 @@ pub fn create_chat_model_with_model_id(
if let Some(result) = try_create_local_runtime_chat_model(role, config) {
return result;
}
// Wire-equivalent BYOK cloud slugs (Anthropic / None / plain-Bearer, no
// codex-oauth or `/v1/responses` fallback) → crate-native `ChatModel`
// (issue #4727 Phase 3, conservative subset). `openai`/codex, custom
// proxy slugs, and the managed entry return `None` and fall through.
if let Some(result) = try_create_cloud_slug_chat_model(role, config) {
return result;
}
}
let (provider, model) = create_chat_provider(role, config)?;
let chat = chat_model_from_provider(provider, model.clone(), temperature);
Expand Down Expand Up @@ -1993,13 +2000,26 @@ fn make_local_openai_provider(
}

/// Look up a `cloud_providers` entry by slug and build the provider.
fn make_cloud_provider_by_slug(
/// The shared resolution for a `<slug>:<model>` cloud provider — the cloud
/// `cloud_providers` entry, the effective model id (with `default_model`
/// fallback + abstract-tier remapping), the resolved API key, and the OpenAI
/// codex-oauth routing. Extracted so the legacy [`Provider`] path
/// ([`make_cloud_provider_by_slug`]) and the crate-native cutover
/// ([`try_create_cloud_slug_chat_model`]) resolve **identically** — the only
/// divergence between them is the wire client they build from this plan.
struct CloudSlugResolution<'a> {
entry: &'a crate::openhuman::config::schema::cloud_providers::CloudProviderCreds,
effective_model: String,
key: String,
codex: crate::openhuman::inference::provider::openai_codex::OpenAiCodexRouting,
}

fn resolve_cloud_slug<'a>(
role: &str,
slug: &str,
model: &str,
temperature_override: Option<f64>,
config: &Config,
) -> anyhow::Result<(Box<dyn Provider>, String)> {
config: &'a Config,
) -> anyhow::Result<CloudSlugResolution<'a>> {
let entry = config.cloud_providers.iter().find(|e| e.slug == slug);

let entry = entry.ok_or_else(|| {
Expand Down Expand Up @@ -2090,9 +2110,33 @@ fn make_cloud_provider_by_slug(
);

let key = lookup_key_for_slug(slug, config)?;
let openai_codex_routing = resolve_openai_codex_routing(config, slug, &entry.endpoint, &key)
let codex = resolve_openai_codex_routing(config, slug, &entry.endpoint, &key)
.map_err(anyhow::Error::msg)?;

Ok(CloudSlugResolution {
entry,
effective_model,
key,
codex,
})
}

/// Look up a `cloud_providers` entry by slug and build the legacy
/// [`Provider`] wire client for it.
fn make_cloud_provider_by_slug(
role: &str,
slug: &str,
model: &str,
temperature_override: Option<f64>,
config: &Config,
) -> anyhow::Result<(Box<dyn Provider>, String)> {
let CloudSlugResolution {
entry,
effective_model,
key,
codex: openai_codex_routing,
} = resolve_cloud_slug(role, slug, model, config)?;

let unsupported = &config.temperature_unsupported_models;
match entry.auth_style {
AuthStyle::Anthropic => {
Expand Down Expand Up @@ -2191,6 +2235,115 @@ fn make_cloud_provider_by_slug(
}
}

/// A `<slug>:<model>` BYOK cloud provider as a crate-native [`ChatModel`] — the
/// Motion B cutover of the wire-equivalent [`make_cloud_provider_by_slug`]
/// branches (issue #4727 Phase 3, conservative subset).
///
/// Returns `None` (fall through to the `Provider` path) unless the role resolves
/// to a **configured** cloud slug whose auth style the crate `OpenAiModel` serves
/// with byte-identical wire semantics:
/// - `Anthropic` / `None` auth → always eligible (their endpoints have no
/// `/v1/responses`, so the host's dormant responses-fallback is behavior-neutral);
/// - `Bearer` → eligible **only** when there is no `/v1/responses` fallback, no
/// `openai-codex` OAuth, and no codex account header — i.e. a plain
/// chat-completions Bearer provider (DeepSeek, Groq, Mistral, xAI, …).
///
/// Everything else — `openai` (codex-oauth + Responses API), custom slugs whose
/// endpoint may proxy `/v1/responses`, and the managed `OpenhumanJwt` entry —
/// stays on the `Provider` path (deferred to the crate `/responses` port). The
/// resolution is shared via [`resolve_cloud_slug`], so eligible slugs resolve
/// identically to the legacy path; only the wire client differs. The **same**
/// access gate the `Provider` path applies (`enforce_local_only_inference` +
/// `verify_session_active`) runs before building. Temperature rides the per-call
/// `ModelRequest` (managed/local parity; the `@<temp>` suffix still bakes a fixed
/// override).
fn try_create_cloud_slug_chat_model(
role: &str,
config: &Config,
) -> Option<anyhow::Result<(Arc<dyn ChatModel<()>>, String)>> {
// Resolve the role's provider string, expanding the empty / "cloud" sentinel
// to the primary cloud target (mirroring create_chat_provider_from_string).
let mut resolved = provider_for_role(role, config);
if resolved.trim().is_empty() || resolved.trim() == "cloud" {
resolved = resolve_primary_cloud_provider_string(config);
}
let p = resolved.trim().to_string();

// Only the "<slug>:<model>[@temp]" cloud form routes here. The managed
// backend, BYOK-incomplete sentinel, and bespoke subprocess providers
// (claude-code / claude_agent_sdk) are handled on the `Provider` path.
if p == PROVIDER_OPENHUMAN
|| p == BYOK_INCOMPLETE_SENTINEL
|| p == CLAUDE_AGENT_SDK_PROVIDER
|| p.starts_with(CLAUDE_AGENT_SDK_PREFIX)
|| p.starts_with(crate::openhuman::inference::provider::claude_code::PROVIDER_PREFIX)
{
return None;
}
let colon = p.find(':')?;
let slug = p[..colon].trim().to_string();
if slug.is_empty() {
return None;
}
let (raw_model, temperature_override) = split_model_and_temperature(&p[colon + 1..]);
// Not a configured cloud slug → let the `Provider` path surface the precise
// "no cloud provider configured" error rather than silently no-op'ing.
if !config.cloud_providers.iter().any(|e| e.slug == slug) {
return None;
}

// Preserve the `Provider` path's gate for custom/cloud providers.
if let Err(e) = enforce_local_only_inference(role, &p) {
return Some(Err(e));
}
#[cfg(not(test))]
if let Err(e) = verify_session_active(config) {
return Some(Err(e));
}

let CloudSlugResolution {
entry,
effective_model,
key,
codex,
} = match resolve_cloud_slug(role, &slug, &raw_model, config) {
Ok(r) => r,
Err(e) => return Some(Err(e)),
};

// Only flip the wire-equivalent auth styles; codex-oauth, the `/v1/responses`
// fallback, and the managed `OpenhumanJwt` entry stay on the `Provider` path.
let auth = match entry.auth_style {
AuthStyle::Anthropic => CompatAuthStyle::Anthropic,
AuthStyle::None => CompatAuthStyle::None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep no-auth custom clouds on the provider path

For an AuthStyle::None cloud slug whose endpoint needs the legacy chat-completions-404 → /v1/responses retry, this arm now unconditionally builds the crate OpenAiModel, which the surrounding comment says has no responses fallback. The repo already guards that exact no-auth custom-cloud behavior in cloud_provider_falls_back_to_responses_on_404, so callers using create_chat_model(_with_model_id) with such a provider will now fail on the first /chat/completions 404 instead of retrying /responses; please apply the same fallback eligibility check used for Bearer or leave AuthStyle::None on the Provider path unless the endpoint is known chat-only.

Useful? React with 👍 / 👎.

AuthStyle::OpenhumanJwt => return None,
AuthStyle::Bearer => {
let responses_fallback = (!is_builtin_cloud_slug(&slug)
|| builtin_cloud_supports_responses_api(&slug))
&& !endpoint_host_is_chat_completions_only(&codex.endpoint);
if responses_fallback || codex.using_oauth || codex.account_id.is_some() {
return None;
}
CompatAuthStyle::Bearer
}
};

let unsupported = &config.temperature_unsupported_models;
let chat = super::crate_openai::make_crate_openai_chat_model(
&slug,
&entry.endpoint,
&key,
auth,
&effective_model,
unsupported,
temperature_override,
// Cloud OpenAI-compatible providers accept a `system` role — no merge
// (parity with `OpenAiCompatibleProvider::new`).
false,
);
Some(Ok((chat, effective_model)))
}

/// Fetch the bearer token for a slug from the workspace `auth-profiles.json`.
///
/// Tries `provider:<slug>` first (new key format), then the bare `<slug>`
Expand Down
79 changes: 79 additions & 0 deletions src/openhuman/inference/provider/factory_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2896,3 +2896,82 @@ fn try_create_local_runtime_returns_none_for_managed_and_cloud() {
cloud.chat_provider = Some("openai:gpt-4o-mini".to_string());
assert!(try_create_local_runtime_chat_model("chat", &cloud).is_none());
}

// ── Motion B (#4727 Phase 3): wire-equivalent BYOK cloud-slug cutover ────────

fn deepseek_entry(id: &str) -> CloudProviderCreds {
CloudProviderCreds {
id: id.to_string(),
slug: "deepseek".to_string(),
label: "DeepSeek".to_string(),
endpoint: "https://api.deepseek.com/v1".to_string(),
auth_style: AuthStyle::Bearer,
default_model: Some("deepseek-chat".to_string()),
..Default::default()
}
}

#[test]
fn create_chat_model_routes_plain_bearer_cloud_slug_to_crate_native() {
use tinyagents::harness::model::ChatModel;

let _guard = crate::openhuman::inference::inference_test_guard();
// DeepSeek is a built-in chat-completions-only Bearer provider: no
// `/v1/responses` fallback and no codex-oauth, so it is wire-equivalent and
// flips crate-native.
let mut config = Config::default();
config.cloud_providers.push(deepseek_entry("p_ds"));
config.chat_provider = Some("deepseek:deepseek-reasoner".to_string());
let (model, model_id) = create_chat_model_with_model_id("chat", &config, 0.7)
.expect("bearer cloud create_chat_model must build");
assert_eq!(model_id, "deepseek-reasoner");
let profile = model
.profile()
.expect("crate-native cloud model exposes a profile");
assert_eq!(profile.provider.as_deref(), Some("deepseek"));
// A generic cloud model keeps native tool calling + vision on (unlike the
// local runtimes), so this is the crate `OpenAiModel` default profile.
assert!(profile.tool_calling);
}

#[test]
fn create_chat_model_routes_anthropic_auth_cloud_slug_to_crate_native() {
use tinyagents::harness::model::ChatModel;

let _guard = crate::openhuman::inference::inference_test_guard();
// Anthropic-auth cloud slugs are always wire-equivalent (their endpoints have
// no `/v1/responses`, so the host's dormant fallback is behavior-neutral).
let mut config = Config::default();
config
.cloud_providers
.push(anthropic_entry("p_anth", "anthropic"));
config.chat_provider = Some("anthropic:claude-sonnet-4-6".to_string());
let (model, model_id) = create_chat_model_with_model_id("chat", &config, 0.7)
.expect("anthropic cloud create_chat_model must build");
assert_eq!(model_id, "claude-sonnet-4-6");
assert_eq!(
model.profile().and_then(|p| p.provider.as_deref()),
Some("anthropic")
);
}

#[test]
fn try_create_cloud_slug_declines_openai_and_non_cloud() {
let _guard = crate::openhuman::inference::inference_test_guard();
// `openai` exposes the Responses API, so its Bearer branch keeps the
// `/v1/responses` fallback — it is NOT wire-equivalent and stays host-side.
let mut openai = Config::default();
openai.cloud_providers.push(openai_entry("p_oai", "openai"));
openai.chat_provider = Some("openai:gpt-4o-mini".to_string());
assert!(try_create_cloud_slug_chat_model("chat", &openai).is_none());

// Managed (default), local runtimes, and unconfigured slugs are not cloud
// slugs — they fall through to their own paths.
assert!(try_create_cloud_slug_chat_model("chat", &Config::default()).is_none());
let mut local = Config::default();
local.chat_provider = Some("ollama:qwen2.5".to_string());
assert!(try_create_cloud_slug_chat_model("chat", &local).is_none());
let mut unconfigured = Config::default();
unconfigured.chat_provider = Some("deepseek:deepseek-chat".to_string());
assert!(try_create_cloud_slug_chat_model("chat", &unconfigured).is_none());
}
Loading