fix: prefer OPENAI_EMBEDDING_API_KEY over OPENAI_API_KEY for embeddings

[w1ndcn]

Closes #863

What changed

File	Change
src/providers/embedding/index.ts	Stop passing OPENAI_API_KEY to OpenAIEmbeddingProvider() constructor; let it resolve its own key from env vars (already done in 0908d6e)
src/config.ts	detectEmbeddingProvider() now also checks OPENAI_EMBEDDING_API_KEY when auto-detecting the embedding provider
.env.example	Document OPENAI_EMBEDDING_API_KEY and OPENAI_EMBEDDING_BASE_URL
src/cli.ts	Passive doctor check hint now includes OPENAI_EMBEDDING_API_KEY

Why

When EMBEDDING_PROVIDER=openai is used with both:

• OPENAI_API_KEY for an OpenAI-compatible chat provider
• OPENAI_EMBEDDING_API_KEY / OPENAI_EMBEDDING_BASE_URL for a separate local embedding endpoint (e.g. LM Studio, vLLM, Ollama)

the embedding provider used OPENAI_API_KEY instead of OPENAI_EMBEDDING_API_KEY, making it impossible to split chat and embedding endpoints.

Root cause

Two issues:

1. Factory override (src/providers/embedding/index.ts): createEmbeddingProvider() passed getEnvVar("OPENAI_API_KEY")! as a constructor argument. Since constructor arguments take highest priority, OpenAIEmbeddingProvider's internal fallback chain (OPENAI_EMBEDDING_API_KEY → OPENAI_API_KEY) was never reached.

2. Auto-detection gap (src/config.ts): detectEmbeddingProvider() only checked OPENAI_API_KEY, not OPENAI_EMBEDDING_API_KEY. If a user only set the latter (e.g. using Anthropic for LLM + local OpenAI-compatible for embeddings), the function returned null and the embedding provider was never created at all.

After the fix

API key resolution for OpenAI embeddings follows this precedence:

1. Explicit constructor argument (if provided)
2. OPENAI_EMBEDDING_API_KEY
3. OPENAI_API_KEY

Base URL resolution follows this precedence:

1. OPENAI_EMBEDDING_BASE_URL
2. OPENAI_BASE_URL

Auto-detection now recognizes OPENAI_EMBEDDING_API_KEY as a valid signal to enable the openai embedding provider.

Reproduction

Config that was broken before this fix:

# Hosted OpenAI-compatible chat provider
OPENAI_API_KEY=hosted-chat-key
OPENAI_BASE_URL=https://example-chat-provider/v1
OPENAI_MODEL=some-chat-model

# Local OpenAI-compatible embedding provider
EMBEDDING_PROVIDER=openai
OPENAI_EMBEDDING_API_KEY=local-embedding-key
OPENAI_EMBEDDING_BASE_URL=http://127.0.0.1:1234/v1
OPENAI_EMBEDDING_MODEL=text-embedding-bge-m3
OPENAI_EMBEDDING_DIMENSIONS=1024

Before: hosted-chat-key was sent to the local embedding endpoint. LM Studio rejects it with an invalid token error.

After: local-embedding-key is used for embeddings, hosted-chat-key is used for chat. Each endpoint gets the correct key.

Backward compatibility

• Users who only set OPENAI_API_KEY continue to work — OpenAIEmbeddingProvider falls back to it.
• Users who don't set any embedding key continue to get BM25-only mode.
• No behavior change for other embedding providers (gemini, voyage, cohere, openrouter).

[vercel]

Someone is attempting to deploy a commit to the rohitg00's projects Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR enables OpenAI embedding provider to use dedicated credentials and endpoints separate from chat credentials. Environment variable documentation is expanded to show OPENAI_EMBEDDING_API_KEY and OPENAI_EMBEDDING_BASE_URL, provider detection now recognizes the embedding-specific key, factory instantiation removes the hardcoded API key argument, and diagnostic hints are updated.

Changes

OpenAI Embedding Provider Configuration

Layer / File(s)	Summary
Environment configuration and provider detection .env.example, src/config.ts	OPENAI_EMBEDDING_API_KEY, OPENAI_EMBEDDING_BASE_URL, and OPENAI_EMBEDDING_MODEL are documented as fallback-capable overrides; detectEmbeddingProvider now treats OPENAI_EMBEDDING_API_KEY as a signal for OpenAI embedding provider selection.
Remove hardcoded API key from embedding factory src/providers/embedding/index.ts	The "openai" case in createEmbeddingProvider instantiates OpenAIEmbeddingProvider() with no arguments, activating the provider's internal key precedence (OPENAI_EMBEDDING_API_KEY → OPENAI_API_KEY).
Update embedding provider diagnostics src/cli.ts	The doctor command's "Embedding provider" failure hint now includes OPENAI_EMBEDDING_API_KEY alongside other embedding provider keys.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

• rohitg00/agentmemory#503: Implemented the OpenAIEmbeddingProvider constructor logic that resolves OPENAI_EMBEDDING_API_KEY internally, which this PR unblocks by removing the hardcoded OPENAI_API_KEY argument.

Suggested reviewers

• rohitg00

Poem

🐇 Two keys, two homes, now side by side,
Chat and embeddings, no longer tied.
OPENAI_EMBEDDING_KEY finds its place,
Local endpoint gets the right embrace.
One small change, the right fix glows — hooray! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: updating the OpenAI embedding provider to prioritize OPENAI_EMBEDDING_API_KEY over OPENAI_API_KEY.
Linked Issues check	✅ Passed	The pull request successfully implements all coding requirements from issue #863: removes the explicit OPENAI_API_KEY argument from OpenAIEmbeddingProvider constructor, updates environment variable documentation, adds OPENAI_EMBEDDING_API_KEY to configuration hints, and enables separate API key handling for embeddings.
Out of Scope Changes check	✅ Passed	All changes in the pull request are directly aligned with issue #863: fixing API key resolution priority for OpenAI embeddings across factory, config detection, documentation, and CLI hints.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}