Skip to content

Adjust/Improve pipeline to be compatible with more models#4

Draft
yuqiannemo wants to merge 13 commits intoXtra-Computing:devfrom
yuqiannemo:dna_dev/run_more_models
Draft

Adjust/Improve pipeline to be compatible with more models#4
yuqiannemo wants to merge 13 commits intoXtra-Computing:devfrom
yuqiannemo:dna_dev/run_more_models

Conversation

@yuqiannemo
Copy link
Copy Markdown
Contributor

@yuqiannemo yuqiannemo commented Mar 22, 2026

  1. Add more dependencies to support more models
  2. Protect sensitive info like openrouter API

@JerryLife
Copy link
Copy Markdown
Collaborator

Thanks for working on broader model compatibility here. Supporting more HF model families is a good direction, and the ModelLoader change is addressing a real misclassification issue.

That said, I don't think this is ready to merge as-is because the dependency changes would make the base package less portable.

Moving these packages into project.dependencies introduces hard platform requirements for every pip install llm-dna user, even if they never use those model families:

  • mlx and mlx-lm are Apple Silicon-specific — this will break installation on Linux/Windows, which is where most of our users run the package
  • optimum, compressed-tensors, etc. are architecture-specific/transitive runtime deps rather than core package requirements

I think the right direction is:

  • Keep the base install minimal
  • Move platform/model-specific packages into optional dependency groups (e.g. apple, quantization)
  • Document which extras are needed for which model families

On ModelLoader, the current fix works for openai/gpt-oss, but it's brittle because it carves out exceptions from a broad openai/gpt- rule — every new HuggingFace model under the openai/ namespace would need another exclusion. Instead, I'd suggest tightening the OpenRouter prefixes to be more specific:

openrouter_prefixes = [
    "openrouter/",
    "openrouter:",
    "anthropic/claude-",
    "deepseek/",
    "openai/gpt-3",
    "openai/gpt-4",
    "google/gemini-",
    "x-ai/grok-",
    "cohere/command",
    "perplexity/",
]

This way openai/gpt-3.5-turbo and openai/gpt-4o still route to OpenRouter, but openai/gpt-oss-* naturally falls through to HuggingFace without needing any exclusion logic.

One smaller process note: the PR description looks incomplete (2. is empty). Please expand that before merge so the packaging/dependency rationale is clear.

Net: I like the goal of the PR, but the dependency strategy needs to change before this can be merged.

@yuqiannemo
Copy link
Copy Markdown
Contributor Author

Thanks for working on broader model compatibility here. Supporting more HF model families is a good direction, and the ModelLoader change is addressing a real misclassification issue.

That said, I don't think this is ready to merge as-is because the dependency changes would make the base package less portable.

Moving these packages into project.dependencies introduces hard platform requirements for every pip install llm-dna user, even if they never use those model families:

  • mlx and mlx-lm are Apple Silicon-specific — this will break installation on Linux/Windows, which is where most of our users run the package
  • optimum, compressed-tensors, etc. are architecture-specific/transitive runtime deps rather than core package requirements

I think the right direction is:

  • Keep the base install minimal
  • Move platform/model-specific packages into optional dependency groups (e.g. apple, quantization)
  • Document which extras are needed for which model families

On ModelLoader, the current fix works for openai/gpt-oss, but it's brittle because it carves out exceptions from a broad openai/gpt- rule — every new HuggingFace model under the openai/ namespace would need another exclusion. Instead, I'd suggest tightening the OpenRouter prefixes to be more specific:

openrouter_prefixes = [
    "openrouter/",
    "openrouter:",
    "anthropic/claude-",
    "deepseek/",
    "openai/gpt-3",
    "openai/gpt-4",
    "google/gemini-",
    "x-ai/grok-",
    "cohere/command",
    "perplexity/",
]

This way openai/gpt-3.5-turbo and openai/gpt-4o still route to OpenRouter, but openai/gpt-oss-* naturally falls through to HuggingFace without needing any exclusion logic.

One smaller process note: the PR description looks incomplete (2. is empty). Please expand that before merge so the packaging/dependency rationale is clear.

Net: I like the goal of the PR, but the dependency strategy needs to change before this can be merged.

Fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants