Skip to content

Release Instruct models#308

Merged
locnguyen1986 merged 11 commits intoreleasefrom
main
Dec 17, 2025
Merged

Release Instruct models#308
locnguyen1986 merged 11 commits intoreleasefrom
main

Conversation

@locnguyen1986
Copy link
Collaborator

@locnguyen1986 locnguyen1986 commented Dec 17, 2025

This PR introduces support for Instruct model fallback functionality and improves conversation title handling. The key changes enable models with reasoning capabilities to automatically switch to a non-reasoning "instruct" variant when users explicitly disable thinking mode via the enable_thinking=false parameter.

Key Changes:

  • Added database schema support for instruct model relationships with supports_instruct flag on model catalogs and self-referencing instruct_model_id on provider models
  • Implemented automatic model switching in chat completions when enable_thinking=false is set and an instruct model is configured
  • Added comprehensive title sanitization utilities to clean URLs, special characters, and markdown from conversation titles
  • Updated CORS configuration to support additional development and production domains, and added MCP protocol headers

Hotifx/conversation titleAdds comprehensive content sanitization for conversation titles to handle URLs, special characters, and markdown links
Extends Kong CORS configuration to include production domains and additional MCP-related headers
Updates prompt orchestration documentation with expanded module descriptions and configuration details
Fix title on conversation create flow too
Improves the title truncation logic in the TruncateTitle function
Copilot AI review requested due to automatic review settings December 17, 2025 02:07
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces support for Instruct model fallback functionality and improves conversation title handling. The key changes enable models with reasoning capabilities to automatically switch to a non-reasoning "instruct" variant when users explicitly disable thinking mode via the enable_thinking=false parameter.

Key Changes:

  • Added database schema support for instruct model relationships with supports_instruct flag on model catalogs and self-referencing instruct_model_id on provider models
  • Implemented automatic model switching in chat completions when enable_thinking=false is set and an instruct model is configured
  • Added comprehensive title sanitization utilities to clean URLs, special characters, and markdown from conversation titles
  • Updated CORS configuration to support additional development and production domains, and added MCP protocol headers

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
services/llm-api/migrations/000014_add_supports_instruct.up.sql Adds supports_instruct column to model catalogs and instruct_model_id foreign key to provider models with indexes
services/llm-api/migrations/000014_add_supports_instruct.down.sql Rollback script for instruct model schema changes
services/llm-api/migrations/000013_fix_project_unique_constraint.up.sql Fixes unique constraint on projects to exclude soft-deleted records using partial index
services/llm-api/migrations/000013_fix_project_unique_constraint.down.sql Rollback script for project unique constraint fix
services/llm-api/internal/utils/stringutils/title.go New utility package with functions to sanitize, truncate, and generate conversation titles from content
services/llm-api/internal/interfaces/httpserver/responses/model/model.go Adds SupportsInstruct field to responses and InstructModelPublicID parameter to provider model responses
services/llm-api/internal/interfaces/httpserver/requests/models/model.go Adds SupportsInstruct and InstructModelPublicID fields to model update requests
services/llm-api/internal/interfaces/httpserver/requests/chat/chat.go Adds EnableThinking parameter to chat completion requests
services/llm-api/internal/interfaces/httpserver/handlers/modelhandler/provider_model_handler.go Implements instruct model resolution logic with batch fetching for list operations
services/llm-api/internal/interfaces/httpserver/handlers/modelhandler/provider_handler.go Adds GetProviderModelByID method for internal ID lookups
services/llm-api/internal/interfaces/httpserver/handlers/modelhandler/model_catalog_handler.go Adds handling for SupportsInstruct field updates
services/llm-api/internal/interfaces/httpserver/handlers/conversationhandler/conversation_handler.go Applies title sanitization when creating and updating conversations
services/llm-api/internal/interfaces/httpserver/handlers/chathandler/chat_handler.go Implements instruct model fallback logic, uses sanitized title generation, and adds detailed message logging
services/llm-api/internal/infrastructure/database/repository/modelrepo/model_catalog_repository.go Adds filter support for SupportsInstruct field
services/llm-api/internal/infrastructure/database/gormgen/model_catalogs.gen.go Generated code updates for SupportsInstruct field
services/llm-api/internal/infrastructure/database/dbschema/provider_model.go Schema updates for InstructModelID field with mapping to domain model
services/llm-api/internal/infrastructure/database/dbschema/model_catalog.go Schema updates for SupportsInstruct field with mapping to domain model
services/llm-api/internal/domain/model/provider_model_service.go Adds FindByID and FindByIDs service methods for instruct model lookups
services/llm-api/internal/domain/model/provider_model.go Adds InstructModelID field to domain model
services/llm-api/internal/domain/model/model_catalog.go Adds SupportsInstruct field to catalog domain model and filter
kong/kong.yml Expands CORS origins to include additional localhost ports and production domains, adds MCP and Accept headers
kong/kong-dev-full.yml Similar CORS configuration updates for development environment
docs/guides/prompt-orchestration.md Comprehensive documentation updates for prompt orchestration system with new module descriptions and examples

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +48 to +70
// TruncateTitle truncates a title to a maximum length, breaking at word boundaries
func TruncateTitle(title string, maxLen int) string {
if len(title) <= maxLen {
return title
}

// Reserve space for ellipsis so the final string never exceeds maxLen
ellipsis := "..."
contentLimit := maxLen - len(ellipsis)
if contentLimit < 0 {
contentLimit = 0
}

truncated := title[:contentLimit]
minLen := contentLimit / 2 // At least half the max content length

// Prefer to cut on a word boundary when possible
if lastSpace := strings.LastIndex(truncated, " "); lastSpace > minLen {
truncated = strings.TrimRight(truncated[:lastSpace], " ")
}

return truncated + ellipsis
}
Copy link

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TruncateTitle function could produce an incorrect result when maxLen is less than the length of the ellipsis (3 characters). When maxLen is 1 or 2, contentLimit becomes -2 or -1 respectively, which is then set to 0. This would result in just "..." being returned even when the title is longer, which exceeds maxLen. Consider adding a check to ensure maxLen is at least the length of the ellipsis, or handle this edge case explicitly.

Copilot uses AI. Check for mistakes.
@locnguyen1986 locnguyen1986 merged commit 255b59c into release Dec 17, 2025
13 of 14 checks passed
@github-project-automation github-project-automation bot moved this to QA in Jan Dec 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: QA

Development

Successfully merging this pull request may close these issues.

3 participants