generated from SAP/repository-template
-
Notifications
You must be signed in to change notification settings - Fork 16
feat: [Orchestration] Support Fallback Modules #743
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
12b5c67
basic functionality
Jonas-Isr d95eeb8
improve code quality
Jonas-Isr 6d088c4
add tests and app endpoint
Jonas-Isr b511219
small fixes
Jonas-Isr 144ace9
streaming
Jonas-Isr 525c6aa
release notes and cleanup
Jonas-Isr 1b0cb69
Merge branch 'main' into orch_fallback_modules
Jonas-Isr 8e8b30b
rearrange tests
Jonas-Isr f36c085
small stuff
Jonas-Isr 482d2e3
fix
Jonas-Isr d162d21
Merge branch 'main' into orch_fallback_modules
Jonas-Isr 0a8ebbb
review
Jonas-Isr 6cb2ffb
review
Jonas-Isr File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
81 changes: 81 additions & 0 deletions
81
orchestration/src/test/resources/__files/fallbackResponse.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| { | ||
| "request_id": "a562703b-7fe5-97ba-b417-e8140a25fb7c", | ||
| "intermediate_results": { | ||
| "templating": [ | ||
| { | ||
| "content": "HelloWorld! Why is this phrase so famous?", | ||
| "role": "user" | ||
| } | ||
| ], | ||
| "llm": { | ||
| "id": "", | ||
| "object": "chat.completion", | ||
| "created": 1769700208, | ||
| "model": "gemini-2.5-flash", | ||
| "choices": [ | ||
| { | ||
| "index": 0, | ||
| "message": { | ||
| "role": "assistant", | ||
| "content": "\"Hello, World!\" is arguably the most famous phrase in the world of programming, and its fame stems from a combination of historical significance, pedagogical utility, and cultural impact.\n\nHere's why it's so famous:\n\n1. **Historical Origin and Standardization:**\n * While simple output programs existed before, the phrase \"Hello, World!\" was popularized and standardized by **Brian Kernighan and Dennis Ritchie** in their seminal 1978 book, **\"The C Programming Language.\"**\n * In the book, the very first example program printed \"hello, world\" (lowercase, no exclamation mark initially). This book became the definitive guide for C, and C became one of the most influential programming languages ever. As C spread, so did its iconic first program.\n * Before K&R, a similar example (\"hello\") appeared in a 1974 Bell Labs internal memo by Kernighan, demonstrating the BCPL language.\n\n2. **Pedagogical Value (Teaching Tool):**\n * **First Step for Beginners:** It's the absolute simplest program that actually *does* something visible. For someone learning to code, successfully running \"Hello, World!\" is an immediate, tangible victory.\n * **Low Barrier to Entry:** It requires minimal syntax and no complex concepts (like variables, loops, functions, or data structures). It focuses solely on the core idea of output.\n * **Immediate Gratification:** Seeing \"Hello, World!\" appear on the screen provides instant feedback and motivation, confirming that the development environment is set up correctly and the basic compilation/execution process works.\n * **Introduction to Core Concepts:** It subtly introduces fundamental ideas like:\n * The concept of a program.\n * The idea of output (printing to the console).\n * The basic structure of a program in a given language.\n\n3. **Practical Utility (Testing and Verification):**\n * **Sanity Check:** When setting up a new programming language, a new development environment (IDE), a new compiler, or a new operating system, \"Hello, World!\" is the go-to program to verify that everything is working correctly. If \"Hello, World!\" doesn't run, you know the problem is fundamental to your setup, not your complex code.\n * **Minimal Viable Product (MVP) for Setup:** It's the quickest way to confirm that the toolchain (editor, compiler/interpreter, linker, runtime) is functional.\n\n4. **Cultural Significance and Rite of Passage:**\n * **Universal Rite of Passage:** Every programmer, regardless of the language they learn, almost certainly starts with \"Hello, World!\" It's a shared experience that connects developers across generations and technologies.\n * **Symbol of Beginning:** It represents the very first step into the vast and complex world of programming. It's a literal \"hello\" to the machine and the coding journey.\n * **Ubiquity:** Its presence in countless tutorials, textbooks, and online guides for virtually every programming language has cemented its status as an iconic and universally recognized phrase.\n\nIn essence, \"Hello, World!\" is famous because it's simple, effective, historically significant, and serves as a universal welcome mat for anyone stepping into the world of computer programming.", | ||
| "tool_calls": [] | ||
| }, | ||
| "finish_reason": "stop" | ||
| } | ||
| ], | ||
| "usage": { | ||
| "completion_tokens": 1728, | ||
| "prompt_tokens": 9, | ||
| "total_tokens": 1737, | ||
| "prompt_tokens_details": { | ||
| "cached_tokens": 0 | ||
| }, | ||
| "completion_tokens_details": { | ||
| "reasoning_tokens": 1024 | ||
| } | ||
| }, | ||
| "citations": [] | ||
| }, | ||
| "output_unmasking": [] | ||
| }, | ||
| "final_result": { | ||
| "id": "", | ||
| "object": "chat.completion", | ||
| "created": 1769700208, | ||
| "model": "gemini-2.5-flash", | ||
| "choices": [ | ||
| { | ||
| "index": 0, | ||
| "message": { | ||
| "role": "assistant", | ||
| "content": "\"Hello, World!\" is arguably the most famous phrase in the world of programming, and its fame stems from a combination of historical significance, pedagogical utility, and cultural impact.\n\nHere's why it's so famous:\n\n1. **Historical Origin and Standardization:**\n * While simple output programs existed before, the phrase \"Hello, World!\" was popularized and standardized by **Brian Kernighan and Dennis Ritchie** in their seminal 1978 book, **\"The C Programming Language.\"**\n * In the book, the very first example program printed \"hello, world\" (lowercase, no exclamation mark initially). This book became the definitive guide for C, and C became one of the most influential programming languages ever. As C spread, so did its iconic first program.\n * Before K&R, a similar example (\"hello\") appeared in a 1974 Bell Labs internal memo by Kernighan, demonstrating the BCPL language.\n\n2. **Pedagogical Value (Teaching Tool):**\n * **First Step for Beginners:** It's the absolute simplest program that actually *does* something visible. For someone learning to code, successfully running \"Hello, World!\" is an immediate, tangible victory.\n * **Low Barrier to Entry:** It requires minimal syntax and no complex concepts (like variables, loops, functions, or data structures). It focuses solely on the core idea of output.\n * **Immediate Gratification:** Seeing \"Hello, World!\" appear on the screen provides instant feedback and motivation, confirming that the development environment is set up correctly and the basic compilation/execution process works.\n * **Introduction to Core Concepts:** It subtly introduces fundamental ideas like:\n * The concept of a program.\n * The idea of output (printing to the console).\n * The basic structure of a program in a given language.\n\n3. **Practical Utility (Testing and Verification):**\n * **Sanity Check:** When setting up a new programming language, a new development environment (IDE), a new compiler, or a new operating system, \"Hello, World!\" is the go-to program to verify that everything is working correctly. If \"Hello, World!\" doesn't run, you know the problem is fundamental to your setup, not your complex code.\n * **Minimal Viable Product (MVP) for Setup:** It's the quickest way to confirm that the toolchain (editor, compiler/interpreter, linker, runtime) is functional.\n\n4. **Cultural Significance and Rite of Passage:**\n * **Universal Rite of Passage:** Every programmer, regardless of the language they learn, almost certainly starts with \"Hello, World!\" It's a shared experience that connects developers across generations and technologies.\n * **Symbol of Beginning:** It represents the very first step into the vast and complex world of programming. It's a literal \"hello\" to the machine and the coding journey.\n * **Ubiquity:** Its presence in countless tutorials, textbooks, and online guides for virtually every programming language has cemented its status as an iconic and universally recognized phrase.\n\nIn essence, \"Hello, World!\" is famous because it's simple, effective, historically significant, and serves as a universal welcome mat for anyone stepping into the world of computer programming.", | ||
| "tool_calls": [] | ||
| }, | ||
| "finish_reason": "stop" | ||
| } | ||
| ], | ||
| "usage": { | ||
| "completion_tokens": 1728, | ||
| "prompt_tokens": 9, | ||
| "total_tokens": 1737, | ||
| "prompt_tokens_details": { | ||
| "cached_tokens": 0 | ||
| }, | ||
| "completion_tokens_details": { | ||
| "reasoning_tokens": 1024 | ||
| } | ||
| }, | ||
| "citations": [] | ||
| }, | ||
| "intermediate_failures": [ | ||
| { | ||
| "request_id": "a562703b-7fe5-97ba-b417-e8140a25fb7c", | ||
| "code": 400, | ||
| "message": "400 - Request Body: Model broken_name not supported.", | ||
| "location": "Request Body", | ||
| "headers": { | ||
| "Content-Type": "application/json" | ||
| } | ||
| } | ||
| ] | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Grounding in
testFallbackModules?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this here to be able to test that the additional configs (like grounding or input filter) are correctly propagated when using the fallback modules. Since this is not an e2e test, this will not add flakyness.