Skip to content
Merged

Dev #2805

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
## Architecture
- [`index.md`](./index.md)
- [`flow-map.md`](./flow-map.md)
- [`operation-pipeline.md`](./operation-pipeline.md)
- [`prompt-map.md`](./prompt-map.md)
- [`configuration.md`](./configuration.md)
- [`implementation-playbook.md`](./implementation-playbook.md)

## Operations
Expand All @@ -12,3 +14,4 @@
- [`operations/application-scoring.md`](./operations/application-scoring.md)
- [`operations/form-mapping.md`](./operations/form-mapping.md)
- [`operations/form-worksheet.md`](./operations/form-worksheet.md)
- [`operations/form-scoresheet.md`](./operations/form-scoresheet.md)
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Runtime Configuration

AI behavior is split between database-owned configuration and deployment
configuration. The database is the source of truth for which model, operation, and
prompt are used; appsettings holds deployment connectivity and operational settings.

## Database configuration

| Record | Owns |
| --- | --- |
| `AIModel` | Provider, deployment name (`Name`), active state, and model settings JSON |
| `AIOperation` | Prompt family (`Name`), selected model, execution mode, completion-token limit, and active state |
| `AIPrompt` | Versioned system/user templates, metadata, active state, and optional tenant ownership |

Host seeders create the built-in models, operations, and global prompts. Operations
select models by ID; `AIModel.Name` is the provider deployment identifier. The runtime
rejects inactive or unsupported configuration rather than choosing a fallback model.

## Prompt selection

For a prompt family, host requests use the newest active global prompt. Tenant requests
use the newest active tenant prompt, then fall back to the newest active global prompt.
Operations do not store a prompt ID or version.

## External configuration

Provider endpoint, API key, and authenticated-user cooldown remain deployment configuration:

```text
Azure:OpenAI:Endpoint
Azure:OpenAI:ApiKey
Azure:Generation:CooldownSeconds
```

Do not add operation defaults, profile maps, or prompt versions to appsettings.
27 changes: 17 additions & 10 deletions applications/Unity.GrantManager/modules/Unity.AI/docs/flow-map.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,21 @@
# Flow Map

## Standard path
UI -> API app service -> queue -> background job -> AI runtime -> persisted result
```text
UI -> AIGenerationAppService -> IApplicationGenerationQueue
automation -------------------> IApplicationGenerationQueue
IApplicationGenerationQueue -> AIGenerationRequest + background job
-> operation executor
-> Unity.AI runtime
-> operation-specific persisted result
```

## Operation families
- Application Analysis: submission -> analysis
- Attachment Summary: attachment ids -> summaries
- Application Scoring: application + scoresheet -> scoring
- Form Mapping: form version -> mapping
- Form Worksheet: form version -> worksheet
The app service authorizes and feature-gates UI requests. Automatic intake checks its
own tenant, form, and feature preconditions before entering the queue. The Grant Manager
queue resolves the active database operation, prevents duplicate active requests,
validates prerequisites, and enqueues work. The background job establishes tenant scope
and records request state; its executor owns operation-specific input and persistence.
The runtime resolves the prompt and model configuration, renders the request, calls the
provider, and parses the response.

## Build Rule
See [`implementation-playbook.md`](./implementation-playbook.md) for the canonical add-a-new-operation sequence.
The form mapping, worksheet, and scoresheet operations require an application form
version. See [operation pipeline](./operation-pipeline.md) for ownership rules.
Original file line number Diff line number Diff line change
Expand Up @@ -10,64 +10,28 @@ Use these existing operations as the canonical references:
3. `AttachmentSummary`
4. `FormMapping`
5. `FormWorksheet`
6. `FormScoresheet`

## Base Pattern
1. Define the prompt type.
2. Add the v2 prompt seed.
3. Add the operation seed.
4. Add the runtime contract method.
5. Add the runtime implementation.
6. Add the app service or queue entry.
7. Add the background job only if the result must be applied or persisted.
8. Add the UI button and status polling only if users trigger the operation from the web app.
9. Add tests for the prompt, runtime parsing, and job or service path.

## Bare Minimum
For the first pass, only add what is required for a working operation:

- prompt type
- prompt seed
- operation seed
- runtime method
- queue/app service entry
- job or direct apply path, if needed

## Optional Pieces
Add these only when the operation needs them:

- feature flag
- permissions
- permission definition provider entries
- menu entry
- UI button
- status polling
- refresh-after-complete behavior
- persistence/import/publish/assign behavior
1. Add the catalog definition, prompt family, model/operation seed, feature, and permissions.
2. Add supported prompt versions; do not assume a specific version number.
3. Add the runtime request/response contract and implementation.
4. Add an executor when Grant Manager must load input or persist a result.
5. Register the executor through the existing transient DI convention.
6. Expose a generate surface and UI only when users need one.
7. Add focused catalog, runtime, executor, and persistence tests.

## Rules
- Keep the prompt as the source of truth.
- Reuse the existing async generation pattern.
- Keep prompt content and operation/model configuration in the database.
- Reuse the shared generation pipeline.
- Do not hardcode field buckets or response shapes in UI code.
- Do not invent new plumbing if an existing operation already does the same job.
- Do not add tenant feature seeding.
- Do not add write-back UI behavior unless the operation already persists output.

## Expected Flow
1. User clicks Generate.
2. UI disables the button and shows generating state, if the operation has UI.
3. API checks permission and feature flag, if the operation uses them.
4. API queues the generation request.
5. Background job loads the operation context.
6. Job builds the prompt payload from existing data.
7. AI runtime renders v2 prompts and logs input/output.
8. Job parses the AI response.
9. Job applies the result if needed.
10. Job stamps status and rate limit state.
11. UI polls status and refreshes after completion, if applicable.
- Keep operation-specific input and persistence in the executor.
- Do not add UI write-back behavior unless the operation persists its output.

## Validation
- Confirm the prompt version is v2.
- Confirm the operation exists in the AI operation seed.
- Confirm the operation exists in the catalog and host seed.
- Confirm every supported prompt version resolves correctly.
- Confirm any required feature flag exists in the host feature definitions.
- Confirm any required permission is wired in the permission definition provider.
- Confirm the UI button uses the same generating/status flow as the other operations, if it is user-triggered.
103 changes: 30 additions & 73 deletions applications/Unity.GrantManager/modules/Unity.AI/docs/index.md
Original file line number Diff line number Diff line change
@@ -1,80 +1,37 @@
# Unity.AI Index
# Unity.AI

## Domain.Shared
AI constants:
- feature flags
- permission names
- localization keys
- prompt type names
`Unity.AI` owns provider-neutral AI contracts, prompt/model/operation configuration,
runtime execution, and the generation API. Grant Manager owns the application data,
queue implementation, operation executors, and persistence of generated results.

## Application.Contracts
Public AI surface:
- app service interfaces
- queue interfaces
- DTOs
- permission definitions
## Boundaries

## Application
AI implementation:
- runtime
- prompt seeding
- generation app services
- validators
- prompt logging
| Area | Responsibility |
| --- | --- |
| `Domain.Shared` | Features, permissions, localization, and prompt family names |
| `Application.Contracts` | Runtime, generation, queue, and DTO contracts |
| `Application` | Prompt/model/operation seeds, provider runtime, API, and status reads |
| `Runtime/Execution` | Prompt rendering, provider calls, response parsing, and prompt logging |
| Grant Manager | Request locking, background jobs, operation executors, and result persistence |
| `Web` | Menus, generate actions, and status polling |

## Web
UI-facing AI bits:
- menus
- generation buttons
- status polling
## Operation catalog

## Files
### Application
- `Operations` - validators and helpers
- `Runtime/Execution` - rendering, parsing, logging, provider calls
- `Runtime/Prompts` - prompt types and template plumbing
- `DataSeed` - seeded prompt and operation data
- `Generation/AIGenerationAppService.cs` - generation API
`AIGenerationOperations` is the single catalog for operation type, prompt family,
feature, permissions, and form-version requirement.

### Application.Contracts
- `IAIService.cs` - runtime contract
- `Generation/IAIGenerationAppService.cs` - generation app service contract
- `Generation/*ResultDto.cs` - queued result DTOs
- `Operations/IAIGenerationPrerequisiteValidator.cs` - queue prerequisites
- `Automation/IApplicationAIGenerationQueue.cs` - queue contract
- `Permissions/*` - permissions

### Domain.Shared
- `Features/AIFeatures.cs` - feature flags
- `Localization/AILocalizationKeys.cs` - messages
- `PromptTypes/AIPromptTypes.cs` - prompt family names

### Web
- `Menus/AIMenuContributor.cs` - menu entries
- `Menus/AIMenus.cs` - menu item names

## Access
| Operation | View | Generate |
| Operation | Type | Requires form version |
| --- | --- | --- |
| Application Analysis | `ViewApplicationAnalysis` | `GenerateApplicationAnalysis` |
| Attachment Summary | `ViewAttachmentSummary` | `GenerateAttachmentSummaries` |
| Application Scoring | `ViewScoringResult` | `GenerateScoring` |
| Form Mapping | `ViewFormMapping` | `GenerateFormMapping` |
| Form Worksheet | `ViewFormWorksheet` | `GenerateFormWorksheet` |

- Features:
- `Unity.AI.ApplicationAnalysis`
- `Unity.AI.AttachmentSummaries`
- `Unity.AI.Scoring`
- `Unity.AI.FormMapping`
- `Unity.AI.FormWorksheet`

- Rule:
- Both permission and feature gate must allow generation.

## AI Notes
- Prompt logging: logs rendered system/user prompts and provider output.
- Response parsing: parses provider output into stable app-facing results.
- Feature gating: disabled features fail early at the API boundary.
- Background jobs: mark failures, then re-throw.
- New operation playbook: see `implementation-playbook.md`.
| Application Analysis | `application-analysis` | No |
| Attachment Summary | `attachment-summary` | No |
| Application Scoring | `application-scoring` | No |
| Form Mapping | `form-mapping` | Yes |
| Form Worksheet | `form-worksheet` | Yes |
| Form Scoresheet | `form-scoresheet` | Yes |

User-triggered generation requires both the catalogued feature and generate permission.
Automatic intake enforces its tenant, form, feature, and generation prerequisites without
user permission authorization. Status reads require the corresponding view permission.

See [configuration](./configuration.md), [pipeline](./operation-pipeline.md), and the
[implementation playbook](./implementation-playbook.md).
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI operation pipeline

AI generation uses a shared operation catalog and queued lifecycle. The catalog owns the operation key, seeded operation name, feature gate, permissions, and whether a form version is required. Submission enters `IAIGenerationAppService.SubmitAsync`, and the Grant Manager queue preserves the existing duplicate-request lock and operation-specific validation.
AI generation uses a shared operation catalog and queued lifecycle. The catalog owns the operation key, seeded operation name, feature gate, permissions, and whether a form version is required. UI submission enters `IAIGenerationAppService.SubmitAsync`; automatic intake checks its own preconditions and enters the Grant Manager queue directly. The queue preserves the duplicate-request lock and operation-specific validation.

The generic background-job base owns tenant scope, structured logging, request state transitions, failure handling, and cooldown stamping. Operation-specific executors remain responsible for loading input, calling the AI contract, validating the response, and persisting the result.

Expand All @@ -14,3 +14,6 @@ The generic background-job base owns tenant scope, structured logging, request s
6. Add focused catalog, lifecycle, executor, and persistence tests.

Do not add another queue branch for shared lifecycle concerns. New operation behavior belongs in its executor; request locking, status transitions, tenant scope, logging, and cooldown behavior stay in the common pipeline.

For Grant Manager queue and executor ownership, see the
[generation hand-off](../../../src/Unity.GrantManager.Application/GrantApplications/Automation/Generation/README.md).
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Generate an AI analysis of an application submission.
- `GET /api/app/ai/generation/status`

## Contract
- Structured analysis output. Returns an immediate queued result via API app service, queue, background job, and AI runtime.
- Structured analysis output. The POST request enqueues generation and returns without the generated payload; clients use the shared status endpoint while the background executor persists the result.

## Notes
- This is a reviewer-oriented summary and recommendation flow.
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Generate scored answers for a submitted application against an assigned scoreshe
- `GET /api/app/ai/generation/status`

## Contract
- Structured scoring output. Returns an immediate queued result via API app service, queue, background job, and AI runtime.
- Structured scoring output. The POST request enqueues generation and returns without the generated payload; clients use the shared status endpoint while the background executor persists the result.

## Notes
- The prompt asks for answers only for the configured section or scoresheet context.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Generate summaries for selected application attachments.
- `GET /api/app/ai/generation/status`

## Contract
- Structured attachment summary output. Returns an immediate queued result via API app service, queue, background job, and AI runtime.
- Structured attachment summary output. The POST request enqueues generation and returns without the generated payload; clients use the shared status endpoint while the background executor persists the result.

## Notes
- Each attachment is processed as part of the generation request.
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Generate recommended CHEFS-to-Unity field mapping for a form version.
- `GET /api/app/application-form-version/{id}`

## Contract
- Structured mapping recommendation JSON output. Returns an immediate queued result via API app service, queue, background job, and AI runtime.
- Structured mapping recommendation JSON output. The POST request enqueues generation and returns without the generated payload; clients use the shared status endpoint while the background executor persists the result.

## Output Shape
- Core field matches.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Form Scoresheet

## Goal

Generate and publish a scoresheet definition for a form version.

## Inputs

- Form version and form context
- Existing linked scoresheet, when present
- Existing scoresheet sections and fields

## Surface

- `POST /api/app/ai/generation/form-scoresheet`
- `GET /api/app/ai/generation/status`

## Result

The executor validates the generated scoresheet JSON, creates or replaces the form's
scoresheet, publishes it, and links it to the application form.
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,13 @@ Generate a recommended worksheet definition for a form version.
- `GET /api/app/ai/generation/status`

## Contract
- Structured Flex worksheet JSON output. Returns an immediate queued result via API app service, queue, background job, and AI runtime.
- Structured worksheet field-suggestion JSON. The POST request enqueues generation and returns without the generated payload; clients use the shared status endpoint while the background executor validates the suggestions and creates an unpublished worksheet for review.

## Output Shape
- Full worksheet definition JSON.
- Include only additional worksheet fields that the form needs beyond core Unity fields.
- Keep the result valid JSON and compatible with Flex import.
- A `fields` collection containing the suggested additional worksheet fields.
- Each suggestion supplies the field key, label, and supported custom-field type.
- The executor builds the worksheet and its `Suggested Fields` section from the validated suggestions.
- Keep the result valid JSON and include only fields that the form needs beyond core Unity fields.

## Notes
- The AI output should stay valid JSON.
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,26 @@
- `ApplicationScoring` - question scoring
- `FormMapping` - CHEFS to Unity mapping
- `FormWorksheet` - worksheet generation
- `FormScoresheet` - scoresheet generation

## Versions
- Built-in `v0`, `v1`, and `v2` prompt rows are defined and seeded by `AIPromptDataSeeder`
- Runtime selects the newest active prompt by family.
- Built-in prompt rows are defined and seeded by `AIPromptDataSeeder`.
- Families may have `v0`, `v1`, and `v2` rows; a new operation only needs the versions it supports.
- Without an explicit request version, runtime selects the newest active prompt by family.

## Tenant selection
- `AIOperation.Name` is the prompt family; operations do not pin a prompt row or version.
- Host requests use the newest active global prompt in the family.
- Tenant requests use the newest active prompt owned by that tenant, falling back to the newest active global prompt.
- An explicit request version selects that active version, with the same tenant/global fallback.
- Otherwise, host requests use the newest active global prompt in the family.
- Otherwise, tenant requests use the newest active prompt owned by that tenant, falling back to the newest active global prompt.
- To roll back a tenant or global prompt, deactivate the active version and leave the prior version active.
- Tenant prompt rows are administrator-created; deployments seed only global prompts and operations.

## Prompt rules
- Versioned prompts are the source of truth.
- Prompt templates define the request shape.
- Structured outputs should stay JSON-shaped.
- New versions should not silently change behavior.
- A new version should be additive and must not silently change an active prompt's behavior.

## Build Rule
Use [`implementation-playbook.md`](./implementation-playbook.md) when adding a new prompt-backed operation.
Loading
Loading