diff --git a/backend/app/api/docs/collections/create.md b/backend/app/api/docs/collections/create.md index 258b49fad..aef52efaa 100644 --- a/backend/app/api/docs/collections/create.md +++ b/backend/app/api/docs/collections/create.md @@ -6,7 +6,7 @@ pipeline: documents stored in the cloud (see the `documents` interface). * Create an OpenAI [Vector Store](https://platform.openai.com/docs/api-reference/vector-stores) - based on those File's. + based on those file(s). * [To be deprecated] Attach the Vector Store to an OpenAI [Assistant](https://platform.openai.com/docs/api-reference/assistants). Use parameters in the request body relevant to an Assistant to flesh out @@ -16,15 +16,15 @@ pipeline: If any one of the OpenAI interactions fail, all OpenAI resources are cleaned up. If a Vector Store is unable to be created, for example, -all File's that were uploaded to OpenAI are removed from +all file(s) that were uploaded to OpenAI are removed from OpenAI. Failure can occur from OpenAI being down, or some parameter -value being invalid. It can also fail due to document types not be +value being invalid. It can also fail due to document types not being accepted. This is especially true for PDFs that may not be parseable. Vector store/assistant will be created asynchronously. The immediate response from this endpoint is `collection_job` object which is going to contain -the collection "job ID" and status.Once the collection has been created, +the collection "job ID" and status. Once the collection has been created, information about the collection will be returned to the user via the callback URL. If a callback URL is not provided, clients can check the -`collection job info` endpoint with the `job_id`, to retrieve the +`collection job info` endpoint with the `job_id`, to retrieve information about the creation of collection. diff --git a/backend/app/api/docs/collections/delete.md b/backend/app/api/docs/collections/delete.md index da2c75cf8..c7f0f2c7a 100644 --- a/backend/app/api/docs/collections/delete.md +++ b/backend/app/api/docs/collections/delete.md @@ -1,14 +1,14 @@ Remove a collection from the platform. This is a two step process: -1. Delete all OpenAI resources that were allocated: File's, the Vector +1. Delete all OpenAI resources that were allocated: file(s), the Vector Store, and the Assistant. -2. Delete the collection entry from the AI platform database. +2. Delete the collection entry from the kaapi database. No action is taken on the documents themselves: the contents of the documents that were a part of the collection remain unchanged, those documents can still be accessed via the documents endpoints. The response from this endpoint will be a `collection_job` object which will contain the collection `job_id` and -status. when you take the id returned and use the collection job -info endpoint, if the job is successful, you will get the status as successful. +status. When you take the id returned and use the `collection job info` endpoint, +if the job is successful, you will get the status as successful. Additionally, if a `callback_url` was provided in the request body, you will receive a message indicating whether the deletion was successful or if it failed. diff --git a/backend/app/api/docs/collections/info.md b/backend/app/api/docs/collections/info.md index f9be78df7..ad862ac52 100644 --- a/backend/app/api/docs/collections/info.md +++ b/backend/app/api/docs/collections/info.md @@ -1,4 +1,4 @@ -Retrieve detailed information about `a specific collection by its ID` from the collection table. This endpoint returns the collection object including its project, organization, -timestamps, and associated LLM service details (`llm_service_id`). +Retrieve detailed information about a specific collection by its collection id. This endpoint returns the collection object including its project, organization, +timestamps, and associated LLM service details (`llm_service_id` and `llm_service_name`). -Additionally, if the `include_docs` flag in the request body is true then you will get a list of document IDs associated with a given collection as well. Documents returned are not only stored by the AI platform, but also by OpenAI. +Additionally, if the `include_docs` flag in the request body is true then you will get a list of document IDs associated with a given collection as well. Note that, documents returned are not only stored by the AI platform, but also by OpenAI. diff --git a/backend/app/api/docs/collections/job_info.md b/backend/app/api/docs/collections/job_info.md index 34d9a342e..ef5589c2c 100644 --- a/backend/app/api/docs/collections/job_info.md +++ b/backend/app/api/docs/collections/job_info.md @@ -3,7 +3,7 @@ Retrieve information about a collection job by the collection job ID. This endpo * Fetching the collection job object, including the collection job ID, the current status, and the associated collection details. * If the job has finished, has been successful and it was a job of creation of collection then this endpoint will fetch the associated collection details from the collection table, including: - - `llm_service_id`: The OpenAI assistant or model used for the collection. + - `llm_service_id` and `llm_service_name`. - Collection metadata such as ID, project, organization, and timestamps. -* If the delete-collection job succeeds, the status is set to “successful” and the `collection_key` contains the ID of the collection that has been deleted. +* If the delete-collection job succeeds, the status is set to “successful” and the `collection` key contains the ID of the collection that has been deleted. diff --git a/backend/app/api/docs/collections/list.md b/backend/app/api/docs/collections/list.md index eec8f312b..cabcd7c61 100644 --- a/backend/app/api/docs/collections/list.md +++ b/backend/app/api/docs/collections/list.md @@ -3,4 +3,4 @@ not deleted If a vector store was created - `llm_service_name` and `llm_service_id` in the response denote the name of the vector store (eg. 'openai vector store') and its id. -[To be deprecated] If an assistant was created, `llm_service_name` and `llm_service_id` in the response denote the name of the model used in the assistant (eg. 'gpt-4o') and assistant id. +[To be deprecated] If an assistant was created, `llm_service_name` and `llm_service_id` in the response denotes the name of the model used in the assistant (eg. 'gpt-4o') and assistant id. diff --git a/backend/app/api/docs/config/create.md b/backend/app/api/docs/config/create.md new file mode 100644 index 000000000..d3c8ff15e --- /dev/null +++ b/backend/app/api/docs/config/create.md @@ -0,0 +1,34 @@ +Create a new LLM configuration with an initial version. + +Configurations allow you to store and manage reusable LLM parameters +(such as temperature, max_tokens, model selection, etc.) with version control. + +**Key Features:** +* Automatically creates an initial version (v1) with the provided configuration +* Enforces unique configuration names per project +* Stores provider-specific parameters as flexible JSON (config_blob) +* Supports optional commit messages for tracking changes +* Provider-agnostic storage - params are passed through to the provider as-is + + +**Example for the config blob: OpenAI Responses API with File Search** + +```json +"config_blob": { + "completion": { + "provider": "openai", + "params": { + "model": "gpt-4o-mini", + "instructions": "You are a helpful assistant for farming communities...", + "temperature": 1, + "tools": [ + { + "type": "file_search", + "vector_store_ids": ["vs_692d71f3f5708191b1c46525f3c1e196"], + "max_num_results": 20 + }]}}} +``` + +The configuration name must be unique within your project. Once created, +you can create additional versions to track parameter changes while +maintaining the configuration history. diff --git a/backend/app/api/docs/config/create_version.md b/backend/app/api/docs/config/create_version.md new file mode 100644 index 000000000..683451a3a --- /dev/null +++ b/backend/app/api/docs/config/create_version.md @@ -0,0 +1,7 @@ +Create a new version for an existing configuration. + +To create a new version, provide the `config_id` in the URL path and the new +configuration parameters in the request body. The system will automatically +create a new version under the same configuration with an incremented version number. +Version numbers are automatically incremented sequentially (1, 2, 3, etc.) +and cannot be manually set or skipped. diff --git a/backend/app/api/docs/config/delete.md b/backend/app/api/docs/config/delete.md new file mode 100644 index 000000000..93303c2e0 --- /dev/null +++ b/backend/app/api/docs/config/delete.md @@ -0,0 +1,5 @@ +Delete a configuration and all its versions. + +This operation performs a delete, marking the configuration and all +associated versions as deleted in the database while retaining records +for audit purposes. diff --git a/backend/app/api/docs/config/delete_version.md b/backend/app/api/docs/config/delete_version.md new file mode 100644 index 000000000..4faa74718 --- /dev/null +++ b/backend/app/api/docs/config/delete_version.md @@ -0,0 +1,4 @@ +Delete a specific version of a configuration. + +Performs a delete on the version, marking it as deleted while +retaining the record for audit purposes. diff --git a/backend/app/api/docs/config/get.md b/backend/app/api/docs/config/get.md new file mode 100644 index 000000000..3421c3be9 --- /dev/null +++ b/backend/app/api/docs/config/get.md @@ -0,0 +1,5 @@ +Retrieve a specific configuration by its ID. + +Returns the configuration metadata including name, description, and +timestamps. This endpoint provides configuration-level details but does +not include version information. diff --git a/backend/app/api/docs/config/get_version.md b/backend/app/api/docs/config/get_version.md new file mode 100644 index 000000000..15b83f49c --- /dev/null +++ b/backend/app/api/docs/config/get_version.md @@ -0,0 +1,4 @@ +Retrieve a specific version of a configuration. + +Returns the complete version details including the full configuration +blob (config_blob) with all LLM parameters. diff --git a/backend/app/api/docs/config/list.md b/backend/app/api/docs/config/list.md new file mode 100644 index 000000000..c4cc3769b --- /dev/null +++ b/backend/app/api/docs/config/list.md @@ -0,0 +1,5 @@ +Retrieve all configurations for the current project. + +Returns a paginated list of configurations ordered by most recently +updated first. Each configuration includes metadata (name, description, +timestamps) but excludes version details for performance. diff --git a/backend/app/api/docs/config/list_versions.md b/backend/app/api/docs/config/list_versions.md new file mode 100644 index 000000000..a77d7d4d6 --- /dev/null +++ b/backend/app/api/docs/config/list_versions.md @@ -0,0 +1,4 @@ +List all versions for a specific configuration. + +Returns versions in descending order (newest first), allowing you to +see the evolution of configuration parameters over time. diff --git a/backend/app/api/docs/config/update.md b/backend/app/api/docs/config/update.md new file mode 100644 index 000000000..c93ce8486 --- /dev/null +++ b/backend/app/api/docs/config/update.md @@ -0,0 +1,5 @@ +Update a configuration's metadata (name or description). + +This endpoint modifies only the configuration-level metadata, not the +LLM parameters themselves. To change LLM parameters, create a new +version instead. diff --git a/backend/app/api/docs/documents/permanent_delete.md b/backend/app/api/docs/documents/permanent_delete.md index ca875a2eb..b179b1fe7 100644 --- a/backend/app/api/docs/documents/permanent_delete.md +++ b/backend/app/api/docs/documents/permanent_delete.md @@ -1,4 +1,6 @@ -This operation soft deletes the document — meaning its metadata and reference are retained in the database, but it is marked as deleted. The actual file stored in cloud storage (e.g., S3) is permanently deleted, and this action is irreversible. +This operation marks the document as deleted in the database while retaining its metadata. However, the actual file is +permanently deleted from cloud storage (e.g., S3) and cannot be recovered. Only the database record remains for reference +purposes. If the document is part of an active collection, those collections will be deleted using the collections delete interface. Noteably, this means all OpenAI Vector Store's and Assistant's to which this document diff --git a/backend/app/api/docs/llm/llm_call.md b/backend/app/api/docs/llm/llm_call.md new file mode 100644 index 000000000..86513bc06 --- /dev/null +++ b/backend/app/api/docs/llm/llm_call.md @@ -0,0 +1,42 @@ +Make an LLM API call using either a stored configuration or an ad-hoc configuration. + +This endpoint initiates an asynchronous LLM call job. The request is queued +for processing, and results are delivered via the callback URL when complete. + +### Request Fields + +**`query`** (required) - Query parameters for this LLM call: +- `input` (required, string, min 1 char): User question/prompt/query +- `conversation` (optional, object): Conversation configuration + - `id` (optional, string): Existing conversation ID to continue + - `auto_create` (optional, boolean, default false): Create new conversation if no ID provided + - **Note**: Cannot specify both `id` and `auto_create=true` + +**`config`** (required) - Configuration for the LLM call (just choose one mode): + +- **Mode 1: Stored Configuration** + - `id` (UUID): Configuration ID + - `version` (integer >= 1): Version number + - **Both required together** + - **Note**: When using stored configuration, do not include the `blob` field in the request body + +- **Mode 2: Ad-hoc Configuration** + - `blob` (object): Complete configuration object (see Create Config endpoint documentation for examples) + - `completion` (required): + - `provider` (required, string): Currently only "openai" + - `params` (required, object): Provider-specific parameters (flexible JSON) + - **Note**: When using ad-hoc configuration, do not include `id` and `version` fields + +**`callback_url`** (optional, HTTPS URL): +- Webhook endpoint to receive the response +- Must be a valid HTTPS URL +- If not provided, response is only accessible through job status + +**`include_provider_raw_response`** (optional, boolean, default false): +- When true, includes the unmodified raw response from the LLM provider + +**`request_metadata`** (optional, object): +- Custom JSON metadata +- Passed through unchanged in the response + +--- diff --git a/backend/app/api/docs/onboarding/onboarding.md b/backend/app/api/docs/onboarding/onboarding.md index 75cae3b95..b6816c60a 100644 --- a/backend/app/api/docs/onboarding/onboarding.md +++ b/backend/app/api/docs/onboarding/onboarding.md @@ -18,10 +18,31 @@ --- -## 🔑 OpenAI API Key (Optional) -- If provided, the API key will be **encrypted** and stored as project credentials. -- If omitted, the project will be created **without OpenAI credentials**. - +## 🔑 Credentials (Optional) +- If provided, the given credentials will be **encrypted** and stored as project credentials. +- The `credential` parameter accepts a list of one or more credentials (e.g., an OpenAI key, Langfuse credentials, etc.). +- If omitted, the project will be created **without credentials**. +- We’ve also included a list of the providers currently supported by kaapi. + ### Example: For sending multiple credentials - + ``` + "credentials": [ + { + "openai": { + "api_key": "sk-proj-..." + } + }, + { + "langfuse": { + "public_key": "pk-lf-...", + "secret_key": "sk-lf-...", + "host": "https://cloud.langfuse.com" + } + } + ] + ``` + ### Supported Providers + - openai + - langfuse --- ## 🔄 Transactional Guarantee diff --git a/backend/app/api/routes/config/config.py b/backend/app/api/routes/config/config.py index cdb71e981..18c3ca84e 100644 --- a/backend/app/api/routes/config/config.py +++ b/backend/app/api/routes/config/config.py @@ -12,7 +12,7 @@ ConfigVersion, Message, ) -from app.utils import APIResponse +from app.utils import APIResponse, load_description from app.api.permissions import Permission, require_permission router = APIRouter() @@ -20,6 +20,7 @@ @router.post( "/", + description=load_description("config/create.md"), response_model=APIResponse[ConfigWithVersion], status_code=201, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], @@ -44,6 +45,7 @@ def create_config( @router.get( "/", + description=load_description("config/list.md"), response_model=APIResponse[list[ConfigPublic]], status_code=200, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], @@ -67,6 +69,7 @@ def list_configs( @router.get( "/{config_id}", + description=load_description("config/get.md"), response_model=APIResponse[ConfigPublic], status_code=200, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], @@ -88,6 +91,7 @@ def get_config( @router.patch( "/{config_id}", + description=load_description("config/update.md"), response_model=APIResponse[ConfigPublic], status_code=200, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], @@ -113,6 +117,7 @@ def update_config( @router.delete( "/{config_id}", + description=load_description("config/delete.md"), response_model=APIResponse[Message], status_code=200, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], diff --git a/backend/app/api/routes/config/version.py b/backend/app/api/routes/config/version.py index aef49ae41..48246dc85 100644 --- a/backend/app/api/routes/config/version.py +++ b/backend/app/api/routes/config/version.py @@ -9,7 +9,7 @@ Message, ConfigVersionItems, ) -from app.utils import APIResponse +from app.utils import APIResponse, load_description from app.api.permissions import Permission, require_permission router = APIRouter() @@ -17,6 +17,7 @@ @router.post( "/{config_id}/versions", + description=load_description("config/create_version.md"), response_model=APIResponse[ConfigVersionPublic], status_code=201, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], @@ -43,6 +44,7 @@ def create_version( @router.get( "/{config_id}/versions", + description=load_description("config/list_versions.md"), response_model=APIResponse[list[ConfigVersionItems]], status_code=200, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], @@ -72,6 +74,7 @@ def list_versions( @router.get( "/{config_id}/versions/{version_number}", + description=load_description("config/get_version.md"), response_model=APIResponse[ConfigVersionPublic], status_code=200, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], @@ -98,6 +101,7 @@ def get_version( @router.delete( "/{config_id}/versions/{version_number}", + description=load_description("config/delete_version.md"), response_model=APIResponse[Message], status_code=200, dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], diff --git a/backend/app/api/routes/documents.py b/backend/app/api/routes/documents.py index a1c2d89dd..ec046aacb 100644 --- a/backend/app/api/routes/documents.py +++ b/backend/app/api/routes/documents.py @@ -109,7 +109,7 @@ async def upload_doc( None, description="Name of the transformer to apply when converting." ), callback_url: str - | None = Form(None, description="URL to call to report endpoint status"), + | None = Form(None, description="URL to call to report doc transformation status"), ): source_format, actual_transformer = pre_transform_validation( src_filename=src.filename, diff --git a/backend/app/api/routes/llm.py b/backend/app/api/routes/llm.py index 26c9ee423..c443941cb 100644 --- a/backend/app/api/routes/llm.py +++ b/backend/app/api/routes/llm.py @@ -5,7 +5,7 @@ from app.api.deps import AuthContextDep, SessionDep from app.models import LLMCallRequest, LLMCallResponse, Message from app.services.llm.jobs import start_job -from app.utils import APIResponse +from app.utils import APIResponse, load_description logger = logging.getLogger(__name__) @@ -32,6 +32,7 @@ def llm_callback_notification(body: APIResponse[LLMCallResponse]): @router.post( "/llm/call", + description=load_description("llm/llm_call.md"), response_model=APIResponse[Message], callbacks=llm_callback_router.routes, ) diff --git a/backend/app/api/routes/onboarding.py b/backend/app/api/routes/onboarding.py index 48b32e0ef..f081c4010 100644 --- a/backend/app/api/routes/onboarding.py +++ b/backend/app/api/routes/onboarding.py @@ -23,4 +23,9 @@ def onboard_project_route( current_user: User = Depends(get_current_active_superuser), ): response = onboard_project(session=session, onboard_in=onboard_in) - return APIResponse.success_response(data=response) + + metadata = None + if onboard_in.credentials: + metadata = {"note": ("Given credential(s) have been saved for this project.")} + + return APIResponse.success_response(data=response, metadata=metadata) diff --git a/backend/app/crud/onboarding.py b/backend/app/crud/onboarding.py index 26777d42e..7cfdddb6f 100644 --- a/backend/app/crud/onboarding.py +++ b/backend/app/crud/onboarding.py @@ -2,7 +2,7 @@ from fastapi import HTTPException from sqlmodel import Session -from app.core.security import encrypt_api_key, encrypt_credentials, get_password_hash +from app.core.security import encrypt_credentials, get_password_hash from app.crud import ( api_key_manager, get_organization_by_name, @@ -102,28 +102,35 @@ def onboard_project( session.add(api_key) - credential = None - if onboard_in.openai_api_key: - creds = {"api_key": onboard_in.openai_api_key} - encrypted_credentials = encrypt_credentials(creds) - credential = Credential( - organization_id=organization.id, - project_id=project.id, - is_active=True, - provider="openai", - credential=encrypted_credentials, - ) - session.add(credential) + created_credentials: list[Credential] = [] - session.commit() + if onboard_in.credentials: + for item in onboard_in.credentials: + (provider_str,) = item.keys() + values = item[provider_str] + + encrypted_credentials = encrypt_credentials(values) + + cred_row = Credential( + organization_id=organization.id, + project_id=project.id, + is_active=True, + provider=provider_str, + credential=encrypted_credentials, + ) + session.add(cred_row) - openai_creds_id = credential.id if credential else None + created_credentials.append(cred_row) + + session.commit() + cred_ids = [c.id for c in created_credentials] logger.info( "[onboard_project] Onboarding completed successfully. " f"org_id={organization.id}, project_id={project.id}, user_id={user.id}, " - f"openai_creds_id={openai_creds_id}" + f"cred_ids={cred_ids}" ) + return OnboardingResponse( organization_id=organization.id, organization_name=organization.name, diff --git a/backend/app/models/onboarding.py b/backend/app/models/onboarding.py index c49963762..43b588b98 100644 --- a/backend/app/models/onboarding.py +++ b/backend/app/models/onboarding.py @@ -1,7 +1,11 @@ import re import secrets +from typing import Any + from sqlmodel import SQLModel, Field -from pydantic import EmailStr, model_validator +from pydantic import EmailStr, model_validator, field_validator + +from app.core.providers import validate_provider, validate_provider_credentials class OnboardingRequest(SQLModel): @@ -48,11 +52,9 @@ class OnboardingRequest(SQLModel): min_length=3, max_length=50, ) - openai_api_key: str | None = Field( + credentials: list[dict[str, Any]] | None = Field( default=None, - description="Optional OpenAI API key to link with this project", - min_length=20, - max_length=256, + description="Optional credential(s) to link with the project", ) @staticmethod @@ -80,6 +82,50 @@ def set_defaults(self): self.password = secrets.token_urlsafe(12) return self + @field_validator("credentials") + @classmethod + def _validate_credential_list(cls, v: list[dict[str, dict[str, str]]] | None): + if v is None: + return v + + if not isinstance(v, list): + raise TypeError( + "credential must be a list of single-key dicts (e.g., {'openai': {...}})." + ) + + errors: list[str] = [] + + for idx, item in enumerate(v): + try: + if not isinstance(item, dict): + raise TypeError( + "must be a dict with a single provider key like {'openai': {...}}." + ) + if len(item) != 1: + raise ValueError( + "must have exactly one provider key like {'openai': {...}}." + ) + + (provider_key,) = item.keys() + values = item[provider_key] + + validate_provider(provider_key) + + if not isinstance(values, dict): + raise TypeError( + f"value for provider '{provider_key}' must be an object/dict." + ) + + validate_provider_credentials(provider_key, values) + + except (TypeError, ValueError) as e: + errors.append(f"[{idx}] {e}") + + if errors: + raise ValueError("credential validation failed:\n" + "\n".join(errors)) + + return v + class OnboardingResponse(SQLModel): """ diff --git a/backend/app/tests/api/routes/test_onboarding.py b/backend/app/tests/api/routes/test_onboarding.py index e275fd3a4..0b3f14971 100644 --- a/backend/app/tests/api/routes/test_onboarding.py +++ b/backend/app/tests/api/routes/test_onboarding.py @@ -1,7 +1,6 @@ from fastapi.testclient import TestClient from sqlmodel import Session -from app.utils import mask_string from app.core.config import settings from app.tests.utils.utils import random_email, random_lower_string from app.tests.utils.test_data import create_test_organization @@ -17,6 +16,9 @@ def test_onboard_project_new_organization_project_user( password = random_lower_string() user_name = "Test User Onboard" openai_key = f"sk-{random_lower_string()}" + langfuse_secret_key = f"sk-lf-{random_lower_string()}" + langfuse_public_key = f"pk-lf-{random_lower_string()}" + langfuse_host = "https://cloud.langfuse.com" onboard_data = { "organization_name": org_name, @@ -24,7 +26,16 @@ def test_onboard_project_new_organization_project_user( "email": email, "password": password, "user_name": user_name, - "openai_api_key": openai_key, + "credentials": [ + {"openai": {"api_key": openai_key}}, + { + "langfuse": { + "secret_key": langfuse_secret_key, + "public_key": langfuse_public_key, + "host": langfuse_host, + } + }, + ], } response = client.post( @@ -158,3 +169,163 @@ def test_onboard_project_with_auto_generated_defaults( assert "@kaapi.org" in data["user_email"] assert "api_key" in data assert len(data["api_key"]) > 0 + + +def test_onboard_project_invalid_provider( + client: TestClient, superuser_token_headers: dict[str, str], db: Session +) -> None: + """Test onboarding fails when an unsupported provider is specified.""" + org_name = "TestOrgOnboard" + project_name = "TestProjectOnboard" + email = random_email() + password = random_lower_string() + + onboard_data = { + "organization_name": org_name, + "project_name": project_name, + "email": email, + "password": password, + "user_name": "User", + "credentials": [{"totally_not_a_provider": {"foo": "bar"}}], + } + + response = client.post( + f"{settings.API_V1_STR}/onboard", + json=onboard_data, + headers=superuser_token_headers, + ) + + assert response.status_code == 422 + error_response = response.json() + assert "error" in error_response + assert "credential validation failed" in error_response["error"] + + +def test_onboard_project_non_dict_values_in_credential( + client: TestClient, superuser_token_headers: dict[str, str], db: Session +) -> None: + """Test onboarding fails when credential value for a provider is not an object/dict.""" + org_name = "TestOrgOnboard" + project_name = "TestProjectOnboard" + email = random_email() + password = random_lower_string() + + onboard_data = { + "organization_name": org_name, + "project_name": project_name, + "email": email, + "password": password, + "user_name": "User", + "credentials": [{"openai": "sk-should-be-inside-object"}], + } + + response = client.post( + f"{settings.API_V1_STR}/onboard", + json=onboard_data, + headers=superuser_token_headers, + ) + + assert response.status_code == 422 + error_response = response.json() + assert "error" in error_response + assert "credential validation failed" in error_response["error"] + assert "must be an object/dict" in error_response["error"] + + +def test_onboard_project_missing_required_fields_for_openai( + client: TestClient, superuser_token_headers: dict[str, str], db: Session +) -> None: + """Test onboarding fails when OpenAI credential is missing required fields.""" + org_name = "TestOrgOnboard" + project_name = "TestProjectOnboard" + email = random_email() + password = random_lower_string() + + onboard_data = { + "organization_name": org_name, + "project_name": project_name, + "email": email, + "password": password, + "user_name": "User", + "credentials": [{"openai": {}}], # missing api_key + } + + response = client.post( + f"{settings.API_V1_STR}/onboard", + json=onboard_data, + headers=superuser_token_headers, + ) + + assert response.status_code == 422 + error_response = response.json() + assert "error" in error_response + assert "credential validation failed" in error_response["error"] + assert "openai" in error_response["error"] + + +def test_onboard_project_missing_required_fields_for_langfuse( + client: TestClient, superuser_token_headers: dict[str, str], db: Session +) -> None: + """Test onboarding fails when Langfuse credential is missing required fields.""" + org_name = "TestOrgOnboard" + project_name = "TestProjectOnboard" + email = random_email() + password = random_lower_string() + + onboard_data = { + "organization_name": org_name, + "project_name": project_name, + "email": email, + "password": password, + "user_name": "User", + "credentials": [ + {"langfuse": {"secret_key": "sk-only"}} + ], # missing public_key/host + } + + response = client.post( + f"{settings.API_V1_STR}/onboard", + json=onboard_data, + headers=superuser_token_headers, + ) + + assert response.status_code == 422 + error_response = response.json() + assert "error" in error_response + assert "credential validation failed" in error_response["error"] + assert "langfuse" in error_response["error"] + + +def test_onboard_project_aggregates_multiple_credential_errors( + client: TestClient, superuser_token_headers: dict[str, str], db: Session +) -> None: + """Test onboarding aggregates multiple credential validation errors with index markers.""" + org_name = "TestOrgOnboard" + project_name = "TestProjectOnboard" + email = random_email() + password = random_lower_string() + + onboard_data = { + "organization_name": org_name, + "project_name": project_name, + "email": email, + "password": password, + "user_name": "User", + "credentials": [ + {"notreal": {"x": "y"}}, + {"openai": "should-be-dict"}, + ], + } + + response = client.post( + f"{settings.API_V1_STR}/onboard", + json=onboard_data, + headers=superuser_token_headers, + ) + + assert response.status_code == 422 + error_response = response.json() + assert "error" in error_response + assert "credential validation failed" in error_response["error"] + assert "[0]" in error_response["error"] + assert "[1]" in error_response["error"] diff --git a/backend/app/tests/crud/test_onboarding.py b/backend/app/tests/crud/test_onboarding.py index 613669d1a..a082fa75e 100644 --- a/backend/app/tests/crud/test_onboarding.py +++ b/backend/app/tests/crud/test_onboarding.py @@ -38,7 +38,7 @@ def test_onboard_project_new_organization_project_user(db: Session) -> None: email=email, password=password, user_name=user_name, - openai_api_key=openai_key, + credentials=[{"openai": {"api_key": openai_key}}], ) response = onboard_project(session=db, onboard_in=onboard_request) @@ -246,7 +246,7 @@ def test_onboard_project_response_data_integrity(db: Session) -> None: email=email, password=password, user_name=user_name, - openai_api_key=openai_key, + credential=[{"openai": {"api_key": openai_key}}], ) response = onboard_project(session=db, onboard_in=onboard_request)