Collection : with assistant flag and additional enhancements

[nishika26]

Summary

Target issue is #398 and also covering this issue - #231 since this is just for writing test cases for existing router endpoints

Checklist

Before submitting a pull request, please ensure that you mark these task.

• Ran fastapi run --reload app/main.py or docker compose up in the repository root and test.
• If you've fixed a bug or added code that is tested and has test cases.

Notes

for reference of request bodies of the job related endpoints of this module you can check this document out.

Create job (create_collection.execute_job) :

• Introduced with_assistant flag: when model & instructions are both present, create an Assistant backed by a Vector Store; otherwise only create a Vector Store.

• We now close the DB session before making external API calls—and reopen a fresh session only to persist results—so we don’t hold connections or risk transaction locks. Same is being done during deletion of collection.

Delete job (delete_collection.execute_job) :

• Detects service type from collection.llm_service_name (Assistant vs Vector Store) and chooses the correct CRUD (OpenAIAssistantCrud / OpenAIVectorStoreCrud) to use delete method of that specific CRUD class.

• Calls local deletion via CollectionCrud.delete_by_id() once remote deletion succeeds.

CRUD refactors :

• CollectionCrud.delete_by_id() sets deleted_at=now() and persists.

• Document→Collection cascade delete now excludes already-deleted collections (deleted_at IS NULL).

API route :

• /collections/create: determines with_assistant by checking both model and instructions. Returns metadata note when creating vector-store-only.

• Validation: if only one of model or instructions is provided, responds 422 with a clear message.

Additional :

• Removed ResponsePayload class because we are now returning collection job object in response of create and delete collections, we use the collection job object for the callback URL as well.

• Rephrased the endpoint docs to remove the indication that user can use any specific endpoint for polling.

Summary by CodeRabbit

• New Features

  • Unified Collections API: list, get (now supports include_docs, skip, limit), create and delete return immediate job metadata, support optional callbacks, and expose new public response shapes including collection-with-docs.

• Bug Fixes

  • Improved cleanup and failure handling, consistent job states, reads ignore deleted items, and deletions correctly route to the appropriate remote backend.

• Documentation

  • Clarified assistant vs vector-only creation, job semantics, callbacks and polling guidance.

• Tests

  • Expanded coverage for create/delete jobs, callbacks, listing, pagination and helper utilities.