Skip to content

Conversation

@Aditya30ag
Copy link
Contributor

@Aditya30ag Aditya30ag commented Oct 30, 2025

πŸš€ Ready to Use
The feature is production-ready and fully integrated:

βœ… Backend routes registered
βœ… Frontend routing configured
βœ… API endpoints connected
βœ… Tests written
βœ… Documentation complete
βœ… Follows all PictoPy conventions

#602

Summary by CodeRabbit

  • New Features

    • Memories page with four views: On This Day, Recent Highlights, People, and Themes (tags).
    • Memory cards showing image grids with expandable Show More/Show Less and image-count badges.
    • Queryable filters for time range, minimum images, and result limits.
  • API

    • New endpoints to fetch memories by category and an aggregated "all" endpoint.
  • Docs & Tests

    • User docs added and comprehensive endpoint tests included.

Signed-off-by: Aditya30ag <adityaagrwal3005@gmail.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 30, 2025

Walkthrough

Adds a Memories feature: backend SQLite retrieval utilities and FastAPI routes, Pydantic schemas, frontend API helpers, React components/pages, types, tests, OpenAPI docs, and feature documentation.

Changes

Cohort / File(s) Summary
Backend β€” DB utilities
backend/app/database/memories.py
New SQLite-backed retrieval functions: db_get_memories_on_this_day(), db_get_recent_memories(), db_get_memories_by_people(), db_get_memories_by_tags() β€” metadata parsing, grouping, deduplication, error logging, connection management.
Backend β€” API & Schemas
backend/app/routes/memories.py, backend/app/schemas/memories.py
New FastAPI router router with endpoints /on-this-day, /recent, /people, /tags, /all and corresponding Pydantic models & response wrappers (OnThisDayMemory, RecentMemory, PersonMemory, TagMemory, AllMemoriesResponse, ErrorResponse).
Backend β€” Integration & Tests
backend/main.py, backend/tests/test_memories.py
Router registered at /memories; comprehensive endpoint tests covering success cases, parameter validation, limits, and error responses.
Frontend β€” API endpoints & functions
frontend/src/api/apiEndpoints.ts, frontend/src/api/api-functions/memories.ts, frontend/src/api/api-functions/index.ts
Adds memoriesEndpoints paths and five fetch helpers (fetchAllMemories, fetchOnThisDayMemories, fetchRecentMemories, fetchPeopleMemories, fetchTagMemories) and re-exports.
Frontend β€” Components & Page
frontend/src/components/Memories/MemoryCard.tsx, frontend/src/components/Memories/MemorySection.tsx, frontend/src/pages/Memories/Memories.tsx
New MemoryCard (thumbnail grid, show more, dispatches image viewer actions) and MemorySection (titled section, empty state); full Memories page rendering sections and fetching data.
Frontend β€” Routing & Types
frontend/src/routes/AppRoutes.tsx, frontend/src/types/Memory.ts
MEMORIES route now renders Memories page; TypeScript interfaces for memory shapes and MemoriesResponse added.
Docs & OpenAPI
docs/backend/backend_python/openapi.json, docs/features/memories.md, docs/stylesheets/output.css
OpenAPI updated with new endpoints and schemas; feature documentation added; trivial CSS formatting tweak.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant Frontend
    participant API
    participant DB
    participant ImageUtil

    User->>Frontend: Open Memories page
    Frontend->>API: GET /memories/all (params)
    API->>DB: db_get_memories_on_this_day(years_back)
    DB->>ImageUtil: parse image metadata
    ImageUtil-->>DB: metadata
    DB-->>API: on_this_day cluster list

    API->>DB: db_get_recent_memories(days,min_images)
    DB->>ImageUtil: parse image metadata
    DB-->>API: recent clusters

    API->>DB: db_get_memories_by_people(limit)
    DB->>ImageUtil: parse image metadata
    DB-->>API: people clusters

    API->>DB: db_get_memories_by_tags(limit)
    DB->>ImageUtil: parse image metadata
    DB-->>API: tag clusters

    API-->>Frontend: AllMemoriesResponse
    Frontend->>Frontend: Render MemorySection + MemoryCard components
    Frontend-->>User: Display memories
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

  • Areas to inspect:
    • SQL/grouping/deduplication and connection handling in backend/app/database/memories.py
    • Consistency between Pydantic schemas and returned data shapes
    • Error handling and logging paths in routes and DB utilities
    • Image metadata parsing usage (image_util) and thumbnail URL handling
    • MemoryCard dispatches and integration with Redux image viewer

Possibly related PRs

Suggested labels

enhancement, backend, frontend, UI, documentation

Suggested reviewers

  • rahulharpal1603

Poem

🐰
I hopped through rows of days and tags,
Collected moments, tail and flags,
From years ago to faces shown,
I stitched a garden, pixel-grown,
Hopβ€”memories bloom, now brightly logged.

Pre-merge checks and finishing touches

βœ… Passed checks (3 passed)
Check name Status Explanation
Description Check βœ… Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check βœ… Passed The title "added the new memory feature" is directly related to the changeset's main objective of introducing a comprehensive Memories feature across both backend and frontend layers, including database queries, API endpoints, schemas, tests, React components, pages, and routing configurations. The title clearly communicates what was addedβ€”a new memory/memories featureβ€”without being misleading or employing vague terms like "misc updates" or "stuff." While it could be slightly more specific about the scope of changes (backend and frontend integration), it effectively conveys the primary change in concise language that teammates would readily understand when scanning commit history.
Docstring Coverage βœ… Passed Docstring coverage is 95.45% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • πŸ“ Generate docstrings
πŸ§ͺ Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (7)
docs/stylesheets/output.css (1)

1-608: Clarify version control strategy for generated CSS output.

This file (output.css) appears to be generated output (Tailwind v3.4.1), and the only change is a formatting adjustment on line 608 (closing brace repositioning). Generated files typically should not require manual edits or commits.

A few questions:

  • Is this output file intentionally committed to version control (e.g., for docs distribution)?
  • Should this change have been generated automatically by a build step rather than manually committed?
  • Is the formatting change a result of running the Tailwind/build process?

If this is auto-generated, consider adding it to .gitignore and regenerating it during the build/deploy process instead.

docs/features/memories.md (1)

54-54: Add language specifiers to code blocks for better rendering.

The fenced code blocks for API endpoints should specify the language (e.g., http or text) to enable proper syntax highlighting and improve documentation rendering.

Apply this pattern to add language specifiers:

-```
+```http
 GET /memories/all

Repeat for lines 83, 92, 102, and 111.


Also applies to: 83-83, 92-92, 102-102, 111-111

</blockquote></details>
<details>
<summary>frontend/src/pages/Memories/Memories.tsx (2)</summary><blockquote>

`44-44`: **Fix redundant gradient colors.**

The gradient is defined with the same color for both start and end (`from-gray-500 to-gray-500`), which produces no gradient effect. Either use a single background color or define a proper gradient with different colors.



Apply this diff to use a proper gradient or solid color:

```diff
-          <div className="rounded-full bg-gradient-to-br from-gray-500 to-gray-500 p-3">
+          <div className="rounded-full bg-gray-500 p-3">

Or use an actual gradient:

-          <div className="rounded-full bg-gradient-to-br from-gray-500 to-gray-500 p-3">
+          <div className="rounded-full bg-gradient-to-br from-purple-500 to-pink-500 p-3">

59-77: Remove redundant isEmpty props.

Each MemorySection is rendered inside a conditional that checks memoriesData.X.length > 0, so the isEmpty={memoriesData.X.length === 0} prop will always be false. This makes the prop redundant and potentially confusing.

Remove the isEmpty prop from all four sections since the outer conditional already ensures the arrays are non-empty:

 {memoriesData?.on_this_day && memoriesData.on_this_day.length > 0 && (
   <MemorySection
     title="On This Day"
     description="Photos from this day in previous years"
-    isEmpty={memoriesData.on_this_day.length === 0}
     emptyMessage="No memories from this day in previous years"
   >

Apply the same change to lines 84, 104, and 125 for the other sections.

Also applies to: 80-97, 100-118, 121-142

frontend/src/api/api-functions/memories.ts (1)

19-49: Restore type-safety for memory fetchers.

We already exposed rich memory models; returning Promise<any> hides schema drift and forces callers to retype the payloads by hand. Please hook up the concrete response types for each endpoint so mistakes surface at compile time.

-import { MemoriesResponse } from '@/types/Memory';
+import {
+  MemoriesResponse,
+  OnThisDayMemory,
+  RecentMemory,
+  PersonMemory,
+  TagMemory,
+} from '@/types/Memory';
+
+type MemoryListResponse<T> = {
+  success: boolean;
+  message: string;
+  data: T[];
+};
@@
-export const fetchOnThisDayMemories = async (
-  years_back: number = 5,
-): Promise<any> => {
-  const response = await apiClient.get(memoriesEndpoints.onThisDay, {
+export const fetchOnThisDayMemories = async (
+  years_back: number = 5,
+): Promise<MemoryListResponse<OnThisDayMemory>> => {
+  const response = await apiClient.get<MemoryListResponse<OnThisDayMemory>>(
+    memoriesEndpoints.onThisDay,
+    {
       params: { years_back },
-  });
+    },
+  );
   return response.data;
 };
@@
-export const fetchRecentMemories = async (
-  days: number = 30,
-  min_images: number = 5,
-): Promise<any> => {
-  const response = await apiClient.get(memoriesEndpoints.recent, {
+export const fetchRecentMemories = async (
+  days: number = 30,
+  min_images: number = 5,
+): Promise<MemoryListResponse<RecentMemory>> => {
+  const response = await apiClient.get<MemoryListResponse<RecentMemory>>(
+    memoriesEndpoints.recent,
+    {
       params: { days, min_images },
-  });
+    },
+  );
   return response.data;
 };
@@
-export const fetchPeopleMemories = async (limit: number = 10): Promise<any> => {
-  const response = await apiClient.get(memoriesEndpoints.people, {
+export const fetchPeopleMemories = async (
+  limit: number = 10,
+): Promise<MemoryListResponse<PersonMemory>> => {
+  const response = await apiClient.get<MemoryListResponse<PersonMemory>>(
+    memoriesEndpoints.people,
+    {
       params: { limit },
-  });
+    },
+  );
   return response.data;
 };
@@
-export const fetchTagMemories = async (limit: number = 10): Promise<any> => {
-  const response = await apiClient.get(memoriesEndpoints.tags, {
+export const fetchTagMemories = async (
+  limit: number = 10,
+): Promise<MemoryListResponse<TagMemory>> => {
+  const response = await apiClient.get<MemoryListResponse<TagMemory>>(
+    memoriesEndpoints.tags,
+    {
       params: { limit },
-  });
+    },
+  );
   return response.data;
 };
docs/backend/backend_python/openapi.json (2)

1759-1820: Consider adding maxItems constraints to memory response arrays.

All memory response schemas (AllMemoriesData, OnThisDayResponse, RecentMemoriesResponse, PeopleMemoriesResponse, TagMemoriesResponse) define unbounded arrays. OpenAPI best practice recommends setting maxItems to prevent unexpectedly large responses and ensure predictable payload sizes.

Verify whether the backend enforces limits via pagination, query parameters (limit/offset), or explicit pagination handling. If limits exist, update the schemas to reflect them.

Also applies to: 2858-3276


2693-2774: Consider consolidating MemoryImageMetadata with existing MetadataModel.

MemoryImageMetadata and MetadataModel are nearly identical but with one key difference: MemoryImageMetadata does not require date_created, while MetadataModel does. This duplication creates maintenance burden and potential confusion.

Options:

  1. Reuse MetadataModel for memory images (if all memory images have creation dates).
  2. Add a clear docstring explaining why MemoryImageMetadata differs (e.g., handle missing dates gracefully).
  3. Create a shared base schema for both.

Verify whether memory images can legitimately lack date_created in the backend.

πŸ“œ Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between 8980d72 and 480037f.

β›” Files ignored due to path filters (1)
  • frontend/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
πŸ“’ Files selected for processing (16)
  • backend/app/database/memories.py (1 hunks)
  • backend/app/routes/memories.py (1 hunks)
  • backend/app/schemas/memories.py (1 hunks)
  • backend/main.py (2 hunks)
  • backend/tests/test_memories.py (1 hunks)
  • docs/backend/backend_python/openapi.json (7 hunks)
  • docs/features/memories.md (1 hunks)
  • docs/stylesheets/output.css (1 hunks)
  • frontend/src/api/api-functions/index.ts (1 hunks)
  • frontend/src/api/api-functions/memories.ts (1 hunks)
  • frontend/src/api/apiEndpoints.ts (1 hunks)
  • frontend/src/components/Memories/MemoryCard.tsx (1 hunks)
  • frontend/src/components/Memories/MemorySection.tsx (1 hunks)
  • frontend/src/pages/Memories/Memories.tsx (1 hunks)
  • frontend/src/routes/AppRoutes.tsx (2 hunks)
  • frontend/src/types/Memory.ts (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (9)
frontend/src/pages/Memories/Memories.tsx (8)
frontend/src/hooks/useQueryExtension.ts (1)
  • usePictoQuery (80-108)
frontend/src/api/api-functions/memories.ts (1)
  • fetchAllMemories (5-17)
frontend/src/features/loaderSlice.ts (2)
  • showLoader (17-20)
  • hideLoader (21-24)
frontend/src/features/infoDialogSlice.ts (1)
  • showInfoDialog (16-30)
backend/app/schemas/memories.py (1)
  • AllMemoriesData (77-81)
frontend/src/types/Memory.ts (1)
  • AllMemoriesData (29-34)
frontend/src/components/Memories/MemorySection.tsx (1)
  • MemorySection (11-42)
frontend/src/components/Memories/MemoryCard.tsx (1)
  • MemoryCard (15-100)
frontend/src/types/Memory.ts (2)
backend/app/schemas/memories.py (5)
  • OnThisDayMemory (27-31)
  • RecentMemory (34-37)
  • PersonMemory (40-44)
  • TagMemory (47-50)
  • AllMemoriesData (77-81)
frontend/src/types/Media.ts (1)
  • Image (13-22)
backend/app/schemas/memories.py (1)
frontend/src/types/Memory.ts (5)
  • OnThisDayMemory (3-8)
  • RecentMemory (10-14)
  • PersonMemory (16-21)
  • TagMemory (23-27)
  • AllMemoriesData (29-34)
frontend/src/api/api-functions/memories.ts (3)
frontend/src/types/Memory.ts (1)
  • MemoriesResponse (36-40)
frontend/src/api/axiosConfig.ts (1)
  • apiClient (5-12)
frontend/src/api/apiEndpoints.ts (1)
  • memoriesEndpoints (33-39)
frontend/src/routes/AppRoutes.tsx (1)
frontend/src/constants/routes.ts (1)
  • ROUTES (1-11)
backend/tests/test_memories.py (1)
backend/tests/test_folders.py (1)
  • client (76-78)
backend/app/routes/memories.py (3)
backend/app/database/memories.py (4)
  • db_get_memories_on_this_day (20-126)
  • db_get_recent_memories (129-224)
  • db_get_memories_by_people (227-341)
  • db_get_memories_by_tags (344-416)
backend/app/schemas/memories.py (7)
  • OnThisDayResponse (53-56)
  • RecentMemoriesResponse (59-62)
  • PeopleMemoriesResponse (65-68)
  • TagMemoriesResponse (71-74)
  • AllMemoriesResponse (84-87)
  • AllMemoriesData (77-81)
  • ErrorResponse (90-93)
frontend/src/types/Memory.ts (1)
  • AllMemoriesData (29-34)
frontend/src/components/Memories/MemoryCard.tsx (2)
frontend/src/types/Media.ts (1)
  • Image (13-22)
frontend/src/features/imageSlice.ts (2)
  • setImages (18-20)
  • setCurrentViewIndex (22-34)
backend/app/database/memories.py (1)
backend/app/utils/images.py (1)
  • image_util_parse_metadata (496-513)
πŸͺ› Checkov (3.2.334)
docs/backend/backend_python/openapi.json

[medium] 1761-1766: Ensure that arrays have a maximum number of items

(CKV_OPENAPI_21)

πŸͺ› LanguageTool
docs/features/memories.md

[grammar] ~7-~7: Ensure spelling is correct
Context: ...face recognition, object detection, and date analysis to create four types of memori...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

πŸͺ› markdownlint-cli2 (0.18.1)
docs/features/memories.md

54-54: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


83-83: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


92-92: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


102-102: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


111-111: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: Backend Tests
  • GitHub Check: Tauri Build Check (windows-latest)
πŸ”‡ Additional comments (7)
frontend/src/api/api-functions/index.ts (1)

7-7: LGTM!

The re-export follows the established pattern and correctly exposes the memories API functions.

frontend/src/routes/AppRoutes.tsx (1)

11-11: LGTM!

The routing configuration correctly wires the new Memories page, replacing the placeholder ComingSoon component.

Also applies to: 23-23

frontend/src/api/apiEndpoints.ts (1)

33-39: LGTM!

The endpoint definitions follow the established pattern and correctly map to the backend API routes.

backend/main.py (1)

29-29: LGTM!

The memories router is correctly imported and registered following the established pattern for other routers in the application.

Also applies to: 136-136

frontend/src/components/Memories/MemorySection.tsx (1)

1-42: LGTM!

The MemorySection component is well-structured with clear TypeScript types, appropriate empty state handling, and consistent styling with dark mode support.

frontend/src/components/Memories/MemoryCard.tsx (1)

1-100: LGTM!

The MemoryCard component is well-implemented with:

  • Proper state management for expand/collapse functionality
  • Correct Redux integration for image viewer
  • Responsive grid layout with lazy loading
  • Good UX with hover effects and smooth transitions
docs/backend/backend_python/openapi.json (1)

1269-1627: Memory endpoints and schemas are well-structured.

The five new memory endpoints follow existing PictoPy conventions:

  • Consistent response patterns (success/message/data).
  • Appropriate error handling (500/422 status codes with ErrorResponse schema).
  • Clear query parameter constraints (min/max bounds).
  • Comprehensive documentation strings.

Schemas are properly defined and referenced across memory response types. The memory aggregation endpoint (/memories/all) logically combines all four memory types, enabling flexible API consumption.

Also applies to: 1759-1820, 2858-3276

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
backend/app/database/memories.py (1)

393-413: The critical GROUP BY bug from the previous review is still present.

This query still groups by both m.class_id and i.id, which means each group contains exactly one image. Therefore, COUNT(DISTINCT ic.image_id) is always 1, the HAVING image_count >= 5 condition never passes, and this endpoint always returns an empty list.

The fix from the previous review should be applied:

  1. First query: aggregate by class_id to find tags with >= 5 images
  2. Second query: fetch images for those qualifying tags

Refer to the detailed fix in the previous review comment on lines 395-452.

🧹 Nitpick comments (1)
backend/app/routes/memories.py (1)

195-199: Consider resilience for partial failures.

The /all endpoint calls all four database functions sequentially. If any one fails (e.g., db_get_memories_by_tags due to the critical GROUP BY bug), the entire request fails with a 500 error.

For better user experience, consider wrapping each call individually and returning partial results:

try:
    on_this_day = db_get_memories_on_this_day(years_back=years_back)
except Exception as e:
    logger.error(f"Error getting on-this-day memories: {e}")
    on_this_day = []

try:
    recent = db_get_recent_memories(days=recent_days, min_images=min_images)
except Exception as e:
    logger.error(f"Error getting recent memories: {e}")
    recent = []

# ... similar for people and tags

This way, users still get memories from working endpoints even if one category fails.

πŸ“œ Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between 480037f and 5786c4c.

πŸ“’ Files selected for processing (3)
  • backend/app/database/memories.py (1 hunks)
  • backend/app/routes/memories.py (1 hunks)
  • backend/tests/test_memories.py (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • backend/tests/test_memories.py
🧰 Additional context used
🧬 Code graph analysis (2)
backend/app/routes/memories.py (3)
backend/app/database/memories.py (4)
  • db_get_memories_on_this_day (20-137)
  • db_get_recent_memories (140-244)
  • db_get_memories_by_people (247-376)
  • db_get_memories_by_tags (379-462)
backend/app/schemas/memories.py (7)
  • OnThisDayResponse (53-56)
  • RecentMemoriesResponse (59-62)
  • PeopleMemoriesResponse (65-68)
  • TagMemoriesResponse (71-74)
  • AllMemoriesResponse (84-87)
  • AllMemoriesData (77-81)
  • ErrorResponse (90-93)
frontend/src/types/Memory.ts (1)
  • AllMemoriesData (29-34)
backend/app/database/memories.py (1)
backend/app/utils/images.py (1)
  • image_util_parse_metadata (496-513)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Backend Tests
πŸ”‡ Additional comments (4)
backend/app/database/memories.py (2)

14-17: LGTM!

The connection helper correctly enables foreign key constraints, which is essential for maintaining referential integrity in SQLite.


247-376: LGTM!

The two-query approach correctly aggregates face clusters first, then fetches images for qualifying clusters. This avoids the GROUP BY pitfall present in db_get_memories_by_tags.

backend/app/routes/memories.py (2)

1-20: LGTM!

Imports are well-organized and all necessary dependencies are present.


23-161: LGTM!

All individual memory endpoints follow a consistent pattern with proper query parameter validation, error handling, and structured responses.

Comment on lines +41 to +56
cursor.execute(
"""
SELECT
i.id,
i.path,
i.thumbnailPath,
i.metadata,
i.isTagged,
m.name as tag_name
FROM images i
LEFT JOIN image_classes ic ON i.id = ic.image_id
LEFT JOIN mappings m ON ic.class_id = m.class_id
WHERE i.metadata IS NOT NULL
ORDER BY i.metadata
"""
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Optimize query to filter dates in SQL rather than Python.

The query fetches all images with metadata, then filters by month/day in Python (lines 65-123). On large databases, this loads the entire images table into memory and performs unnecessary parsing.

Push the date filtering into SQL using SQLite's strftime function:

-        cursor.execute(
-            """
-            SELECT 
-                i.id,
-                i.path,
-                i.thumbnailPath,
-                i.metadata,
-                i.isTagged,
-                m.name as tag_name
-            FROM images i
-            LEFT JOIN image_classes ic ON i.id = ic.image_id
-            LEFT JOIN mappings m ON ic.class_id = m.class_id
-            WHERE i.metadata IS NOT NULL
-            ORDER BY i.metadata
-            """
-        )
+        cursor.execute(
+            """
+            SELECT 
+                i.id,
+                i.path,
+                i.thumbnailPath,
+                i.metadata,
+                i.isTagged,
+                m.name as tag_name
+            FROM images i
+            LEFT JOIN image_classes ic ON i.id = ic.image_id
+            LEFT JOIN mappings m ON ic.class_id = m.class_id
+            WHERE i.metadata IS NOT NULL
+                AND i.metadata LIKE '%"date_created"%'
+                AND json_extract(i.metadata, '$.date_created') IS NOT NULL
+            ORDER BY i.metadata
+            """
+        )

Note: While full date filtering in SQL requires parsing JSON dates (which SQLite doesn't handle natively for ISO8601), pre-filtering images that have date_created will significantly reduce the result set before Python processing.

πŸ“ Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
cursor.execute(
"""
SELECT
i.id,
i.path,
i.thumbnailPath,
i.metadata,
i.isTagged,
m.name as tag_name
FROM images i
LEFT JOIN image_classes ic ON i.id = ic.image_id
LEFT JOIN mappings m ON ic.class_id = m.class_id
WHERE i.metadata IS NOT NULL
ORDER BY i.metadata
"""
)
cursor.execute(
"""
SELECT
i.id,
i.path,
i.thumbnailPath,
i.metadata,
i.isTagged,
m.name as tag_name
FROM images i
LEFT JOIN image_classes ic ON i.id = ic.image_id
LEFT JOIN mappings m ON ic.class_id = m.class_id
WHERE i.metadata IS NOT NULL
AND i.metadata LIKE '%"date_created"%'
AND json_extract(i.metadata, '$.date_created') IS NOT NULL
ORDER BY i.metadata
"""
)
πŸ€– Prompt for AI Agents
In backend/app/database/memories.py around lines 41 to 56, the SQL currently
selects all rows where metadata IS NOT NULL then does month/day filtering in
Python; change the query to pre-filter rows that contain a date_created in the
JSON and (when month/day are provided) apply SQLite strftime on the
JSON-extracted date to limit results in SQL. Specifically, add a WHERE clause
using json_extract(i.metadata, '$.date_created') IS NOT NULL to drop rows
without dates and, if month/day filters are present, include conditions like
strftime('%m', json_extract(i.metadata, '$.date_created')) = ? and
strftime('%d', json_extract(i.metadata, '$.date_created')) = ? (binding the
month/day params) so fewer rows are loaded into Python for further processing.

Comment on lines +158 to +173
cursor.execute(
"""
SELECT
i.id,
i.path,
i.thumbnailPath,
i.metadata,
i.isTagged,
m.name as tag_name
FROM images i
LEFT JOIN image_classes ic ON i.id = ic.image_id
LEFT JOIN mappings m ON ic.class_id = m.class_id
WHERE i.metadata IS NOT NULL
ORDER BY i.metadata DESC
"""
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Optimize query to filter by cutoff date in SQL.

Similar to db_get_memories_on_this_day, this query fetches all images and filters the date range in Python (line 192). This is inefficient for large databases.

Add a pre-filter to reduce the result set:

         cursor.execute(
             """
             SELECT 
                 i.id,
                 i.path,
                 i.thumbnailPath,
                 i.metadata,
                 i.isTagged,
                 m.name as tag_name
             FROM images i
             LEFT JOIN image_classes ic ON i.id = ic.image_id
             LEFT JOIN mappings m ON ic.class_id = m.class_id
             WHERE i.metadata IS NOT NULL
+                AND i.metadata LIKE '%"date_created"%'
+                AND json_extract(i.metadata, '$.date_created') IS NOT NULL
             ORDER BY i.metadata DESC
             """
         )

While exact date comparison still needs Python (due to ISO8601 parsing), this pre-filter eliminates images without dates from the result set.

πŸ“ Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
cursor.execute(
"""
SELECT
i.id,
i.path,
i.thumbnailPath,
i.metadata,
i.isTagged,
m.name as tag_name
FROM images i
LEFT JOIN image_classes ic ON i.id = ic.image_id
LEFT JOIN mappings m ON ic.class_id = m.class_id
WHERE i.metadata IS NOT NULL
ORDER BY i.metadata DESC
"""
)
cursor.execute(
"""
SELECT
i.id,
i.path,
i.thumbnailPath,
i.metadata,
i.isTagged,
m.name as tag_name
FROM images i
LEFT JOIN image_classes ic ON i.id = ic.image_id
LEFT JOIN mappings m ON ic.class_id = m.class_id
WHERE i.metadata IS NOT NULL
AND i.metadata LIKE '%"date_created"%'
AND json_extract(i.metadata, '$.date_created') IS NOT NULL
ORDER BY i.metadata DESC
"""
)

@rahulharpal1603
Copy link
Contributor

#777 will be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants