perf(notion_datasource): speed up get_authorized_pages for large work…

[Kota-Maeda]

Summary

Fixes #3170.

langgenius/notion_datasource@0.1.18 cannot enumerate pages on Notion workspaces with more than ~1k shared items. get_authorized_pages() runs three phases serially (search pages, search databases, resolve every parent block — with recursion and no cache), which routinely exceeds the plugin-daemon SSE deadline (PLUGIN_MAX_EXECUTION_TIMEOUT, default 600s) and the request is killed by timeout.

This PR rewrites the hot path without changing the external behavior:

• Parallel + memoized parent resolution. Resolve block_id parents in a ThreadPoolExecutor (size configurable via the env var NOTION_PARENT_RESOLVE_WORKERS, default 8, clamped to [1, 32]). Results are memoized in a thread-safe dict so sibling pages that share an ancestor don't re-issue the same HTTP request.
• Single /v1/search loop. The previous two filtered loops (one for object="page", one for object="database") are replaced with a single un-filtered pass; items are dispatched by object type. Halves the number of search round-trips.
• Unified retries. The three direct requests.post/get call sites are routed through _make_request, which already handles 429 / transient 5xx with backoff. _make_request gains an allow_status parameter so the existing "404 on a parent block → treat as root" behavior is preserved without re-introducing a direct request.

Backwards compatibility:

• notion_page_search, notion_database_search, notion_block_parent_page_id are kept as thin wrappers delegating to the new internals, so any external caller keeps working.
• OnlineDocumentPage field values are unchanged. The only observable difference is the ordering of the returned list (parallel completion order rather than insertion order), which downstream code does not rely on.

Change Type

• Documentation / non-plugin change
• Non-LLM plugin (tools, extensions, datasource, etc.)
• LLM plugin

Screenshots / Videos

This PR is an internal performance fix with no observable UI change.
Before/After behavior is shown below as plugin-daemon log excerpts.

Before — request killed by the 600s SSE deadline:

plugin_daemon | ERROR ... PluginDaemonInternalServerError error="killed by timeout"
plugin_daemon | HTTP request POST /plugin/<tenant>/dispatch/datasource/get_online_document_pages
              | status=200 latency_ms=600003
api           | ERROR Exception on /console/api/notion/pre-import/pages
              |   httpcore.ReadTimeout: timed out
              |   httpx.ReadTimeout: timed out

After — same workspace, same credential, completes well inside the deadline:

plugin_daemon | HTTP request POST /plugin/<tenant>/dispatch/datasource/get_online_document_pages
              | status=200 latency_ms=63151   # ← 63 s, no timeout

LLM Plugin Checklist

Not applicable — this is a datasource plugin.

Version

• Bumped top-level version in manifest.yaml (0.1.18 → 0.1.19, not the one under meta)
• dify_plugin>=0.5.0 is declared in pyproject.toml and locked in uv.lock

A note on the template wording: the template suggests the literal range dify_plugin>=0.3.0,<0.6.0, but adding an upper bound of <0.6.0 makes the lock unsatisfiable in this plugin — dify_plugin 0.5.x pins werkzeug<3.1.dev0, while this plugin already required werkzeug>=3.1.7 before this PR. The current spec (>=0.5.0, no upper bound) matches the convention used by ~all other datasource plugins in this repo (confluence, dropbox, github, gitlab, onedrive, sharepoint, etc.). uv.lock is left as-is on upstream/main.

Patch-level bump rationale:

• public API of the plugin is unchanged
• no new manifest.yaml capabilities or provider/notion_datasource.yaml fields
• purely an internal performance/reliability fix

Testing

• Local deployment — Dify version: 1.14.1 (self-hosted Docker, plugin-daemon 0.6.0-local)

Verified manually against a multi-thousand-item Notion workspace that was previously hitting the 600s SSE deadline. After this change the same enumeration completes in ~60 seconds end to end with the default worker count. No regressions observed on a small (< 100 items) workspace.

Configuration

Variable	Default	Range	Effect
NOTION_PARENT_RESOLVE_WORKERS	8	1–32 (out-of-range values fall back / are clamped)	Worker count for the parallel parent-block resolution. 1 reproduces the old serial behavior. Increase cautiously — Notion's public rate limit is roughly 3 req/s and exceeding it just amplifies 429 backoff.

[gemini-code-assist]

Code Review

This pull request refactors the Notion client to optimize workspace enumeration by combining page and database searches into a single pass and resolving parent IDs concurrently using a thread pool with memoization. Feedback highlights that error handling in the parent resolution logic should be broadened to prevent crashes from network errors, and the memoization implementation currently allows redundant I/O due to a race condition. Additionally, the shift to in-memory filtering for specific search methods may lead to performance regressions in large workspaces.

[Kota-Maeda]

@cazziwork Could you confirm this PR?