Skip to content

Fix LINE popup OCR menu targeting for copy and context actions#54

Merged
keyang556 merged 1 commit intomainfrom
codex/dev
Apr 14, 2026
Merged

Fix LINE popup OCR menu targeting for copy and context actions#54
keyang556 merged 1 commit intomainfrom
codex/dev

Conversation

@keyang556
Copy link
Copy Markdown
Owner

@keyang556 keyang556 commented Apr 14, 2026

Summary

  • resolve popup OCR matches through menu row geometry instead of raw OCR line index mapping
  • prevent short-message noise lines from shifting 複製 onto 分享 in copy-first reads
  • apply the same popup targeting logic to message context menu actions
  • add regression coverage for noisy OCR popup rows and shared popup click-point resolution

Testing

  • C:\GitHub\linedesktopnvda\.uv-cache-local\archive-v0\gE_Qq9Tk3Gs40iaFLYhBc\Scripts\pytest.exe tests/test_message_context_menu.py tests/test_line_resource_guards.py

Greptile Summary

此 PR 解決了 LINE 桌面版訊息右鍵選單中,OCR 定位因短訊息「雜訊行」(如訊息字數標記 "50")干擾而導致 複製 被誤定位至 分享 按鈕的 bug。

主要變更:

  • 新增 _collectPopupMenuRowRectsline.py:透過 UIA Raw View Walker 遍歷彈出視窗的 UIA 樹,蒐集各行的精確邊界矩形(row rects),以幾何位置替代原本的 OCR 索引映射。
  • 新增 _resolvePopupMenuLabelClickPointline.py:以 _buildMenuElements 建立元素後,依標籤名稱查找對應的 clickPoint,回傳含 clickPointindexcount 的結果物件。
  • 新增 _buildMenuElements_normalizeMenuRowRects_assignRowRectsToElementsmessageContextMenu.py:將 OCR 識別到的選單標籤與 row rects 幾何對齊,優先以 OCR 行中心 Y 最近距離原則(distance-minimizing)分配 row rect,並在 row rects 不足時提供多層 fallback。
  • MessageContextMenu.__init__ 與複製優先讀取路徑(_findCopyMenuItem)及訊息選單動作路徑均已更新,以 popupRowRects=... 傳入 row rects。
  • 新增測試:針對雜訊行過濾(test_build_menu_elements_ignores_noise_lines_before_copy_row)、行解析(test_resolve_popup_menu_label_click_point_returns_aligned_click_target)與選單點擊邏輯均有迴歸測試。

Confidence Score: 5/5

  • 此 PR 可安全合併,核心邏輯正確,測試覆蓋率良好,未發現會影響執行路徑的 bug。
  • 三個選單觸發路徑(鍵盤開啟、複製優先讀取、UIA 滑鼠點選)均已套用 row-rect 定位並加入防禦性 fallback。_assignRowRectsToElements 的「保留剩餘元素所需位置」公式 maxRowIndex = totalRows - remainingElements 可防止越界,且在 row rects 不足時優雅降級至 OCR 衍生座標。測試案例直接驗證雜訊行過濾、click point 對齊及回呼路徑。
  • 所有檔案皆不需要特別關注。

Important Files Changed

Filename Overview
addon/appModules/line.py 新增 _collectPopupMenuRowRects_resolvePopupMenuLabelClickPoint,並在三個選單觸發路徑中正確傳入 popupRowRects;邏輯清晰,防禦性處理完善。
addon/appModules/_virtualWindows/messageContextMenu.py 新增 _buildMenuElements_normalizeMenuRowRects_assignRowRectsToElements;主路徑以距離最近原則對齊 row rects,fallback 在 row rects 不足時降級至 OCR 衍生座標,行為一致且正確。
tests/test_message_context_menu.py 新增雜訊行過濾測試(test_build_menu_elements_ignores_noise_lines_before_copy_row),直接驗證主路徑的 click point 對齊行為。
tests/test_line_resource_guards.py 新增 test_resolve_popup_menu_label_click_point_returns_aligned_click_target 以 mock 驗證 row rects 正確傳遞至 _buildMenuElements;現有資源守護測試未受影響。

Sequence Diagram

sequenceDiagram
    participant LPY as line.py
    participant CRPR as _collectPopupMenuRowRects
    participant RPMLCP as _resolvePopupMenuLabelClickPoint
    participant MCM as messageContextMenu._buildMenuElements
    participant ARRE as _assignRowRectsToElements

    LPY->>CRPR: popupHwnd, popupRectTuple
    CRPR-->>LPY: popupRowRects (UIA row geometry list)

    LPY->>RPMLCP: targetLabel, ocrLines, popupRect, popupRowRects
    RPMLCP->>MCM: "lines, popupRect, rowRects=popupRowRects"
    MCM->>MCM: filter noise lines via _matchMenuLabel
    MCM->>ARRE: elements (with _lineCenterY), rowRects
    ARRE-->>MCM: clickPoints assigned via nearest-row heuristic
    MCM-->>RPMLCP: elements list (name + clickPoint)
    RPMLCP-->>LPY: "{clickPoint, index, count} for targetLabel"

    LPY->>LPY: send mouse click to clickPoint
Loading

Reviews (1): Last reviewed commit: "Fix popup OCR menu alignment for LINE me..." | Re-trigger Greptile

@keyang556 keyang556 merged commit 1a5eea3 into main Apr 14, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant