Feat: Implement job code diff preview and coordinated apply for AI assistant #4283

lmac-1 · 2025-12-31T22:33:14Z

Description

This PR adds a complete interactive code PREVIEW and APPLY workflow for AI-generated job code suggestions. Users can now preview proposed changes in a side-by-side Monaco diff editor, coordinate with collaborators in real-time, and apply code updates in one click.

I have also disabled AI assistant for both workflow and jobs when the version dropdown is != latest. This matches the behaviour of the old editor.

Key features:

PREVIEW button for Job code

New preview button: Shows AI-suggested code in a Monaco diff editor against latest version
Auto preview: Diffs appear automatically when AI responds with code suggestions
Dismiss preview via X button in top-right corner of diff editor
Diff editor overlays the collaborative editor without disrupting Y.Doc sync
Shared Monaco ref architecture using React context for component coordination

APPLY button extended to Job code

Extended existing APPLY to work also on AI job conversations (previously was only on workflow templates)
Multi-user coordination via Phoenix channels prevents conflicts when multiple users apply code
Visual feedback: Shows "APPLYING..." to all collaborators during code application
Message ID tracking ensures the correct message shows apply state across all users
Updates job body in Y.Doc for real-time sync across collaborators

Version management

AI button and Cmd+K shortcut disabled when viewing pinned workflow
Auto close AI panel when version dropdown is updated to something != latest
Auto dismiss diff - clear diff preview when AI chat/panel closes

Closes #3369

Validation steps

Basic PREVIEW/APPLY Flow

Open a workflow, select a job on the workflow diagram, click Code to open code IDE
Click AI assistant button in top right
Ask AI to suggest code changes (e.g., "add a console.log statement")
Verify diff preview appears automatically when AI responds (PREVIEW button should update to PREVIEWING with green styling)
Click X button to dismiss diff, click PREVIEW to show it again
Click APPLY and verify code updates in the editor
Check toast notification confirms "Code applied"

Multi-User Coordination

Open the same workflow in two browser tabs (different users/sessions)
In Tab 1, ask AI for a code suggestion and click APPLY
In Tab 2, verify the APPLY button shows "APPLYING..." and is disabled
Wait for Tab 1 to complete, verify Tab 2 button returns to "APPLY"
Verify the code change from Tab 1 appears in Tab 2's editor

Version-Aware State

Open a workflow at latest version (no ?v= in URL)
Verify AI button is enabled in header
Use version dropdown to switch to an older snapshot
Verify AI button is disabled with tooltip "Switch to the latest version..."
Verify Cmd+K shortcut does nothing
Verify any active diff preview is cleared when switching versions
Switch back to latest, verify AI button is enabled again
Repeat the same steps when the code IDE is open

Updating to historical version during AI chat

Open a workflow, select a job on the workflow diagram, click Code to open code IDE
Click AI assistant button in top right
Ask AI to suggest code changes (e.g., "add a console.log statement")
Make sure that the diff has been auto applied and is activated
Update the version dropdown to something != latest
Diff should close and AI panel should also close

Edge Cases

Monaco diff editor styling: Try resizing your screen and make sure that the diff editor updates to side-by-side / vertical appropriately.
Close panel with active diff: Preview diff, close AI panel, verify diff clears
Session switching: Preview diff, click "← Back to sessions", verify diff clears
New conversation: Start fresh job chat, verify first AI response auto-previews

Additional notes for the reviewer

The diff editor now has the same theme as the standard editor. I don't know if it's me.. but the contrast feels a bit weird when the diff is applied.
The X button to dismiss diff could be improved - open to suggestions and feedback.
I'm wondering if the diff preview state management is overcomplicated. Added MonacoRefContext because the dismiss button lives in CollaborativeMonaco (rendered by FullScreenIDE) but AIAssistantPanelWrapper needs to be notified when clicked - they're in different tree branches. Open to simpler approaches.
Follow up PR/discussion: What happens if User 1 saves changes to a workflow while User 2 has a diff view open? Should the diff automatically close? This is an edge case, but there's a risk of data loss if User 1 updates the code, then User 2 applies their outdated diff, overwriting User 1's changes. We need to decide how to handle this concurrent editing scenario.

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to know!):

You can read more details in our Responsible AI Policy

Pre-submission checklist

I have performed a self-review of my code.
I have implemented and tested all related authorization policies. (e.g., :owner, :admin, :editor, :viewer)
I have updated the changelog.
I have ticked a box in "AI usage" in this PR

Implement backend support for coordinating job code apply operations across multiple collaborators, mirroring the workflow apply pattern. Backend changes: - Extract code field from Apollo responses in ai_assistant.ex - Support Apollo PR #259 'suggested_code' field with markdown fallback - Add channel handlers for start/done job code apply coordination - Broadcast apply state to prevent concurrent modifications Tests: - Add channel tests for job code apply coordination - Add end-to-end coordination test - Verify broadcasts include user metadata

Implement "Generated Job Code" display with APPLY/COPY buttons, coordinating apply operations across all collaborators. Frontend changes: - Add job code apply handler to AIAssistantPanelWrapper - Display "Generated Job Code" section in MessageList for job chat - Add store coordination functions for job code apply state - Disable save button during apply operation - Hide inline ADD buttons when message.code exists - Sync apply state across all connected users via broadcasts Coordination: - All users see "APPLYING..." state during apply - Apply buttons disabled for all users during operation - Graceful degradation when channel unavailable

Create foundational support for inline code diff previews: - Add MonacoRefContext to share Monaco ref without prop drilling - Implement showDiff/clearDiff methods in CollaborativeMonaco - Use dual-container pattern (standard editor + diff overlay) - Add dismiss button on diff overlay This infrastructure enables job chat to preview AI-suggested code changes directly in the Monaco editor.

Enable users to preview AI-suggested code changes: - Add PREVIEW button to job code messages (alongside APPLY/COPY) - Allow PREVIEW in readonly mode (removed isWriteDisabled check) - Auto-preview diffs when AI responds with code - Auto-dismiss diff when user changes workflow version - Track previewing state to prevent duplicate operations Addresses phase 2 enhancements from issue #3369.

codecov · 2025-12-31T22:38:40Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.19%. Comparing base (93647d0) to head (866f248).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4283      +/-   ##
==========================================
- Coverage   89.29%   89.19%   -0.10%     
==========================================
  Files         425      425              
  Lines       19899    19914      +15     
==========================================
- Hits        17768    17762       -6     
- Misses       2131     2152      +21

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

- Add 8 tests for extract_code_from_markdown/1 covering JavaScript/JS code block extraction, edge cases, and nil handling - Add 4 tests for job code apply coordination via Phoenix Channel covering user attribution (first_name vs email fallback), sequential coordination, and retry scenarios

- Add 11 tests for useAutoPreview hook covering mode filtering, session mount behavior, author filtering, duplicate prevention, and message selection - Add 11 tests for Monaco diff display (showDiff/clearDiff methods, dismiss button functionality) - Add monaco-editor mock to avoid 8MB+ package resolution issues - Configure vitest to alias monaco-editor to mock file

Auto-preview was broken for new conversations because the hook checked for any messages instead of assistant code messages. When a user sent the first message, the session had [userMessage], causing hasMessages to be true and preventing hasLoadedSession from being set. This meant the AI's code response would be skipped. Changes: - Check for assistant code messages instead of any messages on session initialization to correctly identify new vs old conversations - Add session ID tracking to properly reset state when switching between conversations - Add cleanup for diff editor on unmount to prevent memory leaks - Add test coverage for session switching behavior

@doc

The function was defined as private (defp) but tests were calling it as a public function, causing "undefined or private" errors. Changes: - Change defp to def to make the function public - Add @doc and @SPEC annotations - Update misleading comment that referenced waiting for Apollo PR #259 (that PR is already deployed; this is now a legitimate fallback) Fixes failing extract_code_from_markdown tests in AiAssistantTest.

Extract shared editor options (theme, fontSize, etc.) into a single baseEditorOptions object to ensure consistent formatting across both the collaborative editor and AI diff preview. Previously, the diff editor used 'vs-dark' theme with default font size while the main editor used custom 'default' theme with fontSize 14, causing visual inconsistencies.

Prevent users from accessing AI Assistant features when viewing older workflow versions (snapshots) to avoid confusion about which version they're modifying. Disables both UI buttons and keyboard shortcuts. Changes: - Add disabledMessage prop to AIButton component - Disable workflow-level AI button (Header) when viewing old snapshot - Disable job-level AI button (WorkflowEditor) when viewing pinned version - Disable Cmd+K keyboard shortcut when viewing pinned version - Show tooltip: "Switch to the latest version of this workflow to use the AI Assistant." Checks params['v'] to determine if viewing a pinned version vs latest.

Extend version change handling to automatically close the AI Assistant panel with animation when users switch to a pinned workflow version. Changes: - Close AI panel and clear session when switching to pinned version - Dismiss any active diff previews - Clear AI session messages and URL params (w-chat, j-chat) - Disable hover effects on AI buttons when disabled - Reuse isPinnedVersion to avoid duplicate version checks The panel remains closed when switching back to latest, requiring manual reopen. This prevents confusion about which version the AI is working with.

When closing the AI assistant panel or navigating back to the session list, clear any active diff preview in Monaco editor. This prevents the diff from persisting after the user has left the chat context. - Update handleClosePanel to clear diff before closing - Update handleShowSessions to clear diff before showing session list - Move both handlers after monacoRef and previewingMessageId declarations

Add applyingJobCodeMessageId to WorkflowStore state to track which AI message is being applied when job code changes are coordinated across collaborators. This ensures the "APPLYING..." indicator displays on the correct message for all users in a collaborative session, matching the existing workflow apply pattern. Changes: - Add applyingJobCodeMessageId field to Workflow.State interface - Store message_id in job_code_applying channel event handler - Clear message_id in job_code_applied channel event handler - Use stored message ID in AIAssistantPanelWrapper for display

taylordowns2000 · 2026-01-06T20:57:17Z

looks great at first glance! added @josephjclark as a reviewer.

josephjclark · 2026-01-07T09:54:42Z

I will try and get this reviewed tomorrow

lmac-1 · 2026-01-07T15:33:04Z

Thanks @josephjclark! I am going to do another self-review of the code today so that works well. This was quite a big bit of work over the Christmas period so want to look at it all with fresh eyes and perspective.

Fixes two issues in AIAssistantPanelWrapper: 1. Replaces direct mutation of aiMode.context with immutable update pattern when attaching current job code to message context 2. Adds user-facing toast notification when Monaco editor ref is unavailable during code preview attempts The immutable update prevents potential bugs from mutating shared context objects. The notification provides helpful feedback instead of silent failure.

Addresses critical issues identified in self code review: 1. Fix memory leak in channel observer cleanup - Add missing cleanup for job_code_applying/applied handlers - Prevents handler accumulation on workflow switches 2. Fix Monaco ref race conditions (TOCTOU bugs) - Capture monacoRef.current value before use in 5 locations - Prevents crashes when ref becomes null between check and use - Locations: handleClosePanel, handleShowSessions, version effect, handlePreviewJobCode, handleApplyJobCode These fixes prevent memory leaks during long sessions and eliminate potential crashes during rapid view switching.

Reset diff editor model to null before disposing text models to prevent "TextModel got disposed before DiffEditorWidget model got reset" error. Monaco's diff editor requires its model references to be cleared before the underlying text models are disposed. Applied fix to clearDiff callback, showDiff cleanup, and unmount effect.

Replace ref mutation pattern with callback registration system for better architectural consistency. The previous implementation had AIAssistantPanelWrapper mutating a ref it didn't own, which violated ownership principles and made the data flow harder to trace. Changes: - MonacoRefContext now owns callback Set and provides registration function - AIAssistantPanelWrapper registers callback instead of mutating ref - CollaborativeMonaco calls context's handleDiffDismissed - Removed onDiffDismissed prop (replaced with context callback) - Pattern now matches CredentialModalContext for consistency Benefits: - Explicit ownership (context owns callbacks) - Automatic cleanup via useEffect return - Better testability and clearer data flow - Consistent with existing codebase patterns

Extract JSX to a named variable for better readability.

lmac-1 · 2026-01-08T00:37:13Z

@josephjclark I did a bit of a self-review today. I was hoping to do a bit more, but I'm a bit under the weather still today.

Check out the "Notes to reviewer" section for extra things I would appreciate an opinion on when reviewing. Also I am still very new to Elixir, so I may have missed something when making changes to backend code.

taylordowns2000

this is great. thank you @lmac-1 .

midigofrank

Nicely done @lmac-1

lmac-1 added 4 commits December 29, 2025 10:48

github-project-automation bot added this to v2 Dec 31, 2025

github-project-automation bot moved this to New Issues in v2 Dec 31, 2025

lmac-1 added 9 commits January 5, 2026 12:15

lmac-1 changed the title ~~[draft] Implement job code diff preview and coordinated apply for AI assistant~~ Feat: Implement job code diff preview and coordinated apply for AI assistant Jan 6, 2026

lmac-1 marked this pull request as ready for review January 6, 2026 19:36

lmac-1 requested review from elias-ba and taylordowns2000 January 6, 2026 19:39

taylordowns2000 requested a review from josephjclark January 6, 2026 20:56

lmac-1 mentioned this pull request Jan 7, 2026

Know when a workflow is in read-only mode OpenFn/apollo#344

Open

lmac-1 added 4 commits January 7, 2026 18:50

Extract tooltip content to variable in AIButton

21c2b02

Extract JSX to a named variable for better readability.

Updates changelog

866f248

taylordowns2000 approved these changes Jan 8, 2026

View reviewed changes

taylordowns2000 requested review from midigofrank and removed request for josephjclark January 8, 2026 08:06

midigofrank approved these changes Jan 8, 2026

View reviewed changes

taylordowns2000 merged commit 58426fe into main Jan 8, 2026
7 of 8 checks passed

taylordowns2000 deleted the 3369-job-ai-chat-improvements branch January 8, 2026 11:36

github-project-automation bot moved this from New Issues to Done in v2 Jan 8, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feat: Implement job code diff preview and coordinated apply for AI assistant #4283

Feat: Implement job code diff preview and coordinated apply for AI assistant #4283

Uh oh!

lmac-1 commented Dec 31, 2025 •

edited

Loading

Uh oh!

codecov bot commented Dec 31, 2025 •

edited

Loading

Uh oh!

taylordowns2000 commented Jan 6, 2026

Uh oh!

josephjclark commented Jan 7, 2026

Uh oh!

lmac-1 commented Jan 7, 2026

Uh oh!

lmac-1 commented Jan 8, 2026

Uh oh!

taylordowns2000 left a comment

Uh oh!

midigofrank left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Feat: Implement job code diff preview and coordinated apply for AI assistant #4283

Feat: Implement job code diff preview and coordinated apply for AI assistant #4283

Uh oh!

Conversation

lmac-1 commented Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key features:

Validation steps

Basic PREVIEW/APPLY Flow

Multi-User Coordination

Version-Aware State

Updating to historical version during AI chat

Edge Cases

Additional notes for the reviewer

AI Usage

Pre-submission checklist

Uh oh!

codecov bot commented Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

taylordowns2000 commented Jan 6, 2026

Uh oh!

josephjclark commented Jan 7, 2026

Uh oh!

lmac-1 commented Jan 7, 2026

Uh oh!

lmac-1 commented Jan 8, 2026

Uh oh!

taylordowns2000 left a comment

Choose a reason for hiding this comment

Uh oh!

midigofrank left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lmac-1 commented Dec 31, 2025 •

edited

Loading

codecov bot commented Dec 31, 2025 •

edited

Loading