Skip to content

Let an existing analysis be extended with further detail (e.g. a follow-up recording) instead of replaced #23

Description

@GiorgioUghini

Summary

Today an analysis is welded to exactly one recording: one session dir → one analysis.json. If the reconstruction is incomplete — you forgot a step, the recording missed the part where you opened the admin page, a step needs a second example to generalize from — the only ways forward are natural-language feedback (no new evidence) or re-analyze (throws away the conversation, the summary, and any manual edits).

We should be able to add further detail to an analysis that already exists, with a follow-up recording as the flagship case: record the missing bit, attach it to the existing analysis, and have the describer fold it in as a revision rather than a replacement.

The interesting part of this feature isn't the agent call — it's when the app should allow it and where the affordance lives. Both are enumerated below.

Why this doesn't work today

Layer Constraint
common/analysis.ts Analysis.sessionId: string — a single recording id. No notion of additional sources; steps carry free-text evidence with no source attribution.
electron/describer/tools.ts get_timeline / get_events / get_frames / get_narration all read from one captured sessionDir. A second recording lives in a directory the agent literally cannot address.
electron/describer/describer.ts createLive() pins workingDirectory: dir and an availableTools allowlist (deliberately — an unsandboxed session plus approveAll would be bad). analyze() calls disposeLive() first, so every explicit analyze is a fresh conversation whose submission overwrites analysis.json wholesale.
common/ipc.ts AnalysisFeedback is { overall?, steps[] } — text only. There is no channel to hand the agent new evidence.
electron/sessions.ts The library is a flat readdir of the sessions root; every recording is its own top-level row. Nothing can be a child/supplement of anything.

The closest existing precedent is the voice-stale card in src/Library.tsx: "Voice transcript added after this analysis — Re-analyze to include it. This replaces the current summary and any edits." That's the exact cost we want to remove: new evidence arriving late should be additive, not destructive.

When can we do this? (the lifecycle question)

The attach action has to be gated on recorder state, analysis state, and downstream artifacts. Proposed rules:

State Behaviour
Recording in progress (RecorderStatus.state === "recording") Attach is unavailable. The recorder is a singleton; a supplement goes through the normal start path (including the pre-recording privacy warning) — no second concurrent capture.
Supplement not post-processed yet (no bundle.json) Attach is accepted but analysis is deferred, mirroring the existing "Still processing this recording… try again in a moment." state.
Analyze / feedback turn in flight (Describer.active) Blocked. Describer already throws "An analysis is already running for this session." — attach must respect the same guard rather than racing it.
No analysis yet Cheapest case: attach first, then the initial analyze reads both recordings. Worth supporting explicitly so a user who knows the take was incomplete doesn't have to analyze first just to extend.
Analysis exists, no manual edits Attach → agent revision (revision += 1) that folds the new evidence in. Straightforward.
Analysis exists with manual edits (analyze:update bypasses the agent) The dangerous case. Extension must preserve user-edited titles/intent/steps, or warn as explicitly as the voice card does. Ideally the extend prompt carries the current (edited) analysis as the baseline — the feedback() path already does this via renderFeedbackPrompt.
Analysis approved (approved / approvedAt) These schema fields exist but nothing writes or reads them yet. Decide now: does extending clear approval and require re-approval before a skill can be built?
A skill / automation already exists (skill.json, built-automation.json, hasSkill / hasAutomation) The built artifact silently becomes stale. Needs a staleness signal + "rebuild from the updated analysis" prompt, analogous to the voice-stale card — not a silent divergence.
Deletion Rules needed for deleteSession on either side: deleting a supplement should demote the analysis to its remaining sources (and mark it stale), not orphan a dangling reference. (See also #13 on delete racing background processing.)

Where does it live? (the surface question)

  • src/Library.tsx, analysis workspace — primary surface. An "Add detail" action alongside Edit / in ws-foot, offering: record more (starts a recording bound to this analysis) and, later, attach an existing recording from the library. Plus a card, styled like voice-card, that says a supplement is attached and not yet folded in.
  • src/Library.tsx, sessions list — a supplement should not read as an unrelated recording. Either nest it under its parent or tag it (tag chips already exist for skill / automation).
  • Window/handoff — the library is a separate window (openLibrary / closeLibrary), and closing it calls Describer.evictIdle(). Recording a supplement takes the user out of the library and back; that round trip needs to land them on the same analysis with the new recording already attached.
  • common/ipc.ts — new channel (e.g. analyze:attach), plus SessionSummary linkage fields so the list can render the relationship.
  • common/analysis.tsAnalysis grows a sources[] (ordered recording ids + when each was attached); revision keeps meaning "nth pass". Per-step evidence should say which recording it came from.
  • electron/describer/LiveSession holds an allowlist of source dirs; describer tools take a source/recording argument resolved only against that allowlist (keep the isValidSessionId / sessionDir traversal guarantees and the availableTools sandbox — this is the main security-relevant change); a new prompt renderer, sibling to renderFeedbackPrompt, that says "here is your current analysis, here is an additional recording, revise holistically".
  • electron/sessions.ts — linkage-aware listing, size accounting, and delete rules.
  • README step 3 ("Analyze") and the analyze disclosure copy in Library.tsx — each attachment sends another recording's timeline and frames to the cloud; the existing warning must clearly cover supplements.

Acceptance criteria

  • From an existing analysis, a user can record (or attach) a further recording and get a revised analysis that references both, without losing the intent, steps, or manual edits they curated.
  • Attach is correctly gated in every state in the table above, with clear UI copy for each blocked case.
  • Describer tools can read only the recordings explicitly attached to that analysis — no widening of the working directory or the tool allowlist.
  • Steps derived from a supplement are attributable to it (evidence identifies its source).
  • A skill/automation built from a now-extended analysis is visibly stale and can be rebuilt.
  • Deleting either recording leaves the analysis in a coherent, non-dangling state.

Open questions

  • Extend vs. re-analyze: does the supplement go into the same live agent conversation (cheap, preserves context, but conversations are evicted — MAX_LIVE_SESSIONS, evictIdle(), app restart), or does it always start a fresh conversation seeded with the current analysis text (deterministic, loses reasoning context)?
  • Merge semantics: is a supplement appended as later steps, interleaved by wall-clock time (the two recordings have unrelated epochs and may overlap in intent), or does the agent freely re-order? Step-id stability matters — renderFeedbackPrompt already asks the agent to keep ids stable.
  • Scope of "further details": plain NL notes are already covered by the feedback round. Is the attachable unit only recordings, or a general evidence type (a screenshot, a file, a pasted log) with recordings as the first implementation?
  • Cardinality: one supplement at a time, or an unbounded chain? Every source multiplies the frames/events the agent reads within TURN_TIMEOUT_MS.
  • Second-example generalization: recording the same task twice with different inputs is arguably the strongest reason to want this (it teaches the builder what varies). Is that the same feature, or a distinct "add another example" concept the skill builder consumes directly?
  • Should approved / approvedAt finally be wired up as part of this, since extension is the thing that most obviously ought to invalidate approval?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions