Integration request: first-class Skyvern adapter

[aaronlab]

BrowserTrace includes a basic Skyvern-shaped tracing wrapper today, plus a no-network example in examples/skyvern_wrapper_example.py. This issue stays open for a deeper first-class Skyvern adapter shaped by real Skyvern failure cases.

Guide:
https://aaronlab.github.io/browsertrace/skyvern-debugging.html

Current support:

• browsertrace.integrations.skyvern.wrap_skyvern wraps Skyvern-shaped clients without importing Skyvern at install/test time.
• Records high-level run_task and run_workflow calls.
• Supports async and sync client methods.
• Records method input, returned run IDs/status, and step-level errors.
• Keeps Skyvern as an optional dependency, not part of the base install.

For public sharing, use:

browsertrace export <run_id> --public -o public.html

--public omits prompt/model I/O, screenshots, and URLs from the standalone HTML export.

Feedback wanted from Skyvern users:

• Which Skyvern workflow failure was hardest to debug?
• Should BrowserTrace capture workflow state, selected elements/selectors, extracted data, retry/history state, screenshots, model output, or task prompts?
• Where does Skyvern expose the browser action or workflow step most reliably?
• Which hook would be easiest to test in a preview adapter?
• Can you share a sanitized trace shape or small repro?

A good first PR should keep Skyvern optional, include tests with fake Skyvern-shaped objects, and avoid network/API-key requirements.

[aaronlab]

For Skyvern users: the most useful feedback is a real failure shape.

Please comment with:

• where Skyvern exposes the browser action or workflow step
• whether screenshot, URL, model/tool decision, or workflow state is hardest to recover after failure
• which hook would be easiest to test in a preview adapter
• whether you can share a sanitized trace shape or small repro

General adapter requests can also use the structured integration form: https://github.com/aaronlab/browsertrace/issues/new?template=integration_request.yml

[aaronlab]

Update: a basic Skyvern tracing wrapper is now implemented on main in browsertrace.integrations.skyvern.

What it covers today:

• wraps Skyvern-shaped clients without importing Skyvern at install/test time
• records high-level run_task and run_workflow calls
• supports async and sync client methods
• records method input, returned run IDs/status, and step-level errors
• documented in the README with a minimal example
• covered by tests/test_skyvern_integration.py

Commit: 132acae feat: add Skyvern tracing wrapper

I am keeping this issue open for a deeper Skyvern adapter. The next useful feedback is which artifacts matter most in real Skyvern failures: workflow state, selected elements/selectors, extracted data, retry/history state, screenshots, model output, or task prompts.

[aaronlab]

Added a no-network Skyvern wrapper example in examples/skyvern_wrapper_example.py and linked it from the README. It uses a fake Skyvern-shaped async client, so people can verify what BrowserTrace records without installing Skyvern or using API keys.\n\nVerification: CI passed on da7af36 with Python 3.11, 3.12, 3.13, plus package build.

[aaronlab]

Progress update: dbac90c improves Skyvern metadata extraction for nested SDK-shaped results. BrowserTrace now recursively finds nested task_run_id, workflow_run_id, run_id, id, task_id, status, or state fields and promotes them into step metadata as skyvern_run_id and skyvern_status.

This keeps the wrapper dependency-free and covered by fake Skyvern-shaped tests.

Local verification before push: uv run --python 3.11 --extra dev pytest -q -> 101 passed.

[aaronlab]

Opened a Skyvern community feedback thread for BrowserTrace's Skyvern wrapper:\n\nhttps://github.com/Skyvern-AI/skyvern/discussions/5931\n\nThe post asks Skyvern users which task/workflow failure artifacts matter most: workflow state, selected elements/selectors, extracted data, retry/history state, screenshots, model output, task prompts, run IDs/status, or other fields. It does not ask for stars or upvotes.

[aaronlab]

Skyvern community feedback update (2026-05-10): posted a targeted technical reply in Skyvern-AI/skyvern#1420 about human-in-the-loop / copilot traces and progressive automation.

Reply: Skyvern-AI/skyvern#1420 (comment)

Why it matters for this issue:

• The discussion asks for user-story traces that can capture human and agent actions while moving from mostly manual workflows toward automation.
• The proposed trace boundary adds actor and handoff_reason to the usual step provenance so a future Skyvern adapter can explain why control moved between human and agent.
• This suggests a deeper adapter direction beyond basic run_task / run_workflow metadata: capture pause/resume, human action, agent action, workflow step state, status, and error boundaries first; avoid full DOM/session recording until a concrete failure requires it.

No stars/upvotes were requested. This was a technical feedback reply only.

[aaronlab]

Skyvern adapter research note (2026-05-11): Skyvern-AI/skyvern#5637 is a useful action-confidence failure shape to track for the deeper adapter.

External thread:
Skyvern-AI/skyvern#5637 (comment)

Relevant signal:

• The question is about whether pre-action confidence is enough to know a click or submit is correct.
• The tricky case is high confidence plus wrong target, wrong context, or an action whose consequence is larger than the model expected.
• A single scalar confidence value is less useful than the evidence bundle that produced it.

Implication for BrowserTrace:

• Keep skyvern_run_id / skyvern_status metadata, but a deeper adapter should also preserve pre-action evidence when Skyvern exposes it.
• Useful fields would include action type, selected element role/name/text, selector if available, bounding box, URL, screenshot id/timestamp, nearby candidate elements, confidence or ranking signal, policy/scoping decision, and post-action status/error.
• For destructive actions such as submit, login, checkout, or payment-like steps, trace rows should make the action boundary explicit: allowed, blocked, escalated, or executed after approval.
• Do not treat confidence as correctness. Treat it as a diagnostic field that lets users find false-high-confidence actions after the run.

No external follow-up is needed right now unless Skyvern maintainers ask for a concrete adapter shape.

[aaronlab]

Skyvern adapter research note (2026-05-11): Skyvern-AI/skyvern#3260 is a useful VNC/CDP debug-integration shape for future deeper Skyvern support.

External thread:
Skyvern-AI/skyvern#3260 (comment)

Relevant signal:

• The proposed Skyvern debugging task combines VNC visual access with CDP DOM, console, network, and performance data.
• Those streams are most useful when linked to the same task/workflow step rather than stored as unrelated logs.
• Connection lifecycle events matter: connect/probe result, capture start/stop, timeout, cleanup, and resource leak detection can explain failures before any screenshot exists.

Implication for BrowserTrace / future Skyvern adapter work:

• Preserve step id, task id, workflow id, URL, frame/page id, action/tool name, status, error, and retry/recovery decision when Skyvern exposes them.
• Link VNC screenshot/recording artifact ids with CDP selected-element or DOM summaries for the same time window.
• Track redaction state for screenshots, URLs, headers, cookies, and form values before any public export.
• Treat browser connection/debug-session lifecycle as traceable evidence, not only task steps after the page is already running.

No external engagement request was made. This was a technical debugging-shape reply only.

[aaronlab]

Skyvern multi-session VNC support research note (2026-05-11): added a technical comment to Skyvern-AI/skyvern#4392 about trace fields for local/self-hosted VNC, CDP target identity, and manual-control lease state.

External comment:
Skyvern-AI/skyvern#4392 (comment)

Relevant trace fields for a future first-class Skyvern adapter:

• workflow_id, task_id, browser_session_id, browser_context_id, page_id, and redacted display/VNC endpoint id
• VNC stream lifecycle: requested, connected, first frame, reconnect, timeout, cleanup, last frame timestamp
• CDP target/session id linked to the same workflow step and time window
• control lease state: agent/manual, owner, acquired/renewed/released timestamps, persisted-across-reconnect flag, release reason
• isolation metadata: own X display/container/browser context vs shared display
• failure cause category: no VNC server, connected without frames, CDP attached but VNC missing, manual-control lease lost, or display conflict

This complements the existing VNC/CDP debug integration note: a Skyvern trace should make VNC stream identity, CDP target identity, and manual-control ownership first-class evidence instead of inferring them from the active UI connection.