Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions SCOPES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Scope Coverage Registry

**The single reference for every data scope, how it can be fulfilled (web vs.
desktop), and the exact connectors behind it.**

This file is **maintained by hand.** It is the source of truth people read when
they need to answer "can the web flow produce `instagram.ads`?" (it can't — see
the table). There is intentionally no generator and no automated sync: the set
of web-fulfillable scopes changes rarely, so when it changes, edit this file
directly. (Background: [BUI-395](https://linear.app/vana-team/issue/BUI-395/instagram-scope-mismatch-found-in-cg-lab-e2e).)

## How to read this

- **Web** — the scope can be produced by the ODL **Data Pipe API**
(`data-pipe.vana.org`), the hosted "light" flow used by the Vana Web app. Today
the Data Pipe API produces only `*.profile`, and only for the five sources it
has been wired for (Instagram, Spotify, YouTube, LinkedIn, GitHub). Update this
column **by hand** when the Data Pipe API gains a new source/scope.
- **Desktop** — the scope can be produced by a Playwright **connector** in this
repo (the heavy flow, run via the Data Connect desktop app). Derived from the
connector manifests + `registry.json`.
- **Connectors** — the exact connector id(s) that fulfill the scope, with status.
A scope may be served by more than one connector.

> A request for a scope marked Desktop-only (e.g. `instagram.ads`) cannot be
> satisfied by the web flow alone — the Personal Server will have no data for it.
> That is the exact mismatch BUI-395 surfaced.

## Coverage

| Source | Scope | Web | Desktop | Connector(s) |
|---|---|:--:|:--:|---|
| amazon | `amazon.orders` | — | ✅ | amazon-playwright (beta) |
| amazon | `amazon.profile` | — | ✅ | amazon-playwright (beta) |
| chatgpt | `chatgpt.conversations` | — | ✅ | chatgpt-playwright (stable) |
| chatgpt | `chatgpt.memories` | — | ✅ | chatgpt-playwright (stable) |
| claude | `claude.conversations` | — | ✅ | claude-playwright (experimental) |
| claude | `claude.projects` | — | ✅ | claude-playwright (experimental) |
| doordash | `doordash.orders` | — | ✅ | doordash-playwright (beta) |
| github | `github.profile` | ✅ | ✅ | github-playwright (stable) |
| github | `github.repositories` | — | ✅ | github-playwright (stable) |
| github | `github.starred` | — | ✅ | github-playwright (stable) |
| heb | `heb.nutrition` | — | ✅ | heb-playwright (experimental) |
| heb | `heb.orders` | — | ✅ | heb-playwright (experimental) |
| heb | `heb.profile` | — | ✅ | heb-playwright (experimental) |
| icloud_notes | `icloud_notes.folders` | — | ✅ | icloud-notes-playwright (experimental) |
| icloud_notes | `icloud_notes.notes` | — | ✅ | icloud-notes-playwright (experimental) |
| instagram | `instagram.ads` | — | ✅ | instagram-ads-playwright (beta); instagram-playwright (stable) |
| instagram | `instagram.following` | — | ✅ | instagram-playwright (stable) |
| instagram | `instagram.posts` | — | ✅ | instagram-playwright (stable) |
| instagram | `instagram.profile` | ✅ | ✅ | instagram-playwright (stable) |
| linkedin | `linkedin.connections` | — | ✅ | linkedin-playwright (stable) |
| linkedin | `linkedin.education` | — | ✅ | linkedin-playwright (stable) |
| linkedin | `linkedin.experience` | — | ✅ | linkedin-playwright (stable) |
| linkedin | `linkedin.languages` | — | ✅ | linkedin-playwright (stable) |
| linkedin | `linkedin.profile` | ✅ | ✅ | linkedin-playwright (stable) |
| linkedin | `linkedin.skills` | — | ✅ | linkedin-playwright (stable) |
| oura | `oura.activity` | — | ✅ | oura-playwright (stable) |
| oura | `oura.readiness` | — | ✅ | oura-playwright (stable) |
| oura | `oura.sleep` | — | ✅ | oura-playwright (stable) |
| shop | `shop.orders` | — | ✅ | shop-playwright (beta) |
| spotify | `spotify.playlists` | — | ✅ | spotify-playwright (stable) |
| spotify | `spotify.profile` | ✅ | ✅ | spotify-playwright (stable) |
| spotify | `spotify.savedTracks` | — | ✅ | spotify-playwright (stable) |
| steam | `steam.friends` | — | ✅ | steam-playwright (experimental) |
| steam | `steam.games` | — | ✅ | steam-playwright (experimental) |
| steam | `steam.profile` | — | ✅ | steam-playwright (experimental) |
| uber | `uber.receipts` | — | ✅ | uber-playwright (beta) |
| uber | `uber.trips` | — | ✅ | uber-playwright (beta) |
| wholefoods | `wholefoods.nutrition` | — | ✅ | wholefoods-playwright (experimental) |
| wholefoods | `wholefoods.orders` | — | ✅ | wholefoods-playwright (experimental) |
| wholefoods | `wholefoods.profile` | — | ✅ | wholefoods-playwright (experimental) |
| youtube | `youtube.history` | — | ✅ | youtube-playwright (beta) |
| youtube | `youtube.likes` | — | ✅ | youtube-playwright (beta) |
| youtube | `youtube.playlistItems` | — | ✅ | youtube-playwright (beta) |
| youtube | `youtube.playlists` | — | ✅ | youtube-playwright (beta) |
| youtube | `youtube.profile` | ✅ | ✅ | youtube-playwright (beta) |
| youtube | `youtube.subscriptions` | — | ✅ | youtube-playwright (beta) |
| youtube | `youtube.watchLater` | — | ✅ | youtube-playwright (beta) |

## Maintaining this file

- **A connector adds/removes a scope** → update the Desktop rows + Connector(s)
column for that source.
- **The Data Pipe API gains a web-fulfillable scope** → flip its **Web** cell to
✅. Keep `unity-surfaces` `web-writable-scopes.ts` consistent with the Web
column by hand (the web app changes infrequently enough that this is fine).
74 changes: 74 additions & 0 deletions docs/specs/2026-05-29-scope-coverage-catalog-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Scope Coverage Registry — Design Spec

**Date:** 2026-05-29
**Author:** Maciej (brainstormed 2026-05-29)
**Status:** Accepted
**Linear:** [BUI-395 — Instagram scope mismatch found in CG lab E2E](https://linear.app/vana-team/issue/BUI-395/instagram-scope-mismatch-found-in-cg-lab-e2e)
**Artifact:** `SCOPES.md` (repo root)

---

## 1. Summary

A single, **hand-maintained** reference file — `SCOPES.md` — in the
`data-connectors` repo. Open it and you see every data scope, whether the **web**
(ODL Data Pipe API) and/or **desktop** (Playwright connectors) flow can fulfill
it, and the exact connector(s) behind each scope.

It is documentation, not machinery: no generator, no codegen, no automated sync.

## 2. Background — the bug

During a CG → Vana Web → Personal Server lab E2E, the lab requested `read:ads`
(→ `instagram.ads`). CG forwarded it correctly, but the **web (light)** flow can
only produce `instagram.profile`, so the Personal Server had no `instagram.ads`
data and the read returned a valid 404.

The architecture is two-tier:
- **Desktop / heavy** — Playwright connectors in `data-connectors`. Already canonical.
- **Web / light** — the ODL-run Data Pipe API (`data-pipe.vana.org`) used by the
Vana Web app. Produces only `*.profile` today, for five wired sources.

"Which tier can fulfill which scope" was undocumented; the only place the web set
existed was a hardcoded list in `unity-surfaces`
(`web-writable-scopes.ts`). The open question on BUI-395 was *where* this should
be documented. This spec answers: one file in data-connectors.

## 3. Decision — manual file, deliberately not generated

We considered generating the catalog from the connector manifests + a declared
web-capabilities input, and generating `unity-surfaces`'
`web-writable-scopes.ts` from it. **Rejected for now**: the web app's set of
web-fulfillable scopes changes infrequently, so the cost/complexity of a
generator, a vendored snapshot, and a sync step is not worth it. A single file a
human reads and edits is enough.

Consequences:
- No scripts are added to any repo.
- `unity-surfaces` `web-writable-scopes.ts` is unchanged and stays
hand-maintained. Keeping it consistent with `SCOPES.md`'s Web column is a
manual responsibility. If it drifts, we fix it by hand.

## 4. The file

`SCOPES.md` at the `data-connectors` repo root. One table:

| Column | Meaning |
|---|---|
| **Source** | source id (e.g. `instagram`) |
| **Scope** | canonical scope id (e.g. `instagram.ads`) |
| **Web** | ✅ if the ODL Data Pipe API can produce it (today: `*.profile` for instagram, spotify, youtube, linkedin, github) |
| **Desktop** | ✅ if a Playwright connector produces it |
| **Connector(s)** | the exact connector id(s) + status that fulfill the scope |

Plus a header explaining how to read it and a footer explaining how to maintain
it (update Desktop rows when a connector changes scopes; flip Web cells when the
Data Pipe API gains a scope; keep `web-writable-scopes.ts` consistent by hand).

## 5. Out of scope

- Any generator, sync script, or codegen.
- Changes to `unity-surfaces` (`web-writable-scopes.ts` stays as-is).
- A Data Pipe API capabilities endpoint.
- CG surfacing the tier at request time (e.g. warning "ads needs Desktop" in the
lab) — a separate follow-on if desired.
Loading