diff --git a/context/skills/self-driving/config.yaml b/context/skills/self-driving/config.yaml index 120d642..a6ac428 100644 --- a/context/skills/self-driving/config.yaml +++ b/context/skills/self-driving/config.yaml @@ -1,6 +1,6 @@ type: skill template: description.md -description: Set up PostHog Self-driving — enable the right signal sources, connect GitHub, tune the scout fleet, and design custom scouts +description: Set up PostHog Self-driving — enable the right signal sources, connect GitHub, tune the scout troop, and design custom scouts tags: [signals, self-driving] references: preamble: "**Read ONLY this file.** Do not read any other reference file until this one tells you to." diff --git a/context/skills/self-driving/description.md b/context/skills/self-driving/description.md index 934f924..3c3adbf 100644 --- a/context/skills/self-driving/description.md +++ b/context/skills/self-driving/description.md @@ -1,6 +1,6 @@ # PostHog Self-driving setup -This skill configures PostHog Signals for a project that already has PostHog installed: it switches on the signal sources (the inbox's "Responders") that match what the product actually uses, makes sure the GitHub integration is connected so Signals can research and fix issues in code, tunes the scout fleet, designs custom scouts for the watchable surfaces the canonical fleet doesn't cover (always proposed to the user first). Organization-level AI data processing approval — which everything downstream depends on — is enforced by the wizard itself before this skill runs. +This skill configures PostHog Signals for a project that already has PostHog installed: it switches on the signal sources (the inbox's "Responders") that match what the product actually uses, makes sure the GitHub integration is connected so Signals can research and fix issues in code, tunes the scout troop, designs custom scouts for the watchable surfaces the canonical troop doesn't cover (always proposed to the user first). Organization-level AI data processing approval — which everything downstream depends on — is enforced by the wizard itself before this skill runs. The wizard's run prompt supplies the project URLs (integrations settings, organization AI settings, new warehouse source, Signals inbox). Use those exact URLs whenever a step sends the user to the browser. @@ -19,7 +19,7 @@ Each step file points to the next. Run them in order. **Start by reading `refere - **Never disable a source the user already enabled.** You only switch things on (and tune scouts off); existing enabled rows are someone's deliberate choice. - **Never enable a connected-tool source the user hasn't confirmed they use.** GitHub Issues, Linear, Zendesk, and pganalyze are ask-then-connect, never blind. - **Stay off the internal surfaces.** Don't call `signals-scout-emit-signal` or any scratchpad-write tool, and don't change a scout's `emit` flag or `run_interval_minutes` — on configs, this skill only flips `enabled`. **Canonical scout bodies are never edited.** New scout skills are created in exactly one place: step 6b, and only ones the user approved there. -- **Keep the scout fleet small.** Every enabled scout is a recurring LLM spend. Step 6 enables only `signals-scout-general` plus the **one or two** specialists for the products this project uses most — never error tracking or session replay (those reach the inbox as native sources) — and step 6b adds **at most two** custom scouts. Everything else stays disabled. +- **Keep the scout troop small.** Every enabled scout is a recurring LLM spend. Step 6 enables only `signals-scout-general` plus the **one or two** specialists for the products this project uses most — never error tracking or session replay (those reach the inbox as native sources) — and step 6b adds **at most two** custom scouts. Everything else stays disabled. - **Batch your questions.** `wizard_ask` has a small per-run budget; one multi-select beats four yes/nos. Don't skip a step or drop a connector (e.g. Linear) or custom scouts setup to save calls. - **Decline goes first.** Every `wizard_ask` that offers choices must include a plain-language decline option (skip / none / "keep what's there"), and it must be the **first** option so it is the default highlight — an accidental `enter` then declines instead of committing the user to something. The **one exception is step 3's GitHub gate**: the run cannot proceed without GitHub, so there the affirmative ("Done — I've installed it") stays first and the decline ("I can't connect right now", which aborts) stays last. diff --git a/context/skills/self-driving/references/2-read-context.md b/context/skills/self-driving/references/2-read-context.md index 3d5f638..36e2ede 100644 --- a/context/skills/self-driving/references/2-read-context.md +++ b/context/skills/self-driving/references/2-read-context.md @@ -20,7 +20,7 @@ Load via `ToolSearch select:Read,Glob,Grep,mcp__posthog-wizard__signals-scout-pr ## Do -1. **Read `./posthog-setup-report.md`.** It is ground truth for what the base integration instrumented **in this repo**: events, error tracking, feature flags. Do not re-derive what it already states. It is NOT authority over project-level facts — session replay in particular may be instrumented in another repo or via the snippet, so the report can rule replay in but never out (step 4 probes the server for that). +1. **Read `./posthog-setup-report.md` if it's there.** It's written only by a recent base-wizard integration run, so it's often absent — a project set up a while ago, manually, or via the snippet won't have one, a run whose report wasn't committed won't either, and a project that never ran the wizard never does. Treat its absence as **no signal** (skip to the profile and probes below), never as "nothing instrumented". When present, it is ground truth for what the base integration instrumented **in this repo**: events, error tracking, feature flags — do not re-derive what it already states. Either way it is NOT authority over project-level facts — session replay in particular may be instrumented in another repo or via the snippet, so the report can rule replay in but never out (step 4 probes the server for that). 2. **Call `signals-scout-project-profile-get`.** It returns products in use, connected integrations, warehouse sources, and the signal source configs split enabled/disabled — one call instead of four. It also carries **relative usage magnitude**: `top_events` (per-event count + distinct users), `recent_activity` (edits per scope), and per-entity active counts (feature flags, experiments, surveys, dashboards). Capture a rough sense of **which products this project uses most** — step 6 enables a scout only for the one or two most-used products, so a usage ranking matters, not just a binary in/out. **Tolerate failure**: it can 404 or error on a team without a profile yet. If it fails, fall back to the step-1 source list and the report; do not retry more than once and do not abort. **Note "profile unavailable" in your checklist** — a profile 404 is expected on a first-run team, so any later decision that relies only on the profile must record "unknown", not a confident negative. diff --git a/context/skills/self-driving/references/3-github.md b/context/skills/self-driving/references/3-github.md index e5463d8..538fbc2 100644 --- a/context/skills/self-driving/references/3-github.md +++ b/context/skills/self-driving/references/3-github.md @@ -33,7 +33,7 @@ Load via `ToolSearch select:mcp__posthog-wizard__integrations-list,mcp__wizard-t ``` { id: "github-connect", - prompt: "Self-driving needs GitHub access to investigate findings in your code and open fixes — setup can't finish without it.\n\nOpen this link to install the PostHog GitHub App in one click, then approve access. Grant it the repos you want Self-driving to work with — include this project's repo so step 5 can also watch its issues:\n\n\n\nThen come back here.\n\n(Need to re-link an existing installation instead? Use your integrations settings: .)", + prompt: "Self-driving needs GitHub access to investigate findings in your code and open fixes — setup can't finish without it.\n\nOpen this link to install the PostHog GitHub App in one click, then approve access. Grant it the repos you want Self-driving to work with — include this project's repo so step 5 can also watch its issues:\n\n\n\nThen come back here.", kind: "single", options: [ { label: "Done — I've installed it", value: "done" }, diff --git a/context/skills/self-driving/references/4-sources.md b/context/skills/self-driving/references/4-sources.md index f3b05a6..8a2e10b 100644 --- a/context/skills/self-driving/references/4-sources.md +++ b/context/skills/self-driving/references/4-sources.md @@ -4,7 +4,7 @@ next_step: 5-connected-tools.md # Step 4 — Enable native signal sources -Switch on the PostHog-native sources (the inbox's "Responders") that match what this product actually uses, per your step-2 checklist. Conditional means conditional: a source for a surface the product doesn't have just adds noise. +Switch on the PostHog-native sources (the inbox's "Responders") that match what this product actually uses, per your step-2 checklist. For most sources, conditional means conditional: one for a surface the product doesn't have just adds noise. **Error tracking and session replay are the exception — lean toward enabling them by default** (see the table), even with no current signal: teams adopt them sooner or later and an idle source costs nothing until data arrives. ## Status @@ -30,9 +30,9 @@ Load via `ToolSearch select:mcp__posthog-wizard__inbox-source-configs-create,mcp | Source | When | Payload | |---|---|---| -| Scout gate | **Always** — it lets the step-6 fleet's findings reach the inbox | `signals_scout` / `cross_source_issue` | -| Error tracking | Error tracking is in use anywhere: instrumented in this repo (report), exception autocapture ON (project-state block), or error issues exist (step-2 probe) | **All three rows**: `error_tracking` / `issue_created`, `error_tracking` / `issue_reopened`, `error_tracking` / `issue_spiking` — the product UI treats them as one switch | -| Session replay | Replay is enabled for the **project**: recording opt-in ON (project-state block) OR recordings exist (step-2 probe) OR the report says this repo instruments it. Opt-in ON with zero recordings still counts (recordings just haven't arrived yet). Skip only when all three say no/unknown, with reason "replay not enabled for this project" | `session_replay` / `session_analysis_cluster` — don't pass a `config`; the server injects the default sample rate. A 400 mentioning AI approval is unexpected (approval is enforced upstream) → skip this source and record a follow-up | +| Scout gate | **Always** — it lets the step-6 troop's findings reach the inbox | `signals_scout` / `cross_source_issue` | +| Error tracking | **Enable by default**, even with no current signal — teams adopt error tracking sooner or later, and with no errors there are no findings and no cost. Evidence (report, exception autocapture ON, or error issues from the step-2 probe) only raises confidence; its absence is **not** a reason to skip | **All three rows**: `error_tracking` / `issue_created`, `error_tracking` / `issue_reopened`, `error_tracking` / `issue_spiking` — the product UI treats them as one switch | +| Session replay | **Enable by default**, same reasoning — arm it now even with no current signal; recordings are only analyzed once they exist, so an idle source costs nothing and teams turn replay on eventually. Evidence (recording opt-in ON, recordings from the step-2 probe, or the report) only raises confidence; its absence is **not** a reason to skip | `session_replay` / `session_analysis_cluster` — don't pass a `config`; the server injects the default sample rate. A 400 mentioning AI approval is unexpected (approval is enforced upstream) → skip this source and record a follow-up | | Support | The team uses PostHog support/conversations (per the profile). If the profile was unavailable (step 2), don't record a confident skip — record "unknown — profile unavailable" + a follow-up to enable Support manually if they use it | `conversations` / `ticket` | ## Skip — do not create diff --git a/context/skills/self-driving/references/5-connected-tools.md b/context/skills/self-driving/references/5-connected-tools.md index b5661f0..e0c0193 100644 --- a/context/skills/self-driving/references/5-connected-tools.md +++ b/context/skills/self-driving/references/5-connected-tools.md @@ -27,7 +27,7 @@ Load via `ToolSearch select:mcp__wizard-tools__wizard_ask,mcp__posthog-wizard__e ``` { id: "connected-tools", - prompt: "Self-driving can also watch your other tools and pull their issues into the inbox. Which of these do you use?", + prompt: "Self-driving can also watch your other tools and investigate and fix the problems they surface. Which of these do you use?", kind: "multi", options: [ { label: "None of these", value: "none" }, diff --git a/context/skills/self-driving/references/5a-github.md b/context/skills/self-driving/references/5a-github.md index 9ae9bfe..4262bf1 100644 --- a/context/skills/self-driving/references/5a-github.md +++ b/context/skills/self-driving/references/5a-github.md @@ -16,8 +16,6 @@ Emit: Load via `ToolSearch select:mcp__posthog-wizard__integrations-github-repos-retrieve,mcp__posthog-wizard__external-data-sources-create`. -If `integrations-github-repos-retrieve` or `external-data-sources-create` isn't available (older server), skip the auto-create and record GitHub Issues as a dormant source (the dormant fallback below). **Not an abort.** - ## Do 1. **Infer the repository.** Run `git remote get-url origin` in the project root and parse `owner/repo` from either form (`git@github.com:owner/repo.git` or `https://github.com/owner/repo[.git]`). No remote, or not a github.com remote → go to the dormant fallback (below). @@ -69,6 +67,6 @@ If `integrations-github-repos-retrieve` or `external-data-sources-create` isn't - 400 mentioning credentials or repository access → dormant fallback (below). - Success returns the source `id` — record "connected by this setup (source id …, first sync started)". -5. **Dormant fallback** (no remote / repo not visible / create failed / tools unavailable): don't redirect the user and don't re-prompt — record **"picked but not connected"** and return to step 5, where the dormant responder is enabled and the follow-up recorded (same harmless posture as Zendesk — it only emits once a warehouse source syncs). When the cause was the repo not being visible to the App, the follow-up also tells the user to grant this repo to the PostHog GitHub App. A failed connector never dead-ends the run. +5. **Dormant fallback** (no remote / repo not visible / create failed): don't redirect the user and don't re-prompt — record **"picked but not connected"** and return to step 5, where the dormant responder is enabled and the follow-up recorded (same harmless posture as Zendesk — it only emits once a warehouse source syncs). When the cause was the repo not being visible to the App, the follow-up also tells the user to grant this repo to the PostHog GitHub App. A failed connector never dead-ends the run. Return to step 5 (responder enabling and class recording happen there). diff --git a/context/skills/self-driving/references/5b-linear.md b/context/skills/self-driving/references/5b-linear.md index 2b37673..1a8a392 100644 --- a/context/skills/self-driving/references/5b-linear.md +++ b/context/skills/self-driving/references/5b-linear.md @@ -14,8 +14,6 @@ Emit: Load via `ToolSearch select:mcp__posthog-wizard__external-data-sources-create` (`integrations-list` from step 3 stays loaded). -If `external-data-sources-create` isn't available (older server), skip this file and treat Linear as picked-but-not-connected — arm the dormant responder and add a follow-up (step 5's picked-but-not-connected path) — instead. **Not an abort.** - ## Do 1. **Check for an existing Linear integration**: call `integrations-list` and look for `kind: "linear"`. Present → skip ahead to create the source (below). diff --git a/context/skills/self-driving/references/6-scouts.md b/context/skills/self-driving/references/6-scouts.md index e4e29d0..7195ffd 100644 --- a/context/skills/self-driving/references/6-scouts.md +++ b/context/skills/self-driving/references/6-scouts.md @@ -2,16 +2,16 @@ next_step: 6b-tailor-scouts.md --- -# Step 6 — Configure the scout fleet +# Step 6 — Configure the scout troop -Scouts are the pull side of Signals: scheduled agents that scan the project on an interval and emit findings as `signals_scout` / `cross_source_issue` signals (which step 4's scout gate lets into the inbox). Every enabled scout is a recurring LLM spend — it costs a full run every tick even when it finds nothing — so the fleet is kept **deliberately small**: the `general` scout, plus the **one or two specialists** for the products this project uses most. Everything else is disabled. +Scouts are the pull side of Signals: scheduled agents that scan the project on an interval and emit findings as `signals_scout` / `cross_source_issue` signals (which step 4's scout gate lets into the inbox). Every enabled scout is a recurring LLM spend — it costs a full run every tick even when it finds nothing — so the troop is kept **deliberately small**: the `general` scout, plus the **one or two specialists** for the products this project uses most. Everything else is disabled. ## Status Emit: ``` -[STATUS] Configuring the scout fleet +[STATUS] Configuring the scout troop ``` ## Tools @@ -20,17 +20,17 @@ Load via `ToolSearch select:mcp__posthog-wizard__signals-scout-config-sync,mcp__ ## Do -1. **Materialize**: call `signals-scout-config-sync`. It is idempotent — it seeds the canonical scout skills for this team and creates any missing configs, then returns the fleet. +1. **Materialize**: call `signals-scout-config-sync`. It is idempotent — it seeds the canonical scout skills for this team and creates any missing configs, then returns the troop. - **Soft-degrade if the tool is missing or fails**: fall back to `signals-scout-config-list`. If that returns rows, tune those. If it returns nothing, the fleet hasn't been materialized yet — record a follow-up ("the scout fleet materializes automatically within ~30 minutes; tune it later in PostHog or re-run this setup") and continue to step 7. **Not an abort.** + **Soft-degrade if the tool is missing or fails**: fall back to `signals-scout-config-list`. If that returns rows, tune those. If it returns nothing, the troop hasn't been materialized yet — record a follow-up ("the scout troop materializes automatically within ~30 minutes; tune it later in PostHog or re-run this setup") and continue to step 7. **Not an abort.** -2. **Decide the enabled set — the whole point of this step is to enable FEW scouts, not many.** Work from the rows `signals-scout-config-sync` actually returned (the fleet grows over time — ~19 scouts today — so never hardcode a list). The enabled set has exactly three parts: +2. **Decide the enabled set — the whole point of this step is to enable FEW scouts, not many.** Work from the rows `signals-scout-config-sync` actually returned (the troop grows over time — ~19 scouts today — so never hardcode a list). The enabled set has exactly three parts: **(a) `general` — always enabled.** `signals-scout-general` watches cross-product correlations and the surfaces no specialist covers; it self-closes cheaply when there's nothing to say. Keep it on for every project. **(b) Never enable the `error-tracking` or `session-replay` scouts.** Step 4 already enables error tracking and session replay as native **sources** — their findings reach the inbox through that pipeline, so a scout on the same surface only duplicates it. Disable `signals-scout-error-tracking` and `signals-scout-session-replay` unconditionally, regardless of evidence. This is an **intentional** exclusion, not an evidence gap, so do **not** record a re-enable follow-up for them — note them as "covered by the native source". - **(c) One or two specialists — for the products this project uses MOST.** This is a judgment call, not a checklist: weigh ALL the step-2 evidence together — the profile's `top_events` (volume + distinct users), recent activity, the active counts for feature flags / experiments / surveys / dashboards, plus any repo signals — and pick the **one or two** product surfaces that are most actually used, then enable each one's scout. The candidate pool is the entire fleet **except** `general` and the two excluded in (b); it includes both the surface-specific scouts and the remaining cross-product ones: + **(c) One or two specialists — for the products this project uses MOST.** This is a judgment call, not a checklist: weigh ALL the step-2 evidence together — the profile's `top_events` (volume + distinct users), recent activity, the active counts for feature flags / experiments / surveys / dashboards, plus any repo signals — and pick the **one or two** product surfaces that are most actually used, then enable each one's scout. The candidate pool is the entire troop **except** `general` and the two excluded in (b); it includes both the surface-specific scouts and the remaining cross-product ones: | Scout | Specialist for | |---|---| @@ -56,16 +56,16 @@ Load via `ToolSearch select:mcp__posthog-wizard__signals-scout-config-sync,mcp__ - **At least one.** Always end with a specialist enabled. If no product surface clearly stands out — e.g. the only products in use are error tracking / session replay (excluded in (b)), or the profile was unavailable and nothing is rankable — **fall back to one universal cross-product scout** (`signals-scout-anomaly-detection` or `signals-scout-health-checks`) as the stand-in. Avoid `signals-scout-inbox-validation` as the fallback on a fresh setup — there are no shipped fixes for it to validate yet. - **A scout the table doesn't name** (posthog keeps adding them): treat it as a specialist candidate — read its description, judge whether its surface is among this project's most-used, and enable it only if it earns one of the ≤2 slots. -3. **Disable every scout you did NOT enable** in (a)–(c) — this is now most of the fleet. Disable via `signals-scout-config-update` with the config `id` and `{ enabled: false }` — **nothing else**. Don't touch `emit` (dry-run posture) or `run_interval_minutes`; the defaults are correct. A failed update is a follow-up, not an abort. +3. **Disable every scout you did NOT enable** in (a)–(c) — this is now most of the troop. Disable via `signals-scout-config-update` with the config `id` and `{ enabled: false }` — **nothing else**. Don't touch `emit` (dry-run posture) or `run_interval_minutes`; the defaults are correct. A failed update is a follow-up, not an abort. For each **surface-specific** scout you disabled, record a re-enable follow-up so the user can switch it on if they do use that surface later (e.g. "enable `signals-scout-logs` in PostHog if you use the logs product"). The error-tracking / session-replay disables are intentional (see (b)) — note them as "covered by the native source", not as a re-enable follow-up. 4. **Show the result.** This step asks the user nothing, so the only in-run visibility is the status line — after tuning, emit one naming the enabled set (short names, no `signals-scout-` prefix): ``` -[STATUS] Scout fleet: 3 active (general, product-analytics, feature-flags); 16 disabled +[STATUS] Scout troop: 3 active (general, product-analytics, feature-flags); 16 disabled ``` -(Adjust counts and names to the actual fleet and your decisions — the enabled set is always `general` + the 1–2 specialists, so "2 active" or "3 active" is expected; error-tracking and session-replay are deliberately among the disabled.) +(Adjust counts and names to the actual troop and your decisions — the enabled set is always `general` + the 1–2 specialists, so "2 active" or "3 active" is expected; error-tracking and session-replay are deliberately among the disabled.) Fresh configs have never run, so they're due immediately — the first scans fire on the next coordinator tick, within ~30 minutes. Record per-scout decisions (enabled / disabled + why) for the report. diff --git a/context/skills/self-driving/references/6b-tailor-scouts.md b/context/skills/self-driving/references/6b-tailor-scouts.md index 88a155d..ad7f60b 100644 --- a/context/skills/self-driving/references/6b-tailor-scouts.md +++ b/context/skills/self-driving/references/6b-tailor-scouts.md @@ -4,7 +4,7 @@ next_step: 7-report.md # Step 6b — Custom scouts for this product -The canonical fleet covers generic surfaces (errors, anomalies, observability gaps, health). You are the only actor in this pipeline that has read the repo — you know what the events *mean*, which ones form a funnel, and which domain surfaces matter. This step turns that into coverage: custom scouts for the watchable surfaces no canonical scout owns. +The canonical troop covers generic surfaces (errors, anomalies, observability gaps, health). You are the only actor in this pipeline that has read the repo — you know what the events *mean*, which ones form a funnel, and which domain surfaces matter. This step turns that into coverage: custom scouts for the watchable surfaces no canonical scout owns. **Canonical scout bodies are never edited** — not here, not anywhere in this setup. Tuning happens in step 6 (`enabled` flags only); new coverage happens here as new, separately-named scouts. This step is **propose-first and fully skippable**: nothing is created until the user approves, and a decline (or any tool failure) means you record the decision and continue to step 7. **Not an abort.** @@ -22,11 +22,11 @@ Load via `ToolSearch select:mcp__posthog-wizard__llma-skill-get,mcp__posthog-wiz ## Do -1. **Read the authoring guide.** `llma-skill-get {"skill_name": "authoring-signals-scouts"}` — step 6's sync seeded it into this team's skills store alongside the fleet. It defines the scout anatomy (quick close-out → orient → discriminator → explore patterns → save-memory → decide → disqualifiers → close-out), the emit contract, and the quality bar. Follow it for every scout you write; pull its bundled references via `llma-skill-file-get` only for the sections you need. +1. **Read the authoring guide.** `llma-skill-get {"skill_name": "authoring-signals-scouts"}` — step 6's sync seeded it into this team's skills store alongside the troop. It defines the scout anatomy (quick close-out → orient → discriminator → explore patterns → save-memory → decide → disqualifiers → close-out), the emit contract, and the quality bar. Follow it for every scout you write; pull its bundled references via `llma-skill-file-get` only for the sections you need. **Soft-degrade if it 404s** (older PostHog deploy that doesn't seed companions): read a canonical scout body via `llma-skill-get` (e.g. `signals-scout-general`) and use it as your only template. If neither is readable, record a follow-up ("add custom scouts once the authoring guide is available") and continue to step 7. -2. **Do the gap analysis — this is the thinking step, take it seriously.** Lay the project evidence (the setup report's event taxonomy above all, plus the step-2 checklist: funnel structure, payment/LLM/survey surfaces, warehouse sources, integrations) against what the canonical fleet already watches. For each candidate surface ask, in order: +2. **Do the gap analysis — this is the thinking step, take it seriously.** Lay the project evidence (the setup report's event taxonomy above all, plus the step-2 checklist: funnel structure, payment/LLM/survey surfaces, warehouse sources, integrations) against what the canonical troop already watches. For each candidate surface ask, in order: - **Is it watchable?** Concrete events with names you can list, a funnel with ordered steps, a domain loop with a success/failure pair. "It's a web app" is not a surface. - **Is it uncovered?** Three things can already own a surface, and a custom scout that duplicates any of them adds noise, not coverage: (1) a canonical scout step 6 kept enabled — the 1–2 specialists or `signals-scout-general`, which sweeps cross-product surfaces every run (e.g. generic anomalies belong to `signals-scout-anomaly-detection` if it was picked); (2) a **native source** — error tracking and session replay are consumed as sources in step 4, so never propose a custom scout for error bursts or replay analysis even though their canonical scouts are now disabled. If the surface is only watched by a canonical scout step 6 *disabled* (not `general`, not a native source), it is genuinely uncovered and fair game. - **Would its scout pass the quality bar?** You must be able to name its signal-vs-noise discriminator and 2–4 concrete explore patterns *before* proposing it. If you can't, the surface isn't ready for a scout — record it as a report note instead. @@ -36,7 +36,7 @@ Load via `ToolSearch select:mcp__posthog-wizard__llma-skill-get,mcp__posthog-wiz 3. **Propose them in ONE `wizard_ask`.** If the gap analysis surfaced **no** candidate, skip this ask entirely and go straight to the status line ("Custom scouts: none"). Otherwise emit one multi-select question — one option per proposed scout (**at most two**), plus a leading "none" option. Write everything for a **human who has never heard the word "scout"**: define the term once in the question `prompt`, in one plain sentence (e.g. "Scouts are scheduled checks that watch your data and flag issues for your inbox."). Each scout option carries a short `label` **and** a `description`: - **`label`** — a plain-language title of what it would watch for, in product terms — e.g. "Watch your signup funnel for conversion drops", not "signals-scout-signup-funnel". One short line. - **`description`** — one or two sentences saying **what it watches and what would make it speak up**, in words a product person reads naturally. This renders dimmed and wrapped beneath the label, so it is where the real explanation lives — **never leave it empty, and never collapse it back into the label.** Do **not** surface raw event names (`run_failed`/`run_started`), internal metric tokens (`p95 duration_s`, `not_matched/candidates_total`), or jargon labels like "Discriminator:" / "Not covered by:" — translate those into plain English. - - **Make the first option an explicit decline** so declining is always one keystroke away and is the safe default: `{ "label": "None — keep the canonical fleet", "value": "none", "description": "Skip custom scouts; the built-in fleet already covers this project." }`. It must be **first** — it is the default highlight, so a user who just presses enter declines rather than accidentally accepting a scout. + - **Make the first option an explicit decline** so declining is always one keystroke away and is the safe default: `{ "label": "None — keep the canonical troop", "value": "none", "description": "Skip custom scouts; the built-in troop already covers this project." }`. It must be **first** — it is the default highlight, so a user who just presses enter declines rather than accidentally accepting a scout. - Keep the machine name `signals-scout-` (prefix mandatory — anything else never runs) **internal**: you still need it for `llma-skill-create`, but it never appears in any text the user reads. Shape (one scout shown; add a second only if a second survived the filters): @@ -47,9 +47,9 @@ Load via `ToolSearch select:mcp__posthog-wizard__llma-skill-get,mcp__posthog-wiz { "id": "custom_scouts", "kind": "multi", - "prompt": "Scouts are scheduled checks that watch your data and flag issues for your inbox. Based on your project I found a gap the built-in fleet doesn't cover — add it, or none.", + "prompt": "Scouts are scheduled checks that watch your data and flag issues for your inbox. Based on your project I found a gap the built-in troop doesn't cover — add it, or none.", "options": [ - { "label": "None — keep the canonical fleet", "value": "none", "description": "Skip custom scouts; the built-in fleet already covers this project." }, + { "label": "None — keep the canonical troop", "value": "none", "description": "Skip custom scouts; the built-in troop already covers this project." }, { "label": "Watch your signup funnel for conversion drops", "value": "signals-scout-signup-funnel", "description": "Speaks up when sign-up completion falls below its recent norm, so a broken or regressed onboarding step gets caught fast." } ] } @@ -69,6 +69,6 @@ Load via `ToolSearch select:mcp__posthog-wizard__llma-skill-get,mcp__posthog-wiz [STATUS] Custom scouts: created run-pipeline; declined: none ``` -(adjust to the actual decisions; if nothing was warranted or the user declined everything, say "Custom scouts: none — canonical fleet covers this project".) +(adjust to the actual decisions; if nothing was warranted or the user declined everything, say "Custom scouts: none — canonical troop covers this project".) Record for the report: each created scout's design rationale (surface, discriminator, why no canonical covers it), surfaces you considered and ruled out (with the filter that killed them), declined proposals, and the noise escape hatch — if a scout turns out noisy, setting `emit: false` on its config in PostHog switches it to dry-run. diff --git a/context/skills/self-driving/references/7-report.md b/context/skills/self-driving/references/7-report.md index cde068f..8862c9c 100644 --- a/context/skills/self-driving/references/7-report.md +++ b/context/skills/self-driving/references/7-report.md @@ -23,7 +23,7 @@ Emit: - **GitHub** — connected (and whether it was already connected or connected during this run). - **Signal sources** — a table of every source you touched or deliberately skipped: `source_product` / `source_type`, action taken (enabled / already enabled / skipped + why / failed). - **Connected tools** — what the user picked, and per tool the step-5 class: "connected by this setup (source id …, first sync started)", "already connected" / "verified connected", "responder enabled but warehouse source not detected (dormant)", or "not used" (only for tools the user didn't pick). Never report a tool as connected unless this run created its source or saw it in `external-data-sources-list`. For sources this run created, note that only the responder-consumed table (issues / tickets) is syncing and more can be enabled in the UI. Any tool the user picked but didn't connect — whether they said "done" or skipped — is "selected but no source detected (dormant)" with a follow-up, never "user confirmed connecting" and never "not used". - - **Scout fleet** — kept-on scouts, disabled scouts with the one-line reason each, or the not-yet-materialized note from step 6. + - **Scout troop** — kept-on scouts, disabled scouts with the one-line reason each, or the not-yet-materialized note from step 6. - **Custom scouts** — from step 6b: each created scout (name, what it watches, its discriminator, and why no canonical scout covers it) or one line on why none was warranted; surfaces considered and ruled out, with the filter that killed each; declined proposals; and the noise escape hatch (set `emit: false` on a scout's config in PostHog to switch it to dry-run). Omit only if step 6b was skipped entirely. - **Follow-ups** — every follow-up recorded along the way, as a checklist. Omit the section if there are none. - **What happens next** — the scout coordinator picks up fresh configs within ~30 minutes; findings cluster into reports in the inbox; immediately-actionable ones can start coding tasks.