Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions context/skills/audit-attribution/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
type: skill
template: description.md
description: Audit a PostHog integration's marketing attribution capture for UTM survival, click-id coverage, cross-subdomain identity, and consent/cookieless interactions
tags: [best-practices]
cli:
role: command
parentCommand: audit
command: attribution
references:
preamble: "**Read ONLY this file.** Do not read any other reference file until this one tells you to."
shared_docs:
- https://posthog.com/docs/data/utm-segmentation.md
- https://posthog.com/docs/libraries/js/config.md
- https://posthog.com/docs/privacy/data-collection.md
- https://posthog.com/docs/getting-started/identify-users.md
variants:
- id: all
display_name: PostHog audit — attribution
tags: [best-practices]
docs_urls: []
70 changes: 70 additions & 0 deletions context/skills/audit-attribution/description.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# PostHog Audit — Attribution

This skill audits a PostHog integration's **marketing attribution capture** for data integrity. **Read-only** — the only file you create is the final audit report.

The check space here covers patterns the other audit skills don't: UTM survival across client-side routing and OAuth redirects, custom click-id capture (`gclid`, `fbclid`, `msclkid`), cross-subdomain identity, cookieless-mode tradeoffs against attribution measurement, and consent-banner load ordering. Failures here look like "ad-spend money disappears between ad click and signup event" — silent, expensive, and hard to debug after the fact.

## Workflow

The audit runs as a 4-step chain: Presence → Attribution capture (fix) → Attribution configuration → Report. Each step file ends with a pointer to the next. Follow them in order. Resolve each in order before any source-tree exploration.

**Start by reading the path relative to this file at `references/1-presence.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md.

`ToolSearch` is only for loading a tool by exact name when the SDK has it deferred (e.g. `select:Grep`). Do **not** use it to browse for other tools — every tool the audit needs (`Glob`, `Grep`, `Read`, `Write`, `Bash`, and the named `mcp__wizard-tools__audit_*` tools) is already named in this skill.

**Do not call `TaskCreate` / `TaskUpdate` / `TaskGet` / `TaskList`.** The audit doesn't track its own task list — progress comes from the audit ledger plus `[STATUS]` lines.

## Live activity — `[STATUS]`

The "Working on …" banner reads from `[STATUS]` lines you emit in plain text. Whenever you start a new sub-step, write a line like:

```
[STATUS] Detecting attribution surfaces
```

The wizard intercepts these and updates the spinner. Use them freely — they are cheap. Each step file lists the exact `[STATUS]` strings to emit at each sub-step.

## Audit checks ledger

The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**:

- `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call.

All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so.

### Check entry shape

- `id` — stable kebab-case slug. Reuse the existing seeded ids exactly when calling `audit_resolve_checks`.
- `area` — short group name. This skill uses `Attribution` (fix) and `Attribution — Configuration` (config).
- `label` — short human name.
- `status` — `pending` | `pass` | `error` | `warning` | `suggestion`.
- `file` — optional `path:line` for findings tied to a location.
- `details` — optional one-line explanation (or compact JSON for structured findings).

After the report is written (Step 4), delete `.posthog-audit-checks.json`.

## Severity levels

- `error`: Must fix. Broken attribution or guaranteed data corruption.
- `warning`: Should fix. Pattern that causes silent attribution loss or measurement degradation.
- `suggestion`: Nice to have. Best-practice improvement or coverage gap on a non-critical attribution surface.

## Investigation standards

Every finding produced by this skill must meet the standards in [posthog-best-practices/references/investigation-standards.md]: provenance on every claim, verification evidence inline, and adversarial self-review per area in the report. The skill's grep patterns and rule prose enforce provenance and evidence; the report step renders the per-area "Assumptions and blind spots" subsection.

## Key principles

- **Read-only**: Do not edit project source files. The only file you create is the audit report.
- **Evidence-based**: Reference specific `file:line` for every non-pass finding.
- **Actionable**: Every finding states what to fix and how.
- **Conservative on tenant-shape inference**: when there's no static signal that a project runs paid ad campaigns (no ad-platform pixels, no campaign-tracking parameters anywhere in the code, no marketing-site routes), resolve campaign-only checks with `pass` and `details: "skip: no paid-acquisition signal detected"` rather than warning on absence.

## Abort statuses

Report abort states with `[ABORT]` prefixed messages. The wizard catches these and terminates the run — do not halt yourself.
- No PostHog SDK initialization found

## Framework guidelines

{commandments}
64 changes: 64 additions & 0 deletions context/skills/audit-attribution/references/1-presence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
next_step: 2-attribution-fix.md
---

# Step 1 — Presence detector + seed the ledger

This step decides whether the rest of the audit has anything to look at, seeds the audit ledger, and records signals later steps need. Run it **before** any other work.

## Tools

Load via `ToolSearch select:Grep,mcp__wizard-tools__audit_seed_checks,mcp__wizard-tools__audit_resolve_checks` once at the start of this step. Subsequent steps reuse `audit_resolve_checks` to patch each check as it resolves, so it stays loaded.

## Status

Emit, in order:

```
[STATUS] Detecting PostHog and attribution surfaces
[STATUS] Seeding audit checklist
```

## Action

### a. Detect presence

Run **two `Grep` calls in parallel**, both with `output_mode: "files_with_matches"`:

1. PostHog init surface — any of:
`posthog\.init\(|new PostHog\(|posthog\.Posthog\(|Posthog\(`
2. Attribution / acquisition signals — any of:
`utm_source|utm_medium|utm_campaign|gclid|fbclid|msclkid|msfclkid|li_fat_id|ttclid|twclid|partner_id|referrer_id`

### b. Abort or continue

- **Init grep returns zero hits anywhere in the project:** emit `[ABORT] No PostHog SDK initialization found` and stop. The wizard catches `[ABORT]` and terminates the run. Do NOT seed the ledger in this case.
- **Init found:** continue to (c). Even projects with no explicit click-id capture rely on PostHog's built-in UTM auto-capture; each step's individual rules decide whether to skip or warn based on the kind of evidence present.

### c. Seed the audit ledger

The ledger lives at `.posthog-audit-checks.json` and renders live in the wizard sidebar / "Audit plan" tab. **The runtime does not pre-seed this skill's ledger** — call `mcp__wizard-tools__audit_seed_checks` directly here with the exact payload below. The tool replaces the file atomically, so calling it once at the start of every run is safe.

```json
{
"checks": [
{ "id": "attribution-utm-survives-landing", "area": "Attribution", "label": "UTM params survive landing-page routing", "status": "pending" },
{ "id": "attribution-survives-auth-redirect", "area": "Attribution", "label": "UTM params survive auth/OAuth redirects", "status": "pending" },
{ "id": "attribution-first-touch-set-once", "area": "Attribution", "label": "First-touch attribution properties use $set_once", "status": "pending" },
{ "id": "attribution-custom-click-ids", "area": "Attribution", "label": "Ad-platform click ids captured (gclid, fbclid, msclkid)", "status": "pending" },
{ "id": "attribution-on-conversion-events", "area": "Attribution", "label": "Conversion events carry attribution context", "status": "pending" },
{ "id": "attribution-cross-subdomain-cookie", "area": "Attribution — Configuration", "label": "cross_subdomain_cookie configured for multi-subdomain projects", "status": "pending" },
{ "id": "attribution-cookieless-mode-impact", "area": "Attribution — Configuration", "label": "cookieless_mode tradeoff acknowledged", "status": "pending" },
{ "id": "attribution-consent-integration", "area": "Attribution — Configuration", "label": "Consent banner wired to PostHog opt_in/opt_out", "status": "pending" },
{ "id": "write-report", "area": "Write report", "label": "Render posthog-audit-attribution-report.md", "status": "pending" }
]
}
```

## Record acquisition signal for later steps

Keep the acquisition-signal grep result in working memory. Step 2's `attribution-custom-click-ids` check uses it to decide whether to warn on absence (project clearly runs paid acquisition but doesn't capture click ids) vs. skip (no paid-acquisition signal anywhere — silence is fine). Step 3's `attribution-cookieless-mode-impact` reuses the same signal.

Do not read any project files in this step. Do not call `audit_resolve_checks`. Do not preload future steps.

Continue to **`2-attribution-fix.md`**.
Loading
Loading