feat: guided onboarding with 18 multi-stage steps

[shuveb]

Summary

• Expand onboarding from 14 to 18 steps by inserting 4 new feature-exploration steps (6-9) between old steps 5 and 6
• Each step uses multi-stage tooltip-guided flow with highlights, pulsing rings, scroll-into-view, and OK buttons instead of direct navigation
• Fix @mfbt AI → @MFBTAI branding consistency
• Add data-onboarding attributes to FeatureRow, ModuleCard, GroundingBranchTabs, AppTopNav, project-settings for tooltip targeting
• Backend: renumber all step references (6→10, 7→11, etc.) across services, workers, MCP metadata; update VALID_STEPS to 1-18
• Global click-to-dismiss tooltips, auto-scroll checklist to next step on completion

New Steps (6-9)

• Step 6: Understand System Features (highlight + OK tooltip on system module card)
• Step 7: Understand User Features (highlight + OK tooltip on user module card)
• Step 8: Notice Missing Specs & Prompt Plans (highlight content indicator icons)
• Step 9: Create a Spec from Conversation (multi-stage: feature row → detail → Create Implementation button)

Multi-Stage Improvements to Existing Steps

• Step 10 (old 6): Highlight feature fix(frontend): use CSS transition for sidebar collapse instead of unm… #11 row first, then auto-prefill + highlight send button
• Steps 11/13: Show waiting message when decision summarizer is running
• Step 12 (old 8): Highlight feature Fix project chat code exploration loop and duplicate messages #12 row first, then scroll to MCQ on detail page
• Step 14 (old 10): 3-stage tour: Settings nav → MCP tab → API Key button
• Step 15 (old 11): Modal with coding agent setup instructions + copyable prompt
• Step 16 (old 12): 2-stage: Grounding nav → main branch tab (completes on click)
• Step 17 (old 13): OK-button tooltip on send button (no auto-dismiss)
• Step 18 (old 14): Dismiss-to-highlight Create Implementation flow

Test plan

• Backend tests pass (uv run pytest tests/test_onboarding.py -v — 14/14)
• Frontend compiles (npx tsc --noEmit)
• ESLint passes (npx eslint --fix exits 0)
• Ruff lint + format pass
• Manual walkthrough of all 18 steps end-to-end
• Verify step renumbering doesn't break mid-onboarding users (reset steps in DB)
• Verify Docker worker restart picks up backend step number changes

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR expands guided onboarding from 14 to 18 steps by inserting 4 new feature-exploration steps (6–9) and renumbering all downstream references, backed by a new OnboardingService, a DB migration, two new API endpoints, a 315-line OnboardingContext, and a 1936-line OnboardingChecklistPanel with multi-stage tooltip flows. It also introduces semaphore-based worker concurrency (WORKER_CONCURRENCY, default 5), fixes phase-less feature deep-linking, and adds is_sample to ProjectResponse for frontend use.

Key findings:

• Duplicate spec-generation onboarding hook with conflicting logic — generation.py adds a spec hook at the start of feature_content_generate_handler using the old feature-key-agnostic assignment. feature_content_version_service.py (same PR) adds a corrected feature-key-aware hook that fires later. Because the generation.py hook commits first, a user who generates a spec for feature-012 before feature-010 will have step 9 incorrectly marked complete — corrupting their onboarding progress.

• Polling-loop cleanup gap in OnboardingChecklistPanel — the step-14 MCP tour effect (and similar loops for steps 1, 10, 12, 16) starts chained setTimeout polling but only cancels the outer timer in the cleanup function.

• Previously flagged issues resolved: is_onboarded guard, stale step-number comments, logger.warning trace.

• Worker concurrency via asyncio.Semaphore is correctly implemented with done_callback for release on both success and exception.

• The phase-less feature deep-link routing fix is clean and well-tested.

Confidence Score: 4/5

Safe to merge after removing the duplicate onboarding hook in generation.py; all other changes are well-guarded and tested.

One P1 remains: the duplicate spec-generation hook in generation.py uses wrong feature-key-agnostic logic and fires before the correct service-level hook, permanently corrupting onboarding step data for users who generate specs out of order. Removing that block is a small change with no risk. All P2 findings are non-blocking.

backend/workers/handlers/generation.py (lines 1573-1601) — remove the duplicate spec-generation onboarding hook

Important Files Changed

Filename	Overview
backend/workers/handlers/generation.py	Adds spec-generation onboarding hook, but uses old feature-key-agnostic step logic that duplicates and conflicts with the corrected hook in feature_content_version_service.py — can mark wrong steps when user generates specs out of order.
backend/app/services/feature_content_version_service.py	Adds corrected feature-key-aware onboarding hook for steps 9/11/13 — correct logic, but fires redundantly after the conflicting generation.py hook.
frontend/components/sidebar/OnboardingChecklistPanel.tsx	1936-line panel with multi-stage tooltip logic for all 18 steps; inner polling-loop timeouts not cleaned up on unmount in several effects.
backend/app/services/onboarding_service.py	New service; idempotent step completion, flag_modified for JSON mutation, dismiss one-way transition — all correct.
frontend/lib/contexts/OnboardingContext.tsx	New context; session-dedup via firedStepsRef, WebSocket subscriptions for step completion and project-list changes, localStorage state — well-structured.
backend/workers/core/worker.py	Adds semaphore-based concurrency (WORKER_CONCURRENCY env var, default 5), getmany batch size, and done_callback for semaphore release — correct implementation.
backend/app/routers/auth.py	Adds /me/onboarding and /me/onboarding/steps endpoints; previously-flagged is_onboarded guard is now correctly applied.
backend/tests/test_onboarding.py	14 tests covering service unit tests and API endpoint tests — idempotency, invalid steps, unauthenticated access all covered.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: backend/workers/handlers/generation.py
Line: 1573-1601

Comment:
**Duplicate onboarding hook uses stale feature-key-agnostic logic**

This block fires at the *start* of `feature_content_generate_handler` — before the spec is written — and uses the old "next-incomplete" assignment (`step = 9 if 9 not in _completed else (11 if … else 13)`) that ignores which feature triggered the generation.

`FeatureContentVersionService.create_or_update` already has a corrected hook (added in this same PR) that maps feature-key suffixes (`-010→9`, `-011→11`, `-012→13`) and fires *after* the spec is saved. Both hooks fire for every spec-generation event, but in this order:

1. **This hook (wrong logic)** — commits first, marks step 9 for *any* feature whose spec is generated when step 9 is not yet done.
2. **Service hook (correct logic)** — re-queries the user, sees step 9 already committed, then correctly maps the feature key and marks the right step.

If a user navigates directly to feature-012 and generates its spec before feature-010, both step 9 (incorrectly) **and** step 13 (correctly) are marked complete. The user's onboarding progress is permanently corrupted — step 9 shows as done without the user ever generating a spec for feature-010.

Since `feature_content_version_service.py` now handles this correctly with feature-key awareness, this entire block in `generation.py` should be removed.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: frontend/components/sidebar/OnboardingChecklistPanel.tsx
Line: 2853-2918

Comment:
**Inner polling timeouts not cancelled on unmount**

The `pollForApiKey` loop chains up to 30 × 500 ms `setTimeout` calls. The effect's cleanup only cancels the outer `timer` (`return () => clearTimeout(timer)`), so if the component unmounts after `timer` has already fired, the inner polling chain keeps running for up to 15 seconds.

The same pattern exists in `tryHighlightBranch` (step 16), `tryFindMcq` (step 12), `tryHighlightSend` (step 10), and `tryHighlight` (step 1). Consider tracking a `cancelled` ref inside each effect and checking it at the top of each recursive call, then setting it in the cleanup function so all pending iterations stop immediately on unmount.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (3): Last reviewed commit: "style: apply ruff format to alembic migr..." | Re-trigger Greptile}