[CI] (97c6d9f) flask/flask3-social-media

[wizard-ci-bot]

Automated wizard CI run

Source: wizard-pr
Trigger ID: 97c6d9f
App: flask/flask3-social-media
App directory: apps/flask/flask3-social-media
Workbench branch: wizard-ci-97c6d9f-flask-flask3-social-media
Wizard branch: release-please--branches--main--components--wizard
Context Mill branch: main
PostHog (MCP) branch: master
Timestamp: 2026-06-23T16:00:47.147Z
Duration: 2643.6s

[wizard-ci-bot]

Now I have all the information needed to write the evaluation.

---

PR Evaluation Report

Summary

This PR integrates PostHog analytics into a Flask 3 social media (microblog) application. PostHog is initialized globally in create_app() using the module-level API, with 15 events captured across authentication, social interactions, content creation, messaging, and API routes. The integration uses the Python SDK's context-based API with new_context() and identify_context().

Files changed	Lines added	Lines removed
9	+118	-1

Confidence score: 4/5 👍

• No error tracking implemented: The app has existing error handlers (errors/handlers.py) but no capture_exception() calls were added. Flask requires manual exception capture per the docs. [CRITICAL]
• identify_context() uses usernames instead of stable IDs: All identify_context() calls pass user.username (a mutable string) rather than str(user.id) or str(user.pk). If a user changes their username, PostHog will treat them as a different person, fragmenting analytics data. [CRITICAL]
• No .env.example updated: The POSTHOG_PROJECT_TOKEN and POSTHOG_HOST env vars are not documented in a .env.example file — only .env (which is gitignored) exists. [MEDIUM]
• Module-level API used instead of instance-based Posthog() constructor: The framework rules specify using the Posthog() class constructor (instance-based API). The PR uses posthog.api_key / posthog.host module-level config. While this follows the Django docs pattern and works, the Flask docs recommend Posthog() constructor, and the rules explicitly state to use it. [MEDIUM]
• enable_exception_autocapture=True not set: The rules require this in the constructor. Since the module-level API is used, there's no constructor call at all, but even so, exception autocapture is not configured. [MEDIUM]

---

File changes

Filename	Score	Description
app/__init__.py	3/5	PostHog initialized globally in create_app() with atexit.register(posthog.shutdown). Uses module-level API instead of recommended Posthog() constructor.
app/auth/routes.py	3/5	5 events for auth flows (login, logout, register, password reset). Uses username as distinct ID.
app/main/routes.py	3/5	8 events for main app actions (posts, follows, messages, search, export). Good property enrichment but uses username as distinct ID.
app/api/tokens.py	3/5	1 event for API token issuance. Uses username as distinct ID.
app/api/users.py	3/5	1 event for API user creation. Uses username as distinct ID.
config.py	4/5	PostHog config vars loaded from environment correctly.
requirements.txt	5/5	posthog package added.
.gitignore	5/5	.env added to gitignore.
posthog-setup-report.md	4/5	Comprehensive setup report with event table and next steps.

---

App sanity check ⚠️

Criteria	Result	Description
App builds and runs	Yes	No syntax errors; all imports are valid standard PostHog SDK imports
Preserves existing env vars & configs	Yes	All existing config preserved; PostHog additions are additive
No syntax or type errors	Yes	All code is syntactically valid Python
Correct imports/exports	Yes	capture, identify_context, new_context, tag imported from posthog correctly
Minimal, focused changes	Yes	All changes directly relate to PostHog integration
Pre-existing issues	None	No pre-existing issues observed

Issues

• No .env.example for PostHog variables: POSTHOG_PROJECT_TOKEN and POSTHOG_HOST are read from environment but not documented in a .env.example file. Only .env exists and is gitignored. Collaborators won't know what to set. [MEDIUM]

Other completed criteria

• App builds without errors — valid Python syntax throughout
• Existing functionality preserved — no code removed or broken
• All imports resolve to the posthog package correctly
• Build configuration valid — requirements.txt is parseable

---

PostHog implementation ❌

Criteria	Result	Description
PostHog SDKs installed	Yes	posthog added to requirements.txt
PostHog client initialized	Yes	Initialized globally in create_app() via posthog.api_key and posthog.host, with atexit.register(posthog.shutdown) for clean shutdown
capture()	Yes	15 meaningful capture() calls across auth, main, and API routes
identify()	N/A	Server-only app
Error tracking	No	No capture_exception() calls added. Flask's built-in error handlers in errors/handlers.py require manual capture per docs. enable_exception_autocapture is also not configured.
Reverse proxy	N/A	Server-only app

Issues

• No error tracking: The Flask docs explicitly state that "PostHog's default exception autocapture won't work" in Flask and you need to "manually capture errors instead using capture_exception()." The existing errors/handlers.py has @bp.app_errorhandler(500) but no PostHog exception capture was added. Additionally, enable_exception_autocapture=True is not set. [CRITICAL]
• Module-level API instead of Posthog() constructor: The framework rules say "Always use the Posthog() class constructor (instance-based API) instead of module-level posthog.api_key config." The Flask docs also show Posthog() constructor usage. The PR uses posthog.api_key = ... instead. [MEDIUM]
• Unstable distinct IDs: All identify_context() calls use user.username which is mutable (users can edit their profile username). The PostHog Django docs recommend str(request.user.pk). Using str(user.id) would be the stable choice. [CRITICAL]

Other completed criteria

• PostHog SDK correctly added to dependencies
• API key loaded from environment variable via Config class
• Host correctly configured with US ingestion endpoint default
• atexit.register(posthog.shutdown) ensures events are flushed on exit
• Context-based API (new_context(), identify_context()) used correctly

---

PostHog insights and events ⚠️

Filename	PostHog events	Description
app/auth/routes.py	user_signed_up, user_logged_in, user_logged_out, password_reset_requested, password_reset_completed	Full auth lifecycle tracked with login method properties
app/main/routes.py	post_created, profile_updated, user_followed, user_unfollowed, post_translated, post_searched, message_sent, posts_export_started	Core social actions tracked with enriched properties (post_length, language, results_count, etc.)
app/api/users.py	api_user_created	API user creation with signup_method property
app/api/tokens.py	api_token_issued	API token issuance tracked

Issues

• user_followed and user_unfollowed missing target user property: These events don't include who was followed/unfollowed. The username variable is available in scope and should be added as a property for funnel analysis. [MEDIUM]
• tag('username', user.username) leaks PII-adjacent data: In login and register routes, tag('username', user.username) adds the username to event properties. While usernames are somewhat public in a social app, this is inconsistent — most other events don't include this tag. [LOW]

Other completed criteria

• Events represent real user actions mapping to actual product flows
• Events enable product insights — full signup-to-engagement funnel is trackable
• Most events include relevant contextual properties (post_length, language, results_count, signup_method, etc.)
• Event names are descriptive and use consistent snake_case convention
• No PII (emails, names, phones) in event properties

---

Reviewed by wizard workbench PR evaluator