ci: add public API snapshot check

[marandaneto]

💡 Motivation and Context

Add a committed public API snapshot and CI check so accidental public API changes are caught during pull requests. The check path is read-only: when the snapshot is stale, it fails with instructions to run mix posthog.public_api --update and includes a generated diff instead of mutating the committed snapshot itself.

💚 How did you test it?

• mix format --check-formatted
• mix posthog.public_api --check
• Verified mix posthog.public_api --check --snapshot <stale temp snapshot> fails with a diff without mutating the stale snapshot file.

📝 Checklist

• I reviewed the submitted code.
• I added tests to verify the changes.
• I updated the docs if needed.
• No breaking change or entry added to the changelog.

If releasing new changes

• Ran sampo add to generate a changeset file

[github-actions]

posthog-elixir Compliance Report

Date: 2026-06-15 15:19:25 UTC
Duration: 107378ms

⚠️ Some Tests Failed

42/45 tests passed, 3 failed

---

Capture Tests

✅ 29/29 tests passed

View Details

Test	Status	Duration
Format Validation.Event Has Required Fields	✅	609ms
Format Validation.Event Has Uuid	✅	611ms
Format Validation.Event Has Lib Properties	✅	610ms
Format Validation.Distinct Id Is String	✅	609ms
Format Validation.Token Is Present	✅	609ms
Format Validation.Custom Properties Preserved	✅	610ms
Format Validation.Event Has Timestamp	✅	610ms
Retry Behavior.Retries On 503	✅	5615ms
Retry Behavior.Does Not Retry On 400	✅	2613ms
Retry Behavior.Does Not Retry On 401	✅	2613ms
Retry Behavior.Respects Retry After Header	✅	5615ms
Retry Behavior.Implements Backoff	✅	15626ms
Retry Behavior.Retries On 500	✅	5614ms
Retry Behavior.Retries On 502	✅	5614ms
Retry Behavior.Retries On 504	✅	5613ms
Retry Behavior.Max Retries Respected	✅	15626ms
Deduplication.Generates Unique Uuids	✅	622ms
Deduplication.Preserves Uuid On Retry	✅	5615ms
Deduplication.Preserves Uuid And Timestamp On Retry	✅	10621ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	5617ms
Deduplication.No Duplicate Events In Batch	✅	616ms
Deduplication.Different Events Have Different Uuids	✅	614ms
Compression.Sends Gzip When Enabled	✅	610ms
Batch Format.Uses Proper Batch Structure	✅	610ms
Batch Format.Flush With No Events Sends Nothing	✅	607ms
Batch Format.Multiple Events Batched Together	✅	615ms
Error Handling.Does Not Retry On 403	✅	2612ms
Error Handling.Does Not Retry On 413	✅	2613ms
Error Handling.Retries On 408	✅	5615ms

Feature_Flags Tests

⚠️ 13/16 tests passed, 3 failed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	9ms
Request Payload.Flags Request Uses V2 Query Param	✅	7ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	7ms
Request Payload.Flags Request Omits Authorization Header	✅	7ms
Request Payload.Token In Flags Body Matches Init	✅	7ms
Request Payload.Groups Round Trip	✅	7ms
Request Payload.Groups Default To Empty Object	✅	7ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It	✅	7ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	7ms
Request Payload.Disable Geoip Omitted Defaults To False	❌	7ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	7ms
Request Lifecycle.No Flags Request On Init Alone	✅	4ms
Request Lifecycle.No Flags Request On Normal Capture	✅	608ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	11ms
Request Lifecycle.Mock Response Value Is Returned To Caller	❌	7ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	❌	609ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['groups', 'distinct_id', 'api_key', 'flag_keys_to_evaluate', 'group_properties', 'person_properties']

request_lifecycle.mock_response_value_is_returned_to_caller

Last action result missing field 'value'. Keys: ['error', 'success']

side_effect_events.get_feature_flag_captures_feature_flag_called_event

Expected 1 events with name '$feature_flag_called', got 0

[greptile-apps]

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
lib/mix/tasks/posthog.public_api.ex:164-193
**`--check` silently mutates the committed snapshot file**

`write_generated_snapshot/2` writes directly to `path`, which is the same committed `public_api.snapshot`. A developer who runs `mix posthog.public_api --check` when their API has changed will see a failure, but the committed snapshot file will already be overwritten with the new content. At that point `git status` shows the file modified, and a hasty `git commit -am "..."` will silently accept the API change — exactly what this guard is designed to prevent.

The `git diff` hint in the error message does work this way (because the file is overwritten in-place), but the same goal can be achieved without touching the committed file. Consider writing to a side-car path (e.g. `path <> ".generated"`) and updating the hint to `git diff #{path} #{path}.generated`, keeping `--check` fully read-only.

_{Reviews (1): Last reviewed commit: "ci: add public API snapshot check" | Re-trigger Greptile}

[greptile-apps]

_{Reviews (2): Last reviewed commit: "fix: keep public API check read-only" | Re-trigger Greptile}

[marandaneto]

#137 (comment)