Skip to content

Fix telemetry normalization edge cases#34

Merged
WellDunDun merged 3 commits intodevfrom
WellDunDun/fix-telemetry-edge
Mar 10, 2026
Merged

Fix telemetry normalization edge cases#34
WellDunDun merged 3 commits intodevfrom
WellDunDun/fix-telemetry-edge

Conversation

@WellDunDun
Copy link
Collaborator

@WellDunDun WellDunDun commented Mar 10, 2026

This follow-up tightens the canonical telemetry boundary after PR #33 by making prompt-state recovery safer, preserving rollback payloads in body evolution, and hardening repair CLI failure handling. It also fixes Codex rollout metadata and prompt merging behavior, and keeps OpenCode adapter-only fields out of the legacy telemetry stream while improving text-only skill detection. Focused tests now cover custom canonical-log recovery, the body-evolution rollback payload, Codex observed-format prompt handling, and OpenCode whole-word matching plus telemetry hygiene. Verified with focused Biome checks and focused Bun tests covering normalization, body evolution, Codex rollout, OpenCode ingest, Claude hooks, and repair skill usage.

@coderabbitai
Copy link

coderabbitai bot commented Mar 10, 2026

Warning

Rate limit exceeded

@WellDunDun has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 10 minutes and 49 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7a353572-cb76-4254-870c-0e8a8398eee5

📥 Commits

Reviewing files that changed from the base of the PR and between 7a60ea7 and 936e4a4.

📒 Files selected for processing (4)
  • cli/selftune/evolution/evolve-body.ts
  • cli/selftune/ingestors/codex-rollout.ts
  • tests/evolution/evolve-body.test.ts
  • tests/ingestors/codex-rollout.test.ts
📝 Walkthrough

Walkthrough

Adds a canonicalLogPath parameter to prompt identity APIs and propagates it through hooks; strengthens prompt-state locking with metadata and heartbeat staleness; centralizes prompt-candidate resolution and skill-detection (whole-word); reuses shared skillUsage in eval set construction and updates audit detail formatting; unifies CLI error handling and adjusts tests.

Changes

Cohort / File(s) Summary
Canonical log & locking
cli/selftune/normalization.ts, cli/selftune/hooks/prompt-log.ts, cli/selftune/hooks/session-stop.ts, cli/selftune/hooks/skill-eval.ts
Extended reservePromptIdentity/getLatestPromptIdentity to accept canonicalLogPath; added lock metadata (owner_id, pid, acquired_at, heartbeat_at) and heartbeat-based staleness; increased lock timeout and propagated canonicalLogPath to call sites.
Eval set & audit details
cli/selftune/evolution/evolve-body.ts, tests/evolution/evolve-body.test.ts
Added createdAuditDetails() helper; read skillUsage earlier via readEffectiveSkillUsageRecords() and reuse for eval set construction; updated audit entries to include original_description: formatted content and adjusted test assertion.
Prompt ingestion / candidate resolution
cli/selftune/ingestors/codex-rollout.ts, tests/ingestors/codex-rollout.test.ts
Introduced rememberPromptCandidate() to consolidate prompt selection across session_meta/event_msg/turn_completed flows; refactored observed meta handling and added tests to ensure first actionable prompt retention and originator tracking.
Skill detection & telemetry
cli/selftune/ingestors/opencode-ingest.ts, tests/ingestors/opencode-ingest.test.ts
Added escapeRegExp() and containsWholeSkillMention() for case-insensitive whole-word skill matching; replaced simple substring checks; switched telemetry writing to explicit object construction and updated tests accordingly.
CLI error handling
cli/selftune/repair/skill-usage.ts
Wrapped main flow in try/catch, replaced direct process.exit usages with thrown Errors, and unified error logging message format while preserving existing CLI behavior and flags.
Tests: normalization canonical path
tests/normalization/normalization.test.ts
Added test to verify recovery of prompt identity/state from a custom canonicalLogPath (writes a custom canonical.jsonl and asserts correct recovery and next index).

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 34.62% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed Title 'Fix telemetry normalization edge cases' uses 'Fix' prefix (valid conventional commit) and clearly summarizes the main objective of the PR.
Description check ✅ Passed Description is detailed and directly related to the changeset, covering prompt-state recovery, rollback payloads, repair CLI, Codex metadata, OpenCode telemetry hygiene, skill detection, and test coverage.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch WellDunDun/fix-telemetry-edge

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 64631ac53c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@cli/selftune/evolution/evolve-body.ts`:
- Around line 184-185: The code calls readEffectiveSkillUsageRecords directly
which breaks the dependency-injection pattern; change the reference to use the
injected dependency (e.g., _deps.readEffectiveSkillUsageRecords) and update any
places that construct or destructure _deps so the function is provided; keep
createdAuditDetails as-is but ensure createdAuditDetails and any callers still
use the injected readEffectiveSkillUsageRecords via _deps to preserve
testability and consistency with other dependencies.

In `@cli/selftune/ingestors/codex-rollout.ts`:
- Around line 205-219: The helper rememberPromptCandidate currently overwrites a
single prompt variable for both the "first actionable" prompt and the
"last_user_query", causing last_user_query to point to the first actionable
turn; fix by introducing a separate variable (e.g., lastPrompt or lastUserQuery)
and update rememberPromptCandidate so it sets prompt only once when the first
actionable message is seen (using hasActionablePrompt and prompt) but always
updates lastPrompt with the most recent non-empty user message; update any
downstream emission sites that used prompt for last_user_query to use the new
lastPrompt variable instead (references: rememberPromptCandidate, prompt,
hasActionablePrompt, last_user_query).
- Around line 234-245: The code uses unsafe TypeScript assertions like
(payload.id as string) to populate observedSessionId/observedCwd and
observedMeta fields; replace these with runtime type checks by testing typeof
payload.id === "string" (and similarly for payload.cwd, payload.model_provider,
payload.model, payload.originator) before assigning to observedSessionId,
observedCwd or setting properties on observedMeta so only real strings are used;
update the assignment logic in the block that touches observedSessionId,
observedCwd and observedMeta to use these typeof guards and leave values
undefined or skip properties when the check fails.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5431ccdd-694e-4fe2-b0ca-695edc2e3248

📥 Commits

Reviewing files that changed from the base of the PR and between 64631ac and 7a60ea7.

📒 Files selected for processing (4)
  • cli/selftune/evolution/evolve-body.ts
  • cli/selftune/ingestors/codex-rollout.ts
  • tests/evolution/evolve-body.test.ts
  • tests/ingestors/codex-rollout.test.ts

@WellDunDun WellDunDun merged commit 0d753ce into dev Mar 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant