Skip to content

docs(ai-evals): Document opt-in evaluation report inbox signals#18140

Open
inkeep[bot] wants to merge 2 commits into
masterfrom
docs-writer-ai-update-2026-07-02T17-43-54-895Z-6699m3
Open

docs(ai-evals): Document opt-in evaluation report inbox signals#18140
inkeep[bot] wants to merge 2 commits into
masterfrom
docs-writer-ai-update-2026-07-02T17-43-54-895Z-6699m3

Conversation

@inkeep

@inkeep inkeep Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Documents the new opt-in behavior for AI observability evaluation report signals introduced in PostHog/posthog#67912.

Changes

contents/docs/ai-evals/index.mdx

  • Added new "### Inbox signals" subsection under "Evaluation reports" explaining:
    • Evaluation reports can appear as signals in the inbox
    • Users must enable the "AI observability" source in inbox signal sources settings
    • This is opt-in and disabled by default

contents/docs/self-driving/inbox/sources.mdx

  • Added "AI observability" to the PostHog sources list
  • Updated intro paragraph to include AI evaluation reports as a signal type

Why

PR #67912 changed evaluation report signals from being always-on to being opt-in via a team-level signal source configuration. Users now need to explicitly enable the "AI observability" source to receive these signals in their inbox. The documentation needs to reflect this new behavior.


This PR was created by the Inkeep Content Writer agent, which is maintained by the Docs and Wizard team. Please reach out on Slack for help if needed.

inkeep Bot added 2 commits July 2, 2026 17:45
Reflects PR #67912 which adds a new evaluation_report signal source
type for AI observability. The source is opt-in and lets teams receive
inbox signals when evaluation report runs complete.
@inkeep inkeep Bot requested a review from oliverb123 July 2, 2026 17:46
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Deploy preview

Status Details Updated (UTC)
🟢 Ready View preview Jul 02, 2026 05:58PM

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Vale prose linter → found 11 errors, 7 warnings, 4 suggestions in your markdown

Full report → Copy the linter results into an LLM to batch-fix issues.

Linter being weird? Update the rules!

contents/docs/ai-evals/index.mdx — 9 errors, 7 warnings, 4 suggestions
Line Severity Message Rule
16:98 warning Avoid trivializing words. 'easy to' can sound dismissive to the reader. PostHogDocs.Trivializers
26:156 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
51:60 warning Capitalize 'Workflows' for PostHog's product. Use 'workflows' for the general industry concept. PostHogBase.ProductNames
98:52 suggestion Address the reader directly. Use 'you' instead of 'the user'. PostHogDocs.DirectAddress
141:4 warning 'Code-based evaluations (Hog)' heading should be in sentence case, and product names should be capitalized. PostHogBase.SentenceCase
155:15 warning 'globals' is a possible misspelling. PostHogBase.Spelling
166:63 suggestion Address the reader directly. Use 'you' instead of 'the user'. PostHogDocs.DirectAddress
182:5 warning 'Writing Hog evaluation code' heading should be in sentence case, and product names should be capitalized. PostHogBase.SentenceCase
184:115 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
266:63 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
278:49 suggestion Address the reader directly. Use 'you' instead of 'the user'. PostHogDocs.DirectAddress
278:220 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
282:245 suggestion Address the reader directly. Use 'you' instead of 'the user'. PostHogDocs.DirectAddress
284:67 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
318:70 warning Capitalize 'Workflows' for PostHog's product. Use 'workflows' for the general industry concept. PostHogBase.ProductNames
338:5 warning 'Cooldown' is a possible misspelling. PostHogBase.Spelling
338:90 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
350:45 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
402:45 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
404:44 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
contents/docs/self-driving/inbox/sources.mdx — 2 errors, 0 warnings, 0 suggestions
Line Severity Message Rule
21:200 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash
21:287 error Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. PostHogBase.EnDash

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Bundle report

Total JS (gzip)

6.29 MiB (+0.0 KiB / +0.0%)

Eager graph (static-import closure per entrypoint)

Entrypoint Eager size Budget Modules
app 24.21 MiB (+1.8 KiB / +0.0%) report-only 5516
Largest modules in the app closure
Module Size
css ./node_modules/.pnpm/css-loader@5.2.7_webpack@5.101.3/node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[1]!./node_modules/.pnpm/postcss-loader@4.3.0_postcss@8.5.6_webpack@5.101.3/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[2]!./src/styles/global.css 708.0 KiB
./src/components/Stickers/Stickers.tsx 696.4 KiB
./.cache/caches/gatsby-plugin-mdx/mdx-scopes-dir/31a094f140f119e73085d847ae81b99b.js + 2 modules 581.1 KiB
./node_modules/.pnpm/@radix-ui+react-icons@1.3.2_react@18.3.1/node_modules/@radix-ui/react-icons/dist/react-icons.esm.js 481.4 KiB
./node_modules/.pnpm/@codemirror+view@6.38.2/node_modules/@codemirror/view/dist/index.js 458.1 KiB
./node_modules/.pnpm/rehype-raw@7.0.0/node_modules/rehype-raw/lib/index.js + 29 modules 395.1 KiB
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.cjs.js 364.8 KiB
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js 354.8 KiB
./src/hooks/useCustomers.tsx + 54 modules 353.9 KiB
./node_modules/.pnpm/react-markdown@8.0.7_@types+react@16.14.66_react@18.3.1/node_modules/react-markdown/lib/react-markdown.js + 88 modules 351.4 KiB
./node_modules/.pnpm/cloudinary-core@2.14.0_lodash@4.17.21/node_modules/cloudinary-core/cloudinary-core.js 281.9 KiB
./src/components/ProductComparisonTable/index.tsx + 116 modules 267.6 KiB
./node_modules/.pnpm/@codesandbox+sandpack-react@2.20.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@codesandbox/sandpack-react/dist/index.mjs 266.6 KiB
./node_modules/.pnpm/d3@7.9.0/node_modules/d3/src/index.js + 208 modules 247.4 KiB
./src/components/Pricing/PricingSlider/Slider.tsx + 87 modules 239.9 KiB

Eager-graph budgets are report-only until a baseline is established. Sizes are gzip of public/**/*.js; eager size is webpack module source bytes.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Hey @oliverb123! This docs PR was generated for you by an agent. You're responsible for reviewing and merging it into production.

  1. Review and approve the PR
  2. Merge the PR
    a. Check the preview env and CI build and merge it yourself
    b. Enable auto-merge if you're confident in the changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants