Skip to content

docs: add A/B testing reference for plugin automations#278

Open
csmith49 wants to merge 1 commit into
mainfrom
docs/ab-testing-plugin-variants
Open

docs: add A/B testing reference for plugin automations#278
csmith49 wants to merge 1 commit into
mainfrom
docs/ab-testing-plugin-variants

Conversation

@csmith49
Copy link
Copy Markdown
Contributor

Summary

Adds documentation for the A/B testing feature introduced in OpenHands/automation#146.

Changes

  • New: skills/openhands-automation/references/ab-testing.md

    • Quick start example
    • How variant selection works (weighted random.choices at runtime)
    • Full API reference: request fields, variant object schema, validation rules
    • Observability via experiment_id / variant conversation tags
    • Three worked examples (version comparison, plugin combos, three-way test)
    • Limitations and link to OpenHands/automation#147 (server-level variant roadmap)
  • Updated: skills/openhands-automation/SKILL.md

    • Added A/B testing row to the "Choosing the Right Preset" table
    • Added references/ab-testing.md to the Reference Files section

Related

This PR was created by an AI agent (OpenHands) on behalf of csmith49.

Add references/ab-testing.md covering variant definitions, weighted
selection, experiment config, observability tags, validation rules,
and examples. Link from SKILL.md Reference Files section and the
Choosing the Right Preset table.

Relates to OpenHands/automation#146 and OpenHands/automation#147.

Co-authored-by: openhands <openhands@all-hands.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant