diff --git a/AGENTS.md b/AGENTS.md index c21b8c54..9c44eb54 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,6 +31,7 @@ - [Reviewing pull requests](#reviewing-pull-requests) - [Keeping evals and mode-economics in sync](#keeping-evals-and-mode-economics-in-sync) - [When the rule fires](#when-the-rule-fires) + - [Voluntary usage-data sharing (opt-in)](#voluntary-usage-data-sharing-opt-in) - [Before submitting](#before-submitting) - [References](#references) @@ -986,6 +987,44 @@ re-run the affected eval suite anyway — it is cheap and protects against the false-negative case where a "clarification" actually changes how the model responds. +## Voluntary usage-data sharing (opt-in) + +Off unless the adopter has opted in via +`/project.md → economics_data_sharing: opt-in` (default +`off`). This is telemetry for the public good: the aggregate feeds +[`docs/mode-economics.md`](docs/mode-economics.md) and the long-term +ASF inference-endpoint capacity model, not any vendor. + +When — and only when — the adopter has opted in, end each skill session +by asking the maintainer whether to submit that session's token-economics +data as a pull request to +[`apache/magpie-economics`](https://github.com/apache/magpie-economics): + +- **Ask, never auto-submit.** The session-end offer is a proposal; the + maintainer confirms or declines each PR and may edit it first. Invoking + a skill is not consent to publish. If the adopter has **not** opted in, + say nothing — do not ask. +- **What the PR may contain:** the public upstream repository name; + token usage per skill invoked in the session (input and output counts); + the model or model-class used; and the theoretical token-use + calculation (counts × a published per-token rate, shown as a worked + estimate). +- **What the PR must never contain (hard rule):** any personal + information — names, handles, emails, or other identifiers for the user + or any contributor — and any subscription, plan, seat, quota, or + billing information. The data is about *token shape*, not what anyone + pays or which tier they are on. When in doubt, drop the field. +- **The submitter's own GitHub account is the one unavoidable exception.** + The PR is opened from the maintainer's own account and attributed to it + as author — that is inherent to how a pull request works, not payload + data. Make sure the maintainer understands the submission is attached to + their account before they confirm; it is their own identity, disclosed + voluntarily, and is separate from the anonymous payload. This never + extends to *third-party* identifiers, which stay excluded. + +One PR per submission. The maintainer edits it before it goes out if +they wish. + ## Before submitting - Re-read the diff and check that every change is intentional. diff --git a/docs/mode-economics.md b/docs/mode-economics.md index c6ce42bf..8be1cf89 100644 --- a/docs/mode-economics.md +++ b/docs/mode-economics.md @@ -18,6 +18,7 @@ - [Model class and mode cost shape](#model-class-and-mode-cost-shape) - [Local and self-hosted inference](#local-and-self-hosted-inference) - [Reducing costs](#reducing-costs) + - [Sharing your usage data (opt-in)](#sharing-your-usage-data-opt-in) - [Long-term: the ASF inference endpoint](#long-term-the-asf-inference-endpoint) - [Cross-references](#cross-references) @@ -249,6 +250,59 @@ paths use identical skill code to hosted paths. --- +## Sharing your usage data (opt-in) + +The ranges on this page are still mostly theoretical estimates. They get +more accurate as real adopters share what their invocations actually +cost. That sharing is **opt-in and never automatic**, and its data +payload carries no personal or billing information (with one unavoidable +caveat about PR authorship — see below). + +**How to opt in.** Set `economics_data_sharing: opt-in` in +`/project.md` (the default is `off`). See the field in the +template at +[`projects/_template/project.md`](../projects/_template/project.md#usage-data-sharing-opt-in). + +**What happens when you opt in.** At the end of each Magpie skill session, +the agent asks whether to submit that session's token-economics data as a +pull request to +[`apache/magpie-economics`](https://github.com/apache/magpie-economics), a +dedicated public repository that aggregates the data feeding this page. It +is a question, not an automatic upload: you confirm — or decline — every +submission, and you can edit the PR before it goes out. If you have not +opted in, the agent never asks. + +**What a submission contains:** + +- the public upstream repository name (already public information); +- token usage per skill invoked in the session (input and output counts); +- the model or model-class used; +- the theoretical token-use calculation — the counts multiplied by a + published per-token rate, shown as a worked estimate. + +**What a submission never contains:** + +- **Any personal information.** No names, handles, emails, or other + identifiers for you or any contributor. +- **Any subscription, plan, seat, quota, or billing information.** The data + describes *token shape*, not what you pay or which tier you are on. A + flat-rate subscriber and a pay-per-token user submitting the same + workload produce identical data. + +**One thing is inherently disclosed: your GitHub account.** Of course, the +submission is a pull request, so it is opened *from your own GitHub +account* and attributed to it as author — that is how a PR works, and it +cannot be otherwise. That is your own identity, revealed voluntarily by +choosing to submit; it is personal data, but it is *yours*, not a third +party's, and it is separate from the anonymous data payload above. If you +would rather not attach your account to the data, do not opt in — or open +the PR from an account you are comfortable associating with it. + +The cross-cutting rule that governs this session-end behaviour lives in +[`AGENTS.md` § Voluntary usage-data sharing](../AGENTS.md#voluntary-usage-data-sharing-opt-in). + +--- + ## Long-term: the ASF inference endpoint [MISSION.md § Affordability](../MISSION.md#affordability-and-vendor-neutrality--the-public-good-commitment) @@ -260,8 +314,10 @@ accepting a vendor's gift. This page's data — token counts per mode, per typical workload — is the quantitative input for the capacity planning and cost models that -endpoint will need. As pilot adopters accumulate real usage data, this -page will be updated with observed ranges rather than theoretical +endpoint will need. The [opt-in data-sharing program](#sharing-your-usage-data-opt-in) +above is how that evidence accumulates: as pilot adopters volunteer real +usage data to [`apache/magpie-economics`](https://github.com/apache/magpie-economics), +this page will be updated with observed ranges rather than theoretical estimates, so the endpoint sizing argument rests on evidence. --- @@ -271,3 +327,5 @@ estimates, so the endpoint sizing argument rests on evidence. - [`MISSION.md` § Affordability](../MISSION.md#affordability-and-vendor-neutrality--the-public-good-commitment) — the policy commitment behind this page. - [`docs/modes.md`](modes.md) — per-mode skill catalogue and maturity status. - [`docs/prerequisites.md`](prerequisites.md) — what you need to run the framework, including model-backend setup. +- [`apache/magpie-economics`](https://github.com/apache/magpie-economics) — the public repository that collects opt-in usage data feeding this page. +- [`AGENTS.md` § Voluntary usage-data sharing](../AGENTS.md#voluntary-usage-data-sharing-opt-in) — the session-end rule that governs opt-in submissions. diff --git a/projects/_template/project.md b/projects/_template/project.md index 10af41a8..b0b0f0b6 100644 --- a/projects/_template/project.md +++ b/projects/_template/project.md @@ -29,6 +29,7 @@ - [Release process](#release-process) - [Roster](#roster) - [Product](#product) + - [Usage-data sharing (opt-in)](#usage-data-sharing-opt-in) - [Pointers to sibling files](#pointers-to-sibling-files) @@ -498,6 +499,16 @@ product: affected_version_extract_prefix: "" ``` +## Usage-data sharing (opt-in) + +Off by default. Governs whether Magpie offers to contribute anonymised +token-economics data that improves +[`docs/mode-economics.md`](../../docs/mode-economics.md#sharing-your-usage-data-opt-in). + +| Key | Value | Notes | +|---|---|---| +| `economics_data_sharing` | `off` | `off` (default) or `opt-in`. When `opt-in`, each Magpie skill session ends by asking whether to submit that session's token-economics data as a pull request to [`apache/magpie-economics`](https://github.com/apache/magpie-economics). Never automatic — the maintainer confirms every submission. The PR carries only the public upstream repo name, per-skill token counts, the model / model-class, and the theoretical-cost calculation. The payload **never** contains personal information, and **never** contains subscription, plan, seat, quota, or billing details. The one unavoidable disclosure is that the PR is opened from the maintainer's own GitHub account (inherent to any PR); it is their own identity, not third-party data. See the hard rule in [`AGENTS.md` § Voluntary usage-data sharing](../../AGENTS.md#voluntary-usage-data-sharing-opt-in). | + ## Pointers to sibling files - [`release-trains.md`](release-trains.md) — fast-moving release state, release-manager attribution, security-team roster.