-
Notifications
You must be signed in to change notification settings - Fork 8
Reserved elements for AI #344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: planned
Are you sure you want to change the base?
Changes from all commits
9d6581c
d4e6244
a1b5633
2a47459
59a63eb
085aedf
113fa86
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| .. The COUNTER Code of Practice © 2017-2024 by COUNTER Metrics | ||
| is licensed under CC BY 4.0. To view a copy of this license, | ||
| visit https://creativecommons.org/licenses/by/4.0/ | ||
|
|
||
| .. _reserved-values: | ||
|
|
||
| Reserved Values Available for Extending Reports | ||
| ------------------------------------------------- | ||
|
|
||
| COUNTER's `Best Practice on Generative and Agentic AI Usage <https://www.countermetrics.org/code-of-practice/best-practice/bp-ai/>`_ created a new Access_Method and new Metric_Types specifically for reporting AI usage. These values are OPTIONAL extensions for inclusion in COUNTER Reports only when called for. | ||
|
|
||
|
|
||
| .. only:: latex | ||
|
|
||
| .. tabularcolumns:: |>{\raggedright\arraybackslash}\Y{0.17}|>{\raggedright\arraybackslash}\Y{0.26}|>{\parskip=\tparskip}\Y{0.46}|>{\raggedright\arraybackslash}\Y{0.11}| | ||
|
|
||
| .. list-table:: | ||
| :class: longtable | ||
| :widths: 15 22 51 12 | ||
| :header-rows: 1 | ||
|
|
||
| * - Element Name | ||
| - Value | ||
| - Description | ||
| - Reports | ||
|
|
||
| * - Access_Method | ||
| - Agent | ||
| - Content and metadata accessed by an AI system. | ||
| - PR, DR, TR, IR | ||
|
|
||
| * - Metric_Type | ||
| - AI_Responses_Generated | ||
| - A response delivered by an AI system in response to a user prompt. The response is likely to be text, but may include images or other multimedia. Each response MUST only be counted once regardless of the number of queries initiated by the AI system. Subsequent prompts within the same user session MUST be counted as a new AI_Responses_Generated. | ||
| - PR | ||
|
|
||
| * - Metric_Type | ||
| - Total_AI_Investigations | ||
| - Total number of times within a user session that a chunk from an Item or information related to an Item was included by an AI system in generating a response to a user prompt. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I still find the "with a user session" for the Total metrics and the Search metric confusing, since the session is only relevant for Unique metrics. The double click filtering does apply, but that information is missing.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Changing this will require going back to the AI working group. Since they are working on phase two of the project now, my recommendation is to hold this PR for a subsequent R5.1.2. |
||
| - PR, DR, TR, IR | ||
|
|
||
|
|
||
| * - Metric_Type | ||
| - Unique_AI_Investigations | ||
| - Unique count of times within a user session that a chunk from an Item or information related to an Item was included by an AI system in generating a response to a user prompt. | ||
| - PR, DR, TR, IR | ||
|
|
||
|
|
||
| * - Metric_Type | ||
| - Total_AI_Requests | ||
| - Total number of times within a user session that a chunk from an Item was requested (i.e. the full text or content was accessible to the AI system) in generating a response to a user prompt, during a user session. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "accessible to" or "accessed by"? Same question for Unique_AI_Requests.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Changing this will require going back to the AI working group. Since they are working on phase two of the project now, my recommendation is to hold this PR for a subsequent R5.1.2. |
||
| - PR, DR, TR, IR | ||
|
|
||
|
|
||
| * - Metric_Type | ||
| - Unique_AI_Requests | ||
| - Unique count of times within a user session that a chunk from an Item was requested (i.e. the full text or content was accessible to the AI system) in generating a response to a user prompt. | ||
| - PR, DR, TR, IR | ||
|
|
||
|
|
||
|
|
||
| Access_Method Agent is an OPTIONAL extension. Where report providers make Access_Method Agent available, it MUST only be included in COUNTER Reports. Where report providers make Access_Method Agent available, it MUST only be included in COUNTER Reports when called for by report consumers. Where Access_Method Agent is included in COUNTER Reports, it MUST be reported against AI Metric_Types. | ||
|
|
||
| The new AI Metric_Types are all OPTIONAL extensions. Where report providers make these AI Metric_Types available, they MUST only be included in COUNTER Reports. Where report providers make these AI Metric_Types available, they MUST only be included in COUNTER Reports when called for by report consumers. Where AI Metric_Types are included in COUNTER Reports, they MUST be reported against Access_Method Agent. | ||
Uh oh!
There was an error while loading. Please reload this page.