Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ Table 3.p (below): List of Data_Type Values
PR_P1

* - Unspecified
- Content that cannot be classified by any of the other Data_Types due to lack of sufficient information. Note that report providers are expected to make all reasonable efforts to classify the content. Using Unspecified will give rise to a Warning in the Validation Tool.
- Content that cannot be classified by any of the other Data_Types due to lack of sufficient information. Note that report providers are expected to make all reasonable efforts to classify the content. Using Unspecified will give rise to a Warning in the COUNTER Validator.
- A&I_Database\ |br|\ |lb|
Aggregated_Full_Content\ |br|\ |lb|
Data_Repository\ |br|\ |lb|
Expand Down Expand Up @@ -972,4 +972,4 @@ The value for an element might be missing or unknown, for example a title might

For clarity, values such as “unknown”, “n/a” or “-” MUST NOT be used.

If a non-empty value is required for an element and the value is empty or the element is omitted, the COUNTER Validation Tool reports a (Critical) Error which would cause the report to fail an audit. If Title, Item or Publisher is empty or Data_Type Unspecified is used, the COUNTER Validation Tool reports a Warning which might affect the result of an audit. See :numref:`validation-tool` for details on the error levels used by the Validation Tool.
If a non-empty value is required for an element and the value is empty or the element is omitted, the COUNTER Validator reports a (Critical) Error which would cause the report to fail an audit. If Title, Item or Publisher is empty or Data_Type Unspecified is used, the COUNTER Validator reports a Warning which might affect the result of an audit. See :numref:`counter-validator` for details on the error levels used by the COUNTER Validator.
4 changes: 2 additions & 2 deletions source/09-audit/01-the-audit-process.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ Stages of the Audit Process

Audits follow a five-stage process.

* **Stage one** - pre-flight preparation. Report providers SHOULD run all of their COUNTER reports (JSON and tabular) through the COUNTER Validation Tool. Report providers SHOULD also seek feedback from large report consumers (e.g. JUSP) in respect of issues they have identified with the reports. Report providers SHOULD fix any problems that are discovered and re-run the Validation Tool before proceeding to Stage two.
* **Stage two** - audit initiation. Auditor and report provider agree on the scope of the audit, which MUST include all COUNTER Reports and Standard Views of COUNTER Reports required by the Code of Practice but SHOULD include all COUNTER Reports and Standard Views of COUNTER Reports offered by the report provider. COUNTER’s Executive Director will adjudicate if the auditor and report provider cannot agree on the audit scope. Report providers MUST provide pre-flight documentation to the auditor, using the template provided in :ref:`Appendix E <appendix-e>`. If Stage one has been followed, report providers MUST provide the Validation Tool results to the auditor.
* **Stage one** - pre-flight preparation. Report providers SHOULD run all of their COUNTER reports (JSON and tabular) through the COUNTER Validator. Report providers SHOULD also seek feedback from large report consumers (e.g. JUSP) in respect of issues they have identified with the reports. Report providers SHOULD fix any problems that are discovered and re-run the COUNTER Validator before proceeding to Stage two.
* **Stage two** - audit initiation. Auditor and report provider agree on the scope of the audit, which MUST include all COUNTER Reports and Standard Views of COUNTER Reports required by the Code of Practice but SHOULD include all COUNTER Reports and Standard Views of COUNTER Reports offered by the report provider. COUNTER’s Executive Director will adjudicate if the auditor and report provider cannot agree on the audit scope. Report providers MUST provide pre-flight documentation to the auditor, using the template provided in :ref:`Appendix E <appendix-e>`. If Stage one has been followed, report providers MUST provide the COUNTER Validator results to the auditor.
* **Stage three** - seeding, audit begins. In this stage the auditors perform actions on the audited platforms in line with :ref:`Appendix E <appendix-e>`. Seeding takes a full calendar month.
* **Stage four** - report reconciliation. The auditors reconcile the COUNTER reports generated by the report provider with the seeding actions they undertook in Stage three. The outputs from this stage may be an Interim Report or a Pass Report, which MUST be shared with the COUNTER Executive Director. If an Interim Report is issued,

Expand Down
2 changes: 1 addition & 1 deletion source/09-audit/03-timelines.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Extensions

Report providers may request extensions to the usual timelines under the following circumstances.

* **Delayed seeding** - where a report provider is able to demonstrate that they have run the COUNTER Validation Tool and are in the process of fixing problems identified by the tool before commencing an audit, they may apply for up to three months delay in the seeding process.
* **Delayed seeding** - where a report provider is able to demonstrate that they have run the COUNTER Validator and are in the process of fixing problems identified by the validator before commencing an audit, they may apply for up to three months delay in the seeding process.
* **Extension to the fix period** - where an Interim Report has identified significant problems with COUNTER reports, the report provider may apply for an extension of the fix period from three to up to six months.

Applications for extensions MUST be sent to both the Executive Director (code@countermetrics.org) and the auditor, and will be considered on a case-by-case basis. Only one extension of each type will be granted for any audit, and all extensions will be tracked in the Registry.
Expand Down
Comment thread
berndoberknapp marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
is licensed under CC BY 4.0. To view a copy of this license,
visit https://creativecommons.org/licenses/by/4.0/

.. _validation-tool:
.. _counter-validator:

COUNTER Validation Tool
-----------------------
COUNTER Validator
-----------------

The `COUNTER Validation Tool <https://www.countermetrics.org/code-of-practice/tools/>`_ allows report providers and auditors to quickly perform compliance checks related to format and consistency of both JSON and tabular reports. Report providers SHOULD use this free tool to check their reports and COUNTER API implementation and fix issues detected by the tool before they begin the audit. It is recommended to use the tool not just when preparing for the audit, but to integrate the testing in the regular QA processes or at least test regularly to make sure the reports stay compliant. We recommend testing reports using the Validation Tool once every four months at a minimum, and monthly where possible.
The `COUNTER Validator <https://validator.countermetrics.org/>`_ allows report providers and auditors to quickly perform compliance checks related to format and consistency of both JSON and tabular reports. Report providers SHOULD use this free validator to check their reports and COUNTER API implementation and fix issues detected by the validator before they begin the audit. It is recommended to use the validator not just when preparing for the audit, but to integrate the testing in the regular QA processes or at least test regularly to make sure the reports stay compliant. We recommend testing reports using the COUNTER Validator once every four months at a minimum, and monthly where possible.

The COUNTER Validation Tool uses the following error levels to report issues with different severity:
The COUNTER Validator uses the following error levels to report issues with different severity:

* **Fatal error** - Validation was aborted because an unrecoverable error was encountered, for example a missing or invalid Reporting_Period. The fatal error MUST be fixed before the report can be fully validated.
* **Critical error** - Validation has detected a serious error that MUST be fixed, for example inconsistent data like a title with more Unique_Item_Requests than Total_Item_Requests or missing data. A critical error indicates that there could be a serious issue that would cause the audit to fail.
Expand Down
2 changes: 1 addition & 1 deletion source/09-audit/05-inclusion-in-registry.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Application for inclusion in the COUNTER Registry

The Registry of report providers and their platforms for which COUNTER-compliant usage reports are available is maintained by COUNTER and publicly available at `Registry <https://registry.countermetrics.org/>`_.

Report providers may apply to the Executive Director (code@countermetrics.org) for their products to be included on the Registry. Report providers will have to provide proof of initial compliance by including the results of COUNTER Validation Tool tests showing compliance for each reports, including testing both the upload of the tabular reports and COUNTER API (formerly sushi) harvesting of the same reports. Upon receipt of the application and proof of compliance, report providers MUST allow at least one large report consumer (e.g. JUSP) to evaluate their usage reports.
Report providers may apply to the Executive Director (code@countermetrics.org) for their products to be included on the Registry. Report providers will have to provide proof of initial compliance by including the results of COUNTER Validator tests showing compliance for each reports, including testing both the upload of the tabular reports and COUNTER API (formerly sushi) harvesting of the same reports. Upon receipt of the application and proof of compliance, report providers MUST allow at least one large report consumer (e.g. JUSP) to evaluate their usage reports.

When the reports are deemed to comply with the COUNTER Code of Practice, the report provider will be asked to sign a Declaration of COUNTER Compliance (:ref:`Appendix C <appendix-c>`), to be sent to the Executive Director, after which the report provider and its platforms will be added to the Registry. These report providers may then use the designation “COUNTER Compliance Pending”.

Expand Down
2 changes: 1 addition & 1 deletion source/09-audit/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,5 @@ COUNTER will recognize an audit carried out by any Certified Public Accountant (
01-the-audit-process
02-categories-of-audit-result
03-timelines
04-counter-validation-tool
04-counter-validator
05-inclusion-in-registry
10 changes: 5 additions & 5 deletions source/appendices/a-glossary-of-terms.rst
Comment thread
berndoberknapp marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -249,8 +249,8 @@ Note: The main Code of Practice document takes precedence in the case of any con
- Status of a vendor who is currently not compliant but whose audit is in progress or scheduled.
-

* - COUNTER Report Validation Tool
- An online tool to validate COUNTER reports in JSON and tabular format.
* - COUNTER Validator
- An online service to validate COUNTER reports in JSON and tabular format.
-

* - COUNTER Reports
Expand Down Expand Up @@ -601,7 +601,7 @@ Note: The main Code of Practice document takes precedence in the case of any con
-

* - ISBN (International Standard Book Number)
- A unique standard identifier (ISO 2108) used to identify monographic publications (books). The format ISBN-13, with hyphens, is required. E-ISBN is the expected value, with print ISBNs provided only where E-ISBN is not available. Inclusion of an ISBN-10, or an incorrectly formatted ISBN-13, will be flagged as an error by the Validation Tool.
- A unique standard identifier (ISO 2108) used to identify monographic publications (books). The format ISBN-13, with hyphens, is required. E-ISBN is the expected value, with print ISBNs provided only where E-ISBN is not available. Inclusion of an ISBN-10, or an incorrectly formatted ISBN-13, will be flagged as an error by the COUNTER Validator.
-

* - ISBN-13
Expand Down Expand Up @@ -953,8 +953,8 @@ Note: The main Code of Practice document takes precedence in the case of any con
- An umbrella term. Includes publishers, aggregators and others who directly provide access to content, as well as organizations that provide specialist reporting services on behalf of one or more organizations.
- Science Direct, Clarivate, JSTOR, ScholarlyIQ

* - Report validation tool
- See COUNTER Report Validation Tool.
* - Report validator
- See COUNTER Validator.
-

* - Reporting period\ |br|\ |lb|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ E.2 General Audit Tests
This section of the appendix outlines tests that MUST be run during all audits, and which are not specific to a particular COUNTER Report.


E.2.1 Validation Tool
"""""""""""""""""""""
E.2.1 COUNTER Validator
"""""""""""""""""""""""

Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST run the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the Validation Tool. This test must include all attributes report providers are required to support, including attributes that are included only if called for.
Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST run the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for.

Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors.

Expand Down