From 8c5db31039395eb7185058717c1dc83e7ba8512f Mon Sep 17 00:00:00 2001 From: COUNTERMetrics <159042634+COUNTERMetrics@users.noreply.github.com> Date: Tue, 26 May 2026 16:11:51 +0100 Subject: [PATCH 01/11] Audit amendments per 303 Minor amends to audit timelines and requirement for COUNTER API to be tested --- source/09-audit/03-timelines.rst | 14 +++++++------- .../02-general-tests.rst | 2 +- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/source/09-audit/03-timelines.rst b/source/09-audit/03-timelines.rst index 4c557ab..eb53a02 100644 --- a/source/09-audit/03-timelines.rst +++ b/source/09-audit/03-timelines.rst @@ -26,23 +26,23 @@ Table 9.a (below): Audit Timelines * - Seeding - 1 calendar month - - February + - Seeding actions in February * - Usage processed, reports available - Up to 28 days from end of seeding month - - March + - Reports available by 28 March * - Report reconciliation - 4 to 6 weeks - - April + - Reconciliation during April * - Fixes after interim report - Up to 3 months - - May to July + - Interim report issued end April, fixes due by end July * - Report reconciliation after fixes - 1 month - - August + - Final report issued during August Extensions @@ -51,11 +51,11 @@ Extensions Report providers may request extensions to the usual timelines under the following circumstances. * **Delayed seeding** - where a report provider is able to demonstrate that they have run the COUNTER Validation Tool and are in the process of fixing problems identified by the tool before commencing an audit, they may apply for up to three months delay in the seeding process. -* **Extension to the fix period** - where an Interim Report has identified significant problems with COUNTER reports, the report provider may apply for an extension of the fix period from three to up to six months. +* **Extension to the fix period** - where an Interim Report has identified significant problems with COUNTER reports, the report provider may apply for an extension of the fix period from the standard three to a maximum of six months (i.e. an additional three months). Applications for extensions MUST be sent to both the Executive Director (code@countermetrics.org) and the auditor, and will be considered on a case-by-case basis. Only one extension of each type will be granted for any audit, and all extensions will be tracked in the Registry. -There may be limited, unforeseen circumstances in which an additional extension may be granted. In these cases the report provider and auditor MUST explain the extenuating circumstances in a short report to the COUNTER Executive Committee. If the additional extension is granted, the report provider will not be required to start seeding an audit until three months after a successful audit pass. +There may be limited, unforeseen circumstances in which an additional extension may be granted. In these cases the report provider and auditor MUST explain the extenuating circumstances in a short report to the COUNTER Executive Committee. Note that being unable to prioritise fixes in the development queue does not qualify as an unforeseen circumstance. If the additional extension is granted, the report provider will not be required to start seeding an audit until three months after a successful audit pass. .. _alternate-year-audits: diff --git a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst index 1d84656..4bb5889 100644 --- a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst +++ b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst @@ -11,7 +11,7 @@ This section of the appendix outlines tests that MUST be run during all audits, E.2.1 Validation Tool """"""""""""""""""""" -Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST run the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the Validation Tool. This test must include all attributes report providers are required to support, including attributes that are included only if called for. +Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report providers' COUNTER API (sushi) configuration ans well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the Validation Tool. This test must include all attributes report providers are required to support, including attributes that are included only if called for. Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors. From d735aa8cf7f05a723606c9754c4640d07f7b20c5 Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Wed, 27 May 2026 00:11:53 +0200 Subject: [PATCH 02/11] Merge changes from planned branch --- source/09-audit/03-timelines.rst | 14 +++++++------- .../02-general-tests.rst | 3 ++- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/source/09-audit/03-timelines.rst b/source/09-audit/03-timelines.rst index ed71b0f..39dc6e0 100644 --- a/source/09-audit/03-timelines.rst +++ b/source/09-audit/03-timelines.rst @@ -26,23 +26,23 @@ Table 9.a (below): Audit Timelines * - Seeding - 1 calendar month - - February + - Seeding actions in February * - Usage processed, reports available - Up to 28 days from end of seeding month - - March + - Reports available by 28 March * - Report reconciliation - 4 to 6 weeks - - April + - Reconciliation during April * - Fixes after interim report - Up to 3 months - - May to July + - Interim report issued end April, fixes due by end July * - Report reconciliation after fixes - 1 month - - August + - Final report issued during August Extensions @@ -51,11 +51,11 @@ Extensions Report providers may request extensions to the usual timelines under the following circumstances. * **Delayed seeding** - where a report provider is able to demonstrate that they have run the COUNTER Validator and are in the process of fixing problems identified by the validator before commencing an audit, they may apply for up to three months delay in the seeding process. -* **Extension to the fix period** - where an Interim Report has identified significant problems with COUNTER reports, the report provider may apply for an extension of the fix period from three to up to six months. +* **Extension to the fix period** - where an Interim Report has identified significant problems with COUNTER reports, the report provider may apply for an extension of the fix period from the standard three to a maximum of six months (i.e. an additional three months). Applications for extensions MUST be sent to both the Executive Director (code@countermetrics.org) and the auditor, and will be considered on a case-by-case basis. Only one extension of each type will be granted for any audit, and all extensions will be tracked in the Registry. -There may be limited, unforeseen circumstances in which an additional extension may be granted. In these cases the report provider and auditor MUST explain the extenuating circumstances in a short report to the COUNTER Executive Committee. If the additional extension is granted, the report provider will not be required to start seeding an audit until three months after a successful audit pass. +There may be limited, unforeseen circumstances in which an additional extension may be granted. In these cases the report provider and auditor MUST explain the extenuating circumstances in a short report to the COUNTER Executive Committee. Note that being unable to prioritise fixes in the development queue does not qualify as an unforeseen circumstance. If the additional extension is granted, the report provider will not be required to start seeding an audit until three months after a successful audit pass. .. _alternate-year-audits: diff --git a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst index df51705..98a608d 100644 --- a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst +++ b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst @@ -11,7 +11,8 @@ This section of the appendix outlines tests that MUST be run during all audits, E.2.1 COUNTER Validator """"""""""""""""""""""" -Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST run the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. +Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report providers' COUNTER API (sushi) configuration ans well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. + Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors. From 4db674a3a096df52101af180d86dbd116af0e2f8 Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Wed, 27 May 2026 00:16:05 +0200 Subject: [PATCH 03/11] Fix spelling and whitespace --- .../e-audit-requirements-and-tests/02-general-tests.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst index 98a608d..1f7d6ce 100644 --- a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst +++ b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst @@ -11,8 +11,7 @@ This section of the appendix outlines tests that MUST be run during all audits, E.2.1 COUNTER Validator """"""""""""""""""""""" -Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report providers' COUNTER API (sushi) configuration ans well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. - +Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report providers' COUNTER API (sushi) configuration as well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors. From ad190078ddc508fc4113169c745accb2fb5759fa Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Wed, 27 May 2026 00:32:31 +0200 Subject: [PATCH 04/11] Changes from the superseeded PR --- .../e-audit-requirements-and-tests/02-general-tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst index 1f7d6ce..306ae02 100644 --- a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst +++ b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst @@ -11,7 +11,7 @@ This section of the appendix outlines tests that MUST be run during all audits, E.2.1 COUNTER Validator """"""""""""""""""""""" -Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report providers' COUNTER API (sushi) configuration as well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. +Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report provider's COUNTER API (sushi) configuration as well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit using the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors. From 97d17efdb60eafb22fdb7fbd87479e2f232a09dc Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Wed, 27 May 2026 00:50:28 +0200 Subject: [PATCH 05/11] Fix formatting --- source/09-audit/03-timelines.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/09-audit/03-timelines.rst b/source/09-audit/03-timelines.rst index 39dc6e0..befadc7 100644 --- a/source/09-audit/03-timelines.rst +++ b/source/09-audit/03-timelines.rst @@ -13,11 +13,11 @@ Table 9.a (below): Audit Timelines .. only:: latex - .. tabularcolumns:: |>{\raggedright\arraybackslash}\Y{0.34}|>{\raggedright\arraybackslash}\Y{0.4}|>{\raggedright\arraybackslash}\Y{0.14}| + .. tabularcolumns:: |>{\raggedright\arraybackslash}\Y{0.34}|>{\raggedright\arraybackslash}\Y{0.28}|>{\raggedright\arraybackslash}\Y{0.38}| .. list-table:: :class: longtable - :widths: 38 38 24 + :widths: 27 32 41 :header-rows: 1 * - Process From 4d517be0a3b0dc4a2d0d004236e4e48355b0c7aa Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Wed, 27 May 2026 00:50:45 +0200 Subject: [PATCH 06/11] Add missing formerly to sushi --- .../e-audit-requirements-and-tests/02-general-tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst index 306ae02..f653fd8 100644 --- a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst +++ b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst @@ -11,7 +11,7 @@ This section of the appendix outlines tests that MUST be run during all audits, E.2.1 COUNTER Validator """"""""""""""""""""""" -Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report provider's COUNTER API (sushi) configuration as well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit using the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. +Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report provider's COUNTER API (formerly sushi) configuration as well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit using the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors. From 03793c0a0a148e287e4e9d9802b4ae69c79d44b5 Mon Sep 17 00:00:00 2001 From: COUNTERMetrics <159042634+COUNTERMetrics@users.noreply.github.com> Date: Thu, 28 May 2026 13:27:21 +0100 Subject: [PATCH 07/11] Update 03-timelines.rst Addition of extremely delayed audit note. This is documenting practice established by LE in 2015 and maintained by TMC --- source/09-audit/03-timelines.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/09-audit/03-timelines.rst b/source/09-audit/03-timelines.rst index eb53a02..e9dfee8 100644 --- a/source/09-audit/03-timelines.rst +++ b/source/09-audit/03-timelines.rst @@ -57,6 +57,8 @@ Applications for extensions MUST be sent to both the Executive Director (code@co There may be limited, unforeseen circumstances in which an additional extension may be granted. In these cases the report provider and auditor MUST explain the extenuating circumstances in a short report to the COUNTER Executive Committee. Note that being unable to prioritise fixes in the development queue does not qualify as an unforeseen circumstance. If the additional extension is granted, the report provider will not be required to start seeding an audit until three months after a successful audit pass. +In rare circumstances, a report provider might receive both a delayed seeding extension, and an extension to the fix period for a single audit. If their seeding month was scheduled for January, the combination of both extensions would result in their audit being completed in December. In this circumstance, COUNTER will reschedule seeding to start three months after completion of the delayed audit. + .. _alternate-year-audits: Alternate year audits From eaa9308b2605e159e51b207f80eb686332c69a5d Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Thu, 28 May 2026 15:27:24 +0200 Subject: [PATCH 08/11] Apply reverted fixes --- source/09-audit/03-timelines.rst | 2 +- .../e-audit-requirements-and-tests/02-general-tests.rst | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/source/09-audit/03-timelines.rst b/source/09-audit/03-timelines.rst index e9dfee8..ab19cef 100644 --- a/source/09-audit/03-timelines.rst +++ b/source/09-audit/03-timelines.rst @@ -50,7 +50,7 @@ Extensions Report providers may request extensions to the usual timelines under the following circumstances. -* **Delayed seeding** - where a report provider is able to demonstrate that they have run the COUNTER Validation Tool and are in the process of fixing problems identified by the tool before commencing an audit, they may apply for up to three months delay in the seeding process. +* **Delayed seeding** - where a report provider is able to demonstrate that they have run the COUNTER Validator and are in the process of fixing problems identified by the validator before commencing an audit, they may apply for up to three months delay in the seeding process. * **Extension to the fix period** - where an Interim Report has identified significant problems with COUNTER reports, the report provider may apply for an extension of the fix period from the standard three to a maximum of six months (i.e. an additional three months). Applications for extensions MUST be sent to both the Executive Director (code@countermetrics.org) and the auditor, and will be considered on a case-by-case basis. Only one extension of each type will be granted for any audit, and all extensions will be tracked in the Registry. diff --git a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst index 4bb5889..94a5aac 100644 --- a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst +++ b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst @@ -8,10 +8,10 @@ E.2 General Audit Tests This section of the appendix outlines tests that MUST be run during all audits, and which are not specific to a particular COUNTER Report. -E.2.1 Validation Tool -""""""""""""""""""""" +E.2.1 COUNTER Validator +""""""""""""""""""""""" -Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report providers' COUNTER API (sushi) configuration ans well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit through the Validation Tool. This test must include all attributes report providers are required to support, including attributes that are included only if called for. +Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report provider's COUNTER API (formerly sushi) configuration ans well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit with the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors. From 2f3f1899695afaf4acf0b55b3538f0c98f6a295c Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Thu, 28 May 2026 15:31:24 +0200 Subject: [PATCH 09/11] Apply reverted formatting --- source/09-audit/03-timelines.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/09-audit/03-timelines.rst b/source/09-audit/03-timelines.rst index ab19cef..7474e58 100644 --- a/source/09-audit/03-timelines.rst +++ b/source/09-audit/03-timelines.rst @@ -13,11 +13,11 @@ Table 9.a (below): Audit Timelines .. only:: latex - .. tabularcolumns:: |>{\raggedright\arraybackslash}\Y{0.34}|>{\raggedright\arraybackslash}\Y{0.4}|>{\raggedright\arraybackslash}\Y{0.14}| + .. tabularcolumns:: |>{\raggedright\arraybackslash}\Y{0.34}|>{\raggedright\arraybackslash}\Y{0.28}|>{\raggedright\arraybackslash}\Y{0.38}| .. list-table:: :class: longtable - :widths: 38 38 24 + :widths: 27 32 41 :header-rows: 1 * - Process From fffda77a9b9b706a435db4cdd57bb0b0b7ac01d9 Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Thu, 28 May 2026 15:34:14 +0200 Subject: [PATCH 10/11] Fix whitespace --- source/09-audit/03-timelines.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/09-audit/03-timelines.rst b/source/09-audit/03-timelines.rst index 7474e58..ff8a9d9 100644 --- a/source/09-audit/03-timelines.rst +++ b/source/09-audit/03-timelines.rst @@ -59,6 +59,7 @@ There may be limited, unforeseen circumstances in which an additional extension In rare circumstances, a report provider might receive both a delayed seeding extension, and an extension to the fix period for a single audit. If their seeding month was scheduled for January, the combination of both extensions would result in their audit being completed in December. In this circumstance, COUNTER will reschedule seeding to start three months after completion of the delayed audit. + .. _alternate-year-audits: Alternate year audits From 3e7d4292d34b65cfec10b783ebed0390b4a57941 Mon Sep 17 00:00:00 2001 From: Bernd Oberknapp Date: Thu, 28 May 2026 15:39:19 +0200 Subject: [PATCH 11/11] Fix spelling --- .../e-audit-requirements-and-tests/02-general-tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst index 94a5aac..62f6230 100644 --- a/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst +++ b/source/appendices/e-audit-requirements-and-tests/02-general-tests.rst @@ -11,7 +11,7 @@ This section of the appendix outlines tests that MUST be run during all audits, E.2.1 COUNTER Validator """"""""""""""""""""""" -Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report provider's COUNTER API (formerly sushi) configuration ans well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit with the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. +Once reports are available for testing (i.e. up to 28 days after the end of the seeding month), auditors MUST test the report provider's COUNTER API (formerly sushi) configuration as well as the COUNTER Reports and Standard Views of COUNTER Reports that are within the scope of the audit with the COUNTER Validator. This test must include all attributes report providers are required to support, including attributes that are included only if called for. Where report providers have elected to follow the pre-flight preparation step outlined in :numref:`audit` of the Code of Practice, this audit test should not result in any errors.