Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 33 additions & 25 deletions docs/coverage-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
This repo runs two coverage gates in CI. They answer different questions, and neither replaces
the other.

| Gate | Config | Scope | Question it answers |
| ---------------- | ------------------------------ | ---------------------------------------- | -------------------------------------------------------- |
| Frick gate | `jest.frick.config.js` | 7 Frick files, run by 7 Frick specs only | Do _these specs alone_ fully cover _these files_? |
| Coverage ratchet | `jest.coverage-gate.config.js` | 413 files, whole suite | Has coverage regressed anywhere it was already complete? |
| Gate | Config | Scope | Question it answers |
| ---------------- | ------------------------------ | ------------------------------------------ | -------------------------------------------------------- |
| Frick gate | `jest.frick.config.js` | 10 Frick files, run by 10 Frick specs only | Do _these specs alone_ fully cover _these files_? |
| Coverage ratchet | `jest.coverage-gate.config.js` | 421 files, whole suite | Has coverage regressed anywhere it was already complete? |

## What the ratchet is, and what it is not

Expand All @@ -16,7 +16,7 @@ file, CI fails.

It is a **regression gate**, not a statement about test quality:

- It does not claim the repo is well tested. Overall coverage is 59.45% of statements and 42.43%
- It does not claim the repo is well tested. Overall coverage is 59.57% of statements and 42.46%
of branches; the pinned files are the subset that happens to be complete today.
- It does not verify that a file's _own_ spec covers it. Under a whole-suite run, coverage may
come from any spec. The Frick gate is the one that makes the stronger per-spec claim, which is
Expand All @@ -25,8 +25,8 @@ It is a **regression gate**, not a statement about test quality:
ratchet only protects files already on the list, and that list grows by hand (see "How the list
grows"). That is the price of the threshold approach.

Of the 413 pinned files, **225 carry real logic** (they have functions and/or branches) and
**188 are purely declarative today** (NestJS modules, constant files with neither). The two groups
Of the 421 pinned files, **232 carry real logic** (they have functions and/or branches) and
**189 are purely declarative today** (NestJS modules, constant files with neither). The two groups
are kept visibly separate in the config so the count is not mistaken for test depth.

Pinning the declarative ones is deliberate and not vacuous. Istanbul reports a metric with a total
Expand All @@ -35,7 +35,7 @@ and fails the threshold. Statements and lines are pinned as well, so even top-le
code that no test reaches turns the gate red.

Test scaffolding is excluded. `shared/utils/test.util.ts` and `shared/utils/test.shared.module.ts`
live outside a `__tests__` directory but are imported only by specs (60 and 28 importers, all
live outside a `__tests__` directory but are imported only by specs (62 and 29 importers, all
`*.spec.ts`). They are filtered out of `collectCoverageFrom`, so an untested change to a test
helper cannot fail a production gate.

Expand Down Expand Up @@ -67,7 +67,7 @@ statement executed by a suite counts as executed no matter which worker ran it.
cannot turn a covered file into an uncovered one, which is why the CI script does not serialise.

The gate runs the whole suite under full compilation, unlike the sharded `test` job that splits
the suite three ways and the Frick gate that runs seven specs. Exact per-file numbers are what
the suite three ways and the Frick gate that runs ten specs. Exact per-file numbers are what
that costs in run time.

## Where the gate runs
Expand Down Expand Up @@ -150,9 +150,9 @@ warm caches, is a good deal slower than the 1.5 min it takes in CI.

## Current state

Measured on develop @ e6139b860.
Measured on develop @ 045e6f8d6 with this PR's tests applied.

The collection glob matches 1,656 files under `src/`. 1,605 of them contain instrumentable code
The collection glob matches 1,657 files under `src/`. 1,606 of them contain instrumentable code
and appear in the report. The remaining 51 compile to no executable statements and therefore
cannot be measured or pinned: 49 are type-only (interfaces, type aliases, response shapes), one
consists entirely of commented-out code (`integration/exchange/services/p2b.service.ts`) and one
Expand All @@ -161,11 +161,11 @@ deleting them would be a separate cleanup.

| Class | Files | Meaning |
| -------- | ----- | ----------------------------------------------- |
| Complete | 413 | Pinned by the ratchet |
| Partial | 1,062 | Some coverage, below 100 on at least one metric |
| None | 130 | No coverage at all |
| Complete | 421 | Pinned by the ratchet |
| Partial | 1,058 | Some coverage, below 100 on at least one metric |
| None | 127 | No coverage at all |

Totals: statements 59.45%, branches 42.43%, functions 34.09%, lines 59.81%.
Totals: statements 59.57%, branches 42.46%, functions 34.16%, lines 59.93%.

Coverage is very unevenly distributed. `subdomains/supporting/payout` has 69 of 102 files
complete; `subdomains/supporting/dex` has 6 of 170, `subdomains/supporting/payin` 6 of 102, and
Expand All @@ -175,29 +175,37 @@ six under `subdomains/generic/admin` have no coverage at all.
## How the list grows

Any PR may add files to `coverageThreshold` once they reach 100%.
`jest.coverage-gate.config.js` holds the 413 paths in two arrays, `PINNED_LOGIC` (logic-carrying
`jest.coverage-gate.config.js` holds the 421 paths in two arrays, `PINNED_LOGIC` (logic-carrying
files) and `PINNED_DECLARATIVE` (purely declarative files), from which `coverageThreshold` is
generated. Adding a file means appending its path to the matching array, not writing out a
`coverageThreshold` object entry by hand.

The intended next step is the set already within reach: **29 files sit at ≥90% on all four
The intended next step is the set already within reach: **26 files sit at ≥90% on all four
metrics**, several of them one or two uncovered branches away. Examples:

| File | branches | functions | lines | statements |
| --------------------------------------------------------------------------- | -------- | --------- | ----- | ---------- |
| `src/subdomains/core/accounting/services/ledger-cutover.service.ts` | 98.55 | 100 | 99.33 | 99.1 |
| `src/subdomains/core/accounting/services/consumers/exchange-tx.consumer.ts` | 96.29 | 100 | 100 | 99.54 |
| `src/subdomains/core/accounting/services/ledger-reconciliation.service.ts` | 95.52 | 100 | 99.41 | 99.48 |
| `src/integration/infrastructure/storage/s3-storage.service.ts` | 95 | 100 | 100 | 100 |
| File | branches | functions | lines | statements |
| ---------------------------------------------------------------------------- | -------- | --------- | ----- | ---------- |
| `src/subdomains/core/accounting/services/ledger-cutover.service.ts` | 98.55 | 100 | 99.33 | 99.1 |
| `src/subdomains/core/accounting/services/consumers/exchange-tx.consumer.ts` | 96.29 | 100 | 100 | 99.54 |
| `src/subdomains/core/accounting/services/ledger-reconciliation.service.ts` | 95.52 | 100 | 99.41 | 99.48 |
| `src/subdomains/core/accounting/services/consumers/payout-order.consumer.ts` | 93.47 | 100 | 100 | 99.44 |

Not every remaining branch is reachable by a test. Some of the open branches are defensive
fallbacks that cannot fire at runtime — for example `name.split('/').pop() ?? name` (split
always returns at least one element) and `+(raw.legCount ?? 0)` over a SQL `COUNT(*)`, which is
never null. Covering such a branch would require inventing a mock the data source cannot
actually produce, which proves nothing. The correct fix is to remove the unreachable fallback,
which is also what the project's rule against silent fallbacks calls for.
`ledger-mark-to-market.service.ts` sits at 92.3% branches for exactly this reason.

To regenerate the full picture, run the gate and read `coverage-gate/coverage-summary.json`.

**Removing a file from the list is not a normal fix.** If a change makes a pinned file drop
below 100, the expected response is to extend the tests. Unpinning is an explicit decision that
belongs in the PR description, not a silent edit.

That rule stays hard for the 225 logic-carrying files. A foreseeable friction case is different:
when one of the 188 purely declarative files (a NestJS module, a constants file) first gains
That rule stays hard for the 232 logic-carrying files. A foreseeable friction case is different:
when one of the 189 purely declarative files (a NestJS module, a constants file) first gains
executable logic — for example a `useFactory` on a module — the function metric jumps from 0/0 to
0/N and the gate turns red. Tests remain the preferred fix, but unpinning that one file is an
allowed outcome if the PR description names and justifies it (not as a silent edit). For
Expand Down
12 changes: 10 additions & 2 deletions jest.coverage-gate.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
// 2. The whole suite runs, because a file is frequently covered by specs other than its own.
//
// The dedicated Frick gate (jest.frick.config.js) stays separate on purpose: it runs ONLY the
// seven Frick specs and therefore proves that those specs alone reach 100% - an assertion this
// ten Frick specs and therefore proves that those specs alone reach 100% - an assertion this
// repo-wide run cannot make, because here any spec may contribute the coverage.
const base = require('./package.json').jest;

Expand Down Expand Up @@ -48,15 +48,19 @@ const PINNED_LOGIC = [
'src/integration/exchange/dto/trade-result.dto.ts',
'src/integration/exchange/enums/exchange.enum.ts',
'src/integration/infrastructure/storage/azure-storage.service.ts',
'src/integration/infrastructure/storage/mock-storage.service.ts',
'src/integration/infrastructure/storage/s3-storage.service.ts',
'src/integration/infrastructure/storage/storage.factory.ts',
'src/integration/infrastructure/storage/storage.service.ts',
'src/integration/kucoin-pay/kucoin-pay.dto.ts',
'src/integration/lightning/dto/lnd.dto.ts',
'src/integration/scorechain/dto/scorechain-screening-dto.mapper.ts',
'src/integration/scorechain/entities/scorechain-screening.entity.ts',
'src/integration/scorechain/exceptions/scorechain-object-not-found.exception.ts',
'src/integration/sift/dto/sift.dto.ts',
'src/polyfills.ts',
'src/shared/auth/allow-tfa-pending.decorator.ts',
'src/shared/auth/get-jwt.decorator.ts',
'src/shared/auth/user-role.enum.ts',
'src/shared/services/typeorm-logger.ts',
'src/shared/utils/bitbox-ascii.util.ts',
Expand Down Expand Up @@ -90,6 +94,7 @@ const PINNED_LOGIC = [
'src/subdomains/core/buy-crypto/routes/buy/dto/personal-iban-provider.enum.ts',
'src/subdomains/core/custody/dto/output/custody-order-history.dto.ts',
'src/subdomains/core/custody/enums/custody.ts',
'src/subdomains/core/custody/mappers/custody-asset-balance-dto.mapper.ts',
'src/subdomains/core/faucet-request/enums/faucet-request.ts',
'src/subdomains/core/history/dto/history.dto.ts',
'src/subdomains/core/history/dto/output/chain-report-history.dto.ts',
Expand All @@ -105,6 +110,8 @@ const PINNED_LOGIC = [
'src/subdomains/core/trading/enums/index.ts',
'src/subdomains/generic/forwarding/controllers/lnurld-forward.controller.ts',
'src/subdomains/generic/forwarding/controllers/lnurlw-forward.controller.ts',
'src/subdomains/generic/gs/dto/db-query.dto.ts',
'src/subdomains/generic/gs/dto/gs-trigger-type.enum.ts',
'src/subdomains/generic/gs/middleware/debug-query-tree-size.middleware.ts',
'src/subdomains/generic/kyc/dto/ident-result-data.dto.ts',
'src/subdomains/generic/kyc/dto/kyc-error.enum.ts',
Expand Down Expand Up @@ -378,6 +385,7 @@ const PINNED_DECLARATIVE = [
'src/subdomains/core/sell-crypto/route/dto/sell.dto.ts',
'src/subdomains/core/sell-crypto/route/dto/unsigned-tx.dto.ts',
'src/subdomains/core/sell-crypto/route/dto/update-sell.dto.ts',
'src/subdomains/generic/gs/dto/support-data.dto.ts',
'src/subdomains/generic/kyc/dto/input/kyc-query.dto.ts',
'src/subdomains/generic/kyc/dto/input/update-kyc-step.dto.ts',
'src/subdomains/generic/kyc/dto/input/update-name-check-log.dto.ts',
Expand Down Expand Up @@ -459,7 +467,7 @@ module.exports = {
'!**/*.d.ts',
'!jest-env.setup.ts',
// Test scaffolding that lives outside a __tests__ directory: imported only by specs
// (60 and 28 importers respectively, all of them *.spec.ts). Pinning them would make an
// (62 and 29 importers respectively, all of them *.spec.ts). Pinning them would make an
// untested change to a test helper fail the production gate.
'!shared/utils/test.util.ts',
'!shared/utils/test.shared.module.ts',
Expand Down
89 changes: 89 additions & 0 deletions migration/1785500000000-ClearDevUserSignatures.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
/**
* @typedef {import('typeorm').MigrationInterface} MigrationInterface
* @typedef {import('typeorm').QueryRunner} QueryRunner
*/

/**
* DEV-ONLY credential rotation: clears stored user wallet signatures that were produced with the
* old environment-independent sign message.
*
* Before the env-scoped sign-message fix, every environment signed the same historical text. A
* signature stored on DEV was therefore also valid on PRD — a DEV database read would yield working
* PRD login credentials. After the fix, non-PRD environments prefix the sign message, so new DEV
* signatures no longer verify on PRD. Existing DEV rows still hold the old (PRD-valid) signatures
* and must be discarded.
*
* Guarded to `ENVIRONMENT === 'dev'` so loc/prd/CI are no-ops. Returning early still records the
* migration as executed, which is the intended no-op outside DEV.
*
* up():
* 1. dev guard (no-op elsewhere)
* 2. audit-then-null via data-modifying CTEs: lock non-null signatures, insert an audit log row
* with md5 fingerprints (not plaintext credentials), then null signatures only if the audit
* insert succeeded (fail-closed)
*
* down() deliberately does NOT restore values: signatures are login credentials and only their
* md5 fingerprint is retained in the audit log, so the prior value is not reconstructible by
* design. Always a no-op in every environment (not a guaranteed inverse).
*
* @class
* @implements {MigrationInterface}
*/
module.exports = class ClearDevUserSignatures1785500000000 {
name = 'ClearDevUserSignatures1785500000000';

/**
* Credential rotation: discard all DEV-stored signatures that were produced with the old,
* environment-independent sign message and would (before this fix) have authenticated on PRD.
*
* Audit trail stores md5("signature") fingerprints rather than plaintext: a candidate value can
* still be checked against the fingerprint during an investigation, without keeping the login
* credential in the database. md5 is used here solely as an audit fingerprint — not a security
* primitive (no password hashing, no salt; pure evidentiary purpose).
*
* EXISTS (SELECT 1 FROM "audit") fail-closes the UPDATE to the audit INSERT: if the insert fails,
* no row is changed (CONTRIBUTING.md "Auditable mutations — no destructive overwrites (CRITICAL)",
* before→after audit before the update).
*
* @param {QueryRunner} queryRunner
*/
async up(queryRunner) {
if (process.env.ENVIRONMENT !== 'dev') return;

await queryRunner.query(`
WITH "affected" AS (
SELECT "id", md5("signature") AS "fingerprint"
FROM "user"
WHERE "signature" IS NOT NULL
FOR UPDATE
),
"audit" AS (
INSERT INTO "log" ("created", "updated", "system", "subsystem", "severity", "message")
SELECT now(), now(), 'User', 'DevSignatureRotation', 'Info',
json_agg(json_build_object(
'id', "id",
'beforeFingerprint', "fingerprint",
'after', null
))::text
FROM "affected"
HAVING count(*) > 0
RETURNING 1
)
UPDATE "user" u
SET "signature" = NULL
FROM "affected" a
WHERE u."id" = a."id" AND EXISTS (SELECT 1 FROM "audit");
`);
}

/**
* Deliberately does not restore signature values: they are login credentials and only their
* fingerprint remains in the audit log, so the prior value is not reconstructible by design.
* Always a no-op in every environment — no env gate needed.
*/
async down() {
// Deliberately not restoring signature values: they are login credentials and only their
// md5 fingerprint is retained in the audit log, so the prior value is not reconstructible by
// design. Not a guaranteed inverse; no-op in every environment.
}
};
11 changes: 11 additions & 0 deletions src/config/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,15 @@ export class Configuration {
resolvers: [{ resolve: () => this.i18n.fallbackLanguage }],
};

// A signature only proves ownership of the exact text that was signed. All environments share the
// same wallets and the same message, so a signature created on a lower environment authenticates on
// PRD as well — a DEV database read would yield working PRD credentials. PRD therefore keeps the
// historical text byte-for-byte (existing PRD signatures stay valid), while every other environment
// signs a distinct text and thus produces a signature PRD cannot verify. An unset or unknown
// ENVIRONMENT is deliberately treated as non-PRD: it must never accidentally yield a PRD-valid
// signature.
signMessagePrefix = this.environment === Environment.PRD ? '' : `[${this.environment}]_`;

auth = {
jwt: {
secret: process.env.JWT_SECRET,
Expand All @@ -312,8 +321,10 @@ export class Configuration {
// are never affected by this flag.
tfaStaffEnforced: process.env.TFA_STAFF_ENFORCED !== 'false',
signMessage:
this.signMessagePrefix +
'By_signing_this_message,_you_confirm_that_you_are_the_sole_owner_of_the_provided_DeFiChain_address_and_are_in_possession_of_its_private_key._Your_ID:_',
signMessageGeneral:
this.signMessagePrefix +
'By_signing_this_message,_you_confirm_that_you_are_the_sole_owner_of_the_provided_Blockchain_address._Your_ID:_',
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,14 @@ describe('MockStorageService', () => {

expect(await service.listKeys('nope/')).toEqual([]);
});

it('lists all entries of the container when no prefix is given', async () => {
const service = new MockStorageService('mock-spec-keys-all');
await service.uploadBlob('x.png', Buffer.from('x'), 'image/png');
await service.uploadBlob('y.png', Buffer.from('y'), 'image/png');

expect((await service.listKeys()).sort()).toEqual(['x.png', 'y.png']);
});
});

describe('getBlob dummy-file fallback', () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,20 @@ describe('S3StorageService', () => {
expect(s3Mock.commandCalls(PutObjectCommand)).toHaveLength(0);
});

it('fails closed with a non-Error rejection from the lock-configuration read', async () => {
const container = 'ep2-worm-non-error-rejection';
s3Mock
.on(GetObjectLockConfigurationCommand, { Bucket: container })
.callsFake(() => Promise.reject('access denied'));
s3Mock.on(PutObjectCommand).resolves({});

await expect(
new S3StorageService(container).uploadWormBlob('settlement.ep2', Buffer.from('<ep2/>'), 'text/xml'),
).rejects.toThrow('could not verify Object Lock is enabled (error: access denied)');

expect(s3Mock.commandCalls(PutObjectCommand)).toHaveLength(0);
});

it('probes Object Lock on every WORM write (no TTL cache)', async () => {
const container = 'ep2-worm-probe-every-write';
s3Mock.on(GetObjectLockConfigurationCommand, { Bucket: container }).resolves({
Expand Down
Loading
Loading