docs(sccm): prepare site-core workflow corpus (#327) - #338
Merged
adamgell merged 4 commits intoJul 30, 2026
Conversation
Owner
Author
|
@coderabbitai review Please review the #327 site-core corpus against the evidence-first workflow contract, especially incomplete/malformed/rotation safety, exact keyed recovery versus contradiction separation, evidence citations, deterministic output, and prohibited cross-side or missing-role claims. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== PR metadata and files ==='
gh pr view 338 --repo adamgell/cmtraceopen --json title,body,baseRefName,headRefName,files \
--jq '.title, .baseRefName, .headRefName, (.files[] | .path)'
echo '=== Repository top-level files ==='
fd -HI -d 2 . | sed -n '1,240p'
echo '=== Candidate corpus/validator references ==='
rg -n -i -C 2 \
'site-core|fragmentComplete|evidenceId|evidence.*line|contradict|recovery|incomplete|malformed|rotation' \
. -g '*.json' -g '*.md' -g '*.yaml' -g '*.yml' -g '*.ts' -g '*.js' -g '*.rs' \
| sed -n '1,1200p'Length of output: 129683 🏁 Script executed: #!/bin/bash
set -euo pipefail
root="crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core"
echo '=== Scenario-level normalized summaries ==='
for manifest in "$root"/*/manifest.json; do
scenario_dir=$(dirname "$manifest")
echo "--- ${scenario_dir#$root/} ---"
jq -c '{
artifacts: [.artifacts[] | {
artifactId, captureState, relativePath, rotation,
configuredPath, bytesCopied
}]
}' "$manifest"
jq -c '{
scenario, resultIds: [.results[]?.resultId],
states: [.results[]?.state],
observationIds: [.unlinkedObservations[]?.observationId],
observationStates: [.unlinkedObservations[]?.state],
coverage: [.coverageGaps[]? | {artifactId,state,diagnosticMeaning}],
nextArtifacts: ([.results[]?.nextArtifacts[]?, .unlinkedObservations[]?.nextArtifacts[]?]
| map({artifactKinds, rotations, maxArtifacts, scope}))
}' "$scenario_dir/expected.json"
done
echo '=== Manifest/evidence/citation consistency audit ==='
python3 - "$root" <<'PY'
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
problems = []
for d in sorted(p for p in root.iterdir() if p.is_dir()):
manifest = json.loads((d / "manifest.json").read_text())
expected = json.loads((d / "expected.json").read_text())
artifacts = {a["artifactId"]: a for a in manifest["artifacts"]}
for aid, a in artifacts.items():
rel = a.get("relativePath")
if a.get("captureState") == "captured":
if not rel or not (d / rel).is_file():
problems.append(f"{d.name}: captured {aid} missing evidence file {rel!r}")
elif rel and (d / rel).exists():
problems.append(f"{d.name}: non-captured {aid} unexpectedly has evidence file")
def walk(x, path=""):
if isinstance(x, dict):
if {"artifactId","entryId","lineStart","lineEnd"} <= x.keys():
aid=x["artifactId"]; entry=x["entryId"]
a=artifacts.get(aid)
if not a:
problems.append(f"{d.name}: citation references unknown {aid} at {path}")
else:
expected_entry=f"{aid}:{x['lineStart']}-{x['lineEnd']}"
if entry != expected_entry:
problems.append(f"{d.name}: {entry} != {expected_entry}")
file=d/a.get("relativePath","")
if not file.is_file():
problems.append(f"{d.name}: citation {entry} file unavailable")
else:
n=sum(1 for _ in file.open())
if not (1 <= x["lineStart"] <= x["lineEnd"] <= n):
problems.append(f"{d.name}: citation {entry} outside 1..{n}")
for k,v in x.items(): walk(v, f"{path}/{k}")
elif isinstance(x, list):
for i,v in enumerate(x): walk(v, f"{path}/{i}")
walk(expected)
# Conservatively flag captured physical fragments that end without newline but lack false.
for aid,a in artifacts.items():
rel=a.get("relativePath")
f=d/rel if rel else None
if a.get("captureState") == "captured" and f and f.is_file():
raw=f.read_bytes()
if raw and not raw.endswith(b"\n") and a.get("rotation",{}).get("fragmentComplete") is not False:
problems.append(f"{d.name}: unterminated captured {aid} lacks fragmentComplete:false")
print("PASS" if not problems else "\n".join(problems))
PY
echo '=== Safety-critical documents ==='
for f in \
"docs/sccm/preparation/issue-327-server-site-core-corpus.md" \
"$root/recovery/manifest.json" "$root/recovery/expected.json" \
"$root/contradictory/manifest.json" "$root/contradictory/expected.json" \
"$root/incomplete/manifest.json" "$root/incomplete/expected.json" \
"$root/malformed/manifest.json" "$root/malformed/expected.json" \
"$root/rotation-boundary/manifest.json" "$root/rotation-boundary/expected.json"
do
echo "--- $f ---"
cat -n "$f"
done
echo '=== Prohibited-claim vocabulary within changed corpus/docs ==='
rg -n -i -C 2 \
'client impact|client|downstream|causality|causal|cause[ds]?|missing role|absent role|cross.side|cross side|healthy|success' \
"$root" "docs/sccm/preparation/issue-327-server-site-core-corpus.md"Length of output: 97259 🏁 Script executed: #!/bin/bash
set -euo pipefail
root="crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core"
python3 - "$root" <<'PY'
import json, pathlib, re, sys
root = pathlib.Path(sys.argv[1])
errors = []
checked = {
"json_documents": 0, "captured_or_capped_evidence": 0,
"citations": 0, "incomplete_references": 0, "result_keys": 0
}
required_prohibited = ["absentDownstreamRole", "clientImpact", "crossSideCausality"]
for d in sorted(p for p in root.iterdir() if p.is_dir()):
try:
manifest = json.loads((d / "manifest.json").read_text())
expected = json.loads((d / "expected.json").read_text())
checked["json_documents"] += 2
except Exception as e:
errors.append(f"{d.name}: JSON parse error: {e}")
continue
artifacts = {a["artifactId"]: a for a in manifest.get("artifacts", [])}
ids = list(artifacts)
if ids != sorted(ids):
errors.append(f"{d.name}: manifest artifacts are not sorted by artifactId")
for aid, a in artifacts.items():
rel = a.get("relativePath")
state = a.get("captureState")
f = d / rel if rel else None
if state in ("captured", "capped"):
if not rel or not f.is_file():
errors.append(f"{d.name}: {state} {aid} lacks its evidence file")
else:
checked["captured_or_capped_evidence"] += 1
if a.get("bytesCopied") != f.stat().st_size:
errors.append(f"{d.name}: {aid} bytesCopied != physical byte count")
elif rel is not None or a.get("bytesCopied") != 0:
errors.append(f"{d.name}: non-evidence {aid} has relativePath or nonzero bytes")
if expected.get("prohibitedClaims") != required_prohibited:
errors.append(f"{d.name}: prohibitedClaims differs from required role-local boundary")
ordering = expected.get("ordering", {})
if ordering.get("resultsBy") != ["resultId"] or ordering.get("evidenceBy") != ["artifactId","lineStart","lineEnd"]:
errors.append(f"{d.name}: ordering contract is incomplete or differs")
result_ids = [r.get("resultId") for r in expected.get("results", [])]
if result_ids != sorted(result_ids):
errors.append(f"{d.name}: results not sorted by resultId")
def scan(node, context=""):
if isinstance(node, dict):
# Evidence nested under coverage gaps inherits its artifact id.
if {"entryId","lineStart","lineEnd"} <= node.keys():
aid = node.get("artifactId") or context
if not aid or aid not in artifacts:
errors.append(f"{d.name}: citation {node['entryId']} has no known artifact")
else:
a = artifacts[aid]
f = d / a["relativePath"] if a.get("relativePath") else None
if not f or not f.is_file():
errors.append(f"{d.name}: citation {node['entryId']} points at unavailable evidence")
else:
lines = f.read_text().splitlines()
lo, hi = node["lineStart"], node["lineEnd"]
if not (1 <= lo <= hi <= len(lines)):
errors.append(f"{d.name}: citation {node['entryId']} outside line range")
if node["entryId"] != f"{aid}:{lo}-{hi}":
errors.append(f"{d.name}: malformed entryId {node['entryId']}")
checked["citations"] += 1
if node.get("completeLogicalRecord") is False:
checked["incomplete_references"] += 1
if a.get("rotation", {}).get("fragmentComplete") is not False:
errors.append(f"{d.name}: incomplete citation {node['entryId']} lacks fragmentComplete:false")
for k, v in node.items():
scan(v, node.get("artifactId", context))
elif isinstance(node, list):
for v in node: scan(v, context)
scan(expected)
for r in expected.get("results", []):
k = r.get("transactionKey", {})
checked["result_keys"] += 1
if [k.get(x) for x in ("profileId","profileVersion","siteCode","componentId","workItemId")] != \
["sccm-site-core", 1, "LAB", k.get("componentId"), k.get("workItemId")] or not k.get("componentId") or not k.get("workItemId"):
errors.append(f"{d.name}: incomplete or incompatible transaction key in {r.get('resultId')}")
ev = r.get("evidence", [])
order = [(x["artifactId"],x["lineStart"],x["lineEnd"]) for x in ev]
if order != sorted(order):
errors.append(f"{d.name}: evidence not ordered in {r.get('resultId')}")
if r.get("findingClass") == "confirmedFailure" and not any(x.get("terminal") for x in ev):
errors.append(f"{d.name}: confirmed failure lacks terminal citation")
print("CHECKED", json.dumps(checked, sort_keys=True))
if errors:
print("FAIL")
print("\n".join(errors))
else:
print("PASS")
PY
echo '=== Exact evidence payloads for incomplete/malformed/rotation ==='
for f in $(fd -t f -p "$root" | rg '/(incomplete|malformed|rotation-boundary)/evidence/'); do
echo "--- $f ---"
cat -n "$f"
doneLength of output: 7678
Validated across all nine scenarios:
The earlier capped-artifact audit message was a checker mistake, not a corpus issue: the documented contract correctly permits capped artifacts to retain exact local evidence. You are interacting with an AI system. |
Owner
Author
|
@coderabbitai full review Please manually override the non-default-base auto-review skip and review the complete change range |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
range="f2caccf..59902aa"
echo '=== Range commits ==='
git log --format='%H %s' "$range"
echo '=== Changed files ==='
git diff --name-status "$range"
echo '=== Diff stat ==='
git diff --stat "$range"
echo '=== Corpus/document paths in range ==='
git diff --name-only "$range" | \
rg '(^docs/|site[_-]core|sccm|fixture|corpus|validator)' || true
echo '=== Repository maps for changed source/text files ==='
while IFS= read -r file; do
case "$file" in
*.rs|*.ts|*.js)
echo "--- outline: $file ---"
ast-grep outline "$file" --items all 2>/dev/null || true
;;
esac
done < <(git diff --name-only "$range")Length of output: 10820 🏁 Script executed: #!/bin/bash
set -euo pipefail
root="crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core"
doc="docs/sccm/preparation/issue-327-server-site-core-corpus.md"
echo '=== JSON, artifact, citation, ordering, and safety audit ==='
python3 - "$root" <<'PY'
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
errors = []
stats = {"scenarios": 0, "json": 0, "artifacts": 0, "evidence_files": 0, "citations": 0}
required_claims = ["absentDownstreamRole", "clientImpact", "crossSideCausality"]
def walk(x, inherited_artifact=None, scenario="?"):
if isinstance(x, dict):
artifact_id = x.get("artifactId", inherited_artifact)
if {"entryId", "lineStart", "lineEnd"} <= x:
yield artifact_id, x
for value in x.values():
yield from walk(value, artifact_id, scenario)
elif isinstance(x, list):
for value in x:
yield from walk(value, inherited_artifact, scenario)
for scenario in sorted(p for p in root.iterdir() if p.is_dir()):
stats["scenarios"] += 1
manifest_path, expected_path = scenario/"manifest.json", scenario/"expected.json"
manifest, expected = json.loads(manifest_path.read_text()), json.loads(expected_path.read_text())
stats["json"] += 2
artifacts = {a["artifactId"]: a for a in manifest.get("artifacts", [])}
stats["artifacts"] += len(artifacts)
if list(artifacts) != sorted(artifacts):
errors.append(f"{scenario.name}: artifact IDs are not sorted")
if expected.get("prohibitedClaims") != required_claims:
errors.append(f"{scenario.name}: prohibitedClaims differs: {expected.get('prohibitedClaims')!r}")
ordering = expected.get("ordering", {})
if ordering.get("resultsBy") != ["resultId"] or ordering.get("evidenceBy") != ["artifactId", "lineStart", "lineEnd"]:
errors.append(f"{scenario.name}: incomplete ordering contract: {ordering!r}")
for artifact_id, artifact in artifacts.items():
rel = artifact.get("relativePath")
physical = scenario/rel if rel else None
state = artifact.get("captureState")
if state in ("captured", "capped"):
if not physical or not physical.is_file():
errors.append(f"{scenario.name}: {artifact_id} missing physical evidence")
continue
stats["evidence_files"] += 1
if artifact.get("bytesCopied") != physical.stat().st_size:
errors.append(f"{scenario.name}: {artifact_id} byte count mismatch")
raw = physical.read_bytes()
if raw and not raw.endswith(b"\n") and artifact.get("rotation", {}).get("fragmentComplete") is not False:
errors.append(f"{scenario.name}: unterminated {artifact_id} lacks fragmentComplete:false")
elif rel is not None or artifact.get("bytesCopied") != 0:
errors.append(f"{scenario.name}: nonphysical {artifact_id} declares a path/bytes")
result_ids = [r.get("resultId") for r in expected.get("results", [])]
if result_ids != sorted(result_ids):
errors.append(f"{scenario.name}: result IDs are not sorted")
for result in expected.get("results", []):
key = result.get("transactionKey", {})
required_key = ["profileId", "profileVersion", "siteCode", "componentId", "workItemId"]
if any(key.get(k) in (None, "") for k in required_key):
errors.append(f"{scenario.name}: {result.get('resultId')} has incomplete transaction key")
evidence = result.get("evidence", [])
positions = [(e.get("artifactId"), e.get("lineStart"), e.get("lineEnd")) for e in evidence]
if positions != sorted(positions):
errors.append(f"{scenario.name}: {result.get('resultId')} evidence is unordered")
if result.get("findingClass") == "confirmedFailure" and not any(e.get("terminal") is True for e in evidence):
errors.append(f"{scenario.name}: {result.get('resultId')} confirmed failure has no terminal evidence")
for artifact_id, citation in walk(expected):
stats["citations"] += 1
artifact = artifacts.get(artifact_id)
if not artifact:
errors.append(f"{scenario.name}: {citation.get('entryId')} uses unknown artifact {artifact_id!r}")
continue
rel = artifact.get("relativePath")
physical = scenario/rel if rel else None
if not physical or not physical.is_file():
errors.append(f"{scenario.name}: {citation.get('entryId')} cites unavailable evidence")
continue
lines = physical.read_text().splitlines()
lo, hi = citation["lineStart"], citation["lineEnd"]
if not (1 <= lo <= hi <= len(lines)):
errors.append(f"{scenario.name}: {citation.get('entryId')} range {lo}-{hi} invalid for {len(lines)} lines")
if citation["entryId"] != f"{artifact_id}:{lo}-{hi}":
errors.append(f"{scenario.name}: malformed entry ID {citation.get('entryId')}")
if citation.get("completeLogicalRecord") is False and artifact.get("rotation", {}).get("fragmentComplete") is not False:
errors.append(f"{scenario.name}: incomplete citation lacks fragmentComplete:false")
print("STATS", json.dumps(stats, sort_keys=True))
print("PASS" if not errors else "FAIL\n" + "\n".join(errors))
PY
echo '=== Normalized manifest and expected-contract summaries ==='
for expected in "$root"/*/expected.json; do
scenario=$(basename "$(dirname "$expected")")
echo "--- $scenario ---"
jq -c '{
scenario,
results: [.results[]? | {
resultId, state, findingClass, transactionKey,
phases, summary, evidence: [.evidence[]? | {entryId, terminal, completeLogicalRecord}],
nextArtifacts
}],
unlinkedObservations: [.unlinkedObservations[]? | {observationId,state,summary,evidence,nextArtifacts}],
coverageGaps,
ordering,
prohibitedClaims
}' "$expected"
done
echo '=== Safety-critical manifests and exact evidence ==='
for scenario in incomplete malformed rotation-boundary contradictory recovery; do
echo "--- manifest: $scenario ---"
cat -n "$root/$scenario/manifest.json"
echo "--- expected: $scenario ---"
cat -n "$root/$scenario/expected.json"
while IFS= read -r evidence; do
echo "--- evidence: $evidence ---"
cat -n "$evidence"
done < <(fd -t f . "$root/$scenario/evidence" | sort)
done
echo '=== Preparation document ==='
cat -n "$doc"
echo '=== Boundary-claim vocabulary in corpus and document ==='
rg -n -i -C 2 \
'client|downstream|cross[- ]side|causal|causality|cause|missing[- ]role|absent[- ]role|merge|join|rotation' \
"$root" "$doc"Length of output: 7167 🏁 Script executed: #!/bin/bash
set -euo pipefail
root="crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core"
doc="docs/sccm/preparation/issue-327-server-site-core-corpus.md"
echo '=== JSON, artifact, citation, ordering, and safety audit ==='
python3 - "$root" <<'PY'
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
errors = []
stats = {"scenarios": 0, "json": 0, "artifacts": 0, "evidence_files": 0, "citations": 0}
required_claims = ["absentDownstreamRole", "clientImpact", "crossSideCausality"]
def walk(node, inherited_artifact=None):
if isinstance(node, dict):
artifact_id = node.get("artifactId", inherited_artifact)
if {"entryId", "lineStart", "lineEnd"}.issubset(node.keys()):
yield artifact_id, node
for value in node.values():
yield from walk(value, artifact_id)
elif isinstance(node, list):
for value in node:
yield from walk(value, inherited_artifact)
for scenario in sorted(p for p in root.iterdir() if p.is_dir()):
stats["scenarios"] += 1
manifest = json.loads((scenario/"manifest.json").read_text())
expected = json.loads((scenario/"expected.json").read_text())
stats["json"] += 2
artifacts = {a["artifactId"]: a for a in manifest.get("artifacts", [])}
stats["artifacts"] += len(artifacts)
if list(artifacts) != sorted(artifacts):
errors.append(f"{scenario.name}: artifact IDs are not sorted")
if expected.get("prohibitedClaims") != required_claims:
errors.append(f"{scenario.name}: prohibitedClaims differs: {expected.get('prohibitedClaims')!r}")
ordering = expected.get("ordering", {})
if ordering.get("resultsBy") != ["resultId"] or ordering.get("evidenceBy") != ["artifactId", "lineStart", "lineEnd"]:
errors.append(f"{scenario.name}: incomplete ordering contract")
for artifact_id, artifact in artifacts.items():
rel = artifact.get("relativePath")
physical = scenario/rel if rel else None
state = artifact.get("captureState")
if state in ("captured", "capped"):
if not physical or not physical.is_file():
errors.append(f"{scenario.name}: {artifact_id} missing physical evidence")
continue
stats["evidence_files"] += 1
if artifact.get("bytesCopied") != physical.stat().st_size:
errors.append(f"{scenario.name}: {artifact_id} byte count mismatch")
raw = physical.read_bytes()
if raw and not raw.endswith(b"\n") and artifact.get("rotation", {}).get("fragmentComplete") is not False:
errors.append(f"{scenario.name}: unterminated {artifact_id} lacks fragmentComplete:false")
elif rel is not None or artifact.get("bytesCopied") != 0:
errors.append(f"{scenario.name}: nonphysical {artifact_id} declares a path/bytes")
results = expected.get("results", [])
if [r.get("resultId") for r in results] != sorted(r.get("resultId") for r in results):
errors.append(f"{scenario.name}: result IDs are not sorted")
for result in results:
key = result.get("transactionKey", {})
required_key = ["profileId", "profileVersion", "siteCode", "componentId", "workItemId"]
if any(key.get(k) in (None, "") for k in required_key):
errors.append(f"{scenario.name}: {result.get('resultId')} has incomplete transaction key")
evidence = result.get("evidence", [])
positions = [(e.get("artifactId"), e.get("lineStart"), e.get("lineEnd")) for e in evidence]
if positions != sorted(positions):
errors.append(f"{scenario.name}: {result.get('resultId')} evidence is unordered")
if result.get("findingClass") == "confirmedFailure" and not any(e.get("terminal") is True for e in evidence):
errors.append(f"{scenario.name}: {result.get('resultId')} confirmed failure has no terminal evidence")
for artifact_id, citation in walk(expected):
stats["citations"] += 1
artifact = artifacts.get(artifact_id)
if not artifact:
errors.append(f"{scenario.name}: {citation.get('entryId')} uses unknown artifact {artifact_id!r}")
continue
rel = artifact.get("relativePath")
physical = scenario/rel if rel else None
if not physical or not physical.is_file():
errors.append(f"{scenario.name}: {citation.get('entryId')} cites unavailable evidence")
continue
lines = physical.read_text().splitlines()
lo, hi = citation["lineStart"], citation["lineEnd"]
if not (1 <= lo <= hi <= len(lines)):
errors.append(f"{scenario.name}: {citation.get('entryId')} range {lo}-{hi} invalid for {len(lines)} lines")
if citation["entryId"] != f"{artifact_id}:{lo}-{hi}":
errors.append(f"{scenario.name}: malformed entry ID {citation.get('entryId')}")
if citation.get("completeLogicalRecord") is False and artifact.get("rotation", {}).get("fragmentComplete") is not False:
errors.append(f"{scenario.name}: incomplete citation lacks fragmentComplete:false")
print("STATS", json.dumps(stats, sort_keys=True))
print("PASS" if not errors else "FAIL\n" + "\n".join(errors))
PY
echo '=== Normalized all-scenario contracts ==='
python3 - "$root" <<'PY'
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
for scenario in sorted(p for p in root.iterdir() if p.is_dir()):
expected = json.loads((scenario/"expected.json").read_text())
print("\n---", scenario.name, "---")
print(json.dumps({
"results": [{
"resultId": r.get("resultId"), "state": r.get("state"),
"findingClass": r.get("findingClass"), "transactionKey": r.get("transactionKey"),
"evidence": [{
"entryId": e.get("entryId"), "terminal": e.get("terminal"),
"completeLogicalRecord": e.get("completeLogicalRecord")
} for e in r.get("evidence", [])],
"nextArtifacts": r.get("nextArtifacts")
} for r in expected.get("results", [])],
"unlinkedObservations": expected.get("unlinkedObservations"),
"coverageGaps": expected.get("coverageGaps"),
"ordering": expected.get("ordering"),
"prohibitedClaims": expected.get("prohibitedClaims")
}, sort_keys=True))
PY
echo '=== Safety-critical manifests, expected contracts, and evidence ==='
for scenario in incomplete malformed rotation-boundary contradictory recovery; do
echo "--- manifest: $scenario ---"
cat -n "$root/$scenario/manifest.json"
echo "--- expected: $scenario ---"
cat -n "$root/$scenario/expected.json"
find "$root/$scenario/evidence" -type f -print | sort | while IFS= read -r evidence; do
echo "--- evidence: $evidence ---"
cat -n "$evidence"
done
done
echo '=== Preparation document ==='
cat -n "$doc"
echo '=== Boundary-claim vocabulary ==='
rg -n -i -C 2 \
'client|downstream|cross[- ]side|causal|causality|cause|missing[- ]role|absent[- ]role|merge|join|rotation' \
"$root" "$doc"Length of output: 114642
Reviewed the entire added corpus and preparation contract:
✅ Action performedFull review finished. Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 56 minutes. |
Contributor
There was a problem hiding this comment.
🟡 Not ready to approve
The contract document currently uses inconsistent terminology/casing (vs the fixture JSON field/enum values), which risks ambiguity when later implementing and validating the reducer contract.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
Prepares the Issue #327 SCCM site-core/status workflow corpus by adding a frozen, scenario-based contract document plus a set of synthetic server-role fixtures (manifests, expected outputs, and sanitized evidence) for later reducer implementation once #318/#335 land.
Changes:
- Adds a preparation document that defines the scenario matrix, phase/terminality rules, transaction-key shape, and bounded next-artifact request rules for #327.
- Introduces 9 synthetic SCCM server scenarios under
crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/with per-scenariomanifest.json+expected.json. - Adds sanitized CCM-log evidence files (including malformed/rotation-boundary/capped fragments) to exercise coverage/ordering/fragment-completeness requirements.
File summaries
| File | Description |
|---|---|
| docs/sccm/preparation/issue-327-server-site-core-corpus.md | Defines the frozen corpus contract and scenario expectations for #327. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/status-processing-failure/manifest.json | Scenario manifest describing captured sitecomp + statmgr artifacts. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/status-processing-failure/expected.json | Expected result contract for status-processing terminal failure. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/status-processing-failure/evidence/sccm/server/site-core/status/current/statmgr.log | Synthetic statmgr.log evidence including terminal failure record. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/status-processing-failure/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Synthetic sitecomp.log evidence showing start/work/inbox phases. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/rotation-boundary/manifest.json | Scenario manifest for split rotation fragments across current + .lo_. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/rotation-boundary/expected.json | Expected contract asserting no cross-rotation join and coverage gaps. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/rotation-boundary/evidence/sccm/server/site-core/sitecomp/lo_/sitecomp.log.lo_ | Opening fragment evidence (incomplete logical record). |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/rotation-boundary/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Closing fragment evidence (must not be concatenated). |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/recovery/manifest.json | Scenario manifest for terminal failure followed by later recovery signal. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/recovery/expected.json | Expected contract for recovered state (historical symptom, no current failure). |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/recovery/evidence/sccm/server/site-core/status/current/statmgr.log | Synthetic status evidence containing recovery terminal record. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/recovery/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Synthetic component evidence containing terminal failure record. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/malformed/manifest.json | Scenario manifest for malformed/unclosed status record fragment. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/malformed/expected.json | Expected contract for parse-gap symptom with bounded follow-up request. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/malformed/evidence/sccm/server/site-core/status/current/statmgr.log | Malformed fragment evidence (incomplete logical record). |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/incomplete/manifest.json | Scenario manifest for capped/absent/access-denied coverage-only inputs. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/incomplete/expected.json | Expected contract for insufficient evidence with next-artifact request. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/incomplete/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Capped evidence including an incomplete trailing fragment. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/inbox-backlog/manifest.json | Scenario manifest for inbox backlog with status source absent. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/inbox-backlog/expected.json | Expected contract for blocked/deferred result + follow-up request. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/inbox-backlog/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Synthetic component evidence including backlog signal. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/healthy/manifest.json | Scenario manifest for fully healthy end-to-end phase completion. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/healthy/expected.json | Expected contract for healthy terminal result with no finding class. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/healthy/evidence/sccm/server/site-core/status/current/statmgr.log | Synthetic status evidence including healthy terminal record. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/healthy/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Synthetic component evidence showing start/work/inbox accepted. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/contradictory/manifest.json | Scenario manifest for simultaneous independent success/failure components. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/contradictory/expected.json | Expected contract asserting no same-minute merge or cross-component recovery. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/contradictory/evidence/sccm/server/site-core/status/current/statmgr.log | Synthetic status evidence for the healthy component transaction. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/contradictory/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Synthetic component evidence containing both component transactions. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/component-failure/manifest.json | Scenario manifest for terminal component failure with status source absent. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/component-failure/expected.json | Expected contract for confirmed failure with coverage-only absent status. |
| crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core/component-failure/evidence/sccm/server/site-core/sitecomp/current/sitecomp.log | Synthetic component evidence including terminal failure record. |
Review details
Comments suppressed due to low confidence (1)
docs/sccm/preparation/issue-327-server-site-core-corpus.md:53
- These state/finding labels (
BlockedOrDeferred,HealthyOrTerminal) are PascalCase, while the fixtures serialize them as lower camelCase inlastSuccessfulPhaseandfindingClass/statevalues. Adding the serialized forms here would make the contract easier to apply consistently.
- Files reviewed: 20/33 changed files
- Comments generated: 2
- Review effort level: Low
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
Owner
Author
|
@coderabbitai full review Commit |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
commit="86ea260"
root="crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core"
echo '=== Commit and changed files ==='
git show -s --format='%H%n%P%n%s%n%b' "$commit"
git diff-tree --no-commit-id --name-status -r "$commit"
echo '=== Diff ==='
git show --format= --find-renames "$commit"
echo '=== JSON contract map ==='
for f in $(git diff-tree --no-commit-id --name-only -r "$commit" | rg '\.json$' || true); do
echo "--- $f ---"
jq -C . "$f"
done
echo '=== Static corpus audit: ID uniqueness, physical citations, ordering ==='
python3 - "$root" <<'PY'
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
errors, notes = [], []
global_ids = {}
stats = {"scenarios": 0, "artifacts": 0, "citations": 0, "physical_citations": 0,
"coverage_refs": 0, "nonphysical_citations": 0}
citation_keys = {"entryId", "lineStart", "lineEnd"}
def walk(value, inherited_artifact=None, path="$"):
if isinstance(value, dict):
aid = value.get("artifactId", inherited_artifact)
if citation_keys <= value.keys():
yield aid, value, path
for k, v in value.items():
yield from walk(v, aid, f"{path}.{k}")
elif isinstance(value, list):
for i, v in enumerate(value):
yield from walk(v, inherited_artifact, f"{path}[{i}]")
for scenario in sorted(p for p in root.iterdir() if p.is_dir()):
manifest_p, expected_p = scenario/"manifest.json", scenario/"expected.json"
if not (manifest_p.is_file() and expected_p.is_file()): continue
stats["scenarios"] += 1
manifest = json.loads(manifest_p.read_text())
expected = json.loads(expected_p.read_text())
artifacts_list = manifest.get("artifacts", [])
artifacts = {}
for a in artifacts_list:
aid = a.get("artifactId")
stats["artifacts"] += 1
if aid in artifacts:
errors.append(f"{scenario.name}: duplicate artifactId {aid!r} in manifest")
artifacts[aid] = a
global_ids.setdefault(aid, []).append(scenario.name)
if [a.get("artifactId") for a in artifacts_list] != sorted(a.get("artifactId") for a in artifacts_list):
errors.append(f"{scenario.name}: artifact list not sorted")
for aid, c, path in walk(expected):
stats["citations"] += 1
artifact = artifacts.get(aid)
if not artifact:
errors.append(f"{scenario.name}: {path}: citation has unknown artifact {aid!r}")
continue
state = artifact.get("captureState")
rel = artifact.get("relativePath")
physical = scenario / rel if rel else None
is_physical = state in ("captured", "capped") and physical and physical.is_file()
if not is_physical:
stats["nonphysical_citations"] += 1
errors.append(f"{scenario.name}: {path}: citation {c['entryId']} targets nonphysical {aid} ({state})")
continue
stats["physical_citations"] += 1
lines = physical.read_text().splitlines()
lo, hi = c["lineStart"], c["lineEnd"]
if not (1 <= lo <= hi <= len(lines)):
errors.append(f"{scenario.name}: {c['entryId']}: invalid physical range")
if c["entryId"] != f"{aid}:{lo}-{hi}":
errors.append(f"{scenario.name}: malformed citation ID {c['entryId']}")
if path.startswith("$.coverageGaps") or c.get("coverageOnly") is True:
stats["coverage_refs"] += 1
if state == "capped" and c.get("terminal") is True:
errors.append(f"{scenario.name}: capped citation {c['entryId']} is terminal")
if c.get("completeLogicalRecord") is False and artifact.get("rotation",{}).get("fragmentComplete") is not False:
errors.append(f"{scenario.name}: incomplete citation {c['entryId']} lacks fragmentComplete:false")
for aid, scenarios in sorted(global_ids.items()):
if len(scenarios) > 1:
notes.append(f"artifactId {aid!r} is reused across manifests: {', '.join(scenarios)}")
print("STATS", json.dumps(stats, sort_keys=True))
print("CROSS_MANIFEST_REUSE", len(notes))
print("\n".join(notes))
if errors:
print("FAIL")
print("\n".join(errors))
else:
print("PASS")
PY
echo '=== Classification and boundary vocabulary in changed corpus/docs ==='
git diff-tree --no-commit-id --name-only -r "$commit" | while IFS= read -r f; do
case "$f" in
*.json|*.md|*.yaml|*.yml|*.rs|*.ts|*.js)
echo "--- $f ---"
rg -n -i -C 3 \
'recognized.?backlog|backlog|classification|deterministic|artifactId|manifest|physical|nonphysical|coverage.?only|coverage.?reference|malformed|capped|terminal' \
"$f" || true
;;
esac
doneLength of output: 46831
The commit addresses the stated contract gaps:
Static audit passed across all nine scenarios: 18 artifacts, 38 citations, zero nonphysical citations, no duplicate artifact IDs within any manifest, and no artifact-ID reuse across manifests. You are interacting with an AI system. |
18 tasks
Owner
Author
|
@coderabbitai full review Please review current head |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
head="1f95d97"
previous="86ea260"
root="crates/cmtraceopen-parser/tests/fixtures/sccm/server/site_core"
echo '=== Commit metadata and change range ==='
git show -s --format='%H%n%P%n%s%n%b' "$head"
git log --oneline "${previous}..${head}"
echo '=== Changed files ==='
git diff --name-status "${previous}..${head}"
echo '=== Full changed diff ==='
git diff --find-renames "${previous}..${head}"
echo '=== Changed JSON/document maps ==='
git diff --name-only "${previous}..${head}" | while IFS= read -r f; do
case "$f" in
*.json)
echo "--- $f ---"
jq -C . "$f"
;;
*.md|*.yaml|*.yml)
echo "--- $f ---"
cat -n "$f"
;;
esac
done
echo '=== Corpus file inventory ==='
fd -t f . "$root" | sort
echo '=== Targeted contract vocabulary ==='
rg -n -i -C 3 \
'siteCode|phase|serialized|recognized.?backlog|blocked.?or.?deferred|artifactId|manifest|physical|nonphysical|coverage.?only|coverage.?reference|malformed|capped|fragmentComplete|terminal|rotation' \
"$root" -g '*.json' -g '*.md'
echo '=== Static corpus audit ==='
python3 - "$root" <<'PY'
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
errors=[]
stats={"scenarios":0,"artifacts":0,"physicalEvidence":0,"citations":0,"coverageCitations":0,"results":0}
required_claims=["absentDownstreamRole","clientImpact","crossSideCausality"]
physical_states={"captured","capped"}
def citations(x, inherited=None, path="$"):
if isinstance(x,dict):
aid=x.get("artifactId", inherited)
if {"entryId","lineStart","lineEnd"} <= x.keys():
yield aid,x,path
for k,v in x.items():
yield from citations(v,aid,f"{path}.{k}")
elif isinstance(x,list):
for i,v in enumerate(x):
yield from citations(v,inherited,f"{path}[{i}]")
for scenario in sorted(p for p in root.iterdir() if p.is_dir()):
mp,ep=scenario/"manifest.json",scenario/"expected.json"
if not (mp.is_file() and ep.is_file()):
errors.append(f"{scenario.name}: missing manifest or expected contract")
continue
stats["scenarios"]+=1
m=json.loads(mp.read_text())
e=json.loads(ep.read_text())
artifacts=m.get("artifacts",[])
artifact_map={}
ids=[]
for a in artifacts:
aid=a.get("artifactId"); ids.append(aid); stats["artifacts"]+=1
if aid in artifact_map: errors.append(f"{scenario.name}: duplicate manifest artifactId {aid!r}")
artifact_map[aid]=a
state=a.get("captureState"); rel=a.get("relativePath")
physical=scenario/rel if rel else None
if state in physical_states:
if not physical or not physical.is_file():
errors.append(f"{scenario.name}: physical {aid} ({state}) has no evidence file")
else:
stats["physicalEvidence"]+=1
if a.get("bytesCopied") != physical.stat().st_size:
errors.append(f"{scenario.name}: {aid} byte count differs from evidence")
if physical.read_bytes() and not physical.read_bytes().endswith(b"\n") and a.get("rotation",{}).get("fragmentComplete") is not False:
errors.append(f"{scenario.name}: unterminated {aid} lacks fragmentComplete:false")
elif rel is not None or a.get("bytesCopied") != 0:
errors.append(f"{scenario.name}: nonphysical {aid} declares path or bytes")
if ids != sorted(ids): errors.append(f"{scenario.name}: artifact list is not deterministic by artifactId")
if e.get("prohibitedClaims") != required_claims:
errors.append(f"{scenario.name}: prohibitedClaims mismatch")
ordering=e.get("ordering",{})
if ordering.get("resultsBy") != ["resultId"] or ordering.get("evidenceBy") != ["artifactId","lineStart","lineEnd"]:
errors.append(f"{scenario.name}: ordering contract mismatch")
results=e.get("results",[])
stats["results"]+=len(results)
if [r.get("resultId") for r in results] != sorted(r.get("resultId") for r in results):
errors.append(f"{scenario.name}: results not sorted by resultId")
for r in results:
key=r.get("transactionKey",{})
for k in ("profileId","profileVersion","siteCode","componentId","workItemId"):
if key.get(k) in (None,""): errors.append(f"{scenario.name}: {r.get('resultId')} lacks {k}")
phases=r.get("phases",[])
if not isinstance(phases,list) or not phases or any(not isinstance(x,str) or not x for x in phases):
errors.append(f"{scenario.name}: {r.get('resultId')} phases are not explicit serialized names")
ev=r.get("evidence",[])
if [(x.get("artifactId"),x.get("lineStart"),x.get("lineEnd")) for x in ev] != sorted((x.get("artifactId"),x.get("lineStart"),x.get("lineEnd")) for x in ev):
errors.append(f"{scenario.name}: {r.get('resultId')} evidence unordered")
if r.get("findingClass")=="confirmedFailure" and not any(x.get("terminal") is True for x in ev):
errors.append(f"{scenario.name}: {r.get('resultId')} failure lacks terminal evidence")
for aid,c,path in citations(e):
stats["citations"]+=1
a=artifact_map.get(aid)
if not a:
errors.append(f"{scenario.name}: {path} has unknown artifactId {aid!r}"); continue
rel=a.get("relativePath"); f=scenario/rel if rel else None
if a.get("captureState") not in physical_states or not f or not f.is_file():
errors.append(f"{scenario.name}: {path} cites nonphysical artifact {aid}"); continue
lines=f.read_text().splitlines()
lo,hi=c["lineStart"],c["lineEnd"]
if not (1<=lo<=hi<=len(lines)):
errors.append(f"{scenario.name}: {c.get('entryId')} has invalid line range")
if c.get("entryId") != f"{aid}:{lo}-{hi}":
errors.append(f"{scenario.name}: malformed citation ID {c.get('entryId')}")
if path.startswith("$.coverageGaps") or c.get("coverageOnly") is True: stats["coverageCitations"]+=1
if a.get("captureState")=="capped" and c.get("terminal") is True:
errors.append(f"{scenario.name}: capped citation {c.get('entryId')} is terminal")
if c.get("completeLogicalRecord") is False and a.get("rotation",{}).get("fragmentComplete") is not False:
errors.append(f"{scenario.name}: incomplete citation {c.get('entryId')} lacks fragmentComplete:false")
print("STATS",json.dumps(stats,sort_keys=True))
print("PASS" if not errors else "FAIL\n"+"\n".join(errors))
PYLength of output: 155194
The documentation-only change correctly aligns the contract with the existing corpus serialization:
One audit sub-check initially expected a You are interacting with an AI system. |
adamgell
added a commit
that referenced
this pull request
Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Issue-scoped synthetic corpus and reducer contract preparation for #327 site-core/status workflows:
rotation.fragmentComplete: falsefor every physically incomplete referenced fragmentNo production reducer or shared interface is added in this PR.
Dependency state
Production #327 implementation remains blocked on reviewed #318 and #335 contracts. The corpus does not infer client impact, absent downstream roles, or cross-side causality.
Verification
cargo test --locked -p cmtraceopen-parser— 571 passednpx tsc --noEmit— passIndependent review’s sole Important finding (missing incomplete-fragment declaration) is fixed in
59902aa.CodeRabbit and final semantic review are requested. Native Windows acceptance is not claimed.
Refs #327
Refs #317