[cuda] implement basic monotone constraint enforcement on the CUDA tree learner

[maxwbuckley]

Problem: the delta

When training with device_type="cuda", monotone_constraints was silently ignored. The CUDA best-split finder and CUDALeafSplits had no constraint plumbing — unlike the CPU FeatureHistogram, which clamps child outputs into the leaf's [min, max] constraint range and rejects monotonicity-violating splits.

Measured delta before this PR (600×3 regression, monotone_constraints_method="basic", gpu_use_dp=true; violations counted on 4,500 grid points per config — 3 base points × 500-point sweeps × constrained features):

constraints	rounds	CPU violations	CUDA violations	CUDA worst violation	max pred delta vs CPU
[1, −1, 0]	30	0	9	3.0e-02	0.52
[1, −1, 0]	100	0	203	1.1e-01	0.32
[1, 1, 1]	30	0	157	3.9e-01	2.76
[1, 1, 1]	100	0	456	3.9e-01	2.81
[−1, 0, 1]	30	0	157	7.0e-01	2.82
[−1, 0, 1]	100	0	389	6.9e-01	3.14

A user requesting monotone constraints on CUDA got a model that violates the requested relationship at hundreds of points — silently. For regulatory/safety uses of monotonicity this is a hard correctness failure.

Fix

Implements the basic monotone constraint method on the CUDA tree learner:

• CalculateSplittedLeafOutputMC / GetSplitGainsMC device functions mirror the CPU USE_MC formulas: candidate child outputs are clamped into the leaf's [min, max] constraint range, and splits whose clamped outputs violate the monotone direction are rejected (gain 0).
• Each SplitFindTask carries its feature's monotone type (config is real-feature-indexed; mapped to inner indices, matching the CPU FeatureMetainfo setup).
• Per-leaf [min, max] constraint ranges are tracked on host — mirroring BasicLeafConstraints::Update midpoint propagation — and passed to the split kernels for the (smaller, larger) leaf pair each iteration.
• Leaf values stored in CUDASplitInfo are the clamped outputs (used for the tree and constraint propagation); leaf gains keep their unconstrained values (they form the child's parent_gain baseline, matching CPU's BeforeNumerical gain-shift computation). Getting this distinction wrong was the prototype's main bug.
• Unsupported monotone configurations fail fast with a clear error: monotone_constraints_method=intermediate/advanced, monotone_penalty>0, and use_quantized_grad + monotone.

This replaces the first commit's Log::Fatal guard: basic-method monotone constraints now work on CUDA.

Result: the delta after the fix

Same measurement, post-fix — across all constraint vectors × {1, 30, 100} rounds × 2 seeds:

constraints	rounds	CUDA violations	CUDA training MSE	CPU training MSE
[1, −1, 0]	30 / 100	0	0.0643 / 0.0122	0.0658 / 0.0129
[1, 1, 1]	30 / 100	0	1.953 / 1.947	1.957 / 1.952
[−1, 0, 1]	30 / 100	0	1.449 / 1.206	1.438 / 1.241
[0, 0, 0] (inactive)	30 / 100	0	bit-identical to CPU (≤ 8.9e-16)	—

Zero monotonicity violations in every configuration (24/24 test configs), with constrained training quality equivalent to CPU (MSE within 3%, often slightly better) and bit-exact parity when constraints are inactive.

Honest scope note: structure parity vs. constraint parity

CPU and CUDA can build different but equally valid constrained trees in multi-round training. The cause: CPU prunes candidate features through its FeatureHistogram::is_splittable_ cache (a feature found unsplittable on a parent is skipped for the whole subtree — a search-space heuristic, not part of the constraint definition). Replicating that cache on GPU requires per-(leaf, feature) state with extra host-device round trips; an experimental version made agreement worse because the cache semantics interact with histogram-subtraction timing. So this PR guarantees the property (zero violations + equivalent quality + bit-parity in the inactive case), not node-by-node tree equality. The PR description and tests state exactly that.

Tests (in test_dual.py, gated on TASK=cuda)

Test	Cases	What it pins down
test_cuda_monotone_constraints_are_enforced	12	zero violations on 4,500-point grids — fails on the old build (up to 456 violations)
test_cuda_monotone_constraints_match_cpu_quality	3	both backends enforce + CUDA MSE within 5% of CPU
test_cuda_monotone_noop_constraints_match_cpu_exactly	2	all-zero constraints → bit-exact CPU parity (atol=1e-10)
test_cuda_monotone_unsupported_configs_raise	1	intermediate/advanced/penalty/quantized configs rejected loudly

Full test_dual.py suite (72 tests) passes.

🤖 Generated with Claude Code

[BelixRogner]

Thank you, Max — and thank you, Claude Code (independently 🙂). This one's a clean bill of health.

I ran a deep correctness review of the device math against the CPU USE_MC path in feature_histogram.hpp and BasicLeafConstraints::Update. Verdict: SOLID, no blocking issues.

• CalculateSplittedLeafOutputMC / GetSplitGainsMC match the CPU clamp + rejection predicate character-for-character ((mc>0 && left>right) || (mc<0 && left<right) → gain 0).
• The delicate part — leaf VALUE clamped vs leaf GAIN unconstrained — is correct. Verified the stored left_gain/right_gain flow into cuda_data_partition.cu as the next level's parent_gain baseline, exactly like CPU's BeforeNumerical. Using clamped outputs there would have inflated child gains; you got it right.
• Host UpdateLeafConstraints mirrors the midpoint propagation (UpdateMin/UpdateMax, parent/new-leaf roles) correctly.
• All three scope guards (intermediate/advanced, monotone_penalty>0, use_quantized_grad) are unconditional Log::Fatal — no silent fall-through — and all tested.

Non-blocking note: path_smooth>0 and lambda_l1>0 combined with monotone aren't exercised by the new tests; the formulas are shared with the already-tested non-MC paths, so low risk.

Only thing standing between this and merge: it fails the required ruff format check, and since all five of your CUDA PRs touch the same files, it'll need a git merge master once the others land. The macOS CI red is the known dask socket flake — unrelated.

P.S. — you wrote a 4,500-point monotonicity grid harness with bit-exact CPU parity… and then didn't run ruff format? 😄 All five PRs tripped it. Exa-row scale, zero-row lint discipline.

[BelixRogner]

Thank you, Max — and thank you, Claude Code (separately 🙂). Rebase + lint nudge 👉

The monotone-constraint device math reviewed out clean. Two things before this can land:

1. Rebase onto current master — it's CONFLICTING now that [cuda] implement forced splits (forcedsplits_filename) on the CUDA tree learner #17 and the [cuda] fix illegal memory access in bagging out-of-bag score update #21–[cuda] order FindBestSplits after histograms with events, not device syncs (~8.6% faster) #27 CUDA stack merged.

2. Its own lint still needs a pass (this is the one PR whose red lint is real code, not infra):

   • cpplint: include-order in src/treelearner/cuda/cuda_single_gpu_tree_learner.cpp (~lines 16-25 — C/C++ system headers after other headers, plus a duplicate algorithm include).
   • ruff PT011: too-broad pytest.raises(lgb.basic.LightGBMError) (~test line 446) — add a match=.

   pre-commit run -a locally catches both in one go (the recurring "ran the kernels but not pre-commit" special 😄).

Write access is live now, so once it's green you can merge it yourself. 🚀

[maxwbuckley]

Rebased onto current master, and both lint issues you flagged are fixed. 👋

Please re-review — there's one behavioural change, called out below.

Your two lint items ✅

• cpplint include-order: the PR had added <algorithm> / <limits> above the cuda_single_gpu_tree_learner.hpp project header, and <algorithm> was then included a second time. Both headers now sit in the existing sorted C++ block below the project header; the duplicate is gone. cpplint is clean on the file with the repo's filters.
• ruff PT011: pytest.raises(lgb.basic.LightGBMError) now carries a match= for each of the four rejected configs (intermediate / advanced / monotone_penalty / use_quantized_grad), so each case asserts it failed for the right reason rather than any LightGBMError. ruff check clean.

(For what it's worth — I'd been running ruff without --config python-package/pyproject.toml, which silently used an 88-char line length and rewrapped unrelated lines. Caught and reverted; the diffs here touch only lines this PR adds.)

⚠️ The behavioural change worth your eyes

FindBestSplitsForLeaf() gained the four monotone-constraint arguments. Since this PR was written, your #17 added a third call site — the forced-split path — which was passing no constraints at all.

Left alone, that call would search for splits ignoring the monotone constraints, cache them in the per-leaf device cache, and FindBestFromAllSplits could later pick one — i.e. a forced-split tree could silently violate monotone_constraints. I extended that call site to pass the same constraints the main search uses.

Validation

RTX 5090, CUDA 13.2, built from source:

• 18 monotone tests pass (enforcement, CPU quality parity, exact no-op parity, and the 4 rejection cases)
• full test_dual.py: 137 passed, 1 skipped — including the forced-split tests, which exercise the call site I changed
• cpplint clean on all 7 changed C++ files; ruff check / ruff format clean under the repo's config

Rebased, re-integrated and validated with Claude Code; real GPU runs, not CI (CI has no GPU, so it never exercises any of this).

[maxwbuckley]

Re-rebased onto current master (the hybrid-growth merges). Both lint items you flagged are still fixed; one new structural change, below.

Monotone constraints are gated off the batched path

Same class of problem as CEGB (#18). Monotone bounds are inherited: a leaf's [min, max] comes from the splits already applied above it. Level-batched growth scores a whole level before applying any of it, so the children of a level's splits would be searched against their parents' stale bounds — and the batched kernels weren't passed constraints at all, so constraints would simply be ignored there.

Fix: fall back to the classic leaf-wise loop when monotone_constraints is set, the same way the batched path is already gated off for categorical features:

if (!config_->monotone_constraints.empty()) {
  use_hybrid_growth_ = false;
}

The batched kernels correspondingly instantiate the inner split kernel with USE_MC = false and identity bounds — reachable only when monotone is off, so it is never a live constraint.

Integration notes

• master hoisted the classic-path FindBestSplitsForLeaf() call into EnqueuePairBestSplitSearch(), so the per-pair constraints are now computed there (one place instead of two).
• The forced-split search path (your [cuda] implement forced splits (forcedsplits_filename) on the CUDA tree learner #17) still gets the constraints explicitly — without them a forced-split tree could silently violate monotone_constraints.
• FindBestSplitsForLeaf() now takes the four constraint args before your defaulted synchronize param.

Your two lint items ✅

• cpplint include-order + duplicate <algorithm>: fixed.
• ruff PT011: each of the four rejection cases now asserts its own message via match=.

Validation

RTX 5090, CUDA 13.2: 18 monotone tests pass (enforcement, CPU quality parity, exact no-op parity, 4 rejection cases); test_dual.py = 125 passed, 12 failed — all 12 are a pre-existing master regression, not this PR (see my note on #35). cpplint / ruff clean.

Validated with Claude Code on real GPU runs; CI has no GPU and never exercises any of this.

[BelixRogner]

Verdict: needs rebase (now conflicting); the device math and host bookkeeping review out correct, so post-rebase this is mergeable pending gates.

Correctness vs the CPU spec — verified line-by-line:

• GetSplitGainsMC mirrors CPU's GetSplitGains<USE_MC=true> exactly: clamp both child outputs into the leaf's [min,max], return 0 on child-order violation, measure gain at the clamped outputs via GetLeafGainGivenOutput. Match.
• UpdateLeafConstraints mirrors BasicLeafConstraints::Update (monotone_constraints.hpp:488-504): new leaf inherits parent entry before tightening, mid = (left+right)/2, decreasing → parent-min/right-max, increasing → parent-max/right-min. (CPU divides by 2.0f, you by 2.0 — division by 2 is exact in binary fp, so bit-identical. Worth a one-word comment so nobody "fixes" it later.)
• The unconstrained left_gain/right_gain retention is the subtle one and you got it right: CPU's gain_shift baseline for the children is the unconstrained GetLeafGain of the child sums, so storing constrained gains would poison parent_gain/min_gain_shift downstream. The comment explains this as a definition — exactly our house style.
• Using the clamped left_value/right_value for the mid-point matches CPU (CPU's best_split_info.left_output is post-clamp).
• Hybrid gating: use_hybrid_growth_ = false under monotone constraints is the correct conservative call (children of a batched level would search against stale parent bounds), and I verified on current master it also disables the selective and graph paths. The batched level kernel passing USE_MC=false + identity bounds with the gating comment is good defensive layering.
• Config guards (basic-only, monotone_penalty=0, no quant) are loud Log::Fatals with actionable messages, and tested. Acceptable staging — intermediate/advanced constraint state is genuinely host-heavy.

Items:

1. Rebase burden is the highest in the stack: cuda_leaf_splits.hpp (fp32-gain rework), the GAIN_T-templated finder inners, the graph L1/L1.5/L2 learner changes, and FindBestSplitsForLeaf's signature all moved last night. The per-leaf scalar constraint args should thread through unchanged, but every touchpoint needs re-doing.
2. ResetConfig/ResetTrainingData don't refresh use_monotone_constraints_/monotone_constraints_ (Init-only). CPU supports changing constraints via parameter reset. Wire it or Fatal on the delta.
3. The quality-parity (not bit-parity) CPU test: the residual divergence you attribute to CPU's is_splittable_ cache is plausibly also plateau tie-breaking — worth revisiting after [cuda] tolerance-based gain tie-break in best-split reductions #13 lands; the test tolerance could likely tighten to bit-parity, which is a much stronger spec check.
4. Two D2H copies of left_value/right_value per split when monotone is active — fine (classic loop already syncs per split), just noting it's on the constrained path only.

Tests are the strongest in the stack: enforcement sweeps (grid probes at 3 base points), CPU quality parity, all-zero no-op bit-parity, and loud-rejection coverage.

md5 locks: unaffected at defaults (empty constraints short-circuits everything; the no-op test proves it).

Gate runs tomorrow (post-rebase): full monotone suite + the all-zero bit-parity test + covtype md5 confirm.

🤖 Review drafted with Claude Code