[cuda] implement feature_contri per-feature gain scaling on the CUDA tree learner

[maxwbuckley]

Problem: the delta

When training with device_type="cuda", feature_contri (per-feature gain scaling, documented as gain[i] = feature_contri[i] * gain[i]) was silently ignored. The scaling is applied only in the CPU FeatureHistogram (output->gain *= meta_->penalty after the per-feature best threshold is found); the CUDA best-split finder compared raw gains with no per-feature multiplier.

Measured delta before this PR (500×4 synthetic data dominated by feature 0; learning_rate=0.1, 5 rounds, gpu_use_dp=true):

feature_contri	features used CPU	features used CUDA	match	max pred delta
[1, 1, 1, 1]	[0, 1, 2]	[0, 1, 2]	✓	0.0
[0, 1, 1, 1]	[1, 2, 3]	[0, 1, 2]	✗	1.22
[1, 1, 1, 0]	[0, 1, 2]	[0, 1, 2]	✓	0.0
[0.5, 1, 1, 1]	[0, 1, 2, 3]	[0, 1, 2]	✗	0.37
[0.5, 1, 2, 1]	[0, 1, 2]	[0, 1, 2]	✓	0.44

CPU honors the scaling (a zero-contri feature is never split on, a down-weighted feature is used less); CUDA ignored it entirely — different features selected, predictions diverging by up to 1.22.

Fix

Implements per-feature gain scaling in the CUDA best-split finder:

• SplitFindTask gains a double penalty field, computed on host from config->feature_contri indexed by the real feature index — mirroring FeatureMetainfo::penalty in the CPU SetFeatureInfo. Set when the task array is built; refreshed (with a device re-copy) in ResetConfig.
• Each kernel multiplies the finalized per-feature gain by the task's penalty after the per-feature best threshold is selected — exactly where CPU applies output->gain *= meta_->penalty. Within-feature threshold selection and the min_gain_to_split checks use the unscaled gain; only the cross-feature / cross-leaf comparison sees the scaled gain. All six gain-finalization sites are covered (numerical, categorical, global-memory, and discretized variants).

The first commit's Log::Fatal guard is removed: feature_contri now works on CUDA instead of being rejected.

Result: the delta after the fix

Same measurement, post-fix:

feature_contri	features used CPU	features used CUDA	match	max pred delta
[1, 1, 1, 1]	[0, 1, 2]	[0, 1, 2]	✓	0.0
[0, 1, 1, 1]	[1, 2, 3]	[1, 2, 3]	✓	8.9e-16
[1, 1, 1, 0]	[0, 1, 2]	[0, 1, 2]	✓	0.0
[0.5, 1, 1, 1]	[0, 1, 2, 3]	[0, 1, 2, 3]	✓	0.0
[0.5, 1, 2, 1]	[0, 1, 2]	[0, 1, 2]	✓	0.0

Identical feature selection on every case, and predictions match bit-for-bit (≤ 8.9e-16 = one fp64 ULP).

Tests (in test_dual.py, gated on TASK=cuda)

test_cuda_feature_contri_matches_cpu — 5 parametrized contri vectors, asserting (a) identical feature-selection sets, (b) zero-contri features excluded on both backends, (c) prediction parity at atol=1e-10. All 5 pass with the fix; the [0,1,1,1] and [0.5,1,1,1] cases fail on the unpatched build. Full test_dual.py suite (59 tests) passes.

Notes

• The CPU code at this commit assigns the raw feature_contri value to the penalty (no max(0, ·) despite the parameter docs); the CUDA implementation mirrors the actual CPU code so both backends stay in lockstep.
• One pre-existing CUDA quirk found (not addressed here, out of scope): the categorical one-hot global-memory kernel never assigns gain at all — that's an upstream issue independent of feature_contri.

🤖 Generated with Claude Code

[BelixRogner]

Thank you, Max — and thank you, Claude Code (separately 🙂). Verdict: SOLID, no blocking bug.

Reviewed against the CPU output->gain *= meta_->penalty site and FeatureMetainfo::penalty setup:

• Real-vs-inner indexing is correct (the most likely place to get this wrong): feature_contri_[real_feature_index_[inner_feature_index]], an exact mirror of CPU's config->feature_contri[train_data->RealFeatureIndex(i)]. Clean.
• Ordering matches CPU: within-feature threshold selection and min_gain_to_split use the unscaled gain; only the final cross-feature/cross-leaf comparison sees the scaled gain.
• All 6 gain-assigning kernel sites got the multiply (numerical, discretized, one-hot + multi-cat categorical, and both global-memory variants). The one site that doesn't (SyncBestSplitForLeafKernel) correctly stays a kMinScore sentinel.
• ResetConfig recomputes penalties and re-copies to device. Raw-value semantics (no max(0,·)) faithfully mirror the actual CPU code, not the docs.

Two non-blocking follow-ups (neither is introduced by this PR):

• You correctly flagged the pre-existing quirk where the global-memory one-hot categorical branch never assigns gain at all. Your PR rightly leaves it untouched (no uninitialized-gain × penalty bug), but it's a real latent bug worth a separate fix.
• The new tests exercise only the shared-memory numerical kernel (the dense 4-feature data keeps use_global_memory_=false). The other 5 scaled sites are verified by reading only — a high-cardinality case (forces global memory) + a categorical case would close that.

Blocker is just the required ruff format + git merge master. macOS red = dask flake.

P.S. — smallest diff of the five (+124/−6) and it still failed lint. 😄 At this point I'm convinced ruff format is the one feature you've left silently ignored.

[BelixRogner]

Thank you, Max — and thank you, Claude Code (independently 🙂). Rebase nudge 👉

This one reviewed out SOLID — it's only CONFLICTING because master moved under it (forced-splits #17 landed, plus a stack of best-split-finder / data-partition merges: #21–#27). A rebase onto current master should clear it cleanly.

You have write access now (accept the repo invite if you haven't), so once it's rebased and green you can merge it yourself. 🚀

[maxwbuckley]

Rebased onto current master. 👋

Please re-review — the branch was rebuilt as master + this one commit, so it isn't byte-for-byte what you reviewed.

What changed. The old branch carried ~28 commits of work that has since landed on master under different SHAs, so a straight replay conflicted. Rebuilt clean: 4 files, 1 commit.

The only real conflict was in cuda_best_split_finder.hpp, where master added SetHistogramEvents() (the histogram event-ordering work) in the same spot this PR adds its host-side feature_contri penalty accessor. Both are additive and independent — both kept.

Good news: unlike #20, this one needed no math re-implementation. It scales gains in the split finder and never touches cuda_leaf_splits.hpp, so your SplitGainMath refactor and this PR are orthogonal. The feature_contri logic itself is unchanged from what you reviewed.

Validation (RTX 5090, CUDA 13.2, built from source): 5 feature_contri tests pass; full test_dual.py = 124 passed, 1 skipped. cpplint / ruff check / ruff format all clean.

Rebased and validated with Claude Code; real GPU runs, not CI.