[cuda] hybrid level-batched tree growth: leaf-wise trees at depth-wise speed

ROADMAP.md

@@ -0,0 +1,158 @@
+# ExaBoost CUDA roadmap
+
+Ideas from the hybrid-growth + benchmark session (2026-07-13/14, PRs #32/#33), roughly
+priority-ordered within groups. Measurements refer to an RTX 5090, per-tree/per-100-tree
+figures from the profiles in the PR discussions.
+
+## Performance
+
+- [ ] **Graphs L1 — device-driven level loop.** Wrap the per-level pipeline in a CUDA
+  conditional-WHILE graph node; a level-controller kernel builds the next level's pair
+  descriptors on-device, updates grid dims via device-updatable node params, and sets the
+  loop condition; ONE host sync per tree. Est. ~1.6-1.7x on small-tree workloads
+  (fraud-class: ~344 us/tree turnaround idle + ~234 us launch API measured), ~1.3x mid
+  (covtype/year), ~nil on roofline-bound data (higgs/epsilon). Depth-limited exact regime
+  first; the budget-limited selective mode keeps its host loop until the selection +
+  tie-break replication is ported to device and re-verified against the quant md5 gates.
+- [ ] **Graphs L2 — per-parent dependency chaining.** A child pair only depends on *its
+  parent's* partition + histogram, not on its level. Launch each pair's chain from the
+  device (fire-and-forget device graph launch) when its parent finishes — a
+  self-scheduling tree that removes phase-boundary tail effects on unbalanced trees.
+  Needs: device-atomic budget reservation + a global final-level top-K join, per-node
+  (not per-level) task/scratch regions, and an order-independent split log (the host
+  rebuild already renumbers canonically, absorbing completion-order nondeterminism).
+  Est. +10-20% over L1 on unbalanced small trees.
+- [ ] **Static planner (auto-tuner tier 0).** At Init the dataset shape (rows,
+  features, actual bins/feature) and config (num_leaves, max_depth, num_class,
+  iterations) determine: expected level geometry (leaf sizes ~ rows/2^level) -> grid
+  configs, small-leaf threshold, speculative bounds; histogram partition packing and
+  quant bit thresholds from the real bin histogram; pipeline selection; and a compact
+  per-tree histogram layout for feature_fraction runs (static version of the 161fe88b
+  dead-entry mask). Precedent: CPU LightGBM's row/col-wise chooser, EFB, multi-val bin
+  packing. Decides only provable shape functions; supplies priors for the ambiguous
+  constants below (GPU cost models are brittle: the construct-floor cap gained
+  year/higgs 35% and regressed covtype 45% -- only measurement caught it).
+- [ ] **Runtime auto-tuner ("JIT optimizer") tier 1 — online policy tuning.** Boosting
+  runs thousands of near-identical trees: measure per-tree wall time (CUDA events) +
+  feedback stats (churn, level widths, imbalance) and bandit-tune the existing
+  dataset-dependent knobs: batched-construct grid sizing (EXABOOST_BATCH_CONSTRUCT_FLOOR
+  -- covtype is latency-bound, year/higgs merge-bound), small-leaf construct threshold
+  (EXABOOST_SMALL_LEAF_ROWS), hist pipeline count, and the selective-growth speculation
+  policy (e.g. gain-margin gating for unbalanced trees). Speculation is model-invariant
+  by monotonicity, and quant-mode integer histograms keep md5 locks valid under any
+  schedule retuning -- the tuner cannot break exactness gates. Hysteresis vs noise;
+  decision logging.
+- [ ] **Runtime auto-tuner tier 2 -- NVRTC shape-specialized kernels.** JIT-compile
+  construct/find kernels at Dataset construction with columns / per-feature bin counts
+  baked in (precedent: LightGBM's OpenCL backend JIT-compiled with #defined bin counts).
+  Star case: numerai's ~5.5-bin features (5 quintiles + a missing-marker bin on ~half
+  of them) waste >90% of the fixed 12288-entry shared histogram; a specialized kernel
+  packs ~10x more features per partition -> fewer
+  partitions, less shared->global merge traffic. One-time ~0.5s compile amortized over
+  thousands of trees; needs AOT fallback.
+- [ ] **Runtime auto-tuner tier 3 -- persisted tuning cache**: store best-found configs
+  keyed by dataset-shape signature (FFTW-wisdom style) so retrains skip exploration.
+- [ ] **Selective-growth churn reduction.** covtype 64/12 applies 2.09x the final split
+  count (52% displaced-then-pruned). Smarter speculation — e.g. only apply candidates
+  with a selection margin / hysteresis — to cut wasted search+apply.
+- [x] **Wide-shape batched search** (161fe88b): multi-block batched find/sync for
+  num_tasks > 1024 (bit-identical reduction), compact-view batched construct,
+  bin-used-mask skipping of dead histogram entries, learner gather from the compact
+  matrix. numerai 4096/13: 39.4 -> 27.2s; example shape per-tree 41.6 -> 34.8ms.
+- [x] **Numerai first-train setup** (22331b18): Booster create 13.9s -> 2.3s. Dense CUDA
+  row data now builds directly from column bins (tiled transpose, no zero-fills) and the
+  host multi-val bin is skipped for dense non-multi-val datasets (-14.6 GB peak RSS).
+  Remaining: ~0.8s pageable 15 GB H2D (pinned staging ring / transpose into pinned) and
+  ~1.2s host transpose (device-side transpose candidate); sparse row-wise CUDA datasets
+  still build the host multi-val bin. Kill switch EXABOOST_FAST_ROWDATA=0.
+- [x] **Native small-int ingestion (int8/int16)** (9c0f5ffa): int8/int16 numpy
+  matrices (row- or col-major) pass zero-copy through the C API and are binned by
+  per-column LUTs via Bin::PushBlock; bins identical to the float path by construction
+  (md5-verified CUDA quant + CPU deterministic). numerai fed int8: construct 38.9 ->
+  15.8s, peak RSS 86.4 -> 43.9 GB. Measured separately from the cross-library matrix
+  (which stays float32-fed for fairness). Follow-ups: EFB FindGroups is now the
+  dominant construct cost (~9s of 15.8s on 2748 features); pandas int frames still
+  convert to float32 (only raw numpy passes through); CSR/CSC + streaming push and
+  uint8 still use the double path.
+- [x] **4-bit packed CUDA row/compact data for <=16-bin datasets** (354ceef3): row
+  matrix AND per-tree compact matrix packed two cells/byte; all construct/fill/gather
+  kernels unpack via IS_4BIT variants. numerai: 2000-tree train 87.6 -> 66.3s (-24%),
+  device memory 19.3 -> 11.5 GB, Booster create 2.25 -> 1.44s, gather source at 1.47
+  TB/s. Kill switch EXABOOST_ROWDATA_4BIT=0; verify env checks unpacked equality.
+  Both follow-ups landed in 45eea4c6 (numerai per-tree 34.4 -> 30.7 ms, -10.6%;
+  2000-tree ~60s): (a) float2 grad/hess interleave (EXABOOST_GH_INTERLEAVE,
+  compile-time-template dispatch -- a runtime branch cost ~1 ms in both modes),
+  modest (-0.5..1 ms) since grad/hess was the smaller half of scattered traffic;
+  (b) split kernels read the packed compact matrix directly
+  (EXABOOST_SPLIT_PACKED_READ), dropping the per-tree 1.5 GB row-to-col gather
+  (-4.2 ms/tree); classic per-split consumers lazily materialize the old view once
+  per tree. Next on this path: CUDAFillCompactData4BitKernel is now the #2 kernel
+  (5.3 ms/tree); the construct kernel (22.7 ms, ~74% of tree time) remains
+  latency-bound on the packed-bin row gathers themselves. Measurement hygiene:
+  desktop wall clocks drift up to ~15% between sessions -- trust same-session A/B
+  per-tree medians (or lock clocks) for future numerai perf work.
+- [ ] **GPU-native dataset construction, default for CUDA (CuPy/cuDF ingestion,
+  QuantileDMatrix parity and beyond).** Whenever device_type=cuda and the input is
+  dense: sample -> host GreedyFindBin (identical boundaries, md5-comparable), then
+  stream row chunks through a pinned staging ring and bin ON DEVICE (LUT for small
+  ints, boundary search for floats) directly into the packed 4-bit partitioned
+  structures -- raw data never fully resident (numerai f32 is 60 GB vs 32 GB VRAM).
+  Accept __cuda_array_interface__ (CuPy; cuDF rides along) to skip the upload
+  entirely. Est. numerai construct: ~1s (CuPy int8), ~2-3s (numpy int8), ~5-6s
+  (numpy f32) vs 15.8/37.4s today; frees ~40 GB host RSS. Host bins become lazy
+  (linear_trees, save_binary, refit fall back). 22331b18/354ceef3 already removed
+  the host multi-val-bin dependency, so CUDARowData/compact are the only consumers.
+- [ ] **EFB density precheck.** FindGroups spends ~9s on numerai's 2748 dense
+  features and (almost certainly) bundles nothing -- bundling exploits sparse
+  mutual exclusivity. Detect dense/no-op cases cheaply and skip; the GPU-native
+  constructor skips it by design.
+- [ ] **Latency-bound construct on tiny-bin wide data**: post-161fe88b numerai construct
+  is scattered-read latency-bound (19ms/tree) -- candidate for NVRTC shape
+  specialization (auto-tuner tier 2) or layout changes.
+- [ ] **Quant one-sync parity.** The quantized path still uses the two-sync level flow;
+  extend the one-sync speculative pipeline to it. Also investigate the per-tree gradient
+  discretization cost on many-tree/small-tree configs (numerai-quant is slower than
+  non-quant today).
+- [ ] **FP32 gain math in the find kernels** (priority up: Felix accepts
+  nondeterminism for speed). find ~= 136 us/tree, FP64-bound and leaf-size-independent;
+  consumer Blackwell runs FP64 at 1/64 the FP32 rate, so this is the one genuinely
+  FLOPs-bound kernel. FP32/mixed gains behind a flag; verify quality via the benchmark
+  harness, not md5.
+- [ ] **FP32-atomic histogram mode** (priority up, same rationale). Non-quant entries
+  are FP64 pairs (16 B/bin); an fp32 pair (8 B/bin) matches quant bandwidth with ZERO
+  per-tree discretization overhead -- candidate fastest mode for many-tree configs
+  (numerai-quant regressed to 155s vs 86s non-quant partly on discretizer cost).
+  Nondeterministic and cancellation-bias risk: judge by measured quality per dataset.
+
+## Correctness / determinism
+
+- [ ] **Deterministic non-quant CUDA mode** (deprioritized: determinism is a
+  verification tool here, not a production requirement -- quant mode already provides
+  it for md5 gates). Fixed-point integer histogram accumulation (XGBoost-style) would
+  eliminate float-atomic run-to-run jitter (measured +-2.6pp on covtype multiclass).
+  Related: `deterministic=true` silently no-ops on CUDA — make it work or warn; it also
+  doesn't pin the timing-based col/row-wise auto-choice on CPU (bimodal md5s; pin
+  force_col_wise in gates).
+
+## Upstream (lightgbm-org/LightGBM) bugs found (documented here for reference;
+## we do not contribute upstream)
+
+- Quantized CUDA int32 histogram-index overflow on wide data (fixed here in
+  6f8402f5; upstream segfaults at scale and silently corrupts below it).
+- Quantized multiclass per-tree random-offset buffer overrun (fixed here in
+  1b28ba03).
+- CUDA-vs-CPU growth parity: upstream CUDA over-grows trees ~3.5x vs its own CPU at
+  identical params (854 vs 237 leaves/tree on covtype; lambda_l2 sweep shows it is
+  systematic).
+- Latent race in the classic loop: child leaf-splits structs point into per-split
+  scratch that the next split overwrites; masked only by per-split syncs (fixed here
+  via point_structs_at_main + copy-event ordering).
+
+## Benchmark
+
+- [ ] Airline (115M rows): kt.ijs.si down; retry, find a mirror, or substitute the
+  benchm-ml 10M-row variant.
+- [ ] Optional: xgboost-lossguide as a seventh benchmark config (leaf-wise
+  apples-to-apples column).
+- [ ] Harness: host-memory guard for runs near RAM limits (xgboost/numerai peaked at
+  102 GB and OOM-killed the host session; its curve cell is skipped).

include/LightGBM/bin.h

@@ -297,6 +297,25 @@ class Bin {
   */
   virtual void Push(int tid, data_size_t idx, uint32_t value) = 0;
 
+  /*! \brief Sentinel for pre-encoded bins that must not be pushed (most frequent bin) */
+  static constexpr uint32_t kSkipBin = std::numeric_limits<uint32_t>::max();
+
+  /*!
+  * \brief Push a block of pre-encoded bins for consecutive records, skipping ``kSkipBin`` entries.
+  *        Values must already be encoded exactly as FeatureGroup::PushData would push them.
+  * \param tid Thread id
+  * \param start_idx Index of the first record
+  * \param count Number of records
+  * \param bins Encoded bin values of the records
+  */
+  virtual void PushBlock(int tid, data_size_t start_idx, data_size_t count, const uint32_t* bins) {
+    for (data_size_t i = 0; i < count; ++i) {
+      if (bins[i] != kSkipBin) {
+        Push(tid, start_idx + i, bins[i]);
+      }
+    }
+  }
+
   virtual void CopySubrow(const Bin* full_bin, const data_size_t* used_indices, data_size_t num_used_indices) = 0;
   /*!
   * \brief Get bin iterator of this bin for specific feature

include/LightGBM/c_api.h

@@ -38,6 +38,7 @@ typedef void* ByteBufferHandle; /*!< \brief Handle of ByteBuffer. */
 #define C_API_DTYPE_INT32   (2)  /*!< \brief int32. */
 #define C_API_DTYPE_INT64   (3)  /*!< \brief int64. */
 #define C_API_DTYPE_INT8    (4)  /*!< \brief int8. */
+#define C_API_DTYPE_INT16   (5)  /*!< \brief int16. */
 
 #define C_API_PREDICT_NORMAL     (0)  /*!< \brief Normal prediction, with transform (if needed). */
 #define C_API_PREDICT_RAW_SCORE  (1)  /*!< \brief Predict raw score. */
@@ -407,7 +408,7 @@ LIGHTGBM_C_EXPORT int LGBM_DatasetCreateFromCSC(const void* col_ptr,
 /*!
  * \brief Create dataset from dense matrix.
  * \param data Pointer to the data space
- * \param data_type Type of ``data`` pointer, can be ``C_API_DTYPE_FLOAT32`` or ``C_API_DTYPE_FLOAT64``
+ * \param data_type Type of ``data`` pointer, can be ``C_API_DTYPE_FLOAT32``, ``C_API_DTYPE_FLOAT64``, ``C_API_DTYPE_INT8`` or ``C_API_DTYPE_INT16``
  * \param nrow Number of rows
  * \param ncol Number of columns
  * \param is_row_major 1 for row-major, 0 for column-major
@@ -429,7 +430,7 @@ LIGHTGBM_C_EXPORT int LGBM_DatasetCreateFromMat(const void* data,
  * \brief Create dataset from array of dense matrices.
  * \param nmat Number of dense matrices
  * \param data Pointer to the data space
- * \param data_type Type of ``data`` pointer, can be ``C_API_DTYPE_FLOAT32`` or ``C_API_DTYPE_FLOAT64``
+ * \param data_type Type of ``data`` pointer, can be ``C_API_DTYPE_FLOAT32``, ``C_API_DTYPE_FLOAT64``, ``C_API_DTYPE_INT8`` or ``C_API_DTYPE_INT16``
  * \param nrow Number of rows
  * \param ncol Number of columns
  * \param is_row_major Pointer to the data layouts. 1 for row-major, 0 for column-major

include/LightGBM/cuda/cuda_column_data.hpp

@@ -113,6 +113,35 @@ class CUDAColumnData {
   // Needed when feature_fraction = 1.0 / no compaction (rare path).
   void RestoreOriginalColumnView();
 
+  // ===== Per-tree packed compact column view (4-bit) =====
+  // Register the histogram constructor's 4-bit packed compact matrix as this
+  // tree's column source: column c's bin for row r is
+  //   (packed_column_data(c)[r * packed_column_stride(c)] >> packed_column_shift(c)) & 0xf.
+  // Only the batched apply path (CUDAHybridApplyDescriptor, bit_type 4) reads
+  // it; callers needing a plain per-column buffer must first re-register one
+  // via SetCompactColumnView (see the tree learner's lazy classic fallback).
+  // Also nulls compact_column_host_view_ so a stale GetColumnData pointer can
+  // never be consumed silently.
+  void SetCompactPackedColumnView(const std::vector<int>& column_to_compact_slot,
+                                  const uint8_t* packed_buf,
+                                  const std::vector<size_t>& slot_base_byte,
+                                  const std::vector<int>& slot_row_stride,
+                                  const std::vector<uint8_t>& slot_shift);
+
+  bool packed_column_view_active() const { return packed_column_view_active_; }
+
+  const uint8_t* packed_column_data(const int column_index) const {
+    return packed_column_ptr_[column_index];
+  }
+
+  int packed_column_stride(const int column_index) const {
+    return packed_column_stride_[column_index];
+  }
+
+  uint8_t packed_column_shift(const int column_index) const {
+    return packed_column_shift_[column_index];
+  }
+
   // Skip per-column allocation in Init? Used when caller will provide compact view.
   bool init_skipped_per_column_alloc_ = false;
 
@@ -160,6 +189,13 @@ class CUDAColumnData {
   // or nullptr for non-sampled columns. Populated by SetCompactColumnView so that
   // GetColumnData has a valid host-readable device pointer for the split column.
   std::vector<uint8_t*> compact_column_host_view_;
+  // Per-tree packed compact view (see SetCompactPackedColumnView): host-side
+  // per-column device base pointer / per-row byte stride / nibble shift. Only
+  // consumed host-side when building batched apply descriptors.
+  bool packed_column_view_active_ = false;
+  std::vector<const uint8_t*> packed_column_ptr_;
+  std::vector<int> packed_column_stride_;
+  std::vector<uint8_t> packed_column_shift_;
 
   CUDAVector<uint8_t> cuda_column_bit_type_;
   CUDAVector<uint32_t> cuda_feature_min_bin_;