[cuda/io] fp32 gain + fp32-atomic histogram modes; uint8/uint16/fp16 + pandas small-int ingestion#34
Open
BelixRogner wants to merge 21 commits into
Open
[cuda/io] fp32 gain + fp32-atomic histogram modes; uint8/uint16/fp16 + pandas small-int ingestion#34BelixRogner wants to merge 21 commits into
BelixRogner wants to merge 21 commits into
Conversation
…_HIST) modes Two opt-in reduced-precision modes for the non-quantized-dominated hot paths on consumer GPUs (fp64 at 1/64 fp32 rate on sm_120): - EXABOOST_FP32_GAIN=1: per-bin split-gain arithmetic in the find kernels runs in fp32 (leaf/quantized-level sums stay double, converted once per task; the once-per-task winner output block stays double). GAIN_T is threaded through the per-leaf, discretized and batched-level kernels; SplitGainMath/CUDALeafSplits helpers gain a type parameter (double-instantiation bit-identical). Categorical and large-bin global-memory kernels stay fp64. - EXABOOST_FP32_HIST=1: non-quantized global histograms are stored as float pairs (8B/bin) in the leading half of each unchanged hist_t leaf slot, halving construct-merge/fix/subtract/find histogram traffic. Shared-memory accumulation was already float (!gpu_use_dp). Auto-falls-back to fp64 for quantized training, gpu_use_dp, sparse row data, large-bin partitions, categorical features, forced splits and NCCL; engagement is logged at debug verbosity. Both flags DEFAULT OFF: quality-gated per dataset (3 repeats each config, 500 rounds, bench params) they are within or above the flag-off noise band everywhere except a ~0.1pp accuracy dip on covtype shallow (inside the cross-seed flag-off band 0.92116-0.92299 over seeds 42-45, i.e. seed-noise scale, but outside the same-seed repeat noise), and numerai (feature_fraction 0.1) is quality-identical but perf-flat. quality (mean over 3, off -> both): fraud shallow AUC .97178 -> .97167 fraud deep AUC .97044 -> .97035 higgs shallow AUC .83075 -> .83084 higgs deep AUC .84843 -> .84859 epsilon deep AUC .94352 -> .94380 year RMSE 8.9718 -> 8.9717 covtype shallow acc .92299 -> .92189 numerai300 RMSE .223662 (equal) per-tree median (off -> gain / hist / both): epsilon deep 92.1 -> 72.7 / 80.9 / 58.9 ms (-36% both) year shallow 1.18 -> 1.10 / 1.02 / 0.96 ms (-18%) covtype shal 5.35 -> 4.62 / 4.57 / 4.50 ms (-16%) fraud deep 0.95 -> 0.86 / 0.87 / 0.82 ms (-14%) higgs deep 19.0 -> 18.8 / 17.2 / 16.7 ms (-12%) fraud shallow 0.62 -> 0.60 / 0.58 / 0.56 ms (-9%) higgs shallow 6.94 -> 6.84 / 6.65 / 6.73 ms (-3%) numerai 300 30.3 -> 30.0 / 30.1 / 30.7 ms (flat) Flags-off bit-identity: covtype 1023/10 quant hybrid/classic md5s 5f4e7bdfff1e / fcb9f6c2ab87 and numerai int8 quant slice 763c75c0d9cb unchanged (non-quantized CUDA training is run-to-run nondeterministic, so its flag-off equivalence is by construction: the double instantiations are line-identical transformations). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
Extend the int8/int16 LUT ingestion (9c0f5ff) to unsigned and half: - C_API_DTYPE_UINT8/UINT16: same seams as int8/int16 (basic.py dtype dispatch, CreateFromMats sample+push, RowFunctionFromDenseMatrix for predict); unsigned LUTs index at the raw value (no +half-range offset). - C_API_DTYPE_FLOAT16: np.float16 matrices pass through as their raw IEEE 754 half bit patterns and bin via a 65536-entry LUT built by Common::HalfBitsToFloat + BinMapper::ValueToBin, so every bit pattern (NaNs -> missing, -0 like 0, subnormals) maps exactly as the float value would; predict converts per value through the same helper. - _data_from_pandas: frames whose columns are all plain numpy-backed small ints (np.result_type in {int8,int16,uint8,uint16}) skip the astype(float) copy and feed the native array; nullable/categorical/ object/float columns keep the old float path (NA needs NaN). - refresh stale _c_float_array dtype tests (predate int16 landing). md5 identity gates (f32-fed vs new-dtype-fed, 20 trees, quant CUDA + deterministic CPU; models identical in every case): uint8 synth 200kx80 C/F: cpu 8b33611219d4, cuda 67f9987da540 uint16 synth 200kx80 C/F: cpu d8d3f2f929d2, cuda 694bfb3784fd uint16 abs(covtype): cpu 5e3f06d6cf15, cuda 16de6c607a42 pandas passthrough (df vs np-int vs np-f32): int8/int16 cpu 610b2e8e1975 cuda 7f2b0842a2cc; uint8/uint16 cpu 766601b843c9 cuda f75aab01a61f; mixed int8+int16->int16 cpu ac82b5caf5f3 cuda c1f8a38efa1e; float-col old path cpu 509c02cc0b56 cuda 31c5d378ec3c; nullable Int8+pd.NA vs f32+NaN cpu 3d6655f88b3d cuda ea7c2b457183 fp16 synth (halves, exact ints, NaN col) C/F: cpu 8c6892eb0665, cuda 64e0542633a2; exhaustive all-65536 bit patterns single feature max_bin=65535 min_data_in_bin=1: 147d8a82f7ee linear_tree raw-data path: fp16 266c793708a8, uint16 736cc35d9266 existing int8/int16 gates unchanged: synth cpu b600e0ab26bb, cuda/fortran 9a815e468afb; covtype cpu f9f7e62eed36, cuda ed39d42901ac; numerai cuda 763c75c0d9cb predict parity: int8/int16/uint8/uint16/fp16/pandas feeds bit-equal to float predictions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…plint) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ # Conflicts: # src/io/dataset.cpp
…dmap fp32/ingestion status Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…uidance); ingestion completion recorded Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dataset::FastFeatureBundling burned ~7.8s on numerai (2748 dense features) in FindGroups conflict counting (GetConflictCount alone was 38% of construct in py-spy) and then bundled nothing. Add a precheck over the already-collected sample: build the exact fixed per-feature non-default index sets (FixSampleIndices semantics) as bitmaps in parallel, and for every feature pair passing the group-size gate compute the exact conflict via word-wise AND+popcount. If no pair could merge in either processing order, FindGroups provably produces one feature per group; emit that structure directly (identical deterministic shuffle) and skip both group searches. Conservative bail-outs: filtered features present, is_sparse second-round regroup possible (any dense rate < 0.4), or bitmaps > 2GB. Measured (RTX 5090, 32 threads, numerai 5.43M x 2748): construct int8: 15.19s -> 7.48s construct f32: 37.51s -> 29.70s Gates: EXABOOST_EFB_PRECHECK_VERIFY=1 (runs both paths, compares group structure) fired+identical on numerai i8/f32, higgs, epsilon, fraud, synth-cuda; correctly did not fire on covtype (bundles) and CPU sparse synth (multi-val second round). Model md5 unchanged: covtype 1023/10 quant GROWTH=1 5f4e7bdfff1e / GROWTH=0 fcb9f6c2ab87, numerai int8 quant slice 763c75c0d9cb, fraud/higgs quant A/B on==off, synth CPU f32==int8. Kill switch: EXABOOST_EFB_PRECHECK=0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
Replace the host per-value binning + packing push loops for dense numpy inputs when device_type=cuda: stream row chunks through a pinned staging ring (the raw matrix is never fully device-resident), bin on device (per-column LUT for int8/16, uint8/16 and fp16 bit patterns - the host tables upload once; exact BinMapper::ValueToBin double binary search for float/double incl. NaN/missing handling), pack the per-group column bins (4-bit where the groups use it) on device, and copy them back into the normal host DenseBin storage so feature groups, save_binary, the fast CUDA row-data build and the CPU predictor all work unchanged. H2D+kernel and bulk D2H run on separate streams; the host gathers chunk k while the GPU processes k-1 and scatters k-2 with non-temporal-store memcpy. EFB-bundled multi-feature groups are handled exactly (ascending-column last-non-skip-wins walk per row); multi-val or sparse bins, linear-tree raw data and categorical features on the float path fall back to the host loops. 4-bit bins drop their odd-nibble merge buffer via the new Bin::SetLoadedFromRawData, skipping the FinishLoad merge. Also two md5-safe host-side fixes the profile surfaced: FeatureGroup creation is parallelized (the sequential zero-fill + page-locking of the full bin storage was ~3s on numerai), and float32/float64 sampling now uses the same column-parallel sampler as the small-int types with a row-gather pass (the sequential per-row lambda loop was ~4s). Measured (RTX 5090, numerai 5.43M x 2748, fresh process): construct int8: 7.79s -> 4.88s (GPU binner itself 1.87s) construct f32: 29.88s -> 7.76s (GPU binner itself 4.35s) higgs f32 1.12s -> 0.64s, epsilon f32 5.24s -> 3.12s (numbers vs stage-0; pre-stage-0 baselines were 15.8s / 37.4s) Gates: EXABOOST_GPU_CONSTRUCT_VERIFY=1 builds both paths and memcmps every group's bin bytes - byte-identical on numerai i8/f32 (full), higgs, epsilon, fraud, covtype (bundled groups) and synthetic i8/u8/i16/u16/f16/f32/f64 in C and Fortran order. Model md5 unchanged: covtype 1023/10 quant GROWTH=1 5f4e7bdfff1e / GROWTH=0 fcb9f6c2ab87; numerai int8 quant slice 763c75c0d9cb with GPU construct on and off, also under bagging 0.8/1 (2915950ca714) and feature_fraction 0.3 (09427318aa51); synth f32==int8 on cpu/cuda/fortran. numerai 100-tree RMSE 0.22367 unchanged; scipy sparse CUDA train unaffected. Kill switch: EXABOOST_GPU_CONSTRUCT=0 reverts to the host push loops. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
lgb.Dataset(...) now accepts any 2-D C- or Fortran-contiguous object exposing __cuda_array_interface__ (CuPy arrays; for cuDF pass .values) for float32/64/16 and int8/16, uint8/16. New C API entry LGBM_DatasetCreateFromMatDevice takes the device pointer: sampling gathers only the sampled rows to the host with a device gather kernel (same deterministic row ids as the host path, fed through the regular samplers with identity indices, so bin boundaries are identical by construction), and the stage-1 device binner then reads the device matrix directly (no staging ring). Datasets the binner cannot handle (and EXABOOST_GPU_CONSTRUCT=0) fall back to streaming row chunks to the host and running the normal push loops, so the kill switch and EXABOOST_GPU_CONSTRUCT_VERIFY=1 byte-comparison work for device inputs too. A device-wide sync on entry covers producer-stream semantics. Also md5-safe sampler tuning that the cupy profile surfaced (helps the numpy paths as well): SampleDenseSmallInt/SampleDenseFloat now count and reserve the per-column sample vectors before filling them - the geometric reallocation of the 6-7GB of appends dominated sampling. Measured (RTX 5090, numerai 5.43M x 2748, fresh process, real cupy wheel cupy-cuda12x 13.6.0): full-shape cupy int8 construct 3.4-3.8s (raw 15GB already device-resident; below the numpy 4.5s but above the aspirational 2s - remaining time is host GreedyFindBin ~0.9s, pinned FeatureGroup allocation ~0.5s, sample gather + samplers ~0.7s, binner ~1.0s). numpy constructs after the sampler tuning: int8 4.5s, f32 7.2s. Gates: model md5 identical numpy vs cupy on 1.5M-row numerai slices for int8 and float32 (both = the standing 763c75c0d9cb lock); synthetic f32 md5 identical across numpy / cupy C-order / cupy F-order / cupy with EXABOOST_GPU_CONSTRUCT=0 (chunked host fallback); verify env byte-identical for cupy i8/f32 numerai slices; full stage-0/1 gate battery re-run green on the final build (covtype hybrid locks, numerai int8 quant/bagging/ff A/B, synth cpu, all dtype/layout byte verifies, numerai 100-tree RMSE 0.22367). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
… construct 37.5->7.2s f32, 3.4s cupy) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tructure Master loosened the rejection-message regex and split the accept tests (int8, int16). Mirror that structure verbatim so the merge is clean, and keep a branch-only accepts_small_dtypes test for what this branch adds on top: uint8/uint16 zero-copy and fp16 (LUT). Reject list drops uint8 (accepted here) and keeps uint32/uint64 coverage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…ion) When nvforest/treelite (cuml-cu12 >= 26.06) are importable, the model was trained with device_type=cuda, and the input is a 2-D numpy/CuPy array, Booster.predict() converts the model in memory (model string -> treelite -> nvforest, no temp files) and predicts on the GPU. Anything FIL cannot serve (pred_leaf/pred_contrib, extra predict kwargs, non-array inputs, objectives outside identity/sigmoid/softmax postprocessing, conversion or GPU errors) falls back silently to the regular predictor. raw_score is served by rebuilding the treelite model with an identity postprocessor. FIL models are cached per (start_iteration, num_iteration, raw_score, precision) and invalidated on update/__boost/rollback_one_iter/ model_from_string/shuffle_models/set_leaf_output; the cache is excluded from pickling. Opt-outs: predict(use_fil=False) and EXABOOST_FIL=0. Residency is preserved (numpy in -> numpy out, CuPy in -> CuPy out) and the CuPy pool is drained after host-input calls so the staged input does not starve non-pool CUDA allocations. Default precision is fp32 (EXABOOST_FIL_PRECISION=single|double|native): fp64 doubles device memory -- 28.3 GB for the 1.29M x 2748 numerai slice, OOM when a device copy of the input already exists -- and is ~5x slower on wide host inputs. Parity vs CPU predictor (fp32 default; max rel err, denominator |ref|>1e-2): fraud binary prob 1.2e-05 raw 1.6e-05 covtype multiclass prob 3.2e-06 raw 5.2e-05 year regression 3.5e-07 categorical binary prob 8.2e-07 int8-fed regression 2.1e-06 numerai reg (ff=.1) 5.2e-07 Caveat: ~0.01% of rows (69/500K on higgs) route to a different leaf where a feature falls inside a split threshold's fp32 rounding gap (higgs AUC unchanged to 7 decimals); EXABOOST_FIL_PRECISION=double is exact (~1e-13). Predict wall time (3 reps, this host: RTX 5090 / 32-thread CPU): numerai 1.29M x 2748: CPU 0.90s | FIL host-in 1.17s | FIL cupy-in 0.046s higgs 500K x 28: CPU 0.37s | FIL host-in 0.009s | FIL cupy-in 0.004s Host-input FIL loses only on very wide inputs with small models (PCIe transfer bound); device-resident inputs are 19-100x faster than CPU. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…d already device-side Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…prefix Run the ENTIRE depth-limited exact prefix of a tree as ONE host graph launch + ONE readback: a CUDA conditional WHILE node (CUDA >= 12.4) repeats the per-level pipeline (batched tree split + partition apply of level L, construct/fix/subtract + find/sync of level L+1), and a device-side level-controller kernel at the start of each iteration replays the host loop's inter-level decisions -- CollectSplittableLeaves (leaf-ascending is_valid scan), ArbitrateLevelBudget (incl. the final partial level's stable gain sort with the exact host tie-break), the apply/pair/tree-split descriptor building (static per-feature metadata tables + per-tree column-source table), terminal-region gap detection (bitonic region sort + parallel gap scan) -- and resizes the body kernels through device-updatable graph node handles (cudaGraphKernelNodeSetGridDim / SetEnabled, one thread per node, with an enabled-state cache), then sets the WHILE condition. No host round trip separates the levels; the host reconstructs all bookkeeping from a journal + the cumulative deferred split-info slab + one final best-split readback, in exactly the host loop's order. Key mechanics: - WHILE body built by cudaStreamBeginCaptureToGraph over the existing launch functions (placeholder grids, same streams/events wiring); nodes collected via cudaStreamGetCaptureInfo and marked device-updatable (the driver returns the devNode handle through the SetAttribute value struct, which must be zeroed per call). - Level-varying state (main/out index buffer swap, split-info slab base) reaches the four buffer-swapping partition kernels through an optional loop-state parameter (nullptr on every host-launched path, bit-for-bit the previous behavior); gap descriptors live at a fixed buffer offset so the gap kernel's frozen descriptor base stays valid. - One instantiated graph per per-tree pointer/shape key, LRU-cached (64): multiclass cycles one gradient pointer per class, the compact column view double-buffers its data and varies its construct BLOCK shape per tree (block dims are not device-updatable) -- each shape gets its own instance captured with the exact host block dims, so results stay bit-identical to the host loop (numerai 100-tree RMSE 0.22366795 and 2000-tree predictions reproduce exactly, both paths). - Scope: depth-limited one-sync prefix only (non-quantized; quantized training keeps the host loop's classic readback ordering), gated on num_leaves <= 2047 / max_depth < 32 / no debug envs; runtime driver check (>= 12.4) and graceful fallback to the host loop on any capture or instantiation failure. EXABOOST_GRAPH_LEVEL_LOOP=0 kill-switch restores the host level loop bit-for-bit. - Budget-limited configs keep the selective grow-then-prune host flow untouched (covtype 64/12 quant md5 01a30b7ff02f unchanged, no graph engagement). Verification (RTX 5090, CUDA 12.9, driver 595.71): - md5 locks (graph env ON and OFF): covtype 1023/10 quant GROWTH=1 5f4e7bdfff1e / GROWTH=0 fcb9f6c2ab87; numerai int8 quant slice 763c75c0d9cb; fraud 63/6 quant 1037798704d1 and year 63/6 quant 7d07dac08d19 identical graph on/off. - numerai 32/5 ff0.1 non-quant: 100-tree RMSE 0.22366795 EXACT both paths; 2000-tree run bit-identical predictions, RMSE 0.22367504. - fraud/covtype/year non-quant A/B: quality + mean_leaves identical. - compute-sanitizer racecheck + memcheck clean with the graph active. - pre-commit --all-files clean. Perf (same-session env A/B, median of 3, nothing else running): - fraud 63/6 1000 trees: 0.63 vs 0.66 s (-4.5% wall, 630 vs 657 us/tree) - fraud 1023/10 1000 trees: 0.93 vs 0.98 s (-5.4%) - covtype 63/6 500 rounds x7: 2.42 vs 2.51 s (-3.5%) - year 63/6 1000 trees: 1.22 vs 1.25 s (-2.6%) - numerai 32/5 ff0.1 300 trees: 10.25 vs 10.22 s (parity; construct-bound) - numerai 2000-tree full train_s: 59.39 vs 59.42 s (parity) - nsys fraud 500 trees: cudaLaunchKernel 35,323 -> 7,022 calls (70.6 -> 14.0 per tree, launch API ~132 -> ~27 us/tree); sync memcpys 21.2 -> 11.1 per tree. numerai: launches 12,926 -> 3,163 per 200 trees, 20 instantiates (shape warmup), 1 graph launch/tree. - device level-controller kernel: 7.2 us/level after warp-shuffle scans + per-thread node updates (was 16.5 us serialized). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
… identified) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
Author
Overnight additions (all gated, pushed to this branch)
Validation eval was verified already device-side (0.22ms/iter at numerai scale) — no work needed. Refreshed official scoreboard at this tip (RTX 5090, medians of 3): 11 wins, 0 ties, 0 losses vs XGBoost — numerai train 59.5s vs 235.9s (3.97×), and construct now also beats XGBoost on the big/wide sets (numerai 7.2 vs 35.5s). All md5 locks unchanged; compute-sanitizer clean on the graph loop. 🤖 Generated with Claude Code |
…ILE node) The depth-limited gate bounds the prefix's level count (max_depth <= 11: 2^max_depth <= num_leaves + 1 <= 2048), so the conditional-WHILE body relaunch was pure overhead. The graph is now max_depth straight-line level bodies plus one epilogue controller, captured back to back with the same stream/event wiring the WHILE body used. Each body owns its device-updatable node slice (stride kHybridGraphMaxNodes); the controller resizes only ITS level's nodes. Early tree completion sets a done flag in the loop state: the remaining bodies' controllers exit immediately and keep their nodes disabled through the per-node enabled cache, so steady-state same-shape trees issue no redundant device graph updates. Also fixes a latent race in the L1 controller that racecheck could not see through the conditional body graph: BlockMax's first s_warp write could race the preceding BlockExclusiveScan's tail read of s_warp under independent thread scheduling. BlockMax now syncs on entry (racecheck-verified: the cuda_hybrid_graph.cu hazards are gone; only the pre-existing upstream ShuffleReduceSum pair remains, which the graph-OFF host path reports too). Gates: covtype 1023/10 quant GROWTH=1/0 md5 5f4e7bdfff1e/fcb9f6c2ab87; numerai int8 quant 763c75c0d9cb; fraud 63/6 + year 63/6 quant graph A/B md5-identical; covtype 63/6 quant bagging 0.8/1 A/B aa6fe97eecb8; numerai 100-tree RMSE 0.22366795 (graph A/B); covtype 64/12 stays on the selective host flow (01a30b7ff02f). Perf (same-session env A/B, median of 3, EXABOOST_GRAPH_LEVEL_LOOP=1 vs 0): fraud 63/6 1000t 0.610 vs 0.670s (-9.0%, was -4.5% at L1); fraud 1023/10 0.900 vs 0.980s (-8.2%); covtype 63/6 500t 2.405 vs 2.510s (-4.2%); year 63/6 1000t 1.210 vs 1.260s (-4.0%); numerai 32/5 ff0.1 300t parity (-0.1%). nsys fraud node trace: controller 16.5 -> 6.9 us avg (115 -> 48.5 us/tree); launch-side API ~30.5 us/tree (graph ON) vs ~162 us/tree host loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…ates Cache each device-updatable node's last-set grid in its slot and skip cudaGraphKernelNodeSetGridDim when unchanged (mirroring the existing enabled-state cache). This only pays off after stage 1's unroll: each level body owns its nodes, and 7 of the ~10 roles' grids depend only on the level's split count, which is stable across same-shape trees -- in steady state a level issues device updates only for the data-dependent extents (partition x = max blocks per split on 2 nodes, construct grid y, gap-copy nodes), i.e. ~3-4 instead of ~10 calls per level. Per-level grid-policy note (stage 2b): static worst-case x-extents for the partition kernels were NOT baked -- the x dims are data-dependent per tree, and padding to the root worst case would add (worst - actual) * level_splits idle blocks per level, which costs more than the ~0.7us update call the cache already skips when the value repeats. The controller stays a separate node (stage 2a not taken): its non-update cost is ~2us; fusing it into the last sync kernel's tail risks the exact host-replay semantics for less than the remaining update cost. Gates: covtype 1023/10 quant GROWTH=1/0 md5 5f4e7bdfff1e/fcb9f6c2ab87; numerai int8 quant 763c75c0d9cb; fraud 63/6 + year 63/6 quant graph A/B md5-identical; covtype 63/6 quant bagging 0.8/1 A/B aa6fe97eecb8; numerai 100-tree RMSE 0.22366795 (graph A/B); covtype 64/12 stays on the selective host flow (01a30b7ff02f). racecheck fraud 63/6: only the pre-existing upstream ShuffleReduceSum hazards (graph-OFF shows the same). Perf (same-session env A/B, median of 3, EXABOOST_GRAPH_LEVEL_LOOP=1 vs 0): fraud 63/6 1000t 0.610 vs 0.660s (-7.6%); fraud 1023/10 0.880 vs 0.970s (-9.3%, stage 1 was -8.2%); covtype 63/6 500t 2.375 vs 2.500s (-5.0%, stage 1 -4.2%); year 63/6 1.200 vs 1.250s (-4.0%); numerai 32/5 ff0.1 parity. nsys fraud node trace: controller kernel 6.85us avg (stage 1 6.93us) -- on fraud the per-tree tree-shape variation defeats the n-keyed cache at deeper levels, so the win concentrates on the wide configs (fraud 1023/10, covtype); no config regressed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…ound Level body B applies at most min(2^B, 1024) splits (each level at most doubles the leaf count from 1), but every controller ran with a fixed 1024-thread block. Block dims are per-node constants of the unrolled graph, so each body's controller now bakes its own bound at capture (64 threads for bodies 0-4 through 2*2^B, capped at 1024): small-level bodies drop from 32 warps to 2 through the ~20 block-wide barriers. The shared-staging guard additionally trips if a body's structural bound is ever exceeded (impossible in the gated regime, as before). Also fixes the three pre-existing cpplint findings in cuda_hybrid_graph.hpp (include order x2, guarded duplicate include). fraud 63/6 nsys (200 trees): controller 6.85 -> 5.97us avg; working 64-thread bodies 6.63us vs 2.0us for the update-free epilogue, i.e. the remaining cost is the serialized device graph updates, not block width. Gates: covtype/numerai/fraud/year/bagging md5 locks unchanged, graph ON/OFF bit-identical, numerai RMSE 0.22366795, selective flow untouched, racecheck clean (0 hazards). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
… slab + best splits) The graph prefix paid three synchronous D2H round trips per tree: the journal readback, FinishSplitBatch's deferred split-info slab, and the tail's SyncAllLeafBestSplitsToHost. Nothing mutates those device buffers between graph completion and the reads (FinishHybridGraphLevels is host-only bookkeeping; the other two are pure reads), so all three copies are now issued async on the graph's stream at host-known upper-bound sizes (num_leaves - 1 splits / num_leaves leaves) right after cudaGraphLaunch, covered by ONE cudaStreamSynchronize. Values are bit-for-bit those of the previous synchronous copies. The full cross-tree overlap (defer the journal walk past the next tree's gradients) stays out: CUDATree::Shrinkage/GBDT consume the host mirror's num_leaves immediately after Train returns, and the leaf-wise tail arbitrates on FinishLevelBookkeeping outputs inside Train, so the walk is on the critical path by contract. fraud 63/6 1000t median: graph ON 0.59s vs OFF 0.67s (-11.9%, was -7.6..-7.7%); fraud 1023/10 -10.9%; covtype 63/6 -8.8%; year 63/6 -5.8%; numerai parity (construct-bound). nsys fraud: sync cudaMemcpy 11 -> 8 per tree, one stream sync per tree carrying the graph wait. Gates: all md5 locks unchanged (covtype 1023/10 GROWTH, numerai int8, fraud/year quant A/B, covtype bagging, selective 64/12), numerai RMSE 0.22366795 ON/OFF, early-stopping valid_sets best_iter parity, racecheck: graph path clean (6 hazards, all the known upstream ShuffleReduceSum pair). Known pre-existing (verified at 2e047b0, before this session): rare intermittent cudaErrorIllegalAddress on multiclass covtype with the graph loop ON (~10-25% of 60-round runs), rate unchanged by this commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…t-cause Pre-existing intermittent cudaErrorIllegalAddress on multiclass with the graph loop active (~7/30 runs at 2e047b0 baseline, machine-state dependent; repro in the session scratchpad; sanitizers cannot attach to device-side graph updates). Keep num_class>1 on the host loop until root-caused. 10/10 multiclass repro runs clean with the guard; covtype quant md5 lock unchanged (5f4e7bdfff1e). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ
…rget training item Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…aints, forced, linear) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #32, for review.
fp32 numerics modes (161dc90) — both default OFF
EXABOOST_FP32_GAIN: fp32/mixed gain math in the find kernels (consumer Blackwell runs fp64 at 1/64 the fp32 rate).EXABOOST_FP32_HIST: fp32-pair (8 B/bin) non-quant CUDA histograms with doubled shared-memory bin capacity per partition; auto-falls back to fp64 for quant/sparse/large-bin/categorical/forced-splits/NCCL/global-memory finder.Ingestion completion (ff77c70)
Also merges master's CI fixes back into the branch (no-OpenMP build fix, swig download hardening, lint).
🤖 Generated with Claude Code
https://claude.ai/code/session_01EYUXNtCAXDd3zYT71m8YSJ