okf: add ML experiment trail sample bundle

okf/bundles/ml_experiment_trail/experiments/baseline_logistic_regression.md

@@ -0,0 +1,39 @@
+---
+type: Experiment
+title: Baseline logistic regression
+description: First end-to-end baseline establishing the validation protocol and a reference score.
+tags:
+- experiment
+- baseline
+- logistic-regression
+timestamp: '2026-01-14T09:00:00+00:00'
+---
+
+A minimal end-to-end run to lock the validation protocol before investing in
+feature engineering. Establishes the reference score every later experiment is
+compared against.
+
+# Setup
+
+| Field | Value |
+| --- | --- |
+| Algorithm | Logistic regression (L2, `C=1.0`) |
+| Features | 11 raw numeric columns, standardized |
+| Validation | 5-fold stratified cross-validation, seed 42 |
+| Metric | ROC AUC |
+
+# Result
+
+| Split | ROC AUC |
+| --- | --- |
+| CV mean | 0.782 |
+| CV std | 0.006 |
+
+# Notes
+
+An early version of this experiment scored a suspiciously high 0.991; that run
+is written up in [target leakage via row id](/lessons/target_leakage_via_row_id.md).
+Superseded as current best by [gradient boosting v2](/experiments/gradient_boosting_v2.md).
+
+# Citations
+- https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression

okf/bundles/ml_experiment_trail/experiments/gradient_boosting_v2.md

@@ -0,0 +1,38 @@
+---
+type: Experiment
+title: Gradient boosting v2
+description: Gradient-boosted trees with engineered features; current best cross-validation score.
+tags:
+- experiment
+- gradient-boosting
+- feature-engineering
+timestamp: '2026-02-11T16:20:00+00:00'
+---
+
+Builds on the baseline by adding ratio and count-encoding features, trained with
+gradient-boosted decision trees. Applies the leakage fix from the baseline
+post-mortem.
+
+# Setup
+
+| Field | Value |
+| --- | --- |
+| Algorithm | Gradient-boosted trees (400 rounds, depth 6, lr 0.03) |
+| Features | 11 raw + 9 engineered (ratios, count encodings) |
+| Validation | 5-fold stratified cross-validation, seed 42 |
+| Metric | ROC AUC |
+
+# Result
+
+| Split | ROC AUC |
+| --- | --- |
+| CV mean | 0.834 |
+| CV std | 0.005 |
+
+# Notes
+
+Explicitly drops `row_id` per [target leakage via row id](/lessons/target_leakage_via_row_id.md).
+The tuned artifact exported from this experiment is [gbdt tuned](/models/gbdt_tuned.md).
+
+# Citations
+- https://xgboost.readthedocs.io/en/stable/

okf/bundles/ml_experiment_trail/experiments/index.md

@@ -0,0 +1,4 @@
+# Experiment
+
+* [Baseline logistic regression](baseline_logistic_regression.md) - First end-to-end baseline establishing the validation protocol and a reference score.
+* [Gradient boosting v2](gradient_boosting_v2.md) - Gradient-boosted trees with engineered features; current best cross-validation score.

okf/bundles/ml_experiment_trail/index.md

@@ -0,0 +1,6 @@
+# Subdirectories
+
+* [experiments](experiments/index.md) - Model-training experiments, each recording its setup, data, and validation result.
+* [models](models/index.md) - Trained models promoted from experiments.
+* [submissions](submissions/index.md) - Scored submissions against the competition leaderboard.
+* [lessons](lessons/index.md) - Durable lessons carried across the experiment trail.

okf/bundles/ml_experiment_trail/lessons/index.md

@@ -0,0 +1,3 @@
+# Lesson
+
+* [Target leakage via row id](target_leakage_via_row_id.md) - A near-perfect early score traced to an ordered row identifier leaking the label.

okf/bundles/ml_experiment_trail/lessons/target_leakage_via_row_id.md

@@ -0,0 +1,32 @@
+---
+type: Lesson
+title: Target leakage via row id
+description: A near-perfect early score traced to an ordered row identifier leaking the label.
+tags:
+- lesson
+- data-leakage
+- validation
+timestamp: '2026-01-15T08:15:00+00:00'
+---
+
+# Symptom
+
+An early version of [baseline logistic regression](/experiments/baseline_logistic_regression.md)
+scored 0.991 ROC AUC in cross-validation — far above anything the available
+features plausibly support.
+
+# Cause
+
+The exported training table was sorted by label before a monotonic `row_id` was
+assigned, so `row_id` was almost perfectly rank-correlated with the target. The
+model learned the identifier, not the signal.
+
+# Fix
+
+Drop `row_id` (and any other index-like column) before training, and shuffle
+rows at export time. Applied in [gradient boosting v2](/experiments/gradient_boosting_v2.md).
+
+# Takeaway
+
+Treat any single feature that produces a near-perfect score as a leak until
+proven otherwise; audit identifier and timestamp columns first.

okf/bundles/ml_experiment_trail/models/gbdt_tuned.md

@@ -0,0 +1,29 @@
+---
+type: Model
+title: GBDT tuned
+description: Tuned gradient-boosted tree model exported from experiment gradient-boosting v2.
+resource: https://example.com/artifacts/gbdt_tuned_2026-02-11.pkl
+tags:
+- model
+- gradient-boosting
+timestamp: '2026-02-11T17:05:00+00:00'
+---
+
+The serialized model artifact selected for submission. Produced by
+[gradient boosting v2](/experiments/gradient_boosting_v2.md).
+
+# Interface
+
+| Property | Value |
+| --- | --- |
+| Format | pickled scikit-learn pipeline |
+| Input | 20 features (see the source experiment) |
+| Output | Probability in `[0, 1]` |
+
+# Notes
+
+Scored on the public leaderboard by
+[leaderboard submission 2026-02-18](/submissions/leaderboard_2026_02_18.md).
+
+# Citations
+- https://scikit-learn.org/stable/model_persistence.html

okf/bundles/ml_experiment_trail/models/index.md

@@ -0,0 +1,3 @@
+# Model
+
+* [GBDT tuned](gbdt_tuned.md) - Tuned gradient-boosted tree model exported from experiment gradient-boosting v2.

okf/bundles/ml_experiment_trail/submissions/index.md

@@ -0,0 +1,3 @@
+# Submission
+
+* [Leaderboard submission 2026-02-18](leaderboard_2026_02_18.md) - Public-leaderboard submission generated from the GBDT tuned model.

okf/bundles/ml_experiment_trail/submissions/leaderboard_2026_02_18.md

@@ -0,0 +1,26 @@
+---
+type: Submission
+title: Leaderboard submission 2026-02-18
+description: Public-leaderboard submission generated from the GBDT tuned model.
+tags:
+- submission
+- leaderboard
+timestamp: '2026-02-18T11:30:00+00:00'
+---
+
+The prediction file submitted to the competition, generated from
+[gbdt tuned](/models/gbdt_tuned.md).
+
+# Result
+
+| Field | Value |
+| --- | --- |
+| Public score | 0.828 |
+| CV score | 0.834 |
+| Public rank | 142 / 1,930 |
+
+# Notes
+
+The public score tracks cross-validation within 0.006, which suggests the
+validation protocol from the [baseline](/experiments/baseline_logistic_regression.md)
+generalizes and there is no public-set overfitting yet.