🔮 probseer 🔮

Warning

probseer is still under active development. APIs, defaults, and examples may change between releases, and some implementations almost certainly still contain errors.

probseer is a Python package for trustworthy probabilistic prediction. It brings together conformal predictive systems, isotonic distributional regression, Venn-Abers calibration, CORP reliability diagrams, and proper scoring rules for classification, regression, causal inference, and right-censored survival data.

The package is designed for workflows where a model returns a full predictive distribution, not just a point prediction. It provides tools to calibrate those distributions, diagnose calibration with recalibration curves, and evaluate them with scoring rules that match the prediction target.

📦 Installation

probseer is available on PyPI under the package name probseer.

pip install probseer

If you use uv in your own project:

uv add probseer

For local development, tests, and notebooks from this repository, install all dependency groups:

uv sync --all-groups
uv run pytest

The survival tutorial uses scikit-survival, which is included in the dev dependency group.

📈 Example: Survival Recalibration Curves

Survival reliability diagrams compare issued survival forecasts with IPCW-adjusted observed outcomes. See the full worked example in examples/tutorial-rel-diagram-survival.ipynb.

The notebook fits a random survival forest, converts predicted survival curves to event-time CDFs with survival_to_cdf, and draws recalibration curves with corp_rel_diagram_survival. It includes CEP functionals such as functional="cep@{1000,2000,3000}" for fixed-horizon event probabilities and quantile functionals such as functional="quantile@{0.1,0.5,0.9}" for predicted event-time quantiles.

🧩 Package Structure

• probseer.cps: conformal predictive systems, including conformal binning, conformal IDR, Venn-Abers predictive systems, Mondrian variants, and weighting utilities to handle distributional shifts.
• probseer.idr: isotonic distributional regression and partial-order helpers.
• probseer.evaluation.metrics: proper scoring rules and calibration metrics for classification, regression, and right-censored survival prediction.
• probseer.evaluation.diagrams: CORP reliability and recalibration diagrams for binary probabilities, quantile forecasts, and survival forecasts.
• probseer.utils: CDF and survival-curve conversions, inverse-CDF utilities, interval extraction, and shared helper functions.

📚 References

If probseer supports your work, please cite the related papers below.

@article{jonkers2026proper,
  title={Proper Scoring Rules for Right-Censored Survival Data},
  author={Jonkers, Jef and Van Wallendael, Glenn and Duchateau, Luc and Van Hoecke, Sofie},
  journal={arXiv preprint arXiv:2606.06393},
  year={2026}
}

@inproceedings{jonkers2026calibrated,
  title={Calibrated Regression-as-Classification for Probabilistic Forecasting},
  author={Jonkers, Jef and Van Wallendael, Glenn and Luc, Duchateau and Van Hoecke, Sofie},
  booktitle={Towards Trustworthy Predictions: Theory and Applications of Calibration for Modern AI},
  year={2026}
}

@article{jonkers2026generalized,
  title={Generalized Conformal Predictive Systems Under Distributional Shifts},
  author={Jonkers, Jef and Ziegel, Johanna},
  journal={arXiv preprint arXiv:2606.11044},
  year={2026}
}

---

👤 Jef Jonkers