BODriving

Bayesian Optimization with human-in-the-loop preference priors, built on BoTorch.

The core idea: a human expert can express preferences over parameter ranges (e.g. "I prefer speed between 50–90 km/h") and these are injected as differentiable prior weights into the acquisition function. The BO loop still explores, but is biased toward the preferred region.

I tried to make it very similar to bo.py so you should be able to implement it relatively direction.

How it works

1. A GP model is fit to observed data.
2. A standard log-space acquisition function (qLogNEI) scores candidate parameters.
3. A PriorWeightedAcquisition wrapper adds log-prior weights to that score — no other changes to the BO pipeline.
4. The human can update their preferences between iterations by passing a new list of priors.

Defining priors

Always use ParameterSpace to express preferences in natural units. Normalization to [0, 1] (required internally) is handled automatically.

from parameter_priors import ParameterSpace

space = ParameterSpace([
    ("speed",   10.0, 130.0),   # km/h
    ("headway",  0.5,   3.0),   # seconds
    ("accel",   -4.0,   3.0),   # m/s²
])

priors = [
    space.soft_bound("speed",   low=50.0, high=90.0),  # prefer this range
    space.gaussian("headway",   mean=1.5, std=0.3),    # prefer near this value
]

Two prior types are available:

Type	Behaviour
soft_bound(name, low, high)	Weight ≈ 1 inside the range, falls off outside via smooth sigmoid walls
gaussian(name, mean, std)	Weight peaks at mean, falls off with a Gaussian shape

Running

uv run main.py

This runs four comparison scenarios (vanilla, soft-bound, Gaussian, online update) on a synthetic Rosenbrock objective and saves a plot to bo_results.png.

For human-in-the-loop use

In production, replace the synthetic objective with a call to your simulator. Each iteration:

1. BO proposes a candidate configuration.
2. The simulator (or a human drive) evaluates it.
3. The human optionally updates their preferences.
4. Pass the new priors into the next acquisition function call.

# Each iteration — priors can change freely between calls
acq_fn = PriorWeightedAcquisition(base_acq, priors)
candidate, _ = optimize_acqf(acq_fn, bounds=bounds, ...)