olap-benchmarks

OLAP database benchmarks

Results

Structure

This repo is divided by database and benchmark suite.

Running locally

Requires Python + uv and Docker.

Install the necessary Python dependencies

uv sync

Install zsh completions once:

uv run olap --install-completion --shell zsh && exec zsh

Run it again if the CLI structure changes and you want the generated completions refreshed.

Results schema migrations

Install dev dependencies (includes Alembic):

uv sync --group dev

Create a new migration from SQLAlchemy models:

uv run --group dev alembic revision --autogenerate -m "describe_change"

Apply migrations to the default results database from .env:

uv run --group dev alembic upgrade head

Apply migrations to a named results revision through the app CLI:

uv run olap results migrate --revision default

Apply migrations to an explicit database path:

uv run --group dev alembic -x db=/absolute/path/to/results.db upgrade head

Attribution

• ClickBench suite is based on ClickBench by ClickHouse
• RTABench suite is based on RTABench by Timescale
• Kaggle Airbnb suite is based on "Testing query speed for DuckDB vs ClickHouse vs StarRocks databases" by Vitaliy

TODO

• Additional step for time series suite with mutate+select
  • Concurrently: insert one row as quickly as possible to large table + run a small number of selects against this table (multiple clients)
  • Simulates actual workloads (single writer + multiple readers)
  • What should this operation be named?