Local Infra

Reusable local infrastructure repo for development across different projects.

Idea

This repo is built around three rules:

• Compose files define services only
• default values live in env/
• the root .env.example stays tiny and is used only for project-specific overrides

Compose files do not contain fallback defaults like ${VAR:-value}.

If a required variable is missing from the merged env set, Docker Compose should fail immediately. That is intentional.

Quick start

1. Optional: create root overrides

cp .env.example .env.project

Then adjust only what is project-specific.

2. Start services

For example, to start postgres and kafka:

python3 local_infra.py up postgres kafka

Start the Confluent Platform Kafka stack:

python3 local_infra.py up cp-kafka

Start the same stack with Kafka REST Proxy enabled:

python3 local_infra.py up cp-kafka-rest

Start a slim Airflow stack for local DAG testing:

python3 local_infra.py up airflow

Start the same Airflow stack in stateful mode:

python3 local_infra.py --stateful up airflow

Start the same stack with persistent volumes:

python3 local_infra.py --stateful up postgres kafka

Start only Postgres:

python3 local_infra.py up postgres

3. Dry Run

Preview the exact Docker Compose command:

python3 local_infra.py --dry-run up postgres kafka

4. Overriding env file

Use an extra override file:

python3 local_infra.py --env-file .env.project up postgres kafka

Keep these extra override files outside env/ so env/ remains reserved for service defaults only.

CLI usage

python3 local_infra.py [--env-file PATH] [--dry-run] [--stateful] <command> [services...]

Commands:

• up
• down
• config
• list

Examples:

python3 local_infra.py list
python3 local_infra.py up kafka postgres
python3 local_infra.py --stateful up kafka postgres
python3 local_infra.py up cp-kafka
python3 local_infra.py config kafka

Current services

Postgres

Service name and aliases: postgres, pg

Description: Single-node PostgreSQL database.

Configurable env	What it does	Default
POSTGRES_IMAGE_TAG	Chooses Postgres image tag.	16-alpine
POSTGRES_PORT	Exposes Postgres on host port.	5432
POSTGRES_DB	Creates default database name.	local
POSTGRES_USER	Creates default database user.	postgres
POSTGRES_PASSWORD	Sets default user password.	postgres

Kafka

Service name and aliases: kafka, kafka-ui, kafkaui

Description: Single-node Kafka, KRaft mode, includes UI.

Configurable env	What it does	Default
KAFKA_IMAGE_TAG	Chooses Kafka image tag.	latest
KAFKA_PORT	Exposes Kafka broker on host.	9092
KAFKA_CONTROLLER_PORT	Exposes Kafka controller on host.	0
KAFKA_CLUSTER_ID	Sets the single-node KRaft ID.	MkU3OEVBNTcwNTJENDM2Qk
KAFKA_UI_IMAGE_TAG	Chooses Kafka UI image tag.	v0.7.2
KAFKA_UI_PORT	Exposes Kafka UI on host.	8080
KAFKA_UI_CLUSTER_NAME	Labels the cluster inside the UI.	local

CP Kafka

Service name and aliases: cp-kafka, cpkafka, cp-kafka-ui

Description: Single-node Confluent Platform Kafka, KRaft mode, includes Schema Registry and UI.

Configurable env	What it does	Default
CP_KAFKA_PLATFORM_TAG	Chooses the Confluent Platform image tag.	7.6.1
CP_KAFKA_UI_IMAGE_TAG	Chooses Kafka UI image tag.	v0.7.2
CP_KAFKA_PORT	Exposes Kafka broker on host.	29092
CP_KAFKA_CONTROLLER_PORT	Exposes Kafka controller on host.	0
CP_SCHEMA_REGISTRY_PORT	Exposes Schema Registry on host.	8081
CP_SCHEMA_REGISTRY_COMPATIBILITY_LEVEL	Sets global Schema Registry compatibility.	BACKWARD
CP_KAFKA_UI_PORT	Exposes Kafka UI on host.	8088
CP_KAFKA_CLUSTER_ID	Sets the single-node KRaft ID.	MkU3OEVBNTcwNTJENDM2Qk
CP_KAFKA_UI_CLUSTER_NAME	Labels the cluster inside the UI.	cp-local

CP Kafka REST

Service name and aliases: cp-kafka-rest, cpkafkarest, cpkafkarest, kafkarest, kafka-rest

Description: Extension for cp-kafka that adds Kafka REST Proxy. Running cp-kafka-rest from the CLI automatically includes the base cp-kafka stack.

Configurable env	What it does	Default
CP_KAFKA_PLATFORM_TAG	Chooses the Confluent Platform image tag.	7.6.1
CP_KAFKA_REST_PORT	Exposes Kafka REST Proxy on host.	8082

Airflow

Service name and aliases: airflow, af

Description: Slim Airflow stack for local DAG development. Includes dedicated Postgres metadata DB, airflow-init, webserver, and scheduler.

Configurable env	What it does	Default
AIRFLOW_IMAGE_TAG	Chooses Airflow image tag.	2.10.5
AIRFLOW_PORT	Exposes Airflow UI on host.	8085
AIRFLOW_EXECUTOR	Selects the Airflow executor.	LocalExecutor
AIRFLOW_UID	Sets the runtime UID for mounted files.	50000
AIRFLOW_ADMIN_USERNAME	The default admin username.	admin
AIRFLOW_ADMIN_PASSWORD	The default admin password.	admin
AIRFLOW_POSTGRES_DB	Creates the Airflow metadata database.	airflow
AIRFLOW_POSTGRES_USER	Creates the Airflow metadata database user.	airflow
AIRFLOW_POSTGRES_PASSWORD	Sets the Airflow metadata database password.	airflow
AIRFLOW__CORE__LOAD_EXAMPLES	Enables or disables Airflow example DAGs.	false
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION	Controls pause state for new DAGs.	true

S3

Service name and aliases: s3

Description: Local S3-compatible object storage powered by LocalStack.

Configurable env	What it does	Default
S3_IMAGE_TAG	Chooses LocalStack image tag.	latest
S3_PORT	Exposes the LocalStack edge API on host.	4566
S3_REGION	Sets the default AWS region for S3 emulation.	us-east-1
S3_ACCESS_KEY_ID	Sets the access key for local AWS clients.	test
S3_SECRET_ACCESS_KEY	Sets the secret key for local AWS clients.	test
S3_DEFAULT_BUCKETS	Creates default bucket names at startup.	local

Complete env files

• airflow.env
• postgres.env
• kafka.env
• cp-kafka.env
• cp-kafka-rest.env
• s3.env