From 84ff36132a0b09c18e7ea8110e9723fc951f32d0 Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Tue, 24 Mar 2026 11:05:44 +0100 Subject: [PATCH 1/8] Replace MinIO with Garage in deployment example --- .gitignore | 2 + README.md | 113 ++++++++++++++++++++++++++++++++++++----- config/application.yml | 8 +-- config/garage.toml | 17 +++++++ create-bucket.sh | 69 +++++++++++++++++++------ docker-compose.yml | 57 ++++++++++++--------- example.env | 18 ++++++- garage-data/.gitkeep | 1 + 8 files changed, 229 insertions(+), 56 deletions(-) create mode 100644 config/garage.toml create mode 100644 garage-data/.gitkeep diff --git a/.gitignore b/.gitignore index 5bd840e..9b4e6c9 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ db-data/data s3-data/data +garage-data/meta +garage-data/data .env diff --git a/README.md b/README.md index a70c105..dc1f5ae 100644 --- a/README.md +++ b/README.md @@ -7,28 +7,117 @@ If you use or plan to use DSW, please let us know via [info@ds-wizard.org](mailt - Join our [**Discord** server](https://discord.gg/MW3H9tdMcT), where you can be notified about important updates and releases + we can discuss your issues and ideas. - Provide us feedback (what is good and bad, [feature requests](https://ideas.ds-wizard.org/), etc.) -This example is intended for **local setup and testing**. For production use there are many more things to do such as authentication, controlling exposed ports (e.g. do not expose ports of `postgres` and `minio`), data backups, or using proxy (with HTTPS and WebSocket enabled). As it is highly dependent on your use case, consult production deployment with your sysadmin or [contact us](https://ds-wizard.org/contact). +This example is intended for **local setup and testing**. For production use there are many more things to do such as authentication, controlling exposed ports (e.g. do not expose ports of `postgres` and `garage`), data backups, or using proxy (with HTTPS and WebSocket enabled). As it is highly dependent on your use case, consult production deployment with your sysadmin or [contact us](https://ds-wizard.org/contact). -## Usage +## Overview -This is an example deployment of the [Data Stewardship Wizard](https://ds-wizard.org) using [docker-compose](https://docs.docker.com/compose/). You can clone the repository, create `.env` file using `example.env` and run it with: +This is an example deployment of the [Data Stewardship Wizard](https://ds-wizard.org) using [Docker Compose](https://docs.docker.com/compose/) and [Garage](https://garagehq.deuxfleurs.fr/) as the S3-compatible object storage. -``` -$ docker-compose up -d -``` +It is intentionally set up as a **single-node local POC**: -Then visit [localhost:8080/wizard](http://localhost:8080/wizard/) and login as `albert.einstein@example.com` with password `password`. +- Garage runs in Docker on `127.0.0.1:9000` +- DSW points to `http://host.docker.internal:9000` so presigned URLs are reachable from the browser +- `create-bucket.sh` performs the one-time Garage bootstrap for this example For information on how to use Data Stewardship Wizard, visit our [guide](https://guide.ds-wizard.org). -## Important notes +## Quick Start + +1. Create the local environment file: + + ```bash + cp example.env .env + ``` + +2. Start the stack: + + ```bash + docker compose up -d + ``` + +3. Bootstrap Garage: + + ```bash + ./create-bucket.sh + ``` + +4. Open DSW: + + [http://localhost:8080/wizard](http://localhost:8080/wizard/) + +5. Log in with: + + - Email: `albert.einstein@example.com` + - Password: `password` + +## What The Bootstrap Does + +`create-bucket.sh` is intended to be safe to rerun for this local setup. It: + +- assigns the single-node Garage layout if the node still has no role +- creates the configured S3 bucket if it does not exist +- imports the configured Garage access key if it does not exist +- grants read, write, and owner permissions on the bucket to that key + +## Verification Checklist + +After setup, use this checklist to confirm the Garage-backed deployment works. + +### Infrastructure checks + +```bash +docker compose ps +docker compose logs garage --tail=100 +docker compose logs server --tail=100 +docker compose logs docworker --tail=100 +``` + +Expected results: + +- `garage` is running +- `server` creates the S3 client successfully +- `docworker` starts without S3 errors +- no authentication, signing, or region errors appear in the logs + +### Application checks + +In the DSW UI, verify: + +1. the application opens and login works +2. a project file can be uploaded +3. the uploaded file can be downloaded +4. a document preview can be generated +5. a document template asset URL works, if applicable + +If these checks pass, Garage is functioning as a drop-in S3-compatible backend for this example deployment. + +## Important Notes * Use `docker compose pull` to get newest image (hotfixes) before starting -* **Do not expose** PostgreSQL and MinIO to the internet (MinIO should be exposed only via proxy) +* **Do not expose** PostgreSQL and Garage to the internet (Garage should be exposed only via proxy in public deployments) * When you want to use DSW publicly, **set up HTTPS proxy** (e.g. Nginx) with a certificate for your domain and change default accounts -* Set up volume mounted to PostgreSQL and MinIO containers for persistent data -* You have to create S3 bucket, either using Web UI (for MinIO, you can expose and use `http://localhost:9000`) or via client: https://docs.min.io/docs/minio-client-complete-guide.html#mb, e.g. use `create-bucket.sh` script -* Always use **strong passwords** and never use default values, **change the secrets** in `config/application.yml` (32 character string in `general.secret` and RSA private key in `general.rsaPrivateKey` via `ssh-keygen -t rsa -b 4096 -m PEM -f jwtRS256.key`) +* Set up volume mounted to PostgreSQL and Garage containers for persistent data +* Garage needs a one-time bootstrap after the stack starts. `create-bucket.sh` assigns the single-node layout, creates the bucket, imports the configured S3 key, and grants bucket permissions +* DSW uses `http://host.docker.internal:9000` as the S3 endpoint so that presigned URLs returned by DSW are reachable from your browser in this local setup +* Always use **strong passwords** and never use default values, **change the secrets** in `config/application.yml` and `.env` (JWT secret, RSA private key, Garage RPC/admin tokens, and S3 credentials) + +## Troubleshooting + +If something does not work: + +```bash +docker compose ps +docker compose logs garage --tail=200 +docker compose logs server --tail=200 +docker compose logs docworker --tail=200 +``` + +Common local issues: + +- Garage was started, but `create-bucket.sh` was not run yet +- `.env` values and the imported Garage key no longer match +- the server is still starting and has not reached a healthy state yet +- an old local data directory contains stale state from a previous attempt ## Security Audit diff --git a/config/application.yml b/config/application.yml index d5831b9..f9a0f30 100644 --- a/config/application.yml +++ b/config/application.yml @@ -59,12 +59,14 @@ general: database: connectionString: postgresql://postgres:postgres@postgres:5432/engine-wizard -# (!!) Change default password +# (!!) Change default credentials s3: + # Use a host-reachable endpoint so browser-facing presigned URLs work in local Docker. url: http://host.docker.internal:9000 - username: minio - password: minioPassword + username: GK0123456789abcdef01234567 + password: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef bucket: engine-wizard + region: garage # (!!) Configure SMTP mail: diff --git a/config/garage.toml b/config/garage.toml new file mode 100644 index 0000000..e4d79d9 --- /dev/null +++ b/config/garage.toml @@ -0,0 +1,17 @@ +metadata_dir = "/var/lib/garage/meta" +data_dir = "/var/lib/garage/data" +db_engine = "sqlite" + +# Single-node local POC settings. +replication_factor = 1 +rpc_bind_addr = "[::]:3901" +rpc_public_addr = "127.0.0.1:3901" + +[s3_api] +# Path-style requests work without extra DNS; root_domain only enables optional vhost-style access. +s3_region = "garage" +api_bind_addr = "[::]:3900" +root_domain = ".s3.garage.localhost" + +[admin] +api_bind_addr = "[::]:3903" diff --git a/create-bucket.sh b/create-bucket.sh index 33238a0..f7d61da 100755 --- a/create-bucket.sh +++ b/create-bucket.sh @@ -1,18 +1,55 @@ #!/bin/sh -set -a; source .env; set +a - -# (!!) Change default password -MINIO_NET="dsw-deployment-example_default" -MINIO_BUCKET="engine-wizard" -MINIO_USER="minio" -MINIO_PASS="minioPassword" - -docker run --rm --net $MINIO_NET \ - -e MINIO_BUCKET=$MINIO_BUCKET \ - -e MINIO_USER=$MINIO_USER \ - -e MINIO_PASS=$MINIO_PASS \ - --entrypoint sh minio/mc:$MINIO_VERSION -c "\ - mc alias set dswminio http://minio:9000 $MINIO_USER $MINIO_PASS && \ - mc mb dswminio/\$MINIO_BUCKET -" +set -eu + +if [ -f .env ]; then + set -a + . ./.env + set +a +fi + +GARAGE_SERVICE="${GARAGE_SERVICE:-garage}" +GARAGE_ZONE="${GARAGE_ZONE:-dc1}" +GARAGE_CAPACITY="${GARAGE_CAPACITY:-1G}" +GARAGE_KEY_NAME="${GARAGE_KEY_NAME:-dsw-engine-wizard}" +S3_URL="${S3_URL:-http://host.docker.internal:9000}" +S3_BUCKET="${S3_BUCKET:-engine-wizard}" +S3_USERNAME="${S3_USERNAME:-GK0123456789abcdef01234567}" +S3_PASSWORD="${S3_PASSWORD:-0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef}" + +garage_exec() { + docker compose exec -T "$GARAGE_SERVICE" /garage "$@" +} + +if ! docker compose ps --status running "$GARAGE_SERVICE" >/dev/null 2>&1; then + echo "Garage service '$GARAGE_SERVICE' is not running. Start the stack first with: docker compose up -d" >&2 + exit 1 +fi + +STATUS="$(garage_exec status)" +NODE_ID="$(printf '%s\n' "$STATUS" | awk '/==== HEALTHY NODES ====/{flag=1; next} flag && $1 ~ /^[0-9a-f]+$/ {print $1; exit}')" + +if [ -z "$NODE_ID" ]; then + echo "Unable to determine the Garage node ID from 'garage status'." >&2 + exit 1 +fi + +if printf '%s\n' "$STATUS" | grep -q "NO ROLE ASSIGNED"; then + garage_exec layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID" + garage_exec layout apply --version 1 +fi + +if ! garage_exec bucket info "$S3_BUCKET" >/dev/null 2>&1; then + garage_exec bucket create "$S3_BUCKET" +fi + +if ! garage_exec key info "$S3_USERNAME" >/dev/null 2>&1; then + garage_exec key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes +fi + +garage_exec bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_USERNAME" + +echo "Garage bootstrap completed." +echo "S3 endpoint: $S3_URL" +echo "Bucket: $S3_BUCKET" +echo "Access key: $S3_USERNAME" diff --git a/docker-compose.yml b/docker-compose.yml index 01078d5..29a011f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -4,22 +4,29 @@ services: image: datastewardshipwizard/wizard-server:${DSW_VERSION} platform: linux/amd64 restart: always + # Local demo only; put an HTTPS proxy in front for shared deployments. ports: - # (!!) Expose only for local deployment, externally use HTTPS proxy - 127.0.0.1:3000:3000 depends_on: - postgres - - minio + - garage volumes: - ./config/application.yml:/app/config/application.yml:ro extra_hosts: - host.docker.internal:host-gateway + # Keep runtime S3 values in .env so they stay aligned with Garage bootstrap. + environment: + S3_URL: ${S3_URL} + S3_USERNAME: ${S3_USERNAME} + S3_PASSWORD: ${S3_PASSWORD} + S3_BUCKET: ${S3_BUCKET} + S3_REGION: ${S3_REGION} client: image: datastewardshipwizard/wizard-client:${DSW_VERSION} restart: always + # Local demo only; put an HTTPS proxy in front for shared deployments. ports: - # (!!) Expose only for local deployment, externally use HTTPS proxy - 127.0.0.1:8080:8080 environment: API_URL: http://localhost:3000/wizard-api @@ -29,12 +36,19 @@ services: restart: always depends_on: - postgres - - minio + - garage - server volumes: - ./config/application.yml:/app/config/application.yml:ro extra_hosts: - host.docker.internal:host-gateway + # Keep runtime S3 values in .env so they stay aligned with Garage bootstrap. + environment: + S3_URL: ${S3_URL} + S3_USERNAME: ${S3_USERNAME} + S3_PASSWORD: ${S3_PASSWORD} + S3_BUCKET: ${S3_BUCKET} + S3_REGION: ${S3_REGION} mailer: image: datastewardshipwizard/mailer:${DSW_VERSION} @@ -48,39 +62,36 @@ services: postgres: image: postgres:${POSTGRES_VERSION} restart: always - # (!!) Expose only for debugging locally + # Local debug access only. ports: - 127.0.0.1:5432:5432 - # (!!) Change default password + # Change defaults before using this outside a local POC. environment: POSTGRES_DB: engine-wizard POSTGRES_USER: postgres POSTGRES_PASSWORD: postgres - # (!!) Mount for persistent data + # Uncomment one option below to persist PostgreSQL data locally. # volumes: # - db-data:/var/lib/postgresql/data # OR # - ./db-data/data:/var/lib/postgresql/data - minio: - image: minio/minio:${MINIO_VERSION} + garage: + image: dxflrs/garage:${GARAGE_VERSION} restart: always - command: server /data --console-address ":9001" - # (!!) Expose only for debugging locally, externally use HTTPS proxy (see MinIO docs) + # Keep the S3 and Admin APIs bound to localhost in this local POC. ports: - - 9000:9000 - - 9001:9001 - # (!!) Change default password + - 127.0.0.1:9000:3900 + - 127.0.0.1:9003:3903 + # Change these defaults before sharing or reusing this setup. environment: - MINIO_ROOT_USER: minio - MINIO_ROOT_PASSWORD: minioPassword - MINIO_VOLUMES: /data - # (!!) Mount and backup for persistent data - # volumes: - # - s3-data:/data - # OR - # - ./s3-data/data:/data + GARAGE_RPC_SECRET: ${GARAGE_RPC_SECRET} + GARAGE_ADMIN_TOKEN: ${GARAGE_ADMIN_TOKEN} + GARAGE_METRICS_TOKEN: ${GARAGE_METRICS_TOKEN} + volumes: + - ./config/garage.toml:/etc/garage.toml:ro + # Uncomment to persist Garage state between local restarts. + # - ./garage-data:/var/lib/garage # volumes: # db-data: -# s3-data: diff --git a/example.env b/example.env index 41064b8..a1e4e24 100644 --- a/example.env +++ b/example.env @@ -1,6 +1,20 @@ DSW_VERSION=4.31 # Versions of dependencies (see their docs) -# - possibly a migration may be needed when upgrading +# Upgrades may require migrations. POSTGRES_VERSION=17.5 -MINIO_VERSION=RELEASE.2025-05-24T17-08-30Z +GARAGE_VERSION=v2.2.0 + +# DSW S3 configuration. +# Keep these aligned with the credentials imported by create-bucket.sh. +S3_URL=http://host.docker.internal:9000 +S3_USERNAME=GK0123456789abcdef01234567 +S3_PASSWORD=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef +S3_BUCKET=engine-wizard +S3_REGION=garage + +# Garage daemon secrets. +# Change these defaults before sharing or reusing this setup. +GARAGE_RPC_SECRET=1111111111111111111111111111111111111111111111111111111111111111 +GARAGE_ADMIN_TOKEN=changeMeGarageAdminToken +GARAGE_METRICS_TOKEN=changeMeGarageMetricsToken diff --git a/garage-data/.gitkeep b/garage-data/.gitkeep new file mode 100644 index 0000000..8b13789 --- /dev/null +++ b/garage-data/.gitkeep @@ -0,0 +1 @@ + From 057b68f8eeb5e54b78b79dd25808faa63ceae234 Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Tue, 24 Mar 2026 11:26:14 +0100 Subject: [PATCH 2/8] Fix Garage accessibility and bootstrap checks --- README.md | 6 +++--- create-bucket.sh | 2 +- docker-compose.yml | 13 +++++++------ example.env | 5 +++++ 4 files changed, 16 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index dc1f5ae..ac11f22 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ This is an example deployment of the [Data Stewardship Wizard](https://ds-wizard It is intentionally set up as a **single-node local POC**: -- Garage runs in Docker on `127.0.0.1:9000` +- Garage runs in Docker on host ports `9000` (S3 API) and `9003` (Admin API) - DSW points to `http://host.docker.internal:9000` so presigned URLs are reachable from the browser - `create-bucket.sh` performs the one-time Garage bootstrap for this example @@ -94,11 +94,11 @@ If these checks pass, Garage is functioning as a drop-in S3-compatible backend f ## Important Notes * Use `docker compose pull` to get newest image (hotfixes) before starting -* **Do not expose** PostgreSQL and Garage to the internet (Garage should be exposed only via proxy in public deployments) +* **Do not expose** PostgreSQL and Garage to the internet in a public deployment (Garage should be behind your proxy, firewall, or private network setup) * When you want to use DSW publicly, **set up HTTPS proxy** (e.g. Nginx) with a certificate for your domain and change default accounts * Set up volume mounted to PostgreSQL and Garage containers for persistent data * Garage needs a one-time bootstrap after the stack starts. `create-bucket.sh` assigns the single-node layout, creates the bucket, imports the configured S3 key, and grants bucket permissions -* DSW uses `http://host.docker.internal:9000` as the S3 endpoint so that presigned URLs returned by DSW are reachable from your browser in this local setup +* DSW uses `http://host.docker.internal:9000` as the S3 endpoint so both the DSW containers and the browser can reach the same local Garage endpoint * Always use **strong passwords** and never use default values, **change the secrets** in `config/application.yml` and `.env` (JWT secret, RSA private key, Garage RPC/admin tokens, and S3 credentials) ## Troubleshooting diff --git a/create-bucket.sh b/create-bucket.sh index f7d61da..914d8a3 100755 --- a/create-bucket.sh +++ b/create-bucket.sh @@ -21,7 +21,7 @@ garage_exec() { docker compose exec -T "$GARAGE_SERVICE" /garage "$@" } -if ! docker compose ps --status running "$GARAGE_SERVICE" >/dev/null 2>&1; then +if [ -z "$(docker compose ps -q --status running "$GARAGE_SERVICE" 2>/dev/null)" ]; then echo "Garage service '$GARAGE_SERVICE' is not running. Start the stack first with: docker compose up -d" >&2 exit 1 fi diff --git a/docker-compose.yml b/docker-compose.yml index 29a011f..18bfdb5 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -67,9 +67,9 @@ services: - 127.0.0.1:5432:5432 # Change defaults before using this outside a local POC. environment: - POSTGRES_DB: engine-wizard - POSTGRES_USER: postgres - POSTGRES_PASSWORD: postgres + POSTGRES_DB: ${POSTGRES_DB} + POSTGRES_USER: ${POSTGRES_USER} + POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} # Uncomment one option below to persist PostgreSQL data locally. # volumes: # - db-data:/var/lib/postgresql/data @@ -79,10 +79,11 @@ services: garage: image: dxflrs/garage:${GARAGE_VERSION} restart: always - # Keep the S3 and Admin APIs bound to localhost in this local POC. + # Publish Garage on the host so both containers and the browser can reach the same endpoint. + # Do not expose this directly in a public deployment. ports: - - 127.0.0.1:9000:3900 - - 127.0.0.1:9003:3903 + - 9000:3900 + - 9003:3903 # Change these defaults before sharing or reusing this setup. environment: GARAGE_RPC_SECRET: ${GARAGE_RPC_SECRET} diff --git a/example.env b/example.env index a1e4e24..7c4d127 100644 --- a/example.env +++ b/example.env @@ -5,6 +5,11 @@ DSW_VERSION=4.31 POSTGRES_VERSION=17.5 GARAGE_VERSION=v2.2.0 +# DSW DB configuration. +POSTGRES_USER=postgres +POSTGRES_PASSWORD=postgres +POSTGRES_DB=engine-wizard + # DSW S3 configuration. # Keep these aligned with the credentials imported by create-bucket.sh. S3_URL=http://host.docker.internal:9000 From 206e18a23a7c420eb1e51e3bbe68b3b7b6a6c738 Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Tue, 30 Jun 2026 13:37:50 +0200 Subject: [PATCH 3/8] Polish Garage local deployment example --- .gitignore | 2 +- README.md | 107 +++++++++++++++++++++++++++++++++++---- config/application.yml | 88 +++++++++++--------------------- config/garage-ui.yaml | 27 ++++++++++ config/garage.toml | 5 ++ config/plugin-proxy.conf | 20 ++++++++ create-bucket.sh | 30 +++++++++-- docker-compose.yml | 27 ++++++++++ example.env | 17 ++++--- s3-data/.gitkeep | 0 10 files changed, 242 insertions(+), 81 deletions(-) create mode 100644 config/garage-ui.yaml create mode 100644 config/plugin-proxy.conf delete mode 100644 s3-data/.gitkeep diff --git a/.gitignore b/.gitignore index 9b4e6c9..4019bd6 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,5 @@ db-data/data -s3-data/data garage-data/meta garage-data/data .env +config/application.local.yml diff --git a/README.md b/README.md index ac11f22..1fce349 100644 --- a/README.md +++ b/README.md @@ -16,12 +16,15 @@ This is an example deployment of the [Data Stewardship Wizard](https://ds-wizard It is intentionally set up as a **single-node local POC**: - Garage runs in Docker on host ports `9000` (S3 API) and `9003` (Admin API) +- Garage UI runs on `127.0.0.1:8081` - DSW points to `http://host.docker.internal:9000` so presigned URLs are reachable from the browser - `create-bucket.sh` performs the one-time Garage bootstrap for this example For information on how to use Data Stewardship Wizard, visit our [guide](https://guide.ds-wizard.org). -## Quick Start +## Required Steps + +These are the only steps needed to run the local example. 1. Create the local environment file: @@ -29,27 +32,42 @@ For information on how to use Data Stewardship Wizard, visit our [guide](https:/ cp example.env .env ``` -2. Start the stack: +2. The tracked `config/application.yml` already contains demo application values for this example. + +3. Start the stack: ```bash docker compose up -d ``` -3. Bootstrap Garage: +4. Bootstrap Garage: ```bash ./create-bucket.sh ``` -4. Open DSW: +5. Open DSW: [http://localhost:8080/wizard](http://localhost:8080/wizard/) -5. Log in with: +6. Log in with: - Email: `albert.einstein@example.com` - Password: `password` +## Optional + +These steps are not required to start the example. They are only useful if you want to inspect the local Garage setup. + +1. Open Garage UI: + + [http://localhost:8081](http://localhost:8081) + +2. Log in to Garage UI with: + + - Username: `admin` + - Password: `admin` + ## What The Bootstrap Does `create-bucket.sh` is intended to be safe to rerun for this local setup. It: @@ -59,15 +77,16 @@ For information on how to use Data Stewardship Wizard, visit our [guide](https:/ - imports the configured Garage access key if it does not exist - grants read, write, and owner permissions on the bucket to that key -## Verification Checklist +## Optional Verification -After setup, use this checklist to confirm the Garage-backed deployment works. +After setup, you can use this checklist to confirm the Garage-backed deployment works. ### Infrastructure checks ```bash docker compose ps docker compose logs garage --tail=100 +docker compose logs garage-ui --tail=100 docker compose logs server --tail=100 docker compose logs docworker --tail=100 ``` @@ -75,6 +94,7 @@ docker compose logs docworker --tail=100 Expected results: - `garage` is running +- `garage-ui` is running - `server` creates the S3 client successfully - `docworker` starts without S3 errors - no authentication, signing, or region errors appear in the logs @@ -89,17 +109,84 @@ In the DSW UI, verify: 4. a document preview can be generated 5. a document template asset URL works, if applicable +In Garage UI, verify: + +1. login works +2. the `engine-wizard` bucket is visible +3. file upload works +4. folder creation works + If these checks pass, Garage is functioning as a drop-in S3-compatible backend for this example deployment. +## Garage Notes + +### Current Local Model + +This repository uses Garage as a **local example only**. Our production setup still uses MinIO. + +- DSW uses a single S3 bucket: `engine-wizard` +- Garage S3 API is available locally on `localhost:9000` +- Garage website endpoint is available locally on `localhost:9002` +- Garage UI is available locally on `localhost:8081` +- A local `plugin-proxy` service exposes plugin assets on `localhost:9004` + +If you want to use locally stored plugin assets with this example, the plugin URL stored in DSW should point to the local proxy, for example: + +```text +http://localhost:9004/plugins/// +``` + +The actual plugin file must then be reachable at: + +```text +http://localhost:9004/plugins///plugin.js +``` + +### Buckets And Plugins + +Garage is S3-compatible, so the same bucket model you use with MinIO can also be used with Garage. + +For this local example: + +- DSW uses one shared bucket +- plugin assets can be exposed through the local `plugin-proxy` +- the plugin URL in the DSW database should point to the public plugin location, not to the Garage UI + +If you use Garage in a self-hosted deployment, keep the same basic rule as with MinIO: + +- private DSW data should stay private +- public plugins should live in a dedicated public location +- do not mix private application data and public plugin assets in the same public bucket + +### Self-Hosted Security Note + +If a self-hosted user wants to expose Garage outside localhost, they should not publish the raw Garage ports directly to the internet. + +Instead, they should put Garage behind an HTTPS reverse proxy or another equivalent security layer that provides: + +- TLS / HTTPS +- authentication for admin access +- controlled exposure of only the endpoints that should be public + +In practice, that usually means: + +- DSW stays behind its normal public HTTPS setup +- public plugin files are exposed through a dedicated public URL +- Garage admin or UI access is protected separately + +This repository does not try to provide a full production Garage deployment. It only illustrates how Garage can be used locally in the deployment example. + ## Important Notes * Use `docker compose pull` to get newest image (hotfixes) before starting -* **Do not expose** PostgreSQL and Garage to the internet in a public deployment (Garage should be behind your proxy, firewall, or private network setup) +* **Do not expose** PostgreSQL or raw Garage ports directly to the internet in a public deployment +* If you self-host Garage publicly, place it behind an HTTPS reverse proxy or another equivalent security layer and protect admin access * When you want to use DSW publicly, **set up HTTPS proxy** (e.g. Nginx) with a certificate for your domain and change default accounts * Set up volume mounted to PostgreSQL and Garage containers for persistent data * Garage needs a one-time bootstrap after the stack starts. `create-bucket.sh` assigns the single-node layout, creates the bucket, imports the configured S3 key, and grants bucket permissions * DSW uses `http://host.docker.internal:9000` as the S3 endpoint so both the DSW containers and the browser can reach the same local Garage endpoint -* Always use **strong passwords** and never use default values, **change the secrets** in `config/application.yml` and `.env` (JWT secret, RSA private key, Garage RPC/admin tokens, and S3 credentials) +* Garage UI is configured with local basic auth defaults for this POC; change them before sharing the setup +* Always use **strong passwords** and never use default values, **change the demo secrets** in `config/application.yml` and `.env` before using this anywhere except local testing ## Troubleshooting @@ -108,6 +195,7 @@ If something does not work: ```bash docker compose ps docker compose logs garage --tail=200 +docker compose logs garage-ui --tail=200 docker compose logs server --tail=200 docker compose logs docworker --tail=200 ``` @@ -115,6 +203,7 @@ docker compose logs docworker --tail=200 Common local issues: - Garage was started, but `create-bucket.sh` was not run yet +- Garage UI started, but Garage itself is not healthy yet - `.env` values and the imported Garage key no longer match - the server is still starting and has not reached a healthy state yet - an old local data directory contains stale state from a previous attempt diff --git a/config/application.yml b/config/application.yml index f9a0f30..95db962 100644 --- a/config/application.yml +++ b/config/application.yml @@ -1,77 +1,49 @@ -# (!!) Change the secret and RDA private key to your own general: clientUrl: http://localhost:8080/wizard - secret: b7zFNUhFYIFLmDSFKFkp0Bnp8Z3uWGnH + secret: insecure-demo-dsw-secret rsaPrivateKey: | - -----BEGIN RSA PRIVATE KEY----- - MIIJKAIBAAKCAgEAyMMyCZe+/oihq702T+Yb2qiqgpmSmH9krH5lX5ZlT94KTiIn - KjNsMg+SLjIWedU+d5R8uUxqXX50MAXYwfi49NDpSi1hDTfxw2O/J03P+SuzwDTU - LMWQjBCT0t1QZOD7ODhPL7+UvZsoujHQJEwJ32Hj99z8HXOJ+pEvLwlmv3te1Tpr - SK9XvzjobVBz+XJL+WkQYFTAnhJHuM7Xzz3wQsiF60vCMe8yFqAlppd+6rybLdl5 - aXn6o4kzpDUI1/izRHW093sOkhAP+Esch7VJpEfLDpqDu2DDml/hL3KXFevau6mt - sWJZ0qyswh1BrdtwKKm2eqzjqw+Q0sThJUA66r1ZzHljqt8E3OSJlFHUgIZXXx1o - G8DNzu5jcjy6T85YSzDg8upIgNgoT14kgYUWnZsIKiXvhAX9WdkSVAHOkM0ADN8P - PTKekgAO4YHbM1hsraGpXJVJf6AdAwYwC4burLrdwiERafXV1FEpcnaSBezjAs3P - PmFGc7tCUocKfxHY3fJqU2n/Ngw8cDxZaqbPR1K+RXPzAYN15PBdtO8VnubKZTBw - x0tPXxm7rz87duP2szLpjaIEvks77As22sOptyzYO39iA57Vs4+onxV/0J/quDTv - Nci4ZdWSPABoyqtoiswrud5hzr75Zo0t8FutkoI9yQXyJJbgHFhSy5A7o0ECAwEA - AQKCAgAPyRFJIc5ZsyKLH2CC3cP46GQnyq11IicnTPTplCi5I+tJg3Py5IIwWXsT - +Mo9oB++RUWdSD3gxg2arQsDDWx1i2uccn78DBcNVu1zyLF+lQoYJc89Cfe4m6ET - 5mTo2avHGt5XEcg/NFD6euozwGrSTp8fIRgNtMT3l6zPFP0oNyoYt/Qxjd/nYDEe - T0IfLfZ9UN6BxIg0pZlQFatwVx1qEtw8yQMm6/jTQJpVQ/otuNC2xS+/tGCZtTMD - SUw9twdZbavh/aICqBKpYjaHESEOMrrrlgiflhcrLzjEg00P4zCaqtfGZ8ZKQ1t+ - E9DTO7rQ5686bBemnp2zhNlzFV7MbGyXHxYUh7hbSHGJwGrSbgDCZEVOllcgALTF - RsUxivBAmkyytYlT79QqSUrMAicf8ys2jW0suHuxT/gNqnb4iGtzoGv3OVXORsPt - TlxfTHIo00yF9OdCYTWrsi2XFwBgaasc7eH4KKTMpFOSjIq3iu+zZiLGv20z04Xi - iSnG4rN/nZS6MYTt50XM1lZb4XE5SWw35Az/BVLeQ4T2yanrNnQr87mdmlrh/LHy - ARInp1rHCaUlodWQCS9loEGGVPxGaetSUPhPPt3xMEdFvjvYHvp/rPRQc5iVZOC7 - +Av8gctVL4t4qsU1aPVVwmyyvZhW0cPrEDd8ap+usAwYW/iY1QKCAQEA+pJuZz+O - tDkUP//p7uNRmc54yWLVMZ1IdrTeqNVRJyHVNvxCzYX3GC2UOQz/6IECf/eCP3Z/ - /IVnYXRsoGKNght7it24hoSGsLSQO6gCYa7Xlmf72VyBWCluE+9KlR4I8LmvfYrC - tYGCazL0qbHwXCBdw0oL1HvSEEGMPt/TpdDkMtyvi+OgmKM6BBiCVC64qVjFGtB1 - CRxchbHN56mETa63m9wC6C/mIl+veysC0nE745h5i+ocu0LLEpfGA6tSp3XyNii1 - fc6LwmA66JsamRqg5IUdj4/AnHvO/XnKyVlzIwLTSWaoGlBHjGHHYqJsoQk/SryX - bAYBcJsB+MYUewKCAQEAzRyKjOPLMvL7IJjbnxNrl3k7IVAAR6o5RgNuMQ7b+Fmb - dPwi6SQJOHKnAHlC3lqUyCQxoW5tOeLE1A2At2LSfNJh8M3tiAhgoamA5yD5x1MC - 1RZK9ADmnDS3ezS3xixHSJl8dPsmgB7M8usrrIVD7xcVbMc6SjkbWPJD3rsJZHXz - b1phu0avUx9kN2Y0yjOGCPSQSDMjY6bOMpJaDmRcKCq1+z22OtuxOYlFttdvRhxf - sODv+FYN+estm2okko5lAhfnZwApp8APmMxKRFzebNMKE6gTKQ+OvquSexx6jakz - PTj8h0aOUQ/sMU/aXWSNDzjE8WHvB/jKr4p1lG5QcwKCAQBHt/3No41u3B+FqfQF - Ih9LDKPZ0GYBC2mgQGiynkvaDX7/e/iK5tswmmsgYS4mGDbk9azcCUjcuDo81Jv6 - 0/p40PDtcETOi8QZc7UMvivgVbDzJdQaBLYoy8LsBcOtJqmcbSljn4HXfbgcgrsw - jvOjCsiLivgT21TmvjwsCtiDT2ThgfTk70QaLC4A2hsDiVPldLkF0cCN2QZKZffv - ZL+vlGCNRc23M26FTCnn8QmfvjVMEWpIt7t9hcKmEbblefXwZHvtMH2yAppSCkUS - KsLUK6D9h3RMkPKqV22Jc4CtXOLtlHNxQObAkVXJQntVKfQ5q3HoLE6S5q0m09ce - o2NdAoIBAEVbflpP+Pk8gsCuD63F8l7hJqaIF+Aj5oQPSuaZ4LYlDtVLIT726YtF - 8cTOPFdWB/N5kbLF1Xd6gBY4jqr0ZrrApU+5QrX6JzrjR2xgkTXekxmdFTFXUKPH - i7fExtPjSyN4i0Y/WvB9+c22yijnU2Z4xszmvYY1s1KrcVnSo0VuQSgJRvz91PFl - rwywqe8p5wwGLvfLGw9qa1VcecleFG+cqbbQUluqyE3tZj6CsOTjWVRFa7/s0YMO - DHj+Nmm8btl7XctNmchYwK1pfz+Iq8nevwyQjBIZaap99+VbZvGaInJ0X6gaqqss - mDDAkNV919kq2CXLePmYHmmnh0Tu3JMCggEBAL1wmSfX55tVCQRSVl7ahNpqGY7C - 7Syyd0z8vE3F+czsd8H0j5KGBgKSUbtiJDhB1BSLcIwaUczJFiFXpegkTXnSz1Id - 2H+riQDon8rVkdP8JP6ny5ncj4PQ5ZAE7gwMPdtjhY4WChfHMXGgyfubNKIGi0pP - U8QopElb2l9/YhTpSvXXHXLYAq8bz3Kye6jQqatoDwRCuHkPc0OUeT9NdtW+x/Y4 - cdU04pDO6DEYKYcD8vgSlEzVCNqb6TvFfRsfQXI5TSkKuX3fTR2wgLR6CxWIejSE - 32jnWA89Tg9wD6ACZo8On4IIVH4oeFuNi8mSb/M3pQ9hTtqIUqwUw+AxD+I= - -----END RSA PRIVATE KEY----- + -----BEGIN PRIVATE KEY----- + MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDAip4YwYPi13kc + UERJiOQPLJfA9GhPvjSqBlyhfmerf+ruMComuWlZQ13QqeCyT1/kOcimRivGM/lC + jjnfQ3FoS5DMUwLrpvPJDH8T27hZ51WDh12bT6lHUhruju1xfQqHYIJ5qm6cigdU + 1951XLESKtJgpkt4uqBK0eDBXY5ZXBBE+MzRqzJKcpMPv6kXEJp0Lenm5Gqtma04 + +iVUh9KU2l1V9vwMqjlyF/DplaABR+YIKTwWKqjHuHTsSuZBAsU5tX0LsGU8uCJd + UYwztfHjQuKFmEWAmXnZeVaFxUqaxCUuKwUSmGjMORcC3zWr7LZoyY/9dS3TAXlW + ln1/5DIlAgMBAAECggEACvWlwh4uj6VZAELHxrFc3RanHK7//dBp1wCKgHbD5LMt + s2Xyc3nVhH5p9j91MGRSpP8IOM60HZGslbYN/UAthS5BjVMJG4WWFgX432LmWQrq + umEXwIg5FOIx8XPmEbIR/PRO+PXs9EqD2FFHssGv715lbMyKbZ84RVbLYE/rxl8Q + LNTYsgsC7y1gY1VMPTLMvwlqaXHCGNgA6aKp+cLGjFhLUcDmjM9aFIygQYvhBm3w + UZjip7vJk7h4m4lXlcjSUkzlyhh+nmM1UKPuXHTvbVWyQN0qU+MjOV6+AW2ZRfSC + xtuCv7JXmjDn6imr2n7aavFgRpdLA+Ns4Mp9/GQUkQKBgQD0dj6avKBG0ITMSIsS + l/tCn673W29387DL+4d7VihlcImhqmnoZZZx8D54XiDWUTBUOcOdNw7a8L8at5Uy + 04EM2jBFzT+3yxRbVgaeXz/8celhhYo5RWFRTpqkAGZPRRWhfXAcvCWQ8Bqxvrsz + wjHeixAQo9omMCeeFaxHk7g7VwKBgQDJoQj7iFp1Sl0k4ifTUb8uTyjRvNr7/ZE/ + jlhWrkbPTxtyrUr0vHe4FkdhgJOhSZwFIPByMfNAq4RkarUoQrYkiDM+VxksHuOz + qLlKzevIYOmJHU09E0LeuL3MznhvUJA20YNnKCXPsrBlRnKKjGDamQDr49/Ducvq + AcST/QyM4wKBgQCRoNMgmNed99/0Vg7LD9C1Wqz/W+/mdDxQ99E4rxW3gDftH7ih + ocriqyx7xg8nvQveUN7wC/mEgcBGaTCvK4IIJv50xpV7prXEx1WACNv0ReOynbu+ + 7E7SAdJrenV4KX5+Slx8MbTrR62PT0Dzz26n1Q3H6+QrtaoUKXMM3Ln2VwKBgQCF + sgICx6czntukWVGPDeV3D4ZBjLin3rsxKXHqCCVr0UiiAcZyAYZgymUKgfdiOyz5 + OZ42GmjxuMlQzHDTAWlFUAqqi5ZlaxYRSw+SSnjNpXU1NfjBRsRQaq9v3nh9Vy5J + KyO2gyiPUC0m+k95Dxwp1sqFn9QANHLB9+ohNunbgQKBgGisecKJ5Ir3XpbWVzjl + NF3DcZonHZ2onGIhO5P/+xR8iIW6St65l1jcmm/oCuOEQh8wMp12yz7Fs0ZmzWaq + jdrXdesJRMB23s+cPfLDJKv/sqZFQfxhvp4c8SIy79hyGhwj2NrhqzBcSTsU1LzB + V4aOMoVXjPOpMHd6TJbmTTRx + -----END PRIVATE KEY----- -# (!!) Change default password database: connectionString: postgresql://postgres:postgres@postgres:5432/engine-wizard -# (!!) Change default credentials s3: - # Use a host-reachable endpoint so browser-facing presigned URLs work in local Docker. url: http://host.docker.internal:9000 username: GK0123456789abcdef01234567 password: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef bucket: engine-wizard region: garage -# (!!) Configure SMTP mail: enabled: false - name: + name: email: provider: smtp smtp: diff --git a/config/garage-ui.yaml b/config/garage-ui.yaml new file mode 100644 index 0000000..c9eb7cc --- /dev/null +++ b/config/garage-ui.yaml @@ -0,0 +1,27 @@ +server: + host: "::" + port: 8080 + environment: "production" + domain: "localhost" + protocol: "http" + root_url: "http://localhost:8081" + +garage: + endpoint: "http://garage:3900" + admin_endpoint: "http://garage:3903" + region: "garage" + +auth: + admin: + enabled: true + token: + enabled: false + session_max_age: 86400 + cookie_name: "garage_session" + cookie_secure: false + cookie_http_only: true + cookie_same_site: "lax" + +logging: + level: "info" + format: "text" diff --git a/config/garage.toml b/config/garage.toml index e4d79d9..5f0e6d5 100644 --- a/config/garage.toml +++ b/config/garage.toml @@ -13,5 +13,10 @@ s3_region = "garage" api_bind_addr = "[::]:3900" root_domain = ".s3.garage.localhost" +[s3_web] +bind_addr = "[::]:3902" +root_domain = ".web.garage.localhost" +index = "index.html" + [admin] api_bind_addr = "[::]:3903" diff --git a/config/plugin-proxy.conf b/config/plugin-proxy.conf new file mode 100644 index 0000000..ec1cddc --- /dev/null +++ b/config/plugin-proxy.conf @@ -0,0 +1,20 @@ +server { + listen 8080; + server_name _; + + location /plugins/ { + proxy_pass http://garage:3902; + proxy_set_header Host engine-wizard.web.garage.localhost; + + add_header Access-Control-Allow-Origin "*" always; + add_header Access-Control-Allow-Methods "GET, HEAD, OPTIONS" always; + add_header Access-Control-Allow-Headers "*" always; + add_header Access-Control-Expose-Headers "Content-Length, Content-Type, ETag" always; + + if ($request_method = OPTIONS) { + add_header Content-Length 0; + add_header Content-Type text/plain; + return 204; + } + } +} diff --git a/create-bucket.sh b/create-bucket.sh index 914d8a3..752f077 100755 --- a/create-bucket.sh +++ b/create-bucket.sh @@ -21,6 +21,18 @@ garage_exec() { docker compose exec -T "$GARAGE_SERVICE" /garage "$@" } +garage_exec_quiet() { + output_file="$(mktemp)" + if garage_exec "$@" >"$output_file" 2>&1; then + rm -f "$output_file" + return 0 + fi + + cat "$output_file" >&2 + rm -f "$output_file" + return 1 +} + if [ -z "$(docker compose ps -q --status running "$GARAGE_SERVICE" 2>/dev/null)" ]; then echo "Garage service '$GARAGE_SERVICE' is not running. Start the stack first with: docker compose up -d" >&2 exit 1 @@ -35,21 +47,29 @@ if [ -z "$NODE_ID" ]; then fi if printf '%s\n' "$STATUS" | grep -q "NO ROLE ASSIGNED"; then - garage_exec layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID" - garage_exec layout apply --version 1 + echo "Assigning single-node Garage layout..." + garage_exec_quiet layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID" + garage_exec_quiet layout apply --version 1 fi if ! garage_exec bucket info "$S3_BUCKET" >/dev/null 2>&1; then - garage_exec bucket create "$S3_BUCKET" + echo "Creating bucket '$S3_BUCKET'..." + garage_exec_quiet bucket create "$S3_BUCKET" fi if ! garage_exec key info "$S3_USERNAME" >/dev/null 2>&1; then - garage_exec key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes + echo "Importing access key '$S3_USERNAME'..." + garage_exec_quiet key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes fi -garage_exec bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_USERNAME" +echo "Granting bucket permissions..." +garage_exec_quiet bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_USERNAME" + +echo "Enabling public website access..." +garage_exec_quiet bucket website --allow "$S3_BUCKET" echo "Garage bootstrap completed." echo "S3 endpoint: $S3_URL" +echo "Public website endpoint: http://$S3_BUCKET.web.garage.localhost:9002/" echo "Bucket: $S3_BUCKET" echo "Access key: $S3_USERNAME" diff --git a/docker-compose.yml b/docker-compose.yml index 18bfdb5..c06006b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -31,6 +31,32 @@ services: environment: API_URL: http://localhost:3000/wizard-api + garage-ui: + image: noooste/garage-ui:${GARAGE_UI_VERSION} + restart: always + # Local admin UI only; do not expose this directly in a public deployment. + ports: + - 127.0.0.1:${GARAGE_UI_PORT}:8080 + depends_on: + - garage + volumes: + - ./config/garage-ui.yaml:/app/config.yaml:ro + environment: + GARAGE_UI_SERVER_ROOT_URL: http://localhost:${GARAGE_UI_PORT} + GARAGE_UI_GARAGE_ADMIN_TOKEN: ${GARAGE_ADMIN_TOKEN} + GARAGE_UI_AUTH_ADMIN_USERNAME: ${GARAGE_UI_ADMIN_USERNAME} + GARAGE_UI_AUTH_ADMIN_PASSWORD: ${GARAGE_UI_ADMIN_PASSWORD} + + plugin-proxy: + image: nginx:1.27-alpine + restart: always + ports: + - 127.0.0.1:9004:8080 + depends_on: + - garage + volumes: + - ./config/plugin-proxy.conf:/etc/nginx/conf.d/default.conf:ro + docworker: image: datastewardshipwizard/document-worker:${DSW_VERSION} restart: always @@ -83,6 +109,7 @@ services: # Do not expose this directly in a public deployment. ports: - 9000:3900 + - 9002:3902 - 9003:3903 # Change these defaults before sharing or reusing this setup. environment: diff --git a/example.env b/example.env index 7c4d127..5a48ac9 100644 --- a/example.env +++ b/example.env @@ -4,11 +4,8 @@ DSW_VERSION=4.31 # Upgrades may require migrations. POSTGRES_VERSION=17.5 GARAGE_VERSION=v2.2.0 - -# DSW DB configuration. -POSTGRES_USER=postgres -POSTGRES_PASSWORD=postgres -POSTGRES_DB=engine-wizard +GARAGE_UI_VERSION=v0.8.4 +GARAGE_UI_PORT=8081 # DSW S3 configuration. # Keep these aligned with the credentials imported by create-bucket.sh. @@ -20,6 +17,10 @@ S3_REGION=garage # Garage daemon secrets. # Change these defaults before sharing or reusing this setup. -GARAGE_RPC_SECRET=1111111111111111111111111111111111111111111111111111111111111111 -GARAGE_ADMIN_TOKEN=changeMeGarageAdminToken -GARAGE_METRICS_TOKEN=changeMeGarageMetricsToken +GARAGE_RPC_SECRET=insecure-demo-rpc-secret +GARAGE_ADMIN_TOKEN=insecure-demo-admin-token +GARAGE_METRICS_TOKEN=insecure-demo-metrics-token + +# Garage UI local admin login. +GARAGE_UI_ADMIN_USERNAME=admin +GARAGE_UI_ADMIN_PASSWORD=admin diff --git a/s3-data/.gitkeep b/s3-data/.gitkeep deleted file mode 100644 index e69de29..0000000 From 5e7d14d5e82a792c689bcdef75a92c6ccf2cb228 Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Tue, 30 Jun 2026 13:51:01 +0200 Subject: [PATCH 4/8] Restore demo Postgres env defaults --- example.env | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/example.env b/example.env index 5a48ac9..3fed1b8 100644 --- a/example.env +++ b/example.env @@ -7,6 +7,11 @@ GARAGE_VERSION=v2.2.0 GARAGE_UI_VERSION=v0.8.4 GARAGE_UI_PORT=8081 +# DSW DB configuration. +POSTGRES_USER=postgres +POSTGRES_PASSWORD=postgres +POSTGRES_DB=engine-wizard + # DSW S3 configuration. # Keep these aligned with the credentials imported by create-bucket.sh. S3_URL=http://host.docker.internal:9000 From 895851d3915a396a809ec2047f49b7dbf741f442 Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Tue, 30 Jun 2026 14:06:46 +0200 Subject: [PATCH 5/8] Simplify Garage example defaults --- README.md | 143 ++--------------------------------------- config/application.yml | 89 ++++++++++++++++--------- create-bucket.sh | 39 +++-------- example.env | 10 +-- 4 files changed, 76 insertions(+), 205 deletions(-) diff --git a/README.md b/README.md index 1fce349..d84bbd8 100644 --- a/README.md +++ b/README.md @@ -55,126 +55,11 @@ These are the only steps needed to run the local example. - Email: `albert.einstein@example.com` - Password: `password` -## Optional +## Notes -These steps are not required to start the example. They are only useful if you want to inspect the local Garage setup. - -1. Open Garage UI: - - [http://localhost:8081](http://localhost:8081) - -2. Log in to Garage UI with: - - - Username: `admin` - - Password: `admin` - -## What The Bootstrap Does - -`create-bucket.sh` is intended to be safe to rerun for this local setup. It: - -- assigns the single-node Garage layout if the node still has no role -- creates the configured S3 bucket if it does not exist -- imports the configured Garage access key if it does not exist -- grants read, write, and owner permissions on the bucket to that key - -## Optional Verification - -After setup, you can use this checklist to confirm the Garage-backed deployment works. - -### Infrastructure checks - -```bash -docker compose ps -docker compose logs garage --tail=100 -docker compose logs garage-ui --tail=100 -docker compose logs server --tail=100 -docker compose logs docworker --tail=100 -``` - -Expected results: - -- `garage` is running -- `garage-ui` is running -- `server` creates the S3 client successfully -- `docworker` starts without S3 errors -- no authentication, signing, or region errors appear in the logs - -### Application checks - -In the DSW UI, verify: - -1. the application opens and login works -2. a project file can be uploaded -3. the uploaded file can be downloaded -4. a document preview can be generated -5. a document template asset URL works, if applicable - -In Garage UI, verify: - -1. login works -2. the `engine-wizard` bucket is visible -3. file upload works -4. folder creation works - -If these checks pass, Garage is functioning as a drop-in S3-compatible backend for this example deployment. - -## Garage Notes - -### Current Local Model - -This repository uses Garage as a **local example only**. Our production setup still uses MinIO. - -- DSW uses a single S3 bucket: `engine-wizard` -- Garage S3 API is available locally on `localhost:9000` -- Garage website endpoint is available locally on `localhost:9002` -- Garage UI is available locally on `localhost:8081` -- A local `plugin-proxy` service exposes plugin assets on `localhost:9004` - -If you want to use locally stored plugin assets with this example, the plugin URL stored in DSW should point to the local proxy, for example: - -```text -http://localhost:9004/plugins/// -``` - -The actual plugin file must then be reachable at: - -```text -http://localhost:9004/plugins///plugin.js -``` - -### Buckets And Plugins - -Garage is S3-compatible, so the same bucket model you use with MinIO can also be used with Garage. - -For this local example: - -- DSW uses one shared bucket -- plugin assets can be exposed through the local `plugin-proxy` -- the plugin URL in the DSW database should point to the public plugin location, not to the Garage UI - -If you use Garage in a self-hosted deployment, keep the same basic rule as with MinIO: - -- private DSW data should stay private -- public plugins should live in a dedicated public location -- do not mix private application data and public plugin assets in the same public bucket - -### Self-Hosted Security Note - -If a self-hosted user wants to expose Garage outside localhost, they should not publish the raw Garage ports directly to the internet. - -Instead, they should put Garage behind an HTTPS reverse proxy or another equivalent security layer that provides: - -- TLS / HTTPS -- authentication for admin access -- controlled exposure of only the endpoints that should be public - -In practice, that usually means: - -- DSW stays behind its normal public HTTPS setup -- public plugin files are exposed through a dedicated public URL -- Garage admin or UI access is protected separately - -This repository does not try to provide a full production Garage deployment. It only illustrates how Garage can be used locally in the deployment example. +- Garage UI is available on [http://localhost:8081](http://localhost:8081) with username `admin` and password `admin` +- For local plugin testing, the plugin URL can point to `http://localhost:9004/plugins///` +- This repository is a local example only; if you self-host Garage publicly, place it behind HTTPS reverse proxy or another equivalent security layer ## Important Notes @@ -188,26 +73,6 @@ This repository does not try to provide a full production Garage deployment. It * Garage UI is configured with local basic auth defaults for this POC; change them before sharing the setup * Always use **strong passwords** and never use default values, **change the demo secrets** in `config/application.yml` and `.env` before using this anywhere except local testing -## Troubleshooting - -If something does not work: - -```bash -docker compose ps -docker compose logs garage --tail=200 -docker compose logs garage-ui --tail=200 -docker compose logs server --tail=200 -docker compose logs docworker --tail=200 -``` - -Common local issues: - -- Garage was started, but `create-bucket.sh` was not run yet -- Garage UI started, but Garage itself is not healthy yet -- `.env` values and the imported Garage key no longer match -- the server is still starting and has not reached a healthy state yet -- an old local data directory contains stale state from a previous attempt - ## Security Audit This repository is used to regularly check vulnerabilities in the latest release of Docker images. [Grype](https://github.com/anchore/grype) tool is used (see [security-audit.yml](.github/workflows/security-audit.yml) file and related GitHub Actions runs). Once a vulnerability is detected, we are notified and start working on a new hotfix version. You should **always use the latest version** you can find used in this repository. diff --git a/config/application.yml b/config/application.yml index 95db962..23244c2 100644 --- a/config/application.yml +++ b/config/application.yml @@ -1,46 +1,73 @@ +# (!!) Change the secret and RDA private key to your own general: clientUrl: http://localhost:8080/wizard - secret: insecure-demo-dsw-secret + secret: b7zFNUhFYIFLmDSFKFkp0Bnp8Z3uWGnH rsaPrivateKey: | - -----BEGIN PRIVATE KEY----- - MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDAip4YwYPi13kc - UERJiOQPLJfA9GhPvjSqBlyhfmerf+ruMComuWlZQ13QqeCyT1/kOcimRivGM/lC - jjnfQ3FoS5DMUwLrpvPJDH8T27hZ51WDh12bT6lHUhruju1xfQqHYIJ5qm6cigdU - 1951XLESKtJgpkt4uqBK0eDBXY5ZXBBE+MzRqzJKcpMPv6kXEJp0Lenm5Gqtma04 - +iVUh9KU2l1V9vwMqjlyF/DplaABR+YIKTwWKqjHuHTsSuZBAsU5tX0LsGU8uCJd - UYwztfHjQuKFmEWAmXnZeVaFxUqaxCUuKwUSmGjMORcC3zWr7LZoyY/9dS3TAXlW - ln1/5DIlAgMBAAECggEACvWlwh4uj6VZAELHxrFc3RanHK7//dBp1wCKgHbD5LMt - s2Xyc3nVhH5p9j91MGRSpP8IOM60HZGslbYN/UAthS5BjVMJG4WWFgX432LmWQrq - umEXwIg5FOIx8XPmEbIR/PRO+PXs9EqD2FFHssGv715lbMyKbZ84RVbLYE/rxl8Q - LNTYsgsC7y1gY1VMPTLMvwlqaXHCGNgA6aKp+cLGjFhLUcDmjM9aFIygQYvhBm3w - UZjip7vJk7h4m4lXlcjSUkzlyhh+nmM1UKPuXHTvbVWyQN0qU+MjOV6+AW2ZRfSC - xtuCv7JXmjDn6imr2n7aavFgRpdLA+Ns4Mp9/GQUkQKBgQD0dj6avKBG0ITMSIsS - l/tCn673W29387DL+4d7VihlcImhqmnoZZZx8D54XiDWUTBUOcOdNw7a8L8at5Uy - 04EM2jBFzT+3yxRbVgaeXz/8celhhYo5RWFRTpqkAGZPRRWhfXAcvCWQ8Bqxvrsz - wjHeixAQo9omMCeeFaxHk7g7VwKBgQDJoQj7iFp1Sl0k4ifTUb8uTyjRvNr7/ZE/ - jlhWrkbPTxtyrUr0vHe4FkdhgJOhSZwFIPByMfNAq4RkarUoQrYkiDM+VxksHuOz - qLlKzevIYOmJHU09E0LeuL3MznhvUJA20YNnKCXPsrBlRnKKjGDamQDr49/Ducvq - AcST/QyM4wKBgQCRoNMgmNed99/0Vg7LD9C1Wqz/W+/mdDxQ99E4rxW3gDftH7ih - ocriqyx7xg8nvQveUN7wC/mEgcBGaTCvK4IIJv50xpV7prXEx1WACNv0ReOynbu+ - 7E7SAdJrenV4KX5+Slx8MbTrR62PT0Dzz26n1Q3H6+QrtaoUKXMM3Ln2VwKBgQCF - sgICx6czntukWVGPDeV3D4ZBjLin3rsxKXHqCCVr0UiiAcZyAYZgymUKgfdiOyz5 - OZ42GmjxuMlQzHDTAWlFUAqqi5ZlaxYRSw+SSnjNpXU1NfjBRsRQaq9v3nh9Vy5J - KyO2gyiPUC0m+k95Dxwp1sqFn9QANHLB9+ohNunbgQKBgGisecKJ5Ir3XpbWVzjl - NF3DcZonHZ2onGIhO5P/+xR8iIW6St65l1jcmm/oCuOEQh8wMp12yz7Fs0ZmzWaq - jdrXdesJRMB23s+cPfLDJKv/sqZFQfxhvp4c8SIy79hyGhwj2NrhqzBcSTsU1LzB - V4aOMoVXjPOpMHd6TJbmTTRx - -----END PRIVATE KEY----- + -----BEGIN RSA PRIVATE KEY----- + MIIJKAIBAAKCAgEAyMMyCZe+/oihq702T+Yb2qiqgpmSmH9krH5lX5ZlT94KTiIn + KjNsMg+SLjIWedU+d5R8uUxqXX50MAXYwfi49NDpSi1hDTfxw2O/J03P+SuzwDTU + LMWQjBCT0t1QZOD7ODhPL7+UvZsoujHQJEwJ32Hj99z8HXOJ+pEvLwlmv3te1Tpr + SK9XvzjobVBz+XJL+WkQYFTAnhJHuM7Xzz3wQsiF60vCMe8yFqAlppd+6rybLdl5 + aXn6o4kzpDUI1/izRHW093sOkhAP+Esch7VJpEfLDpqDu2DDml/hL3KXFevau6mt + sWJZ0qyswh1BrdtwKKm2eqzjqw+Q0sThJUA66r1ZzHljqt8E3OSJlFHUgIZXXx1o + G8DNzu5jcjy6T85YSzDg8upIgNgoT14kgYUWnZsIKiXvhAX9WdkSVAHOkM0ADN8P + PTKekgAO4YHbM1hsraGpXJVJf6AdAwYwC4burLrdwiERafXV1FEpcnaSBezjAs3P + PmFGc7tCUocKfxHY3fJqU2n/Ngw8cDxZaqbPR1K+RXPzAYN15PBdtO8VnubKZTBw + x0tPXxm7rz87duP2szLpjaIEvks77As22sOptyzYO39iA57Vs4+onxV/0J/quDTv + Nci4ZdWSPABoyqtoiswrud5hzr75Zo0t8FutkoI9yQXyJJbgHFhSy5A7o0ECAwEA + AQKCAgAPyRFJIc5ZsyKLH2CC3cP46GQnyq11IicnTPTplCi5I+tJg3Py5IIwWXsT + +Mo9oB++RUWdSD3gxg2arQsDDWx1i2uccn78DBcNVu1zyLF+lQoYJc89Cfe4m6ET + 5mTo2avHGt5XEcg/NFD6euozwGrSTp8fIRgNtMT3l6zPFP0oNyoYt/Qxjd/nYDEe + T0IfLfZ9UN6BxIg0pZlQFatwVx1qEtw8yQMm6/jTQJpVQ/otuNC2xS+/tGCZtTMD + SUw9twdZbavh/aICqBKpYjaHESEOMrrrlgiflhcrLzjEg00P4zCaqtfGZ8ZKQ1t+ + E9DTO7rQ5686bBemnp2zhNlzFV7MbGyXHxYUh7hbSHGJwGrSbgDCZEVOllcgALTF + RsUxivBAmkyytYlT79QqSUrMAicf8ys2jW0suHuxT/gNqnb4iGtzoGv3OVXORsPt + TlxfTHIo00yF9OdCYTWrsi2XFwBgaasc7eH4KKTMpFOSjIq3iu+zZiLGv20z04Xi + iSnG4rN/nZS6MYTt50XM1lZb4XE5SWw35Az/BVLeQ4T2yanrNnQr87mdmlrh/LHy + ARInp1rHCaUlodWQCS9loEGGVPxGaetSUPhPPt3xMEdFvjvYHvp/rPRQc5iVZOC7 + +Av8gctVL4t4qsU1aPVVwmyyvZhW0cPrEDd8ap+usAwYW/iY1QKCAQEA+pJuZz+O + tDkUP//p7uNRmc54yWLVMZ1IdrTeqNVRJyHVNvxCzYX3GC2UOQz/6IECf/eCP3Z/ + /IVnYXRsoGKNght7it24hoSGsLSQO6gCYa7Xlmf72VyBWCluE+9KlR4I8LmvfYrC + tYGCazL0qbHwXCBdw0oL1HvSEEGMPt/TpdDkMtyvi+OgmKM6BBiCVC64qVjFGtB1 + CRxchbHN56mETa63m9wC6C/mIl+veysC0nE745h5i+ocu0LLEpfGA6tSp3XyNii1 + fc6LwmA66JsamRqg5IUdj4/AnHvO/XnKyVlzIwLTSWaoGlBHjGHHYqJsoQk/SryX + bAYBcJsB+MYUewKCAQEAzRyKjOPLMvL7IJjbnxNrl3k7IVAAR6o5RgNuMQ7b+Fmb + dPwi6SQJOHKnAHlC3lqUyCQxoW5tOeLE1A2At2LSfNJh8M3tiAhgoamA5yD5x1MC + 1RZK9ADmnDS3ezS3xixHSJl8dPsmgB7M8usrrIVD7xcVbMc6SjkbWPJD3rsJZHXz + b1phu0avUx9kN2Y0yjOGCPSQSDMjY6bOMpJaDmRcKCq1+z22OtuxOYlFttdvRhxf + sODv+FYN+estm2okko5lAhfnZwApp8APmMxKRFzebNMKE6gTKQ+OvquSexx6jakz + PTj8h0aOUQ/sMU/aXWSNDzjE8WHvB/jKr4p1lG5QcwKCAQBHt/3No41u3B+FqfQF + Ih9LDKPZ0GYBC2mgQGiynkvaDX7/e/iK5tswmmsgYS4mGDbk9azcCUjcuDo81Jv6 + 0/p40PDtcETOi8QZc7UMvivgVbDzJdQaBLYoy8LsBcOtJqmcbSljn4HXfbgcgrsw + jvOjCsiLivgT21TmvjwsCtiDT2ThgfTk70QaLC4A2hsDiVPldLkF0cCN2QZKZffv + ZL+vlGCNRc23M26FTCnn8QmfvjVMEWpIt7t9hcKmEbblefXwZHvtMH2yAppSCkUS + KsLUK6D9h3RMkPKqV22Jc4CtXOLtlHNxQObAkVXJQntVKfQ5q3HoLE6S5q0m09ce + o2NdAoIBAEVbflpP+Pk8gsCuD63F8l7hJqaIF+Aj5oQPSuaZ4LYlDtVLIT726YtF + 8cTOPFdWB/N5kbLF1Xd6gBY4jqr0ZrrApU+5QrX6JzrjR2xgkTXekxmdFTFXUKPH + i7fExtPjSyN4i0Y/WvB9+c22yijnU2Z4xszmvYY1s1KrcVnSo0VuQSgJRvz91PFl + rwywqe8p5wwGLvfLGw9qa1VcecleFG+cqbbQUluqyE3tZj6CsOTjWVRFa7/s0YMO + DHj+Nmm8btl7XctNmchYwK1pfz+Iq8nevwyQjBIZaap99+VbZvGaInJ0X6gaqqss + mDDAkNV919kq2CXLePmYHmmnh0Tu3JMCggEBAL1wmSfX55tVCQRSVl7ahNpqGY7C + 7Syyd0z8vE3F+czsd8H0j5KGBgKSUbtiJDhB1BSLcIwaUczJFiFXpegkTXnSz1Id + 2H+riQDon8rVkdP8JP6ny5ncj4PQ5ZAE7gwMPdtjhY4WChfHMXGgyfubNKIGi0pP + U8QopElb2l9/YhTpSvXXHXLYAq8bz3Kye6jQqatoDwRCuHkPc0OUeT9NdtW+x/Y4 + cdU04pDO6DEYKYcD8vgSlEzVCNqb6TvFfRsfQXI5TSkKuX3fTR2wgLR6CxWIejSE + 32jnWA89Tg9wD6ACZo8On4IIVH4oeFuNi8mSb/M3pQ9hTtqIUqwUw+AxD+I= + -----END RSA PRIVATE KEY----- +# (!!) Change default password database: connectionString: postgresql://postgres:postgres@postgres:5432/engine-wizard +# (!!) Change default password s3: url: http://host.docker.internal:9000 - username: GK0123456789abcdef01234567 - password: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef + username: garage + password: garagePassword bucket: engine-wizard region: garage +# (!!) Configure SMTP mail: enabled: false name: diff --git a/create-bucket.sh b/create-bucket.sh index 752f077..bdabb50 100755 --- a/create-bucket.sh +++ b/create-bucket.sh @@ -14,25 +14,13 @@ GARAGE_CAPACITY="${GARAGE_CAPACITY:-1G}" GARAGE_KEY_NAME="${GARAGE_KEY_NAME:-dsw-engine-wizard}" S3_URL="${S3_URL:-http://host.docker.internal:9000}" S3_BUCKET="${S3_BUCKET:-engine-wizard}" -S3_USERNAME="${S3_USERNAME:-GK0123456789abcdef01234567}" -S3_PASSWORD="${S3_PASSWORD:-0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef}" +S3_USERNAME="${S3_USERNAME:-garage}" +S3_PASSWORD="${S3_PASSWORD:-garagePassword}" garage_exec() { docker compose exec -T "$GARAGE_SERVICE" /garage "$@" } -garage_exec_quiet() { - output_file="$(mktemp)" - if garage_exec "$@" >"$output_file" 2>&1; then - rm -f "$output_file" - return 0 - fi - - cat "$output_file" >&2 - rm -f "$output_file" - return 1 -} - if [ -z "$(docker compose ps -q --status running "$GARAGE_SERVICE" 2>/dev/null)" ]; then echo "Garage service '$GARAGE_SERVICE' is not running. Start the stack first with: docker compose up -d" >&2 exit 1 @@ -47,29 +35,20 @@ if [ -z "$NODE_ID" ]; then fi if printf '%s\n' "$STATUS" | grep -q "NO ROLE ASSIGNED"; then - echo "Assigning single-node Garage layout..." - garage_exec_quiet layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID" - garage_exec_quiet layout apply --version 1 + garage_exec layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID" >/dev/null + garage_exec layout apply --version 1 >/dev/null fi if ! garage_exec bucket info "$S3_BUCKET" >/dev/null 2>&1; then - echo "Creating bucket '$S3_BUCKET'..." - garage_exec_quiet bucket create "$S3_BUCKET" + garage_exec bucket create "$S3_BUCKET" >/dev/null fi if ! garage_exec key info "$S3_USERNAME" >/dev/null 2>&1; then - echo "Importing access key '$S3_USERNAME'..." - garage_exec_quiet key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes + garage_exec key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes >/dev/null fi -echo "Granting bucket permissions..." -garage_exec_quiet bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_USERNAME" +garage_exec bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_USERNAME" >/dev/null -echo "Enabling public website access..." -garage_exec_quiet bucket website --allow "$S3_BUCKET" +garage_exec bucket website --allow "$S3_BUCKET" >/dev/null -echo "Garage bootstrap completed." -echo "S3 endpoint: $S3_URL" -echo "Public website endpoint: http://$S3_BUCKET.web.garage.localhost:9002/" -echo "Bucket: $S3_BUCKET" -echo "Access key: $S3_USERNAME" +echo "Garage is ready." diff --git a/example.env b/example.env index 3fed1b8..3d1040b 100644 --- a/example.env +++ b/example.env @@ -15,16 +15,16 @@ POSTGRES_DB=engine-wizard # DSW S3 configuration. # Keep these aligned with the credentials imported by create-bucket.sh. S3_URL=http://host.docker.internal:9000 -S3_USERNAME=GK0123456789abcdef01234567 -S3_PASSWORD=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef +S3_USERNAME=garage +S3_PASSWORD=garagePassword S3_BUCKET=engine-wizard S3_REGION=garage # Garage daemon secrets. # Change these defaults before sharing or reusing this setup. -GARAGE_RPC_SECRET=insecure-demo-rpc-secret -GARAGE_ADMIN_TOKEN=insecure-demo-admin-token -GARAGE_METRICS_TOKEN=insecure-demo-metrics-token +GARAGE_RPC_SECRET=garageRpcSecret +GARAGE_ADMIN_TOKEN=garageAdminToken +GARAGE_METRICS_TOKEN=garageMetricsToken # Garage UI local admin login. GARAGE_UI_ADMIN_USERNAME=admin From c49a27f1dcd49491908626f1c7e344cc004d195b Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Wed, 1 Jul 2026 11:25:51 +0200 Subject: [PATCH 6/8] Fix Garage plugin bucket bootstrap and proxy --- README.md | 1 + config/application.yml | 6 ++-- config/plugin-proxy.conf | 19 ++++++++++- create-bucket.sh | 73 ++++++++++++++++++++++++++++++---------- example.env | 10 ++++-- 5 files changed, 86 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index d84bbd8..ab72e67 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,7 @@ These are the only steps needed to run the local example. ## Notes - Garage UI is available on [http://localhost:8081](http://localhost:8081) with username `admin` and password `admin` +- `create-bucket.sh` prepares both the private `engine-wizard` bucket and the public `plugins` bucket - For local plugin testing, the plugin URL can point to `http://localhost:9004/plugins///` - This repository is a local example only; if you self-host Garage publicly, place it behind HTTPS reverse proxy or another equivalent security layer diff --git a/config/application.yml b/config/application.yml index 23244c2..80383a7 100644 --- a/config/application.yml +++ b/config/application.yml @@ -62,8 +62,10 @@ database: # (!!) Change default password s3: url: http://host.docker.internal:9000 - username: garage - password: garagePassword + # Garage access key ID must start with GK and contain 24 hex characters after it. + username: GK111111111111111111111111 + # Garage secret key must be 64 hex characters. + password: "1111111111111111111111111111111111111111111111111111111111111111" bucket: engine-wizard region: garage diff --git a/config/plugin-proxy.conf b/config/plugin-proxy.conf index ec1cddc..11e1c02 100644 --- a/config/plugin-proxy.conf +++ b/config/plugin-proxy.conf @@ -3,8 +3,25 @@ server { server_name _; location /plugins/ { + rewrite ^/plugins/(.*)$ /$1 break; proxy_pass http://garage:3902; - proxy_set_header Host engine-wizard.web.garage.localhost; + proxy_set_header Host plugins.web.garage.localhost; + + add_header Access-Control-Allow-Origin "*" always; + add_header Access-Control-Allow-Methods "GET, HEAD, OPTIONS" always; + add_header Access-Control-Allow-Headers "*" always; + add_header Access-Control-Expose-Headers "Content-Length, Content-Type, ETag" always; + + if ($request_method = OPTIONS) { + add_header Content-Length 0; + add_header Content-Type text/plain; + return 204; + } + } + + location / { + proxy_pass http://garage:3902; + proxy_set_header Host plugins.web.garage.localhost; add_header Access-Control-Allow-Origin "*" always; add_header Access-Control-Allow-Methods "GET, HEAD, OPTIONS" always; diff --git a/create-bucket.sh b/create-bucket.sh index bdabb50..dd653ef 100755 --- a/create-bucket.sh +++ b/create-bucket.sh @@ -2,9 +2,11 @@ set -eu -if [ -f .env ]; then +ENV_FILE="${ENV_FILE:-.env}" + +if [ -f "$ENV_FILE" ]; then set -a - . ./.env + . "./$ENV_FILE" set +a fi @@ -14,19 +16,61 @@ GARAGE_CAPACITY="${GARAGE_CAPACITY:-1G}" GARAGE_KEY_NAME="${GARAGE_KEY_NAME:-dsw-engine-wizard}" S3_URL="${S3_URL:-http://host.docker.internal:9000}" S3_BUCKET="${S3_BUCKET:-engine-wizard}" -S3_USERNAME="${S3_USERNAME:-garage}" -S3_PASSWORD="${S3_PASSWORD:-garagePassword}" +PLUGINS_BUCKET="${PLUGINS_BUCKET:-plugins}" +# Garage access key ID must start with GK and contain 24 hex characters after it. +S3_USERNAME="${S3_USERNAME:-GK111111111111111111111111}" +# Garage secret key must be 64 hex characters. +S3_PASSWORD="${S3_PASSWORD:-1111111111111111111111111111111111111111111111111111111111111111}" garage_exec() { - docker compose exec -T "$GARAGE_SERVICE" /garage "$@" + if [ -f "$ENV_FILE" ]; then + docker compose --env-file "$ENV_FILE" exec -T "$GARAGE_SERVICE" /garage "$@" + else + docker compose exec -T "$GARAGE_SERVICE" /garage "$@" + fi +} + +compose_ps() { + if [ -f "$ENV_FILE" ]; then + docker compose --env-file "$ENV_FILE" ps -q --status running "$GARAGE_SERVICE" + else + docker compose ps -q --status running "$GARAGE_SERVICE" + fi +} + +garage_exec_quiet() { + output_file="$(mktemp)" + if garage_exec "$@" >"$output_file" 2>&1; then + rm -f "$output_file" + return 0 + fi + + cat "$output_file" >&2 + rm -f "$output_file" + return 1 +} + +ensure_bucket() { + bucket_name="$1" + website_public="${2:-false}" + + if ! garage_exec bucket info "$bucket_name" >/dev/null 2>&1; then + garage_exec_quiet bucket create "$bucket_name" + fi + + garage_exec_quiet bucket allow --read --write --owner "$bucket_name" --key "$S3_USERNAME" + + if [ "$website_public" = "true" ]; then + garage_exec_quiet bucket website --allow "$bucket_name" + fi } -if [ -z "$(docker compose ps -q --status running "$GARAGE_SERVICE" 2>/dev/null)" ]; then +if [ -z "$(compose_ps 2>/dev/null)" ]; then echo "Garage service '$GARAGE_SERVICE' is not running. Start the stack first with: docker compose up -d" >&2 exit 1 fi -STATUS="$(garage_exec status)" +STATUS="$(garage_exec status 2>/dev/null)" NODE_ID="$(printf '%s\n' "$STATUS" | awk '/==== HEALTHY NODES ====/{flag=1; next} flag && $1 ~ /^[0-9a-f]+$/ {print $1; exit}')" if [ -z "$NODE_ID" ]; then @@ -35,20 +79,15 @@ if [ -z "$NODE_ID" ]; then fi if printf '%s\n' "$STATUS" | grep -q "NO ROLE ASSIGNED"; then - garage_exec layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID" >/dev/null - garage_exec layout apply --version 1 >/dev/null -fi - -if ! garage_exec bucket info "$S3_BUCKET" >/dev/null 2>&1; then - garage_exec bucket create "$S3_BUCKET" >/dev/null + garage_exec_quiet layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID" + garage_exec_quiet layout apply --version 1 fi if ! garage_exec key info "$S3_USERNAME" >/dev/null 2>&1; then - garage_exec key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes >/dev/null + garage_exec_quiet key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes fi -garage_exec bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_USERNAME" >/dev/null - -garage_exec bucket website --allow "$S3_BUCKET" >/dev/null +ensure_bucket "$S3_BUCKET" false +ensure_bucket "$PLUGINS_BUCKET" true echo "Garage is ready." diff --git a/example.env b/example.env index 3d1040b..ac55991 100644 --- a/example.env +++ b/example.env @@ -14,15 +14,19 @@ POSTGRES_DB=engine-wizard # DSW S3 configuration. # Keep these aligned with the credentials imported by create-bucket.sh. +# Garage access key ID must start with GK and contain 24 hex characters after it. +# Garage secret key must be 64 hex characters. S3_URL=http://host.docker.internal:9000 -S3_USERNAME=garage -S3_PASSWORD=garagePassword +S3_USERNAME=GK111111111111111111111111 +S3_PASSWORD=1111111111111111111111111111111111111111111111111111111111111111 S3_BUCKET=engine-wizard +PLUGINS_BUCKET=plugins S3_REGION=garage # Garage daemon secrets. # Change these defaults before sharing or reusing this setup. -GARAGE_RPC_SECRET=garageRpcSecret +# GARAGE_RPC_SECRET must be 64 hex characters. +GARAGE_RPC_SECRET=1111111111111111111111111111111111111111111111111111111111111111 GARAGE_ADMIN_TOKEN=garageAdminToken GARAGE_METRICS_TOKEN=garageMetricsToken From 7f1bba23cd7d0242bb0cb120f1eac6c698771d8c Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Fri, 3 Jul 2026 11:14:34 +0200 Subject: [PATCH 7/8] Document local plugin upload and URL shape --- README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ab72e67..5f318e6 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,11 @@ These are the only steps needed to run the local example. - Garage UI is available on [http://localhost:8081](http://localhost:8081) with username `admin` and password `admin` - `create-bucket.sh` prepares both the private `engine-wizard` bucket and the public `plugins` bucket -- For local plugin testing, the plugin URL can point to `http://localhost:9004/plugins///` +- For local plugin testing, upload plugin files to the `plugins` bucket under `//` +- The plugin file should be stored as `//plugin.js` +- The plugin URL in the DSW database should point to `http://localhost:9004///` +- Example DB value: `http://localhost:9004/6534f8f7-0d43-4c4d-9157-15af17f37649/1.2.0/` +- Usable plugin examples: [replies-importer-plugin](https://github.com/ds-wizard/replies-importer-plugin) and [madmp-importer-plugin](https://github.com/ds-wizard/madmp-importer-plugin) - This repository is a local example only; if you self-host Garage publicly, place it behind HTTPS reverse proxy or another equivalent security layer ## Important Notes From 748cae696dde2178c3b8f69401db30882c5ef647 Mon Sep 17 00:00:00 2001 From: krystofkomanec Date: Fri, 3 Jul 2026 11:17:07 +0200 Subject: [PATCH 8/8] Refresh Garage README details --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5f318e6..d4cd32b 100644 --- a/README.md +++ b/README.md @@ -15,8 +15,9 @@ This is an example deployment of the [Data Stewardship Wizard](https://ds-wizard It is intentionally set up as a **single-node local POC**: -- Garage runs in Docker on host ports `9000` (S3 API) and `9003` (Admin API) +- Garage runs in Docker on host ports `9000` (S3 API), `9002` (web endpoint), and `9003` (Admin API) - Garage UI runs on `127.0.0.1:8081` +- Plugin files are served locally through `127.0.0.1:9004` - DSW points to `http://host.docker.internal:9000` so presigned URLs are reachable from the browser - `create-bucket.sh` performs the one-time Garage bootstrap for this example @@ -73,7 +74,7 @@ These are the only steps needed to run the local example. * If you self-host Garage publicly, place it behind an HTTPS reverse proxy or another equivalent security layer and protect admin access * When you want to use DSW publicly, **set up HTTPS proxy** (e.g. Nginx) with a certificate for your domain and change default accounts * Set up volume mounted to PostgreSQL and Garage containers for persistent data -* Garage needs a one-time bootstrap after the stack starts. `create-bucket.sh` assigns the single-node layout, creates the bucket, imports the configured S3 key, and grants bucket permissions +* Garage needs a one-time bootstrap after the stack starts. `create-bucket.sh` assigns the single-node layout, creates the `engine-wizard` and `plugins` buckets, imports the configured S3 key, and grants bucket permissions * DSW uses `http://host.docker.internal:9000` as the S3 endpoint so both the DSW containers and the browser can reach the same local Garage endpoint * Garage UI is configured with local basic auth defaults for this POC; change them before sharing the setup * Always use **strong passwords** and never use default values, **change the demo secrets** in `config/application.yml` and `.env` before using this anywhere except local testing