Skip to content

README.md

Latest commit

 

History

History
217 lines (172 loc) · 7.25 KB

README.md

File metadata and controls

217 lines (172 loc) · 7.25 KB

Deploy Directory

This directory contains all configurations and artifacts necessary to containerize and orchestrate the SyncDesk application, including the complete observability stack (logging, metrics, and alerting).

📋 Structure

deploy/
├── Dockerfile                 # Application Docker image
├── entrypoint.sh             # Startup script (PostgreSQL healthcheck)
├── alertmanager/             # Alert manager
│   └── alertmanager.yml
├── grafana/                  # Visualization dashboards
│   ├── dashboards/
│   │   └── syncdesk-overview.json
│   └── provisioning/
│       ├── dashboards/
│       │   └── dashboards.yml
│       └── datasources/
│           └── datasources.yml
├── loki/                     # Logging system (log aggregation)
│   └── loki-config.yml
├── prometheus/               # Metrics collection and storage
│   ├── prometheus.yml
│   └── rules.yml
└── promtail/                 # Agent that collects logs for Loki
    └── promtail-config.yml

🐳 Dockerfile

Base image: Python 3.12 (slim)

What it does:

  1. Sets environment variables for optimization (no bytecode cache, unbuffered output)
  2. Installs system dependencies (build-essential, libpq-dev, curl)
  3. Installs Poetry as the dependency manager
  4. Copies pyproject.toml and poetry.lock, installing only production dependencies
  5. Copies application source code
  6. Exposes port 8000 (FastAPI)
  7. Sets up healthcheck and entrypoint

Optimized size: Uses Python slim + removes apt cache after installation.

🚀 Entrypoint Script

The entrypoint.sh script runs before the application starts and:

  1. Waits for PostgreSQL to be ready — Connects in a loop until the database responds
  2. Runs migrations (implemented in app/main.py on startup event)
  3. Starts the API — FastAPI with Uvicorn

This ensures the application never attempts to connect to an unavailable database.

📊 Observability Stack

Prometheus (prometheus/)

  • Port: 9090
  • Purpose: Collects and stores time-series metrics
  • Configuration:
    • prometheus.yml — Endpoints, scrape intervals, alertmanager targets
    • rules.yml — Alert rules (e.g., "API down for 5 minutes")
  • Scrapes: Collects metrics from the API at /metrics every 5 seconds

Grafana (grafana/)

  • Port: 3100 (mapped from 3000)
  • Purpose: Visualization of metrics and logs in dashboards
  • Configuration:
    • provisioning/datasources/datasources.yml — Defines Prometheus and Loki as data sources
    • provisioning/dashboards/dashboards.yml — Points to JSON dashboards
    • dashboards/syncdesk-overview.json — Customized dashboard with panels for health, latency, errors, etc.
  • Username/Password: Configurable via .env (default: admin/admin)

Loki (loki/)

  • Port: 3100
  • Purpose: Storage and querying of structured logs
  • Configuration:
    • Authentication disabled
    • Filesystem storage (/loki/chunks, /loki/rules)
    • Schema v13 with TSDB from 2026-04-09
    • Retention period: 24 hours by default

Promtail (promtail/)

  • Purpose: Agent that collects local logs and sends them to Loki
  • Dependencies: Loki and API (waits for both before starting)
  • Volumes:
    • ./logs/ — Collects JSON logs from the application
    • /var/run/docker.sock — Detects containers via Docker API
  • Configuration: promtail-config.yml defines parsing pipelines and labels

AlertManager (alertmanager/)

  • Port: 9093
  • Purpose: Manages Prometheus alerts, deduplication, and routing
  • Configuration: alertmanager.yml defines how alerts are grouped and routed
  • Integration: Pre-configured to receive alerts from prometheus.yml (targets at alertmanager:9093)

🔄 Startup Order (Docker Compose)

When you run docker compose up:

  1. Parallel (no dependencies):

    • PostgreSQL (db)
    • MongoDB (mongo)
    • Prometheus
    • AlertManager
    • Loki

  2. After (depends on db + mongo healthy):

    • API (api waits for db and mongo with condition: service_healthy)

  3. After (depends on API):

    • Promtail (collects logs generated by the API)
    • Grafana (consumes data from Prometheus, Loki, AlertManager)

🛠️ How to Use

Build the image

docker build -f deploy/Dockerfile -t syncdesk-api:latest .

Run with Docker Compose

docker compose up -d

Access services

Service URL Description
API http://localhost:8000 FastAPI (docs at /docs)
Prometheus http://localhost:9090 Raw metrics
Grafana http://localhost:3100 Dashboards (user: admin, password: from .env)
AlertManager http://localhost:9093 Alert manager
Loki http://localhost:3100 (via Grafana) Logs (no native UI)

View logs

# API logs
docker compose logs -f api

# Prometheus logs
docker compose logs -f prometheus

# All services logs
docker compose logs -f

Stop and cleanup

docker compose down          # Stop containers
docker compose down -v       # Stop + remove volumes (caution: destroys data!)

⚙️ Important Environment Variables

Defined in .env:

# PostgreSQL
POSTGRES_USER=user
POSTGRES_PASSWORD=password
POSTGRES_DB=syncdesk

# MongoDB
MONGO_INITDB_ROOT_USERNAME=user
MONGO_INITDB_ROOT_PASSWORD=password

# Grafana
GF_SECURITY_ADMIN_PASSWORD=admin

📝 Notes and Best Practices

  1. Healthchecks:

    • API and databases have configured healthchecks
    • Prometheus, Grafana, and Loki don't (infrastructure services)

  2. Persistence:

    • postgres_data — PostgreSQL data
    • mongo_data — MongoDB data
    • prometheus_data — Metrics history
    • grafana_data — Customized dashboards
    • loki_data — Aggregated logs

  3. Security in Production:

    • Enable authentication in Loki
    • Use secrets manager instead of .env
    • Configure CORS and authentication in the API
    • Enable TLS/HTTPS (nginx reverse proxy or similar)

  4. Custom Metrics:

    • The API exposes metrics at /metrics (FastAPI + Prometheus client)
    • Add new metrics with decorators in app/core/metrics/decorators.py

  5. Dashboards:

    • Customize dashboards/syncdesk-overview.json to monitor what matters
    • Import pre-built dashboards from Grafana Hub

🔍 Troubleshooting

API cannot connect to PostgreSQL

  • Verify that POSTGRES_HOST, POSTGRES_USER, POSTGRES_PASSWORD are correct in .env
  • Check entrypoint logs: docker compose logs api

Prometheus doesn't collect metrics from API

  • Confirm the API is running and exposing /metrics
  • Check prometheus.yml — target should be api:8000 (container name)
  • Visit http://localhost:9090/targets to see status

Grafana datasources won't connect

  • Confirm Prometheus is running: http://localhost:9090
  • In Grafana, go to Configuration > Data Sources and test the connection
  • Internal URLs must use container names (http://prometheus:9090, not http://localhost:9090)

Logs don't appear in Loki

  • Check if promtail is running: docker compose ps promtail
  • Confirm ./logs/ has .json files
  • Check promtail-config.yml — paths should point to correct logs