Skip to content

deployment.md

Latest commit

 

History

History
362 lines (283 loc) · 8.98 KB

deployment.md

File metadata and controls

362 lines (283 loc) · 8.98 KB

Deployment Guide

Quick Start

Prerequisites

Before running the system, ensure you have the following installed:

  • Docker & Docker Compose: For containerized services
  • Java 17: For Spring Boot services
  • Maven: For building Java services
  • Node.js & npm: For React frontend
  • Git: For version control

System Requirements

  • Memory: Minimum 8GB RAM recommended
  • Storage: At least 10GB free space
  • Network: Internet connection for pulling Docker images
  • Ports: The following ports should be available:
    • 3000 (React Web App)
    • 8080 (HAProxy Load Balancer)
    • 8081, 8083 (Write Layer Services)
    • 8082 (Public API)
    • 8087 (Coupon Service)
    • 5672 (RabbitMQ)
    • 6379 (Redis)
    • 8000 (DynamoDB Local)
    • 9092 (Kafka)

Quick Setup

  1. Clone the repository:
git clone <repository-url>
cd eventually-consistent-highly-scalable-system
  1. Start all services:
docker compose up -d

  1. Wait for services to initialize (approximately 2-3 minutes)

  2. Access the application:

Service Configuration

Docker Compose Services

The system uses Docker Compose to orchestrate multiple services:

Service Port Description Dependencies
web-app 3000 React frontend application keycloak
haproxy 8080 Load balancer and API gateway All backend services
write-layer 8081 Primary write service rabbitmq, dynamodb, keycloak
write-layer-2 8083 Secondary write service rabbitmq, dynamodb, keycloak
public-api 8082 Public read API redis, dynamodb
coupon-service 8087 Coupon management service redis, dynamodb, keycloak
keycloak 8090 Authentication server postgres
postgres 5432 Keycloak database -
rabbitmq 5672, 15672 Message broker -
redis 6379 Cache and session store -
dynamodb 8000 NoSQL database -
kafka 9092 Event streaming platform zookeeper
zookeeper 2181 Kafka coordination service -

Environment Configuration

Development Environment

  • Profile: dev
  • Database: DynamoDB Local
  • Authentication: Keycloak with default realm
  • Caching: Redis (single instance)
  • Message Queue: RabbitMQ (single instance)

Production Environment

  • Profile: prod
  • Database: AWS DynamoDB
  • Authentication: Production Keycloak instance
  • Caching: Redis Cluster
  • Message Queue: RabbitMQ Cluster or AWS SQS
  • Load Balancer: AWS ALB or production HAProxy

Service Startup Order

The services have dependencies and should start in the following order:

  1. Infrastructure Services:

    • PostgreSQL (for Keycloak)
    • Redis
    • DynamoDB Local
    • Zookeeper
    • Kafka
    • RabbitMQ

  2. Authentication Service:

    • Keycloak

  3. Backend Services:

    • Write Layer Services
    • Public API
    • Coupon Service

  4. Gateway and Frontend:

    • HAProxy
    • React Web App

Configuration Management

Environment Variables

Global Configuration

# Environment
ENVIRONMENT=development
SPRING_PROFILES_ACTIVE=dev

# Database
DYNAMODB_ENDPOINT=http://dynamodb:8000
AWS_ACCESS_KEY_ID=dummy
AWS_SECRET_ACCESS_KEY=dummy
AWS_REGION=us-east-1

# Cache
REDIS_HOST=redis
REDIS_PORT=6379

# Message Queue
RABBITMQ_HOST=rabbitmq
RABBITMQ_PORT=5672
RABBITMQ_USERNAME=guest
RABBITMQ_PASSWORD=guest

# Authentication
KEYCLOAK_SERVER_URL=http://haproxy:8080
KEYCLOAK_REALM=event-system

Service-Specific Configuration

Each service has its own configuration file:

  • Spring Boot Services: application.yml or application.properties
  • React App: Environment variables and config.json
  • HAProxy: haproxy.cfg
  • Docker Services: docker-compose.yml

Keycloak Setup

Default Configuration

  • Admin Username: admin
  • Admin Password: admin
  • Realm: event-system
  • Client ID: event-system-client

First-Time Setup

  1. Access Keycloak admin console: http://localhost:8080/auth/admin
  2. Login with admin/admin
  3. Create or import realm configuration
  4. Configure client settings
  5. Set up user roles and permissions

Database Initialization

DynamoDB Tables

The system automatically creates the following tables:

  • Users: User profile information
  • Coupons: Coupon data and status
  • Orders: Order information
  • Events: Event sourcing store

Initial Data

Some services may require initial data setup:

# Create sample data (optional)
docker exec -it write-layer java -jar app.jar --setup-data

Service Health Checks

Health Check Endpoints

Each service provides health check endpoints:

Service Health Check URL Expected Response
Write Layer http://localhost:8081/actuator/health {"status":"UP"}
Public API http://localhost:8082/health {"status":"healthy"}
Coupon Service http://localhost:8087/actuator/health {"status":"UP"}
Web App http://localhost:3000/health {"status":"ok"}

System Health Check Script

#!/bin/bash
# Check all services
./scripts/health-check.sh

# Check specific service
./scripts/health-check.sh write-layer

Scaling Configuration

Horizontal Scaling

Service Replication

Services can be scaled by increasing replicas:

# docker-compose.yml
services:
  write-layer:
    # ... configuration
    scale: 3  # Run 3 instances

Load Balancer Configuration

HAProxy automatically distributes load across service instances:

# haproxy.cfg
backend write_services
    balance roundrobin
    server write1 write-layer-1:8081 check
    server write2 write-layer-2:8081 check
    server write3 write-layer-3:8081 check

Vertical Scaling

Resource Limits

Configure resource limits for containers:

services:
  write-layer:
    deploy:
      resources:
        limits:
          memory: 1G
          cpus: '0.5'
        reservations:
          memory: 512M
          cpus: '0.25'

Auto-Scaling (Production)

For production deployments, consider:

  • Kubernetes Horizontal Pod Autoscaler
  • AWS ECS Service Auto Scaling
  • Docker Swarm Auto Scaling

Troubleshooting

Common Issues

Service Won't Start

  1. Check port conflicts: netstat -tulpn | grep <port>
  2. Verify Docker daemon is running: docker ps
  3. Check service logs: docker compose logs <service-name>
  4. Ensure dependencies are healthy: docker compose ps

Authentication Issues

  1. Verify Keycloak is running: curl http://localhost:8080/auth
  2. Check realm configuration
  3. Verify client credentials
  4. Check JWT token validation logs

Database Connectivity

  1. Test DynamoDB connection: aws dynamodb list-tables --endpoint-url http://localhost:8000
  2. Check Redis connectivity: redis-cli -h localhost ping
  3. Verify network connectivity between services

Message Queue Issues

  1. Check RabbitMQ management interface: http://localhost:15672
  2. Verify queue creation and message flow
  3. Check consumer connection status
  4. Monitor message processing logs

Log Analysis

Centralized Logging

# View all service logs
docker compose logs -f

# View specific service logs
docker compose logs -f write-layer

# Search logs for errors
docker compose logs | grep ERROR

Log Correlation

Services use correlation IDs for distributed tracing:

# Find all logs for a specific request
docker compose logs | grep "correlation-id: 12345"

Performance Monitoring

Key Metrics to Monitor

  • Response Times: API endpoint latency
  • Throughput: Requests per second
  • Error Rates: 4xx/5xx response codes
  • Resource Usage: CPU, memory, disk I/O
  • Queue Depth: Message queue backlog

Monitoring Tools

  • Application Metrics: Spring Boot Actuator endpoints
  • System Metrics: Docker stats, system monitoring
  • Custom Dashboards: Grafana dashboards (if configured)

Backup and Recovery

Data Backup

# Create full system backup
./scripts/backup-system.sh

# Backup specific components
./scripts/backup-database.sh
./scripts/backup-configuration.sh

Disaster Recovery

# Restore from backup
./scripts/restore-system.sh /path/to/backup

# Restore specific service
./scripts/restore-service.sh write-layer /path/to/backup

Production Deployment Considerations

Security Hardening

  • Change default passwords
  • Enable TLS/SSL certificates
  • Configure firewall rules
  • Set up VPN or private networks
  • Enable audit logging

High Availability

  • Deploy across multiple availability zones
  • Configure database replication
  • Set up service redundancy
  • Implement health checks and auto-recovery

Performance Optimization

  • Configure connection pooling
  • Enable caching strategies
  • Optimize database queries
  • Set appropriate timeout values
  • Monitor and tune JVM settings