Skip to content

A secure, high-performance API key management system with dual-hash strategy, distributed caching, and automatic expiration.

License

Notifications You must be signed in to change notification settings

snowmerak/inventory

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

46 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Inventory API - API Key Management System

Tests Docker Bun

A secure, high-performance API key management system with dual-hash strategy, distributed caching, and automatic expiration.

✨ Features

  • πŸ”’ Dual Hash Security: SHA-1 (fast indexing) + Argon2id (secure verification)
  • ⚑ High Performance: Redis cache-first strategy with 95%+ hit rate
  • πŸ”„ Distributed Locking: Race condition prevention with Redis locks
  • πŸ“Š Auto Expiration: MongoDB TTL index for automatic cleanup
  • 🚦 Rate Limiting: Configurable IP-based rate limiting (100 req/min default)
  • πŸ“ˆ Monitoring: Built-in metrics tracking and reporting
  • 🐳 Docker Ready: Complete Docker Compose setup
  • βœ… Fully Tested: Comprehensive integration test suite
  • πŸ“ Structured Logging: JSON logging with LogTape, sensitive data masking, and performance tracking

πŸ—οΈ Tech Stack

πŸš€ Quick Start

Prerequisites

  • Docker and Docker Compose
  • That's it! Everything else runs in containers.

One-Command Setup

docker-compose up -d

That's all! The system automatically:

  • βœ… Starts MongoDB with replica set
  • βœ… Starts Redis cache
  • βœ… Creates TTL index for expiration
  • βœ… Builds and launches the API server

Server running at: http://localhost:3030

Verify Installation

# Check health
curl http://localhost:3030/health

# Expected output:
# {
#   "status": "healthy",
#   "services": {
#     "mongodb": "connected",
#     "redis": "connected"
#   },
#   "timestamp": "2025-10-04T..."
# }

Your First API Key

# Publish a new API key
curl -X POST http://localhost:3030/api/keys/publish \
  -H "Content-Type: application/json" \
  -d '{
    "itemKey": "myapp://users/user123",
    "permission": ["read", "write"],
    "expiresAt": "2025-12-31T23:59:59Z",
    "maxUses": 1000
  }'

# Save the returned apiKey, you'll only see it once!

# Validate the key
curl -X POST http://localhost:3030/api/keys/validate \
  -H "Content-Type: application/json" \
  -d '{
    "apiKey": "YOUR_API_KEY_HERE"
  }'

πŸ“– Table of Contents

  1. Architecture
  2. API Documentation
  3. Development
  4. Testing
  5. Docker Deployment
  6. Configuration
  7. Monitoring
  8. Logging System
  9. Security
  10. Troubleshooting
  11. Contributing

πŸ›οΈ Architecture

System Overview

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                        Client Request                        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                      β”‚
                      β–Ό
         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
         β”‚   Rate Limiter (Redis) β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                      β”‚
                      β–Ό
         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
         β”‚  Distributed Lock      β”‚
         β”‚  (Redis)               β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                      β”‚
                      β–Ό
         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
         β”‚   Cache Layer (Redis)  β”‚
         β”‚   TTL: 15 minutes      β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                      β”‚ Cache Miss
                      β–Ό
         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
         β”‚   MongoDB              β”‚
         β”‚   - Replica Set        β”‚
         β”‚   - TTL Index          β”‚
         β”‚   - Dual Hash Storage  β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Dual Hash Strategy

  1. SHA-1 Hash (8 bytes): Fast indexing for initial lookup
  2. Argon2id Hash: Secure verification of candidates
// Publishing
apiKey (64 chars) 
  β”œβ”€> SHA-1 β†’ searchableHash (indexing)
  └─> Argon2id β†’ hashedApiKey (verification)

// Validation
apiKey β†’ SHA-1 β†’ Find candidates β†’ Argon2id verify each

Data Flow

Publishing Flow:

1. Generate 64-char API key
2. Create SHA-1 hash (searchableHash)
3. Create Argon2id hash (hashedApiKey)
4. Check for duplicates
5. Store in MongoDB
6. Return original key (once!)

Validation Flow:

1. Check rate limit (Redis)
2. Acquire distributed lock
3. Check cache (Redis)
   β”œβ”€ Hit β†’ Return cached data
   └─ Miss ↓
4. Generate SHA-1 from provided key
5. Query MongoDB by searchableHash
6. Verify candidates with Argon2id
7. Check expiration & usage limits
8. Increment usage counter
9. Update cache
10. Release lock

πŸ“‘ API Documentation

Base URL

http://localhost:3030

Public Endpoints

1. Validate API Key

Endpoint: POST /api/keys/validate

Rate Limit: 100 requests/minute per IP

Request:

{
  "apiKey": "your-64-character-api-key"
}

Success Response (200):

{
  "success": true,
  "data": {
    "valid": true,
    "itemKey": "myapp://users/user123",
    "permission": ["read", "write"],
    "expiresAt": "2025-12-31T23:59:59.000Z",
    "usedCount": 5,
    "maxUses": 1000
  }
}

Error Responses:

  • 400 - Invalid request
  • 401 - Invalid API key
  • 403 - Expired or exhausted
  • 404 - Not found
  • 429 - Rate limit exceeded

Private Endpoints

2. Publish API Key

Endpoint: POST /api/keys/publish

Request:

{
  "itemKey": "myapp://users/user123?action=read",
  "permission": ["read", "write"],
  "expiresAt": "2025-12-31T23:59:59Z",
  "maxUses": 1000
}

Item Key Format: <scheme>://<service>/<key>?<query>

Success Response (200):

{
  "success": true,
  "data": {
    "apiKey": "abc...xyz",
    "itemKey": "myapp://users/user123?action=read",
    "permission": ["read", "write"],
    "publishedAt": "2025-10-04T12:00:00.000Z",
    "expiresAt": "2025-12-31T23:59:59.000Z",
    "maxUses": 1000
  }
}

⚠️ Important: The apiKey is shown only once!


Admin Endpoints

⚠️ Security: These endpoints should be protected in production!

3. List Keys by Item

GET /admin/keys/by-item?itemKey=<encoded-item-key>

4. Get Key Statistics

GET /admin/keys/stats/:hashedApiKey

5. Revoke API Key

DELETE /admin/keys/revoke/:hashedApiKey

6. Get Overall Statistics

GET /admin/stats

7. Get Metrics

GET /admin/metrics

Response:

{
  "success": true,
  "data": {
    "keysPublished": 150,
    "keysValidated": 5430,
    "cacheHits": 5200,
    "cacheMisses": 230,
    "cacheHitRate": 95.76,
    "avgValidationTime": 12.5,
    "rateLimitErrors": 15
  }
}

8. Cleanup Expired Keys

POST /admin/keys/cleanup

Note: MongoDB TTL index handles this automatically.

Health Check

GET /health

Response:

{
  "status": "healthy",
  "services": {
    "mongodb": "connected",
    "redis": "connected"
  },
  "timestamp": "2025-10-04T12:34:56.789Z"
}

For detailed API documentation, see docs/API.md.


πŸ’» Development

Local Development Setup

Option 1: Full Docker (Recommended)

# Everything in Docker
docker-compose up -d --build

Option 2: Hybrid (Infrastructure in Docker, App Local)

# 1. Start only infrastructure
docker-compose up -d mongodb redis setup

# 2. Install dependencies
bun install

# 3. Generate Prisma client
bun run db:generate

# 4. Set environment variables
# Windows PowerShell
$env:PORT="3030"
$env:MONGODB_URI="mongodb://localhost:27017/inventory?replicaSet=rs0&directConnection=true"
$env:REDIS_HOST="localhost"
$env:REDIS_PORT="6379"
$env:REDIS_PASSWORD="redis123"
$env:NODE_ENV="development"

# 5. Start dev server with hot reload
bun run dev

Available Scripts

# Development
bun run dev              # Start with hot reload
bun run start            # Production mode

# Database
bun run db:generate      # Generate Prisma client
bun run db:push          # Push schema to database
bun run db:studio        # Open Prisma Studio

# Testing
bun test                 # Run all tests
bun test --watch         # Watch mode

# Docker
bun run docker:up        # Start all services
bun run docker:down      # Stop services
bun run docker:logs      # View logs
bun run docker:restart   # Restart app

Project Structure

inventory/
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ config/
β”‚   β”‚   └── env.ts                 # Environment configuration
β”‚   β”œβ”€β”€ db/
β”‚   β”‚   β”œβ”€β”€ prisma.ts             # Prisma client & health check
β”‚   β”‚   └── api-key-repository.ts # Database operations
β”‚   β”œβ”€β”€ cache/
β”‚   β”‚   β”œβ”€β”€ redis.ts              # Redis client
β”‚   β”‚   β”œβ”€β”€ distributed-lock.ts   # Distributed locking
β”‚   β”‚   └── api-key-cache.ts      # Cache operations
β”‚   β”œβ”€β”€ middleware/
β”‚   β”‚   └── rate-limiter.ts       # Rate limiting
β”‚   β”œβ”€β”€ monitoring/
β”‚   β”‚   └── metrics.ts            # Metrics collection
β”‚   β”œβ”€β”€ services/
β”‚   β”‚   β”œβ”€β”€ publisher.ts          # API key publishing
β”‚   β”‚   β”œβ”€β”€ validator.ts          # API key validation
β”‚   β”‚   └── admin.ts              # Admin operations
β”‚   β”œβ”€β”€ routes/
β”‚   β”‚   β”œβ”€β”€ api.ts                # Public API routes
β”‚   β”‚   └── admin.ts              # Admin routes
β”‚   β”œβ”€β”€ types/
β”‚   β”‚   β”œβ”€β”€ api.ts                # Request/response types
β”‚   β”‚   └── errors.ts             # Custom error classes
β”‚   β”œβ”€β”€ utils/
β”‚   β”‚   β”œβ”€β”€ crypto.ts             # Hashing utilities
β”‚   β”‚   └── logger.ts             # LogTape logging configuration
β”‚   └── index.ts                   # Application entry point
β”œβ”€β”€ tests/
β”‚   └── api.test.ts               # Integration tests
β”œβ”€β”€ prisma/
β”‚   └── schema.prisma             # Database schema
β”œβ”€β”€ scripts/
β”‚   └── setup-ttl-index.js        # TTL index setup
β”œβ”€β”€ docs/                          # Additional documentation
β”œβ”€β”€ docker-compose.yml
β”œβ”€β”€ Dockerfile
└── package.json

πŸ§ͺ Testing

Run Tests

# Make sure services are running
docker-compose up -d

# Run all tests
bun test

# Watch mode
bun test --watch

# Specific test file
bun test tests/api.test.ts

Test Coverage

  • βœ… Health checks
  • βœ… API key publishing (with validation)
  • βœ… API key validation (cache + DB)
  • βœ… Usage counter increment
  • βœ… Admin endpoints
  • βœ… Rate limiting
  • βœ… Error handling

Test Results:

βœ“ 13 tests passed
βœ“ 51 assertions
βœ“ Completed in 1.02s

Manual Testing

With cURL

# Publish
curl -X POST http://localhost:3030/api/keys/publish \
  -H "Content-Type: application/json" \
  -d '{
    "itemKey": "myapp://item/123",
    "permission": ["read"],
    "expiresAt": "2025-12-31T23:59:59Z",
    "maxUses": 100
  }'

# Validate
curl -X POST http://localhost:3030/api/keys/validate \
  -H "Content-Type: application/json" \
  -d '{"apiKey": "YOUR_KEY_HERE"}'

# Metrics
curl http://localhost:3030/admin/metrics | jq

With PowerShell

# Publish
$body = @{
  itemKey = "myapp://item/123"
  permission = @("read", "write")
  expiresAt = "2025-12-31T23:59:59Z"
  maxUses = 100
} | ConvertTo-Json

Invoke-RestMethod -Uri "http://localhost:3030/api/keys/publish" `
  -Method Post -ContentType "application/json" -Body $body

For detailed testing guide, see docs/TESTING.md.


🐳 Docker Deployment

Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚             docker-compose.yml                  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚   mongodb   β”‚    redis     β”‚  setup   β”‚  app   β”‚
β”‚  (rs0)      β”‚  (cache)     β”‚ (init)   β”‚ (api)  β”‚
β”‚  :27017     β”‚   :6379      β”‚  (once)  β”‚ :3030  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Services

Service Purpose Port Health Check
mongodb Database with replica set 27017 mongosh --eval "rs.status()"
redis Cache & distributed locks 6379 redis-cli ping
setup Initialize replica set & indexes - One-time only
app API server 3030 GET /health

Commands

# Start all services
docker-compose up -d --build

# View logs
docker-compose logs -f app        # App logs only
docker-compose logs -f            # All services

# Restart app (after code changes)
docker-compose restart app

# Stop everything
docker-compose down

# Stop and remove volumes (fresh start)
docker-compose down -v

# Check service status
docker-compose ps

# Execute commands in containers
docker-compose exec mongodb mongosh
docker-compose exec redis redis-cli

Environment Variables

Create a .env file in the project root:

# Application
PORT=3030
NODE_ENV=production

# MongoDB
MONGODB_URI=mongodb://mongodb:27017/inventory?replicaSet=rs0&directConnection=true

# Redis
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_PASSWORD=redis123

# Optional: Rate Limiting
RATE_LIMIT_WINDOW=60000          # 1 minute in ms
RATE_LIMIT_MAX_REQUESTS=100      # Max requests per window

Production Considerations

MongoDB Replica Set

For production, use a proper 3-node replica set:

# docker-compose.prod.yml
services:
  mongodb-primary:
    image: mongo:7
    command: mongod --replSet rs0 --bind_ip_all --keyFile /data/mongodb-keyfile
    # ... (see docs/DOCKER.md for full config)
  
  mongodb-secondary:
    # ...
  
  mongodb-arbiter:
    # ...

Security

  • βœ… Enable MongoDB authentication
  • βœ… Use strong Redis password
  • βœ… Set up TLS/SSL certificates
  • βœ… Configure firewall rules
  • βœ… Use Docker secrets for credentials

Scaling

# Scale app instances
docker-compose up -d --scale app=3

# Use load balancer (nginx, traefik)
# ...

For detailed Docker guide, see docs/DOCKER.md.


βš™οΈ Configuration

Environment Variables Reference

Variable Required Default Description
PORT No 3030 Server port
NODE_ENV No development Environment (development/production)
MONGODB_URI Yes - MongoDB connection string with replica set
REDIS_HOST Yes - Redis server host
REDIS_PORT No 6379 Redis server port
REDIS_PASSWORD Yes - Redis password
REDIS_DB No 0 Redis database number
RATE_LIMIT_WINDOW No 60000 Rate limit window (ms)
RATE_LIMIT_MAX_REQUESTS No 100 Max requests per window

MongoDB Connection String

Local Development (Docker)

mongodb://localhost:27017/inventory?replicaSet=rs0&directConnection=true

Production (3-node replica set)

mongodb://user:pass@mongo1:27017,mongo2:27017,mongo3:27017/inventory?replicaSet=rs0&authSource=admin

Important Query Parameters

  • replicaSet=rs0 - Required for transactions
  • directConnection=true - For single-node replica set (dev only)
  • authSource=admin - For authentication
  • retryWrites=true - Enable write retry (default)
  • w=majority - Write concern level

Redis Configuration

// src/cache/redis.ts
const redis = new Redis({
  host: process.env.REDIS_HOST,
  port: parseInt(process.env.REDIS_PORT || "6379"),
  password: process.env.REDIS_PASSWORD,
  db: parseInt(process.env.REDIS_DB || "0"),
  retryStrategy(times) {
    const delay = Math.min(times * 50, 2000);
    return delay;
  }
});

Logging

Structured JSON logging in production:

console.log(JSON.stringify({
  timestamp: new Date().toISOString(),
  level: "info",
  message: "API key validated",
  apiKey: hash.substring(0, 16),
  itemKey: key.itemKey
}));

πŸ“Š Monitoring

Health Check Endpoint

curl http://localhost:3030/health

Response:

{
  "status": "ok",
  "timestamp": "2025-01-20T10:30:00.000Z",
  "services": {
    "mongodb": "connected",
    "redis": "connected"
  }
}

Status Codes:

  • 200 - All services healthy
  • 503 - One or more services unavailable

Metrics Endpoint

curl http://localhost:3030/admin/metrics

Response:

{
  "activeKeys": 42,
  "expiredKeys": 15,
  "exhaustedKeys": 8,
  "totalValidations": 1523,
  "cacheHitRate": 0.87,
  "timestamp": "2025-01-20T10:30:00.000Z"
}

Key Metrics to Monitor

Metric Description Alert Threshold
cacheHitRate Redis cache efficiency < 0.7 (70%)
activeKeys Available API keys < 10
exhaustedKeys Keys at max usage Growing rapidly
Response time API latency > 500ms
Error rate Failed requests > 1%

Integration with Monitoring Tools

Prometheus

// Add prometheus client
import { Registry, Counter, Histogram } from 'prom-client';

const register = new Registry();
const httpRequestDuration = new Histogram({
  name: 'http_request_duration_ms',
  help: 'Duration of HTTP requests in ms',
  labelNames: ['method', 'route', 'status_code'],
  registers: [register]
});

Grafana Dashboard

Create dashboards for:

  • Request rate & latency
  • Cache hit rate
  • Active vs expired keys
  • Error rate by endpoint

For detailed monitoring setup, see docs/MONITORING.md.


πŸ“ Logging System

Overview

The system uses LogTape for enterprise-grade structured logging with the following features:

  • Structured JSON Output: All logs are formatted as JSON for easy parsing and analysis
  • Sensitive Data Masking: Automatic masking of API keys, hashes, passwords, and MongoDB credentials
  • Performance Tracking: Built-in timing utilities for measuring operation duration
  • Caller Information: Every log includes the calling function for full traceability
  • Categorized Loggers: Separate loggers for app, database, cache, service, API, and admin operations

Log Format

{
  "timestamp": "2025-10-05T10:35:51.693Z",
  "level": "info",
  "category": "inventory.api",
  "message": ["API key validated successfully"],
  "itemKey": "myapp://users/123",
  "usedCount": 5,
  "maxUses": 1000,
  "clientIp": "192.168.1.100",
  "caller": "apiRoute.validate"
}

Log Categories

Category Purpose Examples
inventory.app Application lifecycle Startup, shutdown, health checks
inventory.db Database operations Queries, connections, Prisma operations
inventory.cache Redis operations Cache hits/misses, lock operations
inventory.service Business logic Key publishing, validation, admin tasks
inventory.api Public API endpoints Request/response, rate limiting
inventory.admin Admin operations Stats, metrics, revocation

Log Levels

  • DEBUG: Detailed information for debugging (cache hits, lock acquisition)
  • INFO: General informational messages (successful operations)
  • WARN: Warning messages (rate limits, validation failures)
  • ERROR: Error messages (unexpected errors, database failures)

Sensitive Data Masking

The logging system automatically masks sensitive information:

// API Keys: Shows only first 8 characters
"apiKey-abc...def123" β†’ "apiKey-abc***** (masked)"

// Hashes: Shows only first 16 characters  
"$argon2id$v=19$..." β†’ "$argon2id$v=19$m***** (masked)"

// Passwords: Fully masked
"mySecretPassword" β†’ "******* (masked)"

// MongoDB URIs: Credentials removed
"mongodb://user:pass@host/db" β†’ "mongodb://***:***@host/db"

Performance Tracking

Built-in performance timing utilities:

import { performance } from './utils/logger'

// Start timing
const timer = performance.start('operation.name')

try {
  // ... your code ...
  
  // End timing with success
  timer.end({ success: true, additionalData: value })
} catch (error) {
  // Track error with timing
  timer.error(error, { context: 'additional info' })
}

Viewing Logs

Development (Docker)

# Follow all logs
docker-compose logs -f app

# Filter by category
docker-compose logs app | grep "inventory.api"

# Filter by level
docker-compose logs app | grep "\"level\":\"error\""

Production

# Using jq for JSON parsing
docker-compose logs app | jq 'select(.level == "error")'

# Filter by category
docker-compose logs app | jq 'select(.category | contains("cache"))'

# Show only messages and timestamps
docker-compose logs app | jq '{timestamp, message, caller}'

Log Analysis

Common Queries

# Find all errors in the last hour
docker-compose logs --since 1h app | jq 'select(.level == "error")'

# Track API key validation performance
docker-compose logs app | jq 'select(.message[0] | contains("validated")) | {timestamp, duration: .durationMs}'

# Monitor cache hit rate
docker-compose logs app | jq 'select(.message[0] | contains("Cache")) | .message'

# Find rate limit violations
docker-compose logs app | jq 'select(.message[0] | contains("Rate limit")) | {timestamp, clientIp, caller}'

Integration with Log Aggregation

The structured JSON format integrates seamlessly with:

  • ELK Stack (Elasticsearch, Logstash, Kibana)
  • Grafana Loki - Log aggregation system
  • Datadog - Monitoring and analytics
  • CloudWatch - AWS logging service
  • Splunk - Log analysis platform

Example Logstash configuration:

input {
  docker {
    type => "inventory-api"
  }
}

filter {
  json {
    source => "message"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "inventory-logs-%{+YYYY.MM.dd}"
  }
}

Logger Configuration

The logger is configured in src/utils/logger.ts:

import { configureLogger, logger, performance } from './utils/logger'

// Initialize at application startup
await configureLogger()

// Use in your code
logger.api.info('Request received', {
  method: 'POST',
  path: '/api/keys/validate',
  caller: 'apiRoute.validate'
})

// Performance tracking
const timer = performance.start('db.query')
// ... operation ...
timer.end({ success: true, rows: 42 })

Custom Logging

Add custom context to any log:

logger.service.info('Processing batch', {
  batchId: 'batch-123',
  itemCount: 50,
  startTime: new Date(),
  caller: 'BatchProcessor.process'
})

For detailed logging documentation, see docs/LOGGING.md.


πŸ”’ Security

Authentication Strategy

This system uses dual hashing for security:

  1. Argon2id - Memory-hard, resistant to GPU/ASIC attacks
  2. SHA-1 - Fast hash for Redis key generation
// Publishing (once)
const argonHash = await argon2.hash(apiKey);  // Store in MongoDB
const sha1Hash = createHash('sha1').update(apiKey).digest('hex');  // Cache key

// Validation (every request)
const sha1Hash = createHash('sha1').update(apiKey).digest('hex');
const cached = await redis.get(`apikey:${sha1Hash}`);  // Fast lookup
if (!cached) {
  const dbKey = await db.findUnique({ where: { apiKeyHash } });
  await argon2.verify(dbKey.apiKeyHash, apiKey);  // Verify with Argon2
}

Security Best Practices

1. API Key Handling

  • βœ… Never log full API keys - Only log hash prefixes
  • βœ… Generate cryptographically secure keys - Use crypto.randomBytes(32)
  • βœ… Validate input - TypeBox schemas on all endpoints
  • βœ… Rate limiting - Prevent brute force attacks

2. Network Security

# docker-compose.yml
services:
  mongodb:
    networks:
      - backend
    # Don't expose port publicly in production
    
  redis:
    networks:
      - backend
    command: redis-server --requirepass ${REDIS_PASSWORD}
    
  app:
    networks:
      - backend
    ports:
      - "3030:3030"  # Only expose app port

3. Environment Variables

# NEVER commit .env to git
echo ".env" >> .gitignore

# Use strong passwords
REDIS_PASSWORD=$(openssl rand -base64 32)

# Rotate credentials regularly

4. MongoDB Security

// Enable authentication
db.createUser({
  user: "admin",
  pwd: "strong-password",
  roles: ["readWrite", "dbAdmin"]
});

// Use keyfile for replica set
openssl rand -base64 756 > mongodb-keyfile
chmod 400 mongodb-keyfile

5. Production Checklist

  • Enable MongoDB authentication
  • Use TLS/SSL certificates
  • Set up firewall rules
  • Configure CORS properly
  • Enable request logging
  • Set up intrusion detection
  • Regular security audits
  • Dependency vulnerability scanning (bun audit)

πŸ”§ Troubleshooting

Common Issues

1. MongoDB Replica Set Error

Error:

MongoServerError: Transaction numbers are only allowed on a replica set member or mongos

Solution:

# Check replica set status
docker-compose exec mongodb mongosh --eval "rs.status()"

# Reinitialize if needed
docker-compose down -v
docker-compose up -d

2. Redis Connection Refused

Error:

Error: connect ECONNREFUSED 127.0.0.1:6379

Solution:

# Check Redis is running
docker-compose ps redis

# Check logs
docker-compose logs redis

# Restart Redis
docker-compose restart redis

3. CRLF Line Ending Issues (Windows)

Error:

/bin/sh: 1: Syntax error: word unexpected (expecting ")")

Solution:

# Create .gitattributes
echo "*.sh text eol=lf" > .gitattributes
echo "scripts/* text eol=lf" >> .gitattributes

# Convert existing files
dos2unix scripts/*.sh  # Or use editor to convert

# Rebuild
docker-compose down -v
docker-compose up -d --build

4. Port Already in Use

Error:

Error: bind: address already in use

Solution:

# Windows: Find process using port
netstat -ano | findstr :3030
taskkill /PID <PID> /F

# Or change port in .env
PORT=3031

5. Cache Miss Rate Too High

Symptoms:

  • Slow API responses
  • High database load
  • Cache hit rate < 70%

Solution:

# Check Redis memory
docker-compose exec redis redis-cli INFO memory

# Increase Redis max memory
# In docker-compose.yml:
command: redis-server --maxmemory 512mb --maxmemory-policy allkeys-lru

# Monitor cache performance
curl http://localhost:3030/admin/metrics | jq '.cacheHitRate'

6. API Key Validation Fails

Debugging:

# Check if key exists in database
docker-compose exec mongodb mongosh inventory --eval '
  db.apiKey.findOne({ itemKey: "myapp://item/123" })
'

# Check Redis cache
docker-compose exec redis redis-cli KEYS "apikey:*"
docker-compose exec redis redis-cli GET "apikey:<hash>"

# Check application logs
docker-compose logs app | grep "validation failed"

7. High Memory Usage

Solution:

# Limit container resources in docker-compose.yml
services:
  app:
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 512M
        reservations:
          cpus: '0.5'
          memory: 256M

Debug Mode

# Enable verbose logging
NODE_ENV=development docker-compose up

# Or set log level
LOG_LEVEL=debug docker-compose up

Getting Help

  1. Check GitHub Issues
  2. Review docs/ folder for detailed guides
  3. Enable debug logging and check logs
  4. Open a new issue with:
    • Error message
    • Docker logs
    • Environment setup
    • Steps to reproduce

🀝 Contributing

Contributions are welcome! Please follow these guidelines:

Development Workflow

  1. Fork the repository

  2. Create a feature branch

    git checkout -b feature/your-feature-name
  3. Make your changes

    • Follow existing code style
    • Add tests for new features
    • Update documentation
  4. Run tests

    bun test
    bun run lint  # If configured
  5. Commit with clear messages

    git commit -m "feat: add new validation endpoint"
    git commit -m "fix: resolve Redis connection timeout"
    git commit -m "docs: update API documentation"
  6. Push and create PR

    git push origin feature/your-feature-name

Code Style

  • Use TypeScript strict mode
  • Follow existing naming conventions
  • Add JSDoc comments for public APIs
  • Keep functions small and focused

Testing Requirements

  • All new features must include tests
  • Maintain test coverage above 80%
  • Integration tests for API endpoints
  • Unit tests for utility functions

Documentation

  • Update README.md for major changes
  • Add entries to CHANGELOG.md
  • Update API documentation in docs/API.md
  • Include inline code comments

πŸ“„ License

This project is licensed under the MIT License.

MIT License

Copyright (c) 2025 snowmerak

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

πŸ“š Additional Resources


πŸ™ Acknowledgments

Built with:

  • Bun - Fast JavaScript runtime
  • ElysiaJS - Fast and friendly web framework
  • Prisma - Next-generation ORM
  • MongoDB - NoSQL database
  • Redis - In-memory data store
  • LogTape - Structured logging library

Made with ❀️ by snowmerak

Report Bug Β· Request Feature

About

A secure, high-performance API key management system with dual-hash strategy, distributed caching, and automatic expiration.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published