Playground event ticketing distributed system for browsing events, purchase tickets, and process payments to learn distributed systems concepts through hands-on experience.
Done with a build, break then fix approach i.e. steps on how requirements are built, what issues they solve and how it works starting with a monolith and gradually breaking it into distributed components progressively.
- Node.js (v18+ recommended)
node --version # should show v18.x.x or higher - Docker & Docker Compose
docker --version # should show Docker version 20.x.x or higher docker-compose --version # should show docker-compose version 1.29.x or higher
- npm or yarn
npm --version # should show 8.x.x or higher - cURL or HTTPie (for API testing)
- Artillery (for load testing - installed via npm)
# Terminal 1
# clone the repo
git clone https://github.com/oyinetare/ticketflow.git
cd ticketflow
cd monolith
npm install
docker-compose up -d
npm run dev
# Terminal 2 - test basic functionality
# get all events
curl http://localhost:3000/api/v2/events
# purchase a ticket
curl -X POST http://localhost:3000/api/v2/tickets/purchase \
-H "Content-Type: application/json" \
-d '{"eventId": "3", "userId": "test-user"}'
# stop docker
docker-compose down-
Event management
- View all avaialable events (search events)
- View event details
- Create event
-
Ticket purchasing
- Purchase event tickets
- Reserve tickets (prevent others from buying same tickets)
- Process payment
- Confirm ticket purchase
- Handle failed payments gracefully
-
User Features
- View purchased tickets
- Get ticket confirmation
- View purchase history
-
Inventory Management
- Track available tickets per event
- Prevent overselling (no negative inventory)
- Release tickets if payment fails
-
Reliability: The system should prioritize availability i.e. 99.9% uptime (8.7 hours downtime/year) for searching & viewing events, but should prioritize consistency for booking events (no double booking, never sell same ticket twice, charge exactly once per ticket, no lost tickets or payments). Eventual consistency is OK (few seconds delay)
-
Fault tolerant: System stays up if one service fails
-
Scalability: The system should be scalable and able to handle high throughput in the form of popular events (10 million users, one event), Add more servers during peak times, Database Scaling(Handle millions of tickets/events), Queue Scaling(Process thousands of payments per minute)
-
Performance: The system should have low latency search (i.e. Response Time of Browse events < 200ms - 500ms) & purchasing < 3 seconds (including payment)
-
The system is read heavy, and thus needs to be able to support high read throughput (100:1), Handle 1000 concurrent users
-
High concurrency
-
Security: No storing credit cards, Rate limiting to prevent abuse for API, Data Privacy(Secure user information)
- 1 — Monolith
- 2 — Scaling 1: Queues + Redis Locking + API improvements
- 3 — Scaling 2: Microservices
- 4 - Advanced patterns: Saga & Event Sourcing
- REST API for events (create, list, get)
- Ticket purchase endpoint (synchronous)
- Mock payment processing
- load testing
Client → API → Services → SQLite
- TypeScript + Node.js + Express
- SQLite for db (easy to start, single file DB)
- Simple in-memory payment processing
- Event
- Ticket
- Payment
export interface Event {
id: string;
name: string;
venue: string;
date: Date;
totalTickets: number;
availableTickets: number;
price: number;
createdAt: Date;
}
export interface Ticket {
id: string;
eventId: string;
userId: string;
purchaseDate: Date;
price: number;
status: "pending" | "confirmed" | "cancelled";
}
export interface Payment {
id: string;
ticketId: string;
userId: string;
amount: number;
status: "pending" | "completed" | "failed";
processedAt?: Date;
createdAt: Date;
}SqLite In-Memory Database
TABLE events (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
venue TEXT NOT NULL,
date TEXT NOT NULL,
totalTickets INTEGER NOT NULL,
availableTickets INTEGER NOT NULL,
price REAL NOT NULL,
createdAt TEXT NOT NULL
)
TABLE tickets (
id TEXT PRIMARY KEY,
eventId TEXT NOT NULL,
userId TEXT NOT NULL,
purchaseDate TEXT NOT NULL,
price REAL NOT NULL,
status TEXT NOT NULL,
FOREIGN KEY (eventId) REFERENCES events(id)
)
TABLE payments (
id TEXT PRIMARY KEY,
ticketId TEXT NOT NULL,
userId TEXT NOT NULL,
amount REAL NOT NULL,
status TEXT NOT NULL,
processedAt TEXT,
createdAt TEXT NOT NULL,
FOREIGN KEY (ticketId) REFERENCES tickets(id)
)GET /api/events # get all events
GET /api/events/:id # get single event by id
POST /api/events # create event
POST /api/tickets/purchase # purchase ticket
GET /api/tickets/user/:userId # get users tickets- Event Service
- Payment Service
- Ticket Service
- simple Express app
- Middleware
- cors
- express.json(), so that app can read JSON data sent from the client (like in POST or PUT requests) and make it available in req. body. Without it, Express cannot understand JSON data in requests ExpressJS express.json() Function
- logging middleware
- error handler
- init db
- Middleware
- Event management: View all avaialable events (search events) + View event details + Create event
- routes call asynchronous functions in service i.e. await getAllEvents, await getEventById, await createEvent
- services return Promise<> running either
SELECT FROM <table>orINSERT INTO <table>
- Ticket purchasing: Purchase event tickets + Process payment + Confirm ticket
- check event availability (eventService.getEventById) ISSUE: causes RACE CONDITION
- create ticket (ticketService.createTicket) with status pending
- process payment (paymentService.processPayment) ISSUE: SLOW & causes problems
- simulate api call to payment gateway (this will cause race conditions, multiple attempts to pay) & simulate occasional payment failures
- create payment with status completed
- Update ticket status to confirmed or cancelled (ticketService.updateTicketStatus)
- Update event tickets count (eventService.updateEventTickets)
# get all events
curl http://localhost:3000/api/v2/events
# purchase a ticket
curl -X POST http://localhost:3000/api/v2/tickets/purchase \
-H "Content-Type: application/json" \
-d '{"eventId": "3", "userId": "test-user"}'- using nouns for resource names & plural nouns to name collection URIs., i.e.
/eventsinstead of/create-events. The verbal action on a URI is already implied by the HTTPGET,POST,PUT,PATCH, andDELETEmethods - relationships kept simple and flexible
- no API versioning for now
- Interface types dont mirror the internal structure of db, so theres less risk of increasing the attack surface and might lead to data leakage in API and API is an abstraction of the database
- Implement asynchronous methods in services
- ACID Transactions are Critical
BEGIN TRANSACTION;
-- Check availability
SELECT availableTickets FROM events WHERE id = ? FOR UPDATE;
-- Create ticket
INSERT INTO tickets (eventId, userId, status) VALUES (?, ?, 'pending');
-- Update inventory
UPDATE events SET availableTickets = availableTickets - 1 WHERE id = ?;
-- Process payment
INSERT INTO payments (ticketId, amount, status) VALUES (?, ?, 'completed');
COMMIT;- Strong Consistency Requirements
// With SQL, this is guaranteed to be accurate
const event = await db.query(
"SELECT COUNT(*) as soldTickets FROM tickets WHERE eventId = ? AND status = 'confirmed'",
[eventId]
);- NoSQL often provides eventual consistency
- For ticket inventory, you need immediate consistency
- Can't risk overselling due to replication lag
- Complex Relational Queries
-- Find all events a user has attended with payment details
SELECT e.name, e.date, t.purchaseDate, p.amount, p.status
FROM tickets t
JOIN events e ON t.eventId = e.id
JOIN payments p ON p.ticketId = t.id
WHERE t.userId = ?
ORDER BY e.date DESC;- Events → Tickets → Payments → Users all interconnected
- NoSQL would require multiple queries or denormalization
- Financial Data Integrity
-- Ensure payment reconciliation
SELECT
SUM(p.amount) as totalRevenue,
COUNT(DISTINCT t.id) as ticketsSold
FROM payments p
JOIN tickets t ON p.ticketId = t.id
WHERE p.status = 'completed' AND t.eventId = ?;- Payment records must be 100% accurate
- SQL constraints prevent orphaned records
- Foreign keys ensure referential integrity
Using NoSQL db i.e. The Cost of Getting It Wrong in this situation
- Overselling = Angry customers, refunds, reputation damage
- Payment inconsistencies = Financial liability, audit failures
- Lost transactions = Revenue loss, customer frustration
- SQL's strong guarantees make these disasters much less likely.
For the core ticketing logic (inventory, payments, orders), SQL is the clear winner because:
-
ACID transactions prevent race conditions
-
Strong consistency ensures accurate inventory
-
Relational model matches the domain perfectly
-
Financial data requires bulletproof integrity
-
Sqlite
-
Sqlite was not designed for Client/Server Applications: If there are many client programs sending SQL to the same database over a network, then use a client/server database engine instead of SQLite. SQLite will work over a network filesystem, but because of the latency associated with most network filesystems, performance will not be great. Also, file locking logic is buggy in many network filesystem implementations (on both Unix and Windows). If file locking does not work correctly, two or more clients might try to modify the same part of the same database at the same time, resulting in corruption. Because this problem results from bugs in the underlying filesystem implementation, there is nothing SQLite can do to prevent it.
-
High Concurrency: SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time. For many situations, this is not a problem. Writers queue up. Each application does its database work quickly and moves on, and no lock lasts for more than a few dozen milliseconds. But there are some applications that require more concurrency, and those applications may need to seek a different solution.
-
- Race condition happens in process of checking for available tickets and actually decrementing them as slow payment processing creates large time window for concurrent requests
test-race-condition.js
Test race condition
get initial ticket count for event
create array of 10 concurrent purchase attempts
get event and check availability
if not exists return 404, no event
if no available tickets return 400, no tickets left
if event exists and available events
try
create ticket with status pending
process payment
simulate api call to payment gateway (this will cause race conditions, multiple attempts to pay) & simulate occasional payment failures
create payment with status completed
update ticket status to confirmed
update inventory by decrement available tickets for event
catch paymentError
update ticket status to cancelled and throw paymentError
wait for them all to complete
get success count
get final ticket count for event
there’s a race condition if success count isn’t same as (initial - final)
—————————————————————
The Critical Race Condition Path
Thread 1: Checks availableTickets = 5 ✓
Thread 2: Checks availableTickets = 5 ✓ // Same value!
Thread 3: Checks availableTickets = 5 ✓ // Still same!
...
Thread 1: Creates ticket (pending)
Thread 2: Creates ticket (pending)
Thread 1: Processes payment (1-3 seconds)
Thread 2: Processes payment (1-3 seconds)
Thread 1: Updates availableTickets to 4
Thread 2: Updates availableTickets to 4 // Should be 3!
Initial count: Get starting availableTickets
Concurrent attempts: 10 parallel purchases
Success tracking: Count successful purchases
Validation: successCount ≠ (initial - final) proves overselling
- Lost tickets when payments fail
- Multiple users can buy the same ticket, double booking
- No Transaction Boundaries as each operation is independent, allowing partial failures
- Phantom Tickets because tickets can be created even when inventory is exhausted
- Database locks cause timeouts
- Inconsistent inventory counts
- Using SQLite in memory DB whcih doesnt scale well in distributed systems
- different from client/server SQL database engines such as MySQL, Oracle, PostgreSQL, or SQL Server since SQLite is trying to solve a different problem
- Client/server SQL database engines strive to implement a shared repository of enterprise data and emphasize scalability, concurrency, centralization, and control whereas SQLite strives to provide local data storage for individual applications and devices. SQLite emphasizes economy, efficiency, reliability, independence, and simplicity.
- SQLite does not compete with client/server databases. SQLite competes with fopen().
- SQLite database requires no administration, it works well in devices that must operate without expert human support
- Client/server database engines are designed to live inside a lovingly-attended datacenter at the core of the network. SQLite works there too, but SQLite also thrives at the edge of the network, fending for itself while providing fast and reliable data services to applications that would otherwise have dodgy connectivity
- SQLite is a good fit for use in "internet of things" devices.
- Generally speaking, any site that gets fewer than 100K hits/day should work fine with SQLite
- Reference: https://sqlite.org/whentouse.html
- better error handling
- Implement versioning in API: imporvements to API
- Redis for distributed locking
- Bull queue for async payment processing
- Idempotency keys for payments: imporvements to API
- query result caching to speed up frequently repeated search queries and reduce load on our search infrastructure?: imporvements to API
- Implement data pagination and filtering: imporvements to API
- HATEOS link: imporvements to API
Client → API → [Redis Lock] → Services → Queues → SQLite
- Race conditions - Multiple users checking/buying simultaneously
- Inconsistent inventory counts - Stale reads lead to wrong counts
- Slow payments - 1-3 second payment processing blocks everything
- Phantom Tickets - Tickets created even when inventory exhausted
- Lost tickets when payments fail - No proper rollback
- Double booking - Same ticket sold multiple times
- Duplicate charges
- Database locks/timeouts
Ideas
-
How do we improve the booking experience by reserving tickets?: Dealing w contention (Race condition)
- pessimistic locking
- status & expiration time with cron
- mpicit status with tatus and time expiration
- distirbuted lock with TTL
- Redis for distributed locking with TTL to prevent race conditions
-
How to prefent duplicate charges
- Idempotent payment handling: Use Idempotency keys for payments
- Proper rollback on failures
-
How can you speed up frequently repeated search queries and reduce load on our search infrastructure?
- implement caching trategies using redis and memcached
- impoklement query result caching and edge caching techniques
- Distributed Locking (Redis with TTL) Solves:
- Race conditions
- Double booking
- Inconsistent inventory counts
- Phantom Tickets
- Async Payment Queue (Bull/Redis) Solves:
- Slow payments
- Database locks/timeouts
- Partially helps with scalability
- Idempotency for Payments Solves:
- Duplicate charges
- Partial failure recovery
- Redis cache
- speed up frequently repeated search queries and reduce load on our search infrastructure?
How idempotency works
- Idempotent payment handling: Use Idempotency keys for payments
- idempotency
- handler
- if not POST, then next
- get idempotency key from redis cache or generate if one doesnt exist
- Generate key based on user, endpoint, and request body
- check for cached response with idempotency key
- store original json method using redis cache
- ovveride json method to cache response
- attach key to request for use in handlers
- idempotency
- Proper rollback on failures, Partial failure recovery
Problem appears: Single database bottleneck
Move SQLite down here
- Single Database Bottleneck (SQLite)
🔴 Still can't handle true horizontal scaling 🔴 Write operations still bottlenecked 🔴 No replication/failover
-
Additional Issues in Phase 2: markdownNew Problems in Phase 2:
-
Distributed System Complexity
- Lock failures/deadlocks
- Redis single point of failure
- Queue processing delays
-
Consistency Challenges
- Cache invalidation complexity
- Eventual consistency between Redis/SQLite
- Lock orphaning if process crashes
-
Operational Overhead
- Need to monitor Redis, queues, cron jobs
- Debugging distributed transactions
- Managing TTLs and expirations
-
Still Limited by SQLite
- No read replicas
- No sharding capability
- File-based = single server
-
Real time updates: How will the system ensure a good user experience during high-demand events with millions simultaneously booking tickets? + Reduceunnecessary API call + Fair access to tickets
- SSE for real time seat updates
- Virtual waiting queue for xtrmemly popular events
- How can you improve search to ensure we meet our low latency requirements?
- indexinfg & sql qurery ptimization
- full-text idnexes in db
- se a full text search engine liem elastic db
- Scaling reads: How is the view API going to scale to support 10s of millions of concurrent requests during popular events?
- cachign, lb, horizontal scaling
- Ticket Reservation System (Status + Expiration) Solves:
✅ Lost tickets when payments fail ✅ Phantom Tickets ✅ Better UX during high contention
- No Transaction Boundaries - Operations aren't atomic
- Database locks cause timeouts - SQLite locks entire DB
- No scalability - SQLite single-file limitation
- move to PostgreSQL
- Separate services can scale independently
- Payment service gets more servers
- New capabilities: Multi-region, analytics, real-time updates
The lockfile is important because it:
Ensures consistent installs across different machines Locks specific versions of all dependencies and sub-dependencies Speeds up CI/CD builds Helps prevent security vulnerabilities by pinning versions