Scaling a Payment Gateway to 10M+ Daily Transactions
Re-architected a monolithic legacy payment engine into a high-throughput microservices cluster using Kubernetes, Go, and Redis. Isolated transaction processing into stateless pods with Redis Streams for async settlement, achieving 99.999% uptime with sub-50ms p99 latency across three availability zones.
The Challenge
A mid-size digital payments company processing 2M daily transactions hit a hard ceiling. Their monolithic Java application, originally built for card-present transactions, could not scale horizontally. Every deployment required a full system restart, causing 4–8 minutes of downtime. Settlement reconciliation ran as a nightly batch job that routinely failed on high-volume days, requiring manual intervention from the finance team.
Peak-hour latency spiked to 800ms at the 99th percentile, triggering timeout cascades across merchant integrations. The compliance team flagged that the single-database architecture made PCI DSS audit scoping nearly impossible — the entire application was in scope because cardholder data flowed through every module.
Our Approach
We decomposed the monolith into five bounded-context microservices: Transaction Ingestion, Risk Scoring, Payment Routing, Settlement, and Merchant Reporting. Each service was written in Go for predictable latency and deployed as stateless pods on Amazon EKS with horizontal pod autoscaling tied to custom Prometheus metrics.
Transaction state was decoupled from the application layer using Redis Streams. Each transaction event (initiated, authorised, captured, settled) was published to a stream, allowing downstream services to process asynchronously without blocking the critical authorisation path. This reduced the p99 latency of the authorisation endpoint from 800ms to 38ms.
For settlement, we replaced the nightly batch with a continuous micro-batch processor consuming from the Redis Stream. Settlements now reconcile within 15 minutes of capture, eliminating the manual intervention that previously consumed 12 hours of finance team effort per week.
PCI DSS scope was reduced by 70% by isolating cardholder data within a dedicated tokenisation service behind a network policy boundary. Only the ingestion service touches raw card data, and it writes tokenised references for all downstream processing.
Technical Stack
- Languages: Go 1.21 (services), TypeScript (merchant dashboard)
- Infrastructure: Amazon EKS, Terraform, Helm
- Data: Redis Streams, PostgreSQL 15 with Citus extension, TimescaleDB for metrics
- Observability: Prometheus, Grafana, Jaeger distributed tracing
- CI/CD: GitHub Actions with Argo Rollouts for canary deployments
Results
| Metric | Before | After |
|---|---|---|
| Daily transaction capacity | 2M | 10M+ |
| Authorisation p99 latency | 800ms | 38ms |
| Deployment downtime | 4–8 min | Zero (rolling) |
| Settlement reconciliation | Nightly batch | Continuous (< 15 min) |
| PCI DSS scope | 100% of codebase | 30% of codebase |
| System uptime | 99.95% | 99.999% |