Building an API gateway from scratch in Go
A reverse proxy is the easy part. The real work starts when traffic needs to be shared fairly, failures contained, and every request made explainable.
Why build a gateway?
The gateway is the one service that sees every request before it reaches an application. That makes it an excellent place to centralize concerns that otherwise get reimplemented — often inconsistently — in each backend: overload protection, backend selection, resilience, and visibility.
For this project, the goal was not to replace nginx or a managed gateway. It was to make the machinery familiar by implementing it with Go's standard library and small, focused interfaces.
Start with a careful proxy
Forwarding an HTTP request means more than changing its destination. A proxy needs sensible connection reuse and timeouts, must copy bodies safely, and must remove hop-by-hop headers such as Connection and Keep-Alive. Those headers describe one network hop and should not leak to the next one.
The proxy package owns those details. The rest of the system can think in terms of a selected backend and a response, instead of repeating transport code in every feature.
Choosing a backend is a policy decision
Round robin is a useful baseline, but it assumes all backends have equal capacity and request cost. The gateway adds three alternatives because traffic has different shapes:
All four strategies satisfy the same small balancer interface. That keeps backend selection replaceable rather than coupled to proxying.
Health is not a single signal
A backend can look healthy while idle and fail only under real traffic. It can also be perfectly capable but receive no traffic, leaving passive monitoring blind. The project uses both signals: active HTTP probes and passive error-rate tracking over a sliding window.
The combined health checker requires both to agree. The health pool can also choose fail-open behavior when every backend appears unhealthy — useful when a false negative is more harmful than attempting a request — or fail closed when strict isolation is the safer choice.
Circuit breakers protect the caller
Retrying a failing backend harder is how a small dependency problem becomes a system-wide outage. Each backend gets its own circuit: it begins closed, opens after enough failures, and later permits a limited trial request in the half-open state. A successful trial closes the circuit; another failure opens it again.
The important detail is isolation. A bad backend should not stop healthy peers from serving traffic.
Make every decision visible
When an API gateway makes decisions on behalf of the system, it must leave an audit trail. The project exposes Prometheus metrics for requests, latency, limiter hits, circuit state, backend health, and active connections. Structured logs attach method, path, client IP, status, latency, and trace ID. The trace ID is generated securely when absent and returned in X-Request-ID, so an incident can be followed across services.
Graceful shutdown completes the operational story: stop accepting new work, drain in-flight requests within a deadline, then close background workers such as health checks and reloaders.
The lesson
These mechanisms reinforce one another. Rate limiting protects capacity, health checks change the candidate pool, the balancer selects from it, and circuit breakers prevent a bad choice from turning into a cascade. Observability tells you why each choice was made. Building them independently was useful; seeing how their boundaries meet was the actual lesson.
Read the API Gateway case study or explore the source on GitHub ↗.