API Gateway in System Design: The Front Door of Microservices

A monolith has one front door. Split it into twelve services and the same clients now face twelve hostnames, twelve TLS configurations, and (unless someone stops it) twelve half-different authentication implementations. The cross-cutting concerns never disappear when a system is decomposed; they multiply. The API gateway is the standard answer: one enforced entry point where those concerns live once.

An API gateway is a server that receives every external API request for a backend, authenticates the caller, applies policy such as rate limits and quotas, routes the request to the appropriate service, and returns the response. It is the single, enforced entry point between clients and a microservices backend.

Where the gateway sits

A gateway is a tier on the request path, usually reached through the same front-of-house infrastructure as everything else:

Client
   ↓
DNS
   ↓
CDN                    ← static content served at the edge
   ↓
Load balancer          ← spreads traffic across gateway instances
   ↓
API gateway            ← authentication, policy, routing
   ↓
Microservices          ← business logic, one service per route

Static assets are answered before this tier by the CDN. The load balancer in front of the gateway keeps the entry point itself redundant, a gateway deployed as a single instance caps the availability of every service behind it. And the gateway is best understood as a relative of the reverse proxy: the same software fronting servers, grown a policy layer. How the two relate (and where the balancer’s job ends) is drawn later, in the comparison below.

API gateway responsibilities

A gateway earns its hop by doing, in one place, what would otherwise be repeated in every service:

• Routing. Requests are dispatched by path, host, and header, /users/* to the user service, /orders/* to orders, /v2/ to the new version while /v1/ drains. The mapping from public contract to internal topology lives in one place, so services can be renamed, split, and redeployed without clients noticing.
• Gateway authentication. API keys, JWTs, and OAuth tokens are validated at the edge, before expensive work happens. Authorization at the resource level still belongs to the services; the gateway answers “is this caller allowed in at all.”
• Rate limiting and quotas. Per-client budgets are enforced in one place; the algorithms and their distributed enforcement are covered in rate limiting in system design.
• TLS termination. Certificates and handshakes are managed at the edge, so backend services spend their cycles on application work rather than cryptography.
• Request and response transformation. The gateway translates between what clients speak and what services speak: REST to gRPC, header rewrites, payload shaping, versioned contracts.
• Telemetry. Request logs, latency metrics, and tracing context are recorded at the tier that sees every call; the gateway is often the richest observability point in the system, and the vocabulary for exploiting that position is covered in monitoring and observability.

Gateway authentication in practice

Three mechanisms cover most real deployments, in ascending order of rigor:

• API keys, opaque strings issued per client. Simple to operate, weak on their own: a key is a bearer credential, and whoever holds it can impersonate the client.
• JWTs, signed tokens carrying claims. The gateway validates a signature and an expiry without a database lookup, which makes authentication cheap and stateless; revocation is the weak point, handled with short-lived tokens and a revocation list.
• mTLS: the client presents a certificate, the gateway verifies it and terminates the connection. Common for partner and service-to-service traffic, where the operational cost buys cryptographic identity.

Once the gateway authenticates a caller, it forwards identity downstream: a trusted header such as X-User-Id, or the validated token itself. The internal network then carries an assumption: services trust what the gateway says about the caller. That assumption is only as good as the network’s isolation, which is why zero-trust deployments re-validate the token in each service and treat the gateway as the first check rather than the only one.

API gateway vs reverse proxy vs load balancer

The three components are relatives, and the confusion is legitimate, the same software often plays two or three of the roles. The cleanest way to separate them is the question each one answers:

A gateway is a reverse proxy grown a policy layer. Fronting (TLS, routing, hiding backends) is what a reverse proxy does and remains the substrate here; what separates a gateway is that its decisions are made about the caller: identity, budget, contract, not only about a request’s destination. The load balancer’s job, distributing across instances, is orthogonal and still happens; the division of labor between it and a proxy is drawn in load balancing in system design.

API gateway design decisions

• One gateway or many. A single front door keeps policy uniform and is the default starting point. Backend-for-frontend (BFF) variants give each client type (mobile, web, partner) its own gateway when their contracts genuinely diverge; the cost is one more component per client type.
• Who owns the configuration. Centralizing every route with a platform team creates a queue: every new endpoint waits on someone outside the owning team. Self-service route ownership (the platform runs the gateway, teams register routes) is what keeps the pattern compatible with the team autonomy that motivates microservices architecture in the first place.
• The latency budget. The gateway is a hop on every request. Keep the per-request work small (an auth validation, a rate-limit check, a route lookup) and watch the gateway tier’s p99 like any other tier, in the terms established in latency vs throughput.
• Long-lived connections. WebSockets and streaming connections pass through or bypass the gateway, and the choice must be deliberate: a gateway designed for request-response can hold worker threads hostage to idle connections. The mechanics of those connections are in what are websockets.
• Failing fast. A gateway that waits out a dead backend amplifies the failure instead of containing it. Per-route timeouts, bounded retries, and a circuit breaker on each route (the pattern explained in the circuit breaker pattern) keep one sick service from consuming the shared front door.

Failure modes

• The gateway becomes the system’s availability ceiling. One instance (or a fleet without health-checked redundancy) and the front door’s uptime caps every service behind it; the definitions of availability used across this library are in availability vs reliability vs durability. Production gateways run as a fleet behind the balancer, deployed and rolled like any other tier.
• The distributed monolith trap. Business logic creeps into gateway configuration: a price calculation, an eligibility rule, “just this once.” The services are now coupled to the gateway’s release cycle, and whoever owns routing changes owns a business rule. Gateways carry policy, never business logic.
• Head-of-line blocking. One slow backend plus unbounded concurrency holds gateway workers until the whole front door starves. The fixes are per-route timeouts and per-route concurrency limits, set deliberately rather than inherited from defaults.
• The trust cliff. Services that skip their own authorization because “the gateway checks it” are one internal path away from exposure; the gateway is a perimeter control, and a perimeter is only as strong as what it encloses.

Common mistakes

• Business logic in the gateway. The fastest way to turn a policy tier into a bottleneck owned by the wrong team.
• No per-route timeout policy. Default timeouts on shared infrastructure are inherited from nobody’s requirements and fail everybody’s at once.
• Treating gateway telemetry as optional. The tier that sees every request is the cheapest place to answer “which endpoints are slow, and for whom”, leaving it uninstrumented wastes the position.
• One global rate limit. A policy that cannot distinguish a cheap read from an expensive export protects neither; budgets belong per client and per endpoint.
• Routing internal traffic through the external gateway. East-west calls pay the public hop’s latency for none of its benefits; internal services find each other by their own means, service discovery.

When you do not need a gateway

A gateway is a platform component with a release cycle, a configuration surface, and a failure domain. With a single service and a handful of endpoints, a reverse proxy does the fronting and a gateway adds only the hop. The pattern pays for itself when the number of services or client types makes cross-cutting policy genuinely shared, and the shape of that tipping point is part of the microservices architecture decision itself.

FAQ

What is an API gateway in system design?

A server that receives every external API request for a backend, authenticates the caller, applies policy such as rate limits and quotas, transforms requests, and routes each call to the appropriate service. It centralizes the cross-cutting concerns (routing, authentication, limits, translation, telemetry) that would otherwise be reimplemented in every service.

What is the difference between an API gateway and a reverse proxy?

A reverse proxy fronts servers (TLS, routing, buffering, hiding backends) whether there is one backend or many. An API gateway is that fronting plus a policy layer whose decisions are about the caller: authentication, per-client budgets, request transformation, contract translation. The same software often implements both; the gateway is the pattern with policy added.

Should authentication happen in the gateway or in each service?

Both, at different resolutions. The gateway performs the coarse check (is this caller valid at all) cheaply, before expensive work happens. Services enforce resource-level authorization, because only they know what a specific operation means. Treating the gateway’s check as the only check works only for as long as nothing can reach a service except through the gateway.

Does every microservices system need an API gateway?

No. With a handful of services and one client type, direct exposure behind a reverse proxy is simpler, and a gateway can be introduced when the number of routes, client types, or shared policies makes the front door earn its hop. The failure mode is adopting the pattern early and letting it centralize the ownership the decomposition was meant to distribute.

Can an API gateway be a single point of failure?

Yes; one instance of the tier that receives every request caps the availability of everything behind it. Production gateways run as redundant fleets behind a load balancer, with health checks and rolling deploys, so the front door fails like any other tier: partially, and in a way that gets routed around.

Related articles

• Next read: rate limiting in system design, the algorithms and distributed enforcement behind the quotas a gateway imposes.
• microservices architecture; the style that multiplies front doors, and the reason the pattern exists.
• what is a reverse proxy; the substrate a gateway is grown from.
• load balancing in system design, keeping the gateway itself redundant.
• what are websockets; the long-lived connections a gateway must consciously handle or bypass.

Last updated on 7 September 2026