Lyft
Lyft: Envoy and the service mesh
One proxy on every host: how Lyft turned polyglot L7 chaos into a uniform, observable service mesh.
Why Lyft built Envoy
As Lyft broke its monolith into a service-oriented architecture, it hit the recurring failure mode of every large SOA: the hard problems of networking (service discovery, load balancing, retries, timeouts, and above all observability) got re-solved, badly and differently, inside each language runtime. Lyft ran services in PHP, Python, Go, and more, and every language had its own HTTP client library with its own idea of connection pooling, retry behavior, and metrics. When a request failed somewhere in a call chain, there was no consistent way to see which hop was slow or erroring.
Envoy's founding thesis, stated plainly in the project's own docs, is that "the network should be transparent to applications." Rather than ship yet another library per language, Lyft moved the networking concerns out of the application entirely and into a separate process. Envoy is described as an L7 proxy and communication bus for modern service-oriented architectures.
By the early summer of 2016 Envoy was fully deployed at Lyft, handling all edge and service-to-service networking, forming a mesh between over a hundred services and transiting millions of requests per second. Lyft open-sourced it in September 2016, and it later became a graduated CNCF project. The direct payoff Lyft called out was operational: robust observability and easy debugging, the two things most stacks lose when they leave the monolith.
The sidecar mesh: one proxy per host
Envoy runs as a self-contained, out-of-process sidecar co-located with each application instance. Every request into and out of a service passes through its local Envoy over localhost, so the application talks to what looks like a simple local endpoint while Envoy handles the real network. Because the proxy is a separate process, its features work identically no matter what language the app is written in, and it can be deployed and upgraded without touching application code. This is the core tradeoff of the sidecar model: you pay for an extra hop and an extra process per host in exchange for uniform, centrally controlled L7 behavior.
The same Envoy binary also runs as an edge (front) proxy: a pool of Envoys terminates TLS at the edge and performs L7 routing into the mesh, so the north-south and east-west paths share one codebase and one set of features. Envoy is a self-contained process forming a transparent communication mesh, which means adding a service to the mesh is a deployment concern, not an application rewrite.
Envoy's data plane is fed by a control plane. Configuration (endpoints, clusters, routes, listeners) is distributed to every proxy over the dynamic xDS APIs. Internally each Envoy is single-process, multi-threaded: a main thread handles xDS config updates, stats flushing, and the admin interface, while a configurable number of worker threads handle connections (a connection is bound to one worker for its lifetime). Config is pushed to workers via thread-local storage, giving lock-free reads at the cost of eventual consistency across worker threads. See the diagram for how ingress, egress, control plane, and observability wire together.
Envoy service mesh at Lyft: data plane, control plane, observabilityApplication-layer load balancing and resilience (task 1.3)
Because Envoy speaks HTTP/2 and gRPC natively (with transparent HTTP/1.1-to-HTTP/2 proxying), it load-balances at the application layer rather than blindly at L4. It supports weighted round robin, weighted least request, random, and the consistent-hashing families (ring hash and Maglev), plus zone-aware and locality-weighted routing, priority levels, and a panic threshold that keeps sending traffic when too many hosts look unhealthy. On top of raw load balancing, Envoy adds the resilience primitives Lyft needed everywhere: automatic retries, timeouts, circuit breaking, global rate limiting, and request shadowing/mirroring, all applied per request and configured centrally instead of per client library.
Service discovery is deliberately eventually consistent. Envoy was designed from the beginning with the idea that service discovery does not require full consistency, so it does not need a strongly consistent store like ZooKeeper or etcd. Endpoint Discovery Service (EDS, part of xDS) gives Envoy explicit knowledge of each upstream host, which lets it make smarter load-balancing decisions than DNS alone; strict DNS, logical DNS, static, and original-destination discovery are also supported. Crucially, EDS membership is combined with active health checks and passive outlier detection. When a host crashes between discovery refreshes, health checking drops it from rotation using a 2x2 discovery-by-health matrix, so the mesh tolerates stale membership and network partitions instead of failing hard.
This is exactly how the exam frames application-layer load balancing: the choice of an L7 load balancer buys you content-based routing, protocol awareness (HTTP/2, gRPC), and per-request resilience, whereas an L4 load balancer only sees connections. The cost is that an L7 proxy must parse and buffer requests and terminate connections, which is why capacity and CPU planning for the proxy tier matters.
Per-hop observability: the real payoff (task 3.2)
Observability was Lyft's primary motivation, and it is the clearest reason to prefer a sidecar over per-language libraries. Every Envoy emits the same statistics for every service: counters (e.g. total requests), gauges (e.g. active requests), and histograms (e.g. upstream request time), broken out per upstream cluster and split into downstream, upstream, and server categories. These flow through pluggable sinks such as statsd. Because the numbers come from the proxy, a Go service and a PHP service produce identical, comparable metrics with zero application instrumentation.
Envoy also provides distributed tracing. It generates an x-request-id and, for tracers like Zipkin, the B3 trace headers, then reports spans to backends such as Zipkin, Jaeger, Datadog, or AWS X-Ray. A span captures the originating and upstream cluster, timing, HTTP method/URL/status, and error tags for 5xx responses. The one thing Envoy cannot do alone is stitch hops together: each service must propagate the incoming trace-context headers onto its outbound calls, or the trace fragments into disconnected single-hop pieces. That propagation requirement is a common real-world debugging trap.
Contrast this with lower-layer telemetry. VPC Flow Logs record L3/L4 metadata (IPs, ports, bytes, accept/reject) and can tell you two hosts talked, but never the HTTP status, latency percentile, or which route was taken. Edge access logs (for example ALB access logs) capture the front door but not internal east-west hops. The exam's traffic-observability objective is about matching the tool to the layer: use flow logs and mirroring for L3/L4 forensics, and an L7 proxy's stats and traces for application-level performance and error visibility.
How AWS productized this: App Mesh, ALB, and the exam framing
AWS App Mesh is a service mesh built directly on the Envoy proxy: you add an Envoy sidecar (AWS vends the aws-appmesh-envoy container image) to each ECS task, EKS/Kubernetes pod, or EC2 instance, and the App Mesh control plane distributes routing config to those Envoys over xDS. You model traffic with virtual services, virtual nodes, virtual routers, and routes; a virtual router can weight traffic across nodes (for example 90% to serviceB v1, 10% to a v2 canary) and route on HTTP headers, paths, or gRPC method, and it can apply retry policies. The services keep their original discovery names, so introducing the mesh changes no application code and needs no redeploy to change routing. The diagram traces a client request through an ALB into an Envoy-fronted task and then a weighted split to two backend versions. Note that AWS has announced it will end App Mesh support on September 30, 2026 and steer customers to Amazon ECS Service Connect, so treat App Mesh as the canonical Envoy example rather than a greenfield recommendation.
The Application Load Balancer is the managed L7 building block the exam leans on most. It operates at OSI layer 7, evaluates listener rules in priority order, and routes on path, host, HTTP header, method, query string, or source IP to different target groups. It speaks HTTP/2 and gRPC, terminates TLS with ACM certificates, injects X-Amzn-Trace-Id for request tracing, writes access logs, and reports CloudWatch metrics and health checks per target group. That per-target-group granularity is the managed analog of Envoy's per-cluster stats.
For ANS-C01, the mental model is a spectrum: an ALB gives you managed L7 routing and observability at the edge or between tiers, while a sidecar mesh (Envoy/App Mesh) pushes the same L7 features (content routing, retries, circuit breaking, uniform per-hop metrics and traces) all the way down to every service-to-service call. Choose based on where you need application-layer control and how much east-west visibility the design requires.
AWS App Mesh: Envoy data plane with weighted canary routingSources
Practice what you just read