Dapper: How Google Made Distributed Tracing Cheap Enough to Leave On
Sigelman et al. on Dapper: trace trees, sampling, and the production constraints that still shape OpenTelemetry and Zipkin.
Engineering at scale
20 articles · 76 min of reading
Sigelman et al. on Dapper: trace trees, sampling, and the production constraints that still shape OpenTelemetry and Zipkin.
Malewicz et al. SIGMOD 2010: vertex-centric supersteps, combiners, and why graph algorithms stopped looking like giant MapReduce shuffles.
Inside Zanzibar, the globally consistent authorization system that decides who can access what across Google Drive, Calendar, Photos, and Maps.
Jeff Dean and Sanjay Ghemawat's embedded key-value store: tables, compaction, and why so many databases still speak its file format.
How Brin and Page turned the web into an eigenvector problem, what the damping factor is for, and which lessons still apply to ranking inside products.
How Google's internal Protocol Buffers serialization format and Stubby RPC framework evolved into gRPC, the open-source RPC standard many companies now run on.
The columnar storage and execution-tree design behind Google's Dremel engine, and how it became the foundation for BigQuery.
How Google's internal Blaze build system, and its open-source descendant Bazel, made fast, reproducible builds possible across a codebase of billions of lines.
Inside Piper and Client in the Cloud, the infrastructure that lets Google run a single monorepo at billions of lines of code without collapsing under its own weight.
How Google's Jupiter network fabric used Clos topologies and centralized control to scale datacenter bandwidth by orders of magnitude.
How Google's Site Reliability Engineering practice used error budgets and a hard cap on toil to align reliability work with product velocity.
How Google's Maglev system moved network load balancing off specialized hardware and onto commodity servers without sacrificing speed or reliability.
Inside Borg, Google's internal cluster manager, and how its ideas about scheduling, priority, and bin packing became the design for Kubernetes.
How Google's Percolator system replaced batch MapReduce reindexing with incremental, transactional updates and cut search index latency dramatically.
How Google's Bigtable paper defined the wide-column data model that HBase, Cassandra, and much of the NoSQL movement built on.
Why Google's original GFS design hit a scaling wall and how its successor, Colossus, fixed the single-master bottleneck underneath nearly every Google product.
How Google's Chubby lock service solved distributed coordination and became the foundation GFS, Bigtable, and countless internal systems depend on.
How Google's 2004 MapReduce paper turned a painful distributed-systems problem into a simple programming model — and accidentally created Hadoop.
How Google used GPS and atomic clocks to bound time uncertainty and give Spanner globally consistent ACID transactions across continents.
How Google's 2003 GFS paper rejected the assumptions of traditional file systems and built storage around cheap hardware that fails constantly.