By the early 2000s, Google was crawling and indexing a meaningful fraction of the web, and the storage layer underneath that effort was buckling under assumptions that made sense for a single reliable server but not for thousands of cheap ones. Commercial file systems assumed disks rarely failed, files were small, and most access patterns were random reads and writes. None of that matched what Google actually had: racks of inexpensive commodity machines where component failure was a routine, daily event rather than an exception, and workloads dominated by huge files — crawl data, logs — that were mostly appended to and read sequentially rather than edited in place. The 2003 SOSP paper "The Google File System," by Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, described the system built around those realities instead of fighting them.

Design for failure, not around it

GFS's starting premise was blunt: on a cluster of thousands of machines built from commodity parts, disk failures, machine crashes, and network hiccups aren't rare edge cases to handle defensively — they're constant background conditions the system has to tolerate as a matter of course. Rather than trying to prevent failure, GFS assumed it and built monitoring, replication, and automatic recovery into the core design. Every chunk of data was replicated across multiple machines — commonly three — so that losing a disk or a whole server didn't mean losing data, just triggering a re-replication to restore the target count.

The second departure from convention was optimizing for the workload Google actually had. Files were often huge — gigabytes was common — and most writes were appends from many concurrent clients (think many web-crawling processes writing results into the same log-like file) rather than random-offset overwrites. GFS made appends a first-class, atomic operation, so multiple writers could safely append to the same file concurrently without corrupting each other's data, a pattern that matched Google's batch-processing pipelines directly.

One master, many chunkservers

GFS split responsibility between a single master node holding filesystem metadata — the namespace, file-to-chunk mapping, and chunk locations — and many chunkservers that stored the actual 64MB chunks the files were broken into. Centralizing metadata in one master simplified the design considerably: the master always had a global view and could make placement and rebalancing decisions without complex distributed consensus for every operation. Google mitigated the obvious single-point-of-failure risk with operation logs, checkpoints, and a shadow master that could take over if the primary failed, and by keeping metadata small and in-memory so the master stayed fast even at scale.

Clients talked to the master only to get chunk locations, then read and wrote chunk data directly with chunkservers — keeping the master out of the data path entirely so it never became a bandwidth bottleneck. This master/chunkserver split, and the general shape of centralized metadata with distributed data, went on to directly influence Hadoop's HDFS, which mirrored GFS's architecture closely enough that early Hadoop documentation cited the GFS paper as its direct inspiration.

What broke when they scaled

GFS (Ghemawat, Gobioff, Leung, SOSP 2003) assumed component failure was normal: replicate 64MB chunks, relax POSIX, optimize for large sequential scans. The single master was the simplicity that later forced Colossus. Append-heavy, write-once-ish workloads fit; small-file random I/O and huge namespaces did not, over time. Applications had to tolerate stale replicas and record append semantics that are not local UNIX.

A master restart that rebuilds state from chunkservers is a scaling event at cluster size. Clients caching metadata can serve wrong locations after moves. The paper is honest about these tradeoffs; cargo-culting GFS for a POSIX home directory is the failure mode.

HDFS copied the design and inherited the NameNode wall — history repeating until HA and federation. Concurrent record-append from many mappers onto one log file was a GFS-specific bet: the master leased chunks, chunkservers serialized appends, and clients lived with at-least-once duplicate records. That is a systems API, not POSIX write(). When the workload mix shifted toward many small files, 64MB chunks wasted space and inflated metadata — the Colossus-era pressure already visible in the original paper's assumptions.

A smaller-team version of the same idea

Object storage with replication, not a custom GFS. If you run HDFS, plan NameNode HA. Design apps for immutable blobs and cheap sequential throughput. Treat disks as dying. Do not require POSIX locks on petabytes.

What you can borrow

  • Design your storage layer around your actual workload's access pattern (append-heavy vs. random-write, large files vs. small) rather than defaulting to a general-purpose file system's assumptions.
  • Treat hardware failure as a constant, expected condition at scale, not an exception — build replication and automatic recovery in from day one rather than bolting it on later.
  • A single, simple coordinator (like GFS's master) can outperform a fully distributed design for metadata, as long as you keep it out of the high-bandwidth data path.
  • Atomic append operations remove a whole class of coordination problems for concurrent writers sharing one log-like output.
  • Replicate at the chunk or shard level, not just the whole-node level, so recovery from a single failure is fast and localized.