Files On-Demand: OneDrive's Trick for Making a Cloud Namespace Look Local
How OneDrive placeholders, reparse points, and a cloud filter driver let Windows show entire libraries without downloading every byte.
Tag
18 articles · 66 min of reading
How OneDrive placeholders, reparse points, and a cloud filter driver let Windows show entire libraries without downloading every byte.
How Microsoft replaced three full copies of blob data with a local reconstruction code that keeps durability high without tripling disk.
How GitLab extracted Git disk operations into Gitaly so the Rails app no longer spoke filesystem paths to a shared NFS server.
How MongoDB's WiredTiger engine brought document-level concurrency, compression, and checkpointing — replacing MMAPv1's collection-level locks.
Why databases write the log before the page, how REDO and UNDO recover, and the fsync choices that turn a power loss into silent corruption or lost commits.
What read-after-write means on S3 today, why list-after-delete still bites, and how to design keys, retries, and multipart uploads around that.
How micro-partitions make undrop, time travel, and clone cheap at first — and why Fail-safe plus retained history still shows up on the bill.
Jeff Dean and Sanjay Ghemawat's embedded key-value store: tables, compaction, and why so many databases still speak its file format.
How Bloom filters avoid disk reads in LSM trees and caches, how false positives behave, and when a cuckoo or ribbon filter is the better structure.
Why Facebook replaced InnoDB with MyRocks, an LSM-tree storage engine built on RocksDB, to shrink storage footprint across its huge MySQL fleet.
How Facebook forked Google's LevelDB into RocksDB to target fast flash storage, and how an embedded engine built for one problem became infrastructure for many.
How AWS's EBS team solved a control-plane consistency problem by giving every volume its own small, independently placed replicated database.
Key design patterns for using S3-style object storage well, from key naming schemes to lifecycle policies, and the mistakes that cause throttling at scale.
Why Google's original GFS design hit a scaling wall and how its successor, Colossus, fixed the single-master bottleneck underneath nearly every Google product.
How Facebook's Haystack replaced a filesystem-per-photo approach with a purpose-built object store to eliminate the metadata bottleneck of billions of images.
What Amazon S3's famous '99.999999999% durability' figure actually means statistically, and the engineering that backs the promise.
How Dropbox built Magic Pocket, its own exabyte-scale storage system, and migrated hundreds of petabytes of user data off S3 without downtime.
How Google's 2003 GFS paper rejected the assumptions of traditional file systems and built storage around cheap hardware that fails constantly.