Multi-Cluster Warehouses: Snowflake's Answer to Concurrency Spikes
How Snowflake scales a virtual warehouse out with extra clusters so queued queries get hardware instead of waiting behind a dashboard storm.
3 min read
Engineering at scale
4 articles · 11 min of reading
How Snowflake scales a virtual warehouse out with extra clusters so queued queries get hardware instead of waiting behind a dashboard storm.
Snowflake stores tables as compressed micro-partitions with per-column min/max metadata so most queries never open most files.
Dageville et al. at SIGMOD 2016 described a warehouse that stores data in object storage and spins independent virtual warehouses for compute.
How micro-partitions make undrop, time travel, and clone cheap at first — and why Fail-safe plus retained history still shows up on the bill.