LinkedIn
Why a Job Site Invented Kafka
How LinkedIn's tangle of point-to-point data pipelines led to Kafka's commit-log abstraction, and how one internal tool became industry infrastructure.
3 min read
Category
6 articles
LinkedIn
Venice: Serving Derived Data from LinkedIn's Batch and Stream Pipelines
Why LinkedIn built Venice to bridge offline-computed and streaming-computed data into a low-latency read-only serving store for machine learning features.
3 min read
LinkedIn
Inside LinkedIn's A/B Testing Platform and Experimentation Culture
How LinkedIn built T-REX, its internal experimentation platform, to make running and trusting thousands of concurrent A/B tests routine rather than risky.
3 min read
LinkedIn
Rest.li: Standardizing APIs Across Hundreds of LinkedIn Teams
How LinkedIn's move to a service-oriented architecture forced a standardized REST framework, and why schema-driven API contracts paid off at scale.
3 min read
LinkedIn
Espresso: The Document Store Behind LinkedIn's Member Data
Why LinkedIn built its own distributed document database to replace Oracle for online member data, and how it married timeline consistency with Kafka.
3 min read
LinkedIn
Samza: Making Stream Processing a First-Class Citizen Next to Kafka
How LinkedIn built Apache Samza to pair with Kafka, using partitioned logs and local state to make stateful stream processing durable and rebalanceable.
3 min read