Streaming Systems: The What, Where, When, and How of Large-Scale Data Processing

Streaming data is a big deal in big data these days. As more and more businesses seek to tame the massive unbounded data sets that pervade our world, streaming systems have finally reached a level of maturity sufficient for mainstream adoption. With this practical guide, data engineers, data scientists, and developers will learn how to work with streaming data in a conceptual and platform-agnostic way.

Expanded from Tyler Akidau’s popular blog posts "Streaming 101" and "Streaming 102", this book takes you from an introductory level to a nuanced understanding of the what, where, when, and how of processing real-time data streams. You’ll also dive deep into watermarks and exactly-once processing with co-authors Slava Chernyak and Reuven Lax.

You’ll explore:

• How streaming and batch data processing patterns compare
• The core principles and concepts behind robust out-of-order data processing
• How watermarks track progress and completeness in infinite datasets
• How exactly-once data processing techniques ensure correctness
• How the concepts of streams and tables form the foundations of both batch and streaming data processing
• The practical motivations behind a powerful persistent state mechanism, driven by a real-world example
• How time-varying relations provide a link between stream processing and the world of SQL and relational algebra

• Visualizing Streaming Data: Interactive Analysis Beyond Static Limits
• Stream Processing with Apache Flink: Fundamentals, Implementation, and Operation of Streaming Applications
• Foundations for Architecting Data Solutions: Managing Successful Data Projects
• Database Internals: A Deep Dive into How Distributed Data Systems Work
• Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
• Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale
• Designing Distributed Systems: Patterns and Paradigms for Scalable, Reliable Services
• Building Microservices: Designing Fine-Grained Systems
• Kafka Streams in Action: Real-time apps and microservices with the Kafka Streams API
• Distributed Systems