Overview (Bonus)

So far, we've spent quite some time understanding the vast set of technologies used in Data Engineering and focusing primarily on batch processing. In this section, we'll talk about another type of architecture, streaming, which will allow you to scale and handle more real-time data.

We'll first cover Apache Kafka, a common tool used in the streaming space. Then we'll talk about what some common cloud offerings are and how they work and compare to Apache Kafka. Lastly, we'll do into some more advanced streaming processing and analytics concepts before jumping into our stateful streaming exercise.