Driving Streaming in Production

We've talked a lot about the theoretics (and even walked through some exercises) of Spark Streaming in this section. You might recall that many of our examples involved writing input to netcat, from which our Streaming Application picked up data. In production settings, we'll need to leverage tools that offer Pub/Sub architectures that are both addressable and can scale with lead. One popular tool is Kafka.

There are other Cloud-native streaming technologies as well which offer similar capabilities. The next section Streaming Technologies (Bonus) gives an overview of these technologies in addition to a deep dive into Kafka topics (pun-intended, which you will understand after reading that section). The Streaming Technologies section is marked as Bonus because it requires a pre-requisite to multi-system cloud architectures, which isn't a pre-requisite to this training. We suggest going back to that section at a later time, but it remains in the narrative for the subset of our readers who want to venture in that direction.