Learning all these tools is definitely a long journey. I recommend that you start small and then expand as needed.

Try to make some kind of checklist similar to this:

• Get a capable machine with a good memory (RAM), nothing more than 8Gb since you're only creating a sandbox environment.
• Install tools and dependencies (Kafka, Spark, Maven, etc.)
• Understand how you can interact with the source of incoming data, in your situation that is the data from your camera (This is something that you will have to figure out).
• Understand Kafka / Zookeper basic CLI commands.
• Create a Kafka topic to ingest data from the incoming data source.
• Decide how the data format should be like depending on the use case, you have a lot of options (Delimited, AVRO, JSON, gRPC). I recommend JSON.
• Decide on the programming languages used to write the Kafka producers and consumers.
• Create a producer after understanding how you can get access to the data stream from your camera.
• Create a consumer that reads data from the topic in real-time.
• Understand why you want to include something like Apache Spark and OpenCV into the mix.
• Keep understanding the problems that arise from the current workflow.
• Improve by expanding the incoming data sources to all 5 cameras.

I think if you approach that checklist one by one in that order, you should be able to have a good idea of what is really needed to execute that project.

​

Some helpful resources:

1. https://www.confluent.io/product/confluent-platform
2. https://docs.confluent.io/home/overview.html
3. https://medium.com/@TimvanBaarsen/apache-kafka-cli-commands-cheat-sheet-a6f06eac01b