Apache Kafka is a distributed stream processing software developed by LinkedIn and written in Scala and Java.  

Chapters 
0:00 Intro 
2:32 Kafka Broker
5:39 Kafka Producer
7:11 Kafka Consumer
8:34 Kafka Partitions
12:50 Queue vs Pub-Sub
16:47 Consumer Group
23:06 ZooKeeper
29:45 Coding Example
1:02:48 Kafka Pros-Cons
1:17:00 Summary

Download slides here 
https://payhip.com/b/Y0Cq
(Members get all slides for all my video courses for free)


In this video I want to focus on the basics of Kafka, talk about how it works give finally spin up a Kafka cluster and write a producer and a consumer. 

Kafka Components
_ producer
_consumer
_topic 
_partition
How Kafka works
Example ( Ride sharing ) 
Spin Kafka cluster 
Write a producer 
Write a consumer
Kafka Applications
Pros and Cons 



Source Code 
https://github.com/hnasr/javascript_playground/tree/master/kafka

Docker commands spin up kafka
//if you ever mess up or you want to start fresh just stop and remove the containers and rerun the command 

docker stop zookeeper kafka
docker rm zookeeper kafka

docker run --name zookeeper  -p 2181:2181 -d zookeeper


docker run -p 9092:9092 --name kafka  -e KAFKA_ZOOKEEPER_CONNECT=husseinmac:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://husseinmac:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 -d confluentinc/cp-kafka 

Get my fundamentals of database engineering course link redirects to udemy with coupon applied (https://database.husseinnasser.com)

Resources
 KIP 500 (Remove Zookeeper)
https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum
 
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join

🧑‍🏫 Courses I Teach
https://husseinnasser.com/courses 

🏭 Backend Engineering Videos in Order
https://backend.husseinnasser.com

💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2

🎙️Listen to the Backend Engineering Podcast
https://husseinnasser.com/podcast 

Gears and tools used on the Channel (affiliates)

🖼️ Slides and Thumbnail Design 
Canva
https://partner.canva.com/c/2766475/647168/10068

🎙️ Mic Gear
Shure SM7B Cardioid Dynamic Microphone  
https://amzn.to/3o1NiBi

Cloudlifter
https://amzn.to/2RAeyLo

XLR cables
https://amzn.to/3tvMJRu

Focusrite Audio Interface
https://amzn.to/3f2vjGY


📷 Camera Gear
Canon M50 Mark II
https://amzn.to/3o2ed0c

Micro HDMI to HDMI 
https://amzn.to/3uwCxK3

Video capture card
https://amzn.to/3f34pyD

AC Wall for constant power
https://amzn.to/3eueoxP


Stay Awesome,
Hussein

Apache Kafka Crash Course

Weekly system design newsletter: https://bit.ly/3tfAlYD

Checkout our bestselling System Design Interview books: 
Volume 1: https://amzn.to/3Ou7gkd
Volume 2: https://amzn.to/3HqGozy

Other things we made:
Digital version of System Design Interview books: https://bit.ly/3mlDSk9
Twitter: https://bit.ly/3HqEz5G
LinkedIn:  https://bit.ly/39h22JK

ABOUT US: 
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.