Let's discuss today how Kafka handles message consumption. We'll start by explaining Consumer Groups and how they manage consumers and data distribution. Then, go deeper into partition assignments, fault tolerance, and the trade-offs involved. We'll wrap it up with real-world implications of Kafka’s design, comparing it to other solutions and discussing if it's always as great as pictured.
Great write-up Oskar. Thanks for a quick ending ;) Question I have is how consumers effectively communicate with the broker. How a consumer knows when rebalancing happen? Does it get info in a heartbeat request or a broker maintains a consumer registry and uses it for that purpose (push)?
Not that I'm complaining but I think the default PlantUML sequence diagrams are more readable than the ones used here (especially the italics). Sorry for complaining:)
Great write-up Oskar. Thanks for a quick ending ;) Question I have is how consumers effectively communicate with the broker. How a consumer knows when rebalancing happen? Does it get info in a heartbeat request or a broker maintains a consumer registry and uses it for that purpose (push)?
Michał, again, thanks for the good question and a bit of Nostradamus skills, as that's what I wanted to discuss and actually did in the latest additions: https://www.architecture-weekly.com/p/understanding-kafkas-consumer-protocol
I'm curious your next question 😀
Not that I'm complaining but I think the default PlantUML sequence diagrams are more readable than the ones used here (especially the italics). Sorry for complaining:)
Yeah, some monospace font without italics would be much better :)
Good feedback guys, in the latest edition, I stopped using italics :)