[apache-kafka] When to use RabbitMQ over Kafka?

I know it's a bit late and maybe you already, indirectly, said it, but again, Kafka is not a queue at all, it's a log (as someone said above, poll based).

To make it simple, the most obvious use case when you should prefer RabbitMQ (or any queue techno) over Kafka is the following one :

You have multiple consumers consuming from a queue and whenever there is a new message in the queue and an available consumer, you want this message to be processed. If you look closely at how Kafka works, you'll notice it does not know how to do that, because of partition scaling, you'll have a consumer dedicated to a partition and you'll get into starvation issue. Issue that is easily avoided by using simple queue techno. You can think of using a thread that will dispatch the different messages from same partition, but again, Kafka does not have any selective acknowledgment mechanisms.

The most you could do is doing as those guys and try to transform Kafka as a queue : https://github.com/softwaremill/kmq

Yannick