Recommendations to others considering Apache Kafka:
it's a good platform for streaming your data in huge it's easily scalable. Only issue when you start scaling is you have to manually rebalance the partitions when you add a node or delete any existing nodes be careful about systems when restarts are frequent because it takes considerable time to restart, however it is the best solution you can get among all open-source solutions available Another aspect worth considering is its parallelism level, kafka is having pull mechanism to read the messages, partitions are per consumer, so it provides parallelism at partitions level Review collected by and hosted on G2.com.
What problems is Apache Kafka solving and how is that benefiting you?
data streaming from different IOT devices to processing engine, storm is connected to kafka and used to process and store in elastic search we also stream logs with the system, events within the system. we also log from various devices, these logs are huge in number. almost millions per second. Review collected by and hosted on G2.com.