scalable

data is partitioned

brokered

stable versions of kafka enables to upgrade the kafka at any time

Kafka parallelism at the topic level is very useful to achieve performance and scale to million requests per second

kafka has provided very well defined storage management, it provides many options to configure them by size & by time Review collected by and hosted on G2.com.

load balancing requires all ips to be known by producer

When the kafka is hosted in any on prem environment, usually users does not want to expose all three ip address due to seccurity concerns. But kafka client needs all ip addresses of kafka cluster to load balance

Load time of the data when any node restart is huge when we have large data in kafka before restart

when one node goes down in cluster, it takes lot of time to rebalance Review collected by and hosted on G2.com.

Kafka is known the best streaming platform, it has the capability to handle data in pull mechanism makes it highly performant. Kafka provides capabilities to segregate data via topics, it supports binary data and text data. It has the capability to distribute data to partitions, a notion of shards

Kafka has consumer - kafka streams which is built on the principles of kafka to give better performance

we considered replacing storm with kafka streams and could achieve all the required functionalities

kafka provides various tune ups to achieve HA Review collected by and hosted on G2.com.

One thing I observed is we can not switch off replication in multi node setup. suppose say i want only load distribution but its okay if i lose data. I dont want any replication. If any partition assigned node goes down, kafka should identify and reassign the partitions at some point to other nodes instead it goes on toss expecting isr to be available

another point is when any node goes down it takes a while to reassign the primary partitions and make kafka available, though config is there to set the time to trigger rebalancing, usually it kicks the rebalancing but the rebalancing itself takes lot of time Review collected by and hosted on G2.com.

The nature of message streaming is guaranteed to all consumers who subscribed and good thing is the message will never missed which guarantees data delivery Review collected by and hosted on G2.com.

No issues with Kafka in terms of topics and streaming and couldn’t any drawbacks Review collected by and hosted on G2.com.

The way we can scale it, as opposed to rabbitmq, the persistent intermediate storage with ksql, and paralleling data by topics Review collected by and hosted on G2.com.

The official sink and source connectors, from debezium have issues with some advanced data types, especially arrays and son, when reading from postgres WAL files. Review collected by and hosted on G2.com.

Apache Kafka is a very fast and scalable messaging system. I used it for all the microservices asynchronous communication channels. It is extremely easy to setup and configure according to your needs. It also has sdks to help using it with wide range of technology stack. It also has a central broker which governs the master instance and takes care of failure and scaling. Review collected by and hosted on G2.com.

The fact that it broker is not inbuilt and requires zookeper. But its not a dealbreaker. Also that you do not have a dedicated gui for tracking and maintenance purposes. Review collected by and hosted on G2.com.

I like the simplicity of setup of Kafka broker in cluster mode. It permits to have a three powerful Kafka brokers by using even a single virtual low cost machine. The interaction with the cluster is quite simple and you can customize the topics functionalities by setting replication factors and main copy broker.

I like also the flexibility of Kafka. It can be used for managing high quantity of messages from multiple sensors, but also to stream a single big video to multiple subscribers. Review collected by and hosted on G2.com.

I have not found anything that I dislike until now. Maybe the confluent support can be improved, since they offer only support for payed Confluent licenses and not for on-premises Kafka installation. Review collected by and hosted on G2.com.

Very fast messaging platform - consumer groups are a great feature - So is replication. Kafka streams are fantastic. Review collected by and hosted on G2.com.

Whatever I have used worked well. Some more standard monitoring tools should be part of the basic setup. Review collected by and hosted on G2.com.

Ability to use kafka as database, capability to handle large volume of data efficiently. Review collected by and hosted on G2.com.

Very less monitoring tools , not all the scripts available for monitoring. Review collected by and hosted on G2.com.

Consumer group is one of my favourite feature which let multiple consumers to consume at the same time but avoid reading duplicates. Review collected by and hosted on G2.com.

It’s highly configurable, I mean with configuration settings we can manage Kafka. This is both good and bad. It’s really easy to set up, but if we miss something, we get screwed. One good example: Our Kafka consumer will not poll until it process all the currently running jobs. In this case the running jobs takes up to an hour. So, Kafka consumer will not do any activities for an hour. In this case, zookeeper assumes my Kafka consumer died and recreates the new one and read the same message which processed already. This was finally a confit settings Review collected by and hosted on G2.com.

We were looking for an open-source solution to perform stream processing. When our team found this, it was immediately accepted to be used as our permanent solution to achieve the goal we had. Review collected by and hosted on G2.com.

We haven't found any issues so far in Kafka. Review collected by and hosted on G2.com.