Recommendations to others considering Apache Flink:
Use it wisely, tune the memory parameters and parallelism wisely. Otherwise you end up back pressure or under utilising the resources
Lot of tuning with respect to num of threads and memory allotment is required
do not overwrite the processors, which will lead to a lot of parallelism and simply data transfer between the nodes and can lead to slow down
Need to archirect cautiously Review collected by and hosted on G2.com.
What problems is Apache Flink solving and how is that benefiting you?
We use flink for both online streaming and offline batch processing
Mainly to enrich the incoming data, integrated with elastic search to store it. We also do aggregation using tumbling winfdow. We use flink views
For batch processing, we do use to learn some thresholds, like cpu, memory thresholds etc.,
Deployed with 100's of nodes, highly scalable
deployed in aws using kubernetese container
We also use flink ui to debug high level issues
We dont do sql on streaming Review collected by and hosted on G2.com.