I have used Fluentd on real-time distributed logging machines hosted on cloud to gather raw logs, filter logs, parse raw logs to well-formed JSON, forward all well-formed JSON logs to central fluentd machine (with backup node) central fluentd machine acts as an intermediate node which further forwards logs to the elastic search cluster master node. I had configured Kibanna and Elast alerts on master elastic search node to view and get real time business alerts. Review collected by and hosted on G2.com.

Performance is little show - each Fluentd instance can only process around 18,000 events per second.

We can enable multi-process workers to increase throughput, but this may cause problems with plugins that don’t support this feature. Review collected by and hosted on G2.com.