Google Cloud Dataflow Reviews & Product Details

Best thing about Dataflow about its fully managed capability so that we don't need to manage infrastructure and scales easily. It also provides lot of templates which is useful for beginner and intermediate level developers and top of that they can easily update the configuration and pipeline and can run process petabyte of data. Also it supports Yaml SDK which removes Apache Beam dependencies as well. Review collected by and hosted on G2.com.

When we are working with distributed processing, its difficult to get correct configuration especially for new user its very complex to set it up and most of the time it charges a lot if not set properly. And as it supports only Apache Beam, some of the concepts are very difficult to understand. Also they can work on monitoring and logging, sometime its not clear. Review collected by and hosted on G2.com.

Google Cloud Dataflow is extremely easy to use for processing stream of events. Building complex streaming pipelines is simple and effiicent with Dataflow. Offers real time monitoring of the streaming pipeline with important metrics such as Throughput, CPU and memory utilisation.

It is easy to integrate with Data sources and sinks such as Cloud Pub-sub, Kafka and Cloud Spanner. Review collected by and hosted on G2.com.

It is costly as compared to other solutions such as Apache Flink.

Difficult to implement watermarks. Review collected by and hosted on G2.com.

- using jyupter notebook to write code and not to worry about backend services.

GCP Dataflow effortlessly manages big data tasks, unifying code with Apache Beam. It's a Google Cloud team player, excelling with services like BigQuery. Review collected by and hosted on G2.com.

- not much documentation and resources available on it

Cost intricacies demand vigilance, and external connectors may need custom attention. While it harmonizes within the Google Cloud realm, dependence on the ecosystem may not suit those in other cloud neighborhoods. Review collected by and hosted on G2.com.

Cloud dataflow allows you to have a daemon that performs ETL while providing top tier observability. Prior to this I was accustomed to long running jobs with poor observability, or low/no code tools that didnt allow me to configure tasks the way I wanted to using code. Dataflow allows me to utilize apache beam on python to great effect to make a repeatable and easily monitored solution. Review collected by and hosted on G2.com.

The python examples could be a lot more comprehensive. also the concurrency model is difficult to understand, I sometimes experience lock contention with simultaneously running DoFn instances and its not entirely clear how many concurrent threads are processing my workload Review collected by and hosted on G2.com.

Google cloud dataflow is automatically optimize and manages resources for you this platform supports multiple programming languages including Python, java and SQL and makes it easy for developers to focus on writing codes Review collected by and hosted on G2.com.

Till now I have not found any kind of drawback.I just love this and every update of this platform makes me fall again Review collected by and hosted on G2.com.

Fault tolerance is the best thing about dataflow with the easy to launch the job and monitor Review collected by and hosted on G2.com.

Python sdk seems less evolved. Kafka integration for python is not ready for production Review collected by and hosted on G2.com.

Visualisations of your nodes, and insights during runtime regarding execution, how many nodes have been spun up. Review collected by and hosted on G2.com.

Once you trigger a run, there is no way to cancel/kill it. This means if you notice any issues during execution, you are pretty much stuck. Review collected by and hosted on G2.com.

I really like Apache Beam, which I use with Dataflow. It works well, on Google Cloud, for both Batch and Stream processing. The compute just works, and let's our Data Engineers get things done. Review collected by and hosted on G2.com.

Some of the marketing - years ago - was overkill. These days it lives up to the hype. I am saddened that more people don't use, because it is so powerful. Review collected by and hosted on G2.com.

The UI is very simplistic and the schedules and jobs can be created very easily alongside. Also the support for templates in Java and python are incredible. Review collected by and hosted on G2.com.

The parameter and templates are very rigid and further debugging is hard as the logs aren't very resourceful. If a pipeline fails often the errors and logs are poor. Review collected by and hosted on G2.com.

Google Dataflow is great as it provides the easiest and smoothest experience of loading data, be it in Batch or Streaming. The best aspects are ;

1. It provides 100% efficient Big Data processing along with easy data migration.

2. Ability to create custom Apps and design APIs.

3. Its developer-friendly user- interface and working. Review collected by and hosted on G2.com.

I think it is one of the best platforms to work on Big Data but the fact that it is very difficult to understand. Even the helping guides and documentation are limited and the community is not so big. I know in the coming years it can be solved and I hope that it is achieved as soon as possible. Review collected by and hosted on G2.com.