G2

Ajay Kumar S.

Analyst

Enterprise (> 1000 emp.)

Originally submitted 7/14/2020

"Azure Data Factory For Delta Processing And Moving Data in Azure"

5/5

What do you like best about Azure Data Factory?

The best part of the Azure Data Factory is that by using Service or User Authentication (Azure Key Vault to fetch Secrets), one can connect to many Databases all around the technologies. You can connect to various databases like Mongo DB, Google Big Query, Azure Data Lake Gen1, Azure Synapse, Azure SQL, Postgre SQL, Oracle, EDW, etc. Also in this, we create pipelines that are easy to implement either by writing codes or by dragging/dropping the fields and attributes and then providing values for them. One does not need to manually run the pipeline, it can be scheduled as needed and many pipelines can work in parallel if we use different self-hosted Integrated runtime machine. Azure Data Factory can be integrated with various tools like Azure Functions, Azure Web Jobs, Azure Data Bricks. Review collected by and hosted on G2.com.

What do you dislike about Azure Data Factory?

Till now I have nothing to mention here as I am using the DataFactory for various purposes. But sometimes it takes a while to load the Azure Data Factory Interface which can be due to Slow Network or machine. Review collected by and hosted on G2.com.

Recommendations to others considering Azure Data Factory:

If you are working on the projects which involve lots of Data Movement and Data Modeling and you want things to happen in a sequential manner, then I would request to use the Azure Data Factory to create pipelines that can be scheduled to run as required and also save time and resource. Various process/projects where you can use Azure Data Factory are:- 1) Setting up od Delta Lake Setup in Azure Data lake to do the Delta Processing. 2) Refreshing Azure Analysis Model by triggering Logic Apps to refresh tables with real-time data for Analysis and Reporting. 3) To Stop or Start Azure Analysis Server to save some cost in which it triggers the run book of Azure Automation Account which pauses the server over weekends. 4) Bringing data from Google Analytics and Google BigQuery and store the data in the form of tables in Azure Data Ware House. 5) Union of Data Pipes from two workstreams following the same schema. 6) Moving data from various Databases ad then storing them on Azure Data Ware House. 7) Running Azure Function without any manual intervention. 8) Running Azure Web Jobs without any manual intervention. 9) For Automation of some tasks and processes required for software development and maintenance. I will mention that the use of Azure Data Factory in Data Science Projects is very much needed and priceless. Review collected by and hosted on G2.com.

What problems is Azure Data Factory solving and how is that benefiting you?

I have used the Azure Data Factory for various purposes. You can consider Azure Data Factory the same as SSIS. We have used Data Factory to bring data from enormous Databases to Azure DataLake. The Problems that Azure Data Factory solved for us are:- 1) Setting up od Delta Lake Setup in Azure Data lake to do the Delta Processing. 2) Refreshing Azure Analysis Model by triggering Logic Apps to refresh tables with real-time data for Analysis and Reporting. 3) To Stop or Start Azure Analysis Server to save some cost in which it triggers the run book of Azure Automation Account which pauses the server over weekends. 4) Bringing data from Google Analytics and Google BigQuery and store the data in the form of tables in Azure Data Ware House. 5) Union of Data Pipes from two workstreams following the same schema. 6) Moving data from various Databases ad then storing them on Azure Data Ware House. 7) Running Azure Function without any manual intervention. 8) Running Azure Web Jobs without any manual intervention. 9) For Automation of some tasks and processes required for software development and maintenance. Review collected by and hosted on G2.com.