G2

Harshal A.

Azure Data Engineer

Mid-Market (51-1000 emp.)

10/29/2026

"Flexible Azure Data Engineering with Databricks: Delta Lake, PySpark & Spark SQL"

4/5

What do you like best about Databricks?

As an Azure data engineer, I like Databricks because it gives me the flexibility to build data pipelines that can handle both structured and semi-structured data. I can use PySpark and Spark SQL to clean raw files, apply business rules, and prepare curated datasets for analytics. Working with Delta Lake is especially useful when managing incremental loads and maintaining reliable data across different processing stages. My favorite features are Delta Lake, PySpark notebooks, Spark SQL, workflow scheduling, and the medallion architecture. Delta Lake supports reliable table operations, and the workspace provides the best environment for developing and testing data pipelines. Notebooks make it easy to combine code, SQL, and output results in one place. Review collected by and hosted on G2.com.

What do you dislike about Databricks?

Some configurations require a good understanding of compute resources, access permissions, and workspace settings. Troubleshooting slow jobs can also take time when data is skewed or transformations are not optimized. Monitoring compute usage is important because poorly managed workloads can increase cloud costs. Review collected by and hosted on G2.com.

What problems is Databricks solving and how is that benefiting you?

It helps me to process data from multiple sources and transform it into a consistent format for downstream reporting. In an Azure environment, I can work with files stored in Azure Data Lake Storage, apply transformations using PySpark, and organize the results into bronze, silver, and gold layers. This makes it easier to implement data quality checks and maintain a repeatable ETL process. Review collected by and hosted on G2.com.

Response from SoHanna Minami of Databricks

edit

Thanks for the thoughtful review. We’re glad PySpark, Spark SQL, notebooks, and Delta Lake support your data engineering workflows, from raw-file processing to curated analytics datasets. Your comments about compute configuration, skew, and cloud costs are valuable insights for teams optimizing their Databricks environments.