# Spark SQL Reviews **Vendor:** The Apache Software Foundation **Category:** [Relational Databases](https://www.g2.com/categories/relational-databases) **Average Rating:** 4.5/5.0 **Total Reviews:** 45 ## About Spark SQL Spark SQL lets you query structured data inside Spark programs, using either SQL or a familiar DataFrame API. Usable in Java, Scala, Python and R. ## Spark SQL Reviews ### 1. When the data is huge, sparksql is the answer **Rating:** 5.0/5.0 stars **Reviewed by:** Manjunath K. | Machine Learning Intern, Enterprise (> 1000 emp.) **Reviewed Date:** March 28, 2024 **What do you like best about Spark SQL?** querying data from both in RDDs and external sources at ease. And able to query historical data with only Spark engine reducing the dependency of different engine. **What do you dislike about Spark SQL?** limitations in real time processing of the data. **What problems is Spark SQL solving and how is that benefiting you?** For querying/analysing marketing data, we need to connect to sources which is huge in data. SparkSQL processes this at ease. ### 2. Spark SQL revolutionized data processing with seamless SQL for efficient, scalable analytics. **Rating:** 4.5/5.0 stars **Reviewed by:** Kumar A. | Data Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** November 18, 2023 **What do you like best about Spark SQL?** I appreciate Spark SQL's robust query optimization and unified data processing capabilities, providing a streamlined and high-performance solution for complex analytics tasks." **What do you dislike about Spark SQL?** While Spark SQL impresses with robust query optimization and unified data processing, occasional challenges in resource management and the absence of some conventional SQL functions may require careful consideration in specific analytics scenarios. **What problems is Spark SQL solving and how is that benefiting you?** Spark SQL plays a pivotal role in solving business problems by offering a unified platform for efficient data processing and analytics. Its unique strengths in handling diverse data sources, executing complex queries, and providing high-performance processing contribute significantly to accelerating decision-making, fostering data-driven insights, and ultimately enhancing business performance and agility. ### 3. Excellent **Rating:** 4.5/5.0 stars **Reviewed by:** ROSHAN S. | Small-Business (50 or fewer emp.) **Reviewed Date:** January 17, 2024 **What do you like best about Spark SQL?** The ability to manipulate the spark data frame using SQL queries. **What do you dislike about Spark SQL?** The syntax can seem a bit different at the start. **What problems is Spark SQL solving and how is that benefiting you?** I am using Spark SQL in databricks. I am using it to build ETL pipelines. I has been excellent for my usage be it extracting data from Delta tables or manipulating the Spark Dataframes. ### 4. Spark SQL is a module which can run on both dataframes and SQL queries. **Rating:** 5.0/5.0 stars **Reviewed by:** Sneh H. | Software Engineer, Small-Business (50 or fewer emp.) **Reviewed Date:** June 05, 2023 **What do you like best about Spark SQL?** It can run on dataframe as well as act as a distributed query engine. Unlike the normal SQL, it is a module which is used for structured databases. It allows Hadoop Hive queries to run like 100 times faster on existing deployment of data and therefore allowing big data to be handled efficiently. **What do you dislike about Spark SQL?** There is no file management system of it's own and so it needs to be connected with one. There are no automatuc file optimisation techniques and we have to optimise our codes manylually. There is no support for real time processing and also there are issues with small files while working with hadoop. **What problems is Spark SQL solving and how is that benefiting you?** It quickly performs processing tasks on big data sets and can also distrubute data processing tasks on multiple platforms. In my last organisation, I was working on a project which needed large datasets such parquet files containg million records and on querring it on Microsoft SQL server it took much time and my seniors were also not able to help me in real time. But Spark SQL solved this problem. ### 5. Faster SQL processing **Rating:** 5.0/5.0 stars **Reviewed by:** Nitish K. | Big Data Engineer, Computer Software, Small-Business (50 or fewer emp.) **Reviewed Date:** October 03, 2023 **What do you like best about Spark SQL?** It allows all types of SQL commands like mysql,mssql,posgresql and many with faster processing **What do you dislike about Spark SQL?** There isn't anything i disliked about Spark SQL **What problems is Spark SQL solving and how is that benefiting you?** It processes big queries using sql within Python environment. After querying it became easier to make machine learning predictions using pyspark mlib ### 6. 4 yrs of experience in Pyspark, Python, SQL, GCP. Worked for Telecom and Entertainment Industry **Rating:** 5.0/5.0 stars **Reviewed by:** Atul P. | Data Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** June 05, 2023 **What do you like best about Spark SQL?** Partitioning and The way to use Memory as well as Disk. Cache is one of the best features of Spark SQL. And use of temporary table. **What do you dislike about Spark SQL?** Limitated support for transaction. Performance impact with small Dataset.lack of native support of all SQL functions like proprietary extension. **What problems is Spark SQL solving and how is that benefiting you?** The use UDF in SQL queries. It helps me to write some python functions and use the same in Spark SQL. Scalability: Spark SQL scales horizontally, allowing you to process large datasets that do not fit into memory on a single machine. It automatically partitions and distributes data across a cluster, enabling parallel processing. ### 7. Integrated Spark SQL with other Spark components for real-time processing & machine learning tasks. **Rating:** 4.0/5.0 stars **Reviewed by:** Verified User in Computer Software | Enterprise (> 1000 emp.) **Reviewed Date:** June 05, 2023 **What do you like best about Spark SQL?** What I love about Spark SQL is its seamless integration with the Spark ecosystem, enabling me to leverage distributed computing capabilities and work with structured data using SQL syntax. The optimiser and query planner, Catalyst, ensures efficient execution. At the same time, its wide range of data source support and integration with other Spark components make it a powerful tool for end-to-end data processing. **What do you dislike about Spark SQL?** What I dislike are : Debugging complexity: Challenging to debug complex queries and optimize query plans. Performance tuning: Fine-tuning query performance requires in-depth knowledge and experimentation. Compatibility limitations: Not fully compatible with all SQL dialects and databases. Limited support for complex analytics: Some advanced SQL features may be unsupported or require custom implementations. Despite these challenges, I think Spark SQL is a powerful tool for distributed data processing with efficient and scalable capabilities. **What problems is Spark SQL solving and how is that benefiting you?** Spark SQL is solving several problems and benefiting me in multiple ways.It solves the problem of querying and analyzing large-scale structured and semi-structured data using SQL syntax. It benefits me by eliminating the need to learn new languages, seamlessly integrating with other Spark components, optimizing query performance, and supporting multiple programming languages. This allows for efficient data processing, faster insights, and improved productivity in my work. ### 8. Describe spark sql capabilities **Rating:** 5.0/5.0 stars **Reviewed by:** Tanmay A. | Data Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** April 23, 2023 **What do you like best about Spark SQL?** Easy to understand and using spark power via easy sqlskils ,this terminology easy to learn and any one who have the basic sql skills can easily work ,only some of the things from sql is different but most of the things are same **What do you dislike about Spark SQL?** In terms of downside spark sql , the thing which I don't like is that if I want to use some custom function either I need to use udf or udaf which is a little hectic think and it need good coding knowledge **What problems is Spark SQL solving and how is that benefiting you?** In terms of benefit this helps to reduce are lines of code and easy of understanding ,so me as a data engineer it helps me to debug and tune the spark jobs also helps me improve my sql skills ### 9. Simplified interface for querying structured data **Rating:** 4.5/5.0 stars **Reviewed by:** Harish K. | Senior Software Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** June 19, 2023 **What do you like best about Spark SQL?** Allowing users to seamlessly switch between different data processing and works with structured and semi structured data efficiently. Uses catalyst engine to enable spark sql to deliver fast and efficient query processing across large datasets. **What do you dislike about Spark SQL?** Error messages during the query execution provided by Spark sql can be challenging to interpret. Users who are not familiar with databases and sql concepts require some time and effort to grasp. **What problems is Spark SQL solving and how is that benefiting you?** Big data processing can be handled using Spark sql. Large scale datasets can be processed with high speed data processing. Helps to analyse massive volumes of data in parallel. ### 10. Powering Efficient Data Analysis and Querying **Rating:** 5.0/5.0 stars **Reviewed by:** Verified User in Telecommunications | Small-Business (50 or fewer emp.) **Reviewed Date:** June 15, 2023 **What do you like best about Spark SQL?** I have been using Spark SQL for quite some time now, and I must say it has completely revolutionized the way I analyze and query large-scale datasets. With its impressive capabilities and seamless integration with Apache Spark, Spark SQL has become an essential tool in my data processing toolkit. **What do you dislike about Spark SQL?** One area where Spark SQL can be challenging is its learning curve. **What problems is Spark SQL solving and how is that benefiting you?** Unified Data Processing: Traditionally, working with different data processing engines and languages for various data formats (e.g., SQL for structured data, MapReduce for unstructured data) was cumbersome and time-consuming. ### 11. Easy and efficient Analysis of data **Rating:** 4.0/5.0 stars **Reviewed by:** vimal k. | Dotnet Developer, Mid-Market (51-1000 emp.) **Reviewed Date:** July 22, 2023 **What do you like best about Spark SQL?** if you are familiar with sql queries, It is one of the best tool for you to analyse any data with minimum coding. it is effective of any type of data **What do you dislike about Spark SQL?** it is mostly awesome tool, but can provide intellisense, charts for user **What problems is Spark SQL solving and how is that benefiting you?** it provide best way for analysing data of any type if effective manner through which businesses can improve its services, recommendation and lots of things. ### 12. It is an amazing experience to use Spark SQL. **Rating:** 5.0/5.0 stars **Reviewed by:** Hitesh S. | Machine Learning Engineer, Small-Business (50 or fewer emp.) **Reviewed Date:** June 11, 2023 **What do you like best about Spark SQL?** The best thing about Spark SQL is Its ability to work with Large dataset with parallel distribution. Currently, I am working on 80 GB dataset and It is very useful in that. **What do you dislike about Spark SQL?** As of now my experience is very good with spark. So, I don't dislike anything about Spark SQL so far. **What problems is Spark SQL solving and how is that benefiting you?** Working with large dataset. As I mentioned earlier in my job most of the datasets are very large. So, It is really beneficial for us to work with those large datasets. ### 13. Spark SQL review **Rating:** 4.0/5.0 stars **Reviewed by:** SHWETA R. | Enterprise (> 1000 emp.) **Reviewed Date:** June 05, 2023 **What do you like best about Spark SQL?** I like it's easy SQL like syntax when querying from tables **What do you dislike about Spark SQL?** I've been using it for around a year, so nothing to dislike **What problems is Spark SQL solving and how is that benefiting you?** It is avoiding the need to use dataframes for retrieving data ### 14. Easy and Understandable queries **Rating:** 5.0/5.0 stars **Reviewed by:** Divya S. | Software Developer, Small-Business (50 or fewer emp.) **Reviewed Date:** June 19, 2023 **What do you like best about Spark SQL?** The UI is very user friendly and easy to navigate through all the tools. **What do you dislike about Spark SQL?** There is no such thing I dislike about Spark SQL. **What problems is Spark SQL solving and how is that benefiting you?** Instead of writing huge queries in other databases. Spark SQL does the work in one or two lines of queries with a very high processing speed. ### 15. Easy to learn **Rating:** 5.0/5.0 stars **Reviewed by:** Sateesh W. | Programmer trainee, Enterprise (> 1000 emp.) **Reviewed Date:** June 24, 2023 **What do you like best about Spark SQL?** The visual of editor in the website is too good, it helps me to find the errors happened in the actual line of code. **What do you dislike about Spark SQL?** Nothing, everything is good enough only. **What problems is Spark SQL solving and how is that benefiting you?** We can make a very good Data visualization in this platform. ### 16. Spark SQL review **Rating:** 5.0/5.0 stars **Reviewed by:** Verified User in Computer Software | Enterprise (> 1000 emp.) **Reviewed Date:** June 04, 2023 **What do you like best about Spark SQL?** Spark SQL made easy querying in Spark and it is easy to integrate with Hive. Also, it is easy to connect to any data source, and mainly JDBC connections are simple which is much needed for Application developers or Data Engineers. **What do you dislike about Spark SQL?** As of now, errors related to oversized varchars are not shown. Apart from that, everything is good. **What problems is Spark SQL solving and how is that benefiting you?** Spark SQL is solving issues with processing large amounts of data. It made data querying easy in the application ### 17. Its a great tool for all spark and sql queries **Rating:** 4.5/5.0 stars **Reviewed by:** Srawan Singh R. | Data Engineer II, Enterprise (> 1000 emp.) **Reviewed Date:** June 16, 2023 **What do you like best about Spark SQL?** Its very comprehensive tool for all big data and data modelling. It was well managed **What do you dislike about Spark SQL?** Nothing. All good. I liked everything about it **What problems is Spark SQL solving and how is that benefiting you?** Easy to handle db and schema ### 18. Most important Hadoop tool **Rating:** 5.0/5.0 stars **Reviewed by:** Verified User in Retail | Enterprise (> 1000 emp.) **Reviewed Date:** June 20, 2023 **What do you like best about Spark SQL?** Spark SQL is very user friendly even for someone who isn't from core coding background. **What do you dislike about Spark SQL?** So far I din't find any downside of using Spark SQL in my job role. **What problems is Spark SQL solving and how is that benefiting you?** I am basically using Spark SQL to analyse my client database in various marketplaces. ### 19. User friendly for people for working in Big Data Projects. **Rating:** 4.5/5.0 stars **Reviewed by:** Verified User in Apparel & Fashion | Mid-Market (51-1000 emp.) **Reviewed Date:** June 19, 2023 **What do you like best about Spark SQL?** Spark SQL doesn't require schema or table-like structure for performing advanced aggregations Code customization will be complicated when the dataset's schema changes, so Spark SQL supports customization. **What do you dislike about Spark SQL?** Most of the databases, like Hive, don't support this. Cost-Effective. **What problems is Spark SQL solving and how is that benefiting you?** Large Datasets. ### 20. Good technology overall **Rating:** 4.0/5.0 stars **Reviewed by:** Verified User in Financial Services | Mid-Market (51-1000 emp.) **Reviewed Date:** June 06, 2023 **What do you like best about Spark SQL?** It's faster processing time and abstract over low level structures with dataframes. **What do you dislike about Spark SQL?** Spark SQL documentation needs to be improved. **What problems is Spark SQL solving and how is that benefiting you?** While working with machine learning to do faster experiments we need fast engines like spark SQL for querying relational databases. For that we are using this. ### 21. It is a good course for beginners to understand basics of spark **Rating:** 4.0/5.0 stars **Reviewed by:** Verified User in Retail | Enterprise (> 1000 emp.) **Reviewed Date:** June 12, 2023 **What do you like best about Spark SQL?** Explanation about rdd and how we perform all the operations in distributed worker node **What do you dislike about Spark SQL?** Very basic level not much was there about connecting spark withbother data source **What problems is Spark SQL solving and how is that benefiting you?** I was using hive in replacement of cosmos db for cost benifits ### 22. Really brings the spark on **Rating:** 5.0/5.0 stars **Reviewed by:** Verified User in Information Technology and Services | Small-Business (50 or fewer emp.) **Reviewed Date:** June 19, 2023 **What do you like best about Spark SQL?** Caching Temporary table usage It can be used for structured databases as well **What do you dislike about Spark SQL?** No automatic optimisation techniques High learning curve, better documentation needed **What problems is Spark SQL solving and how is that benefiting you?** It helps processing huge volume of data in a simplistic manner ### 23. Easy to use mainly functionality like Nomal Sql **Rating:** 3.0/5.0 stars **Reviewed by:** Verified User in Management Consulting | Enterprise (> 1000 emp.) **Reviewed Date:** September 11, 2022 **What do you like best about Spark SQL?** I like the most about spark sql is it is easy to use and we can easily apply normal sql query for it and we can use it in btw pyspark notebook also using %sql by register dataframe as temp table. **What do you dislike about Spark SQL?** Not any particular dislike some time I faced difficulty in performance and run time of cell if we run any complex query that has lot join and other condition to reduce this I use cte and make the query in chunks. **What problems is Spark SQL solving and how is that benefiting you?** Sometimes if someone not have knowledge in python about transformation and if he has knowledge om sql he can easily do the transformation using sql spark. I also did lot of time sometime query complex in pyspark to reduce the efforts I used spark sql. ### 24. "How convenient it is to use is amazing!" **Rating:** 4.0/5.0 stars **Reviewed by:** Mubeen M. | Full Stack Developer, Small-Business (50 or fewer emp.) **Reviewed Date:** July 13, 2022 **What do you like best about Spark SQL?** Spark SQL has a few capabilities that I particularly like: Spark SQL is a go-to product because big data technologies interface with it so seamlessly. It has a high efficiency and can quickly process plenty of data. Because of how similar its query syntax is to standard SQL, learning a new language is made easier. **What do you dislike about Spark SQL?** My main gripe with Spark SQL is its limitations, which include latency issues, minor file issues, and the lack of real-time data processing. Apache Apex, an alternative solution, has already resolved some issues. However, these issues must be resolved at Spark SQL, as an alternative is acceptable, but some features provided by Spark SQL are not available with Apex. **What problems is Spark SQL solving and how is that benefiting you?** We are solving of the problem of using this for our AI and Machine learning algorithms, but also as a data storage and retrieval system in general.Spark sql requires a large amount of RAM to process data, resulting in excessive memory consumption. In addition, we are unable to create tables in Spark for union types. ### 25. How spark sql is so use full for data fetching and analysis. **Rating:** 4.5/5.0 stars **Reviewed by:** Arpan s. | System Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** February 05, 2022 **What do you like best about Spark SQL?** Spark sql is nothing but an query language which gives the developer for fetching and analysis the data in HDFS.spark sql support the indexing features as well as the partitions features so that the data is which is going to load is arranged in a manner so it can be easily fetch and good to go for analysis.spark core provide the platform or and terminal where we can write our queries and perform any operational task. So easy to learn and implement. **What do you dislike about Spark SQL?** Spark core doesn't support the cache features so it can't store any queries results or a query in cache so every time a developer perform a query its allways use the warehouse or full scan of data.so i feel it should give the cache features so the computation cost will be low when someone using the spark framework. **Recommendations to others considering Spark SQL:** Who wants to implement there python skills in backend you can use spark framework using python aswell. **What problems is Spark SQL solving and how is that benefiting you?** Shifting the data from warehouses to cloud storage we have used the spark sql for data transfering and analysis purposes in hdfs.so we have used spark in hadoop cluster system using the scala language. ### 26. A light wait extremally fast distribution engine to project large data sets. **Rating:** 4.5/5.0 stars **Reviewed by:** Gaurav G. | Senior Data Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** April 21, 2022 **What do you like best about Spark SQL?** As it helps to run a large number of data sets in a distribution manner. Spark SQL provides a programming abstraction called Data Frames and can also act as a distributed SQL query engine. **What do you dislike about Spark SQL?** Nothing As such. But joining in this manner is difficult if you are joining unbalance data sets. It creates a large skewed data set that does not use for a suitable data model. **What problems is Spark SQL solving and how is that benefiting you?** Joining multiple data frames and doing the transformation on top of that. Apart from that for data analysis using the windowing function. While using the Unified dataset. ### 27. One of the fastest computation tools out there **Rating:** 3.5/5.0 stars **Reviewed by:** Aaryan S. | Project Engineer - Turbo, Enterprise (> 1000 emp.) **Reviewed Date:** March 30, 2022 **What do you like best about Spark SQL?** The integration of sql queries with the spark programs is one the best features. Speed of processing the big queries as well as optimization of storage space is second to none. **What do you dislike about Spark SQL?** The cost of the tool is on the expensive side in comparison to what it offers. If you are on intranet, there could be a latency issue while executing the queries. **What problems is Spark SQL solving and how is that benefiting you?** I am using this in my spark programs to seamlessly fetch the data from queries. It is very easy to use the query structure of spark sql inside your code specially for beginners ### 28. Best software Spark **Rating:** 5.0/5.0 stars **Reviewed by:** Niyazahmedraza M. | A, Mid-Market (51-1000 emp.) **Reviewed Date:** April 06, 2022 **What do you like best about Spark SQL?** Spark works more effectively compared to other database like MySQL. It can load more data compared to other database and is better when working with huge datasets **What do you dislike about Spark SQL?** I wouldn't say it's disadvantage, but the premier version of Spark is more costly compared to other databases like MySQL, SQLite but Spark SQL provides premier experience **What problems is Spark SQL solving and how is that benefiting you?** I could load and work with huge datasets with Spark SQL. I couldn't do it with other regularly databes .Spark SQL has helped to me work with huge data and provide results more effectively ### 29. SparkSQL made our work easy **Rating:** 5.0/5.0 stars **Reviewed by:** Kubendra Reddy M. | Data Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** February 21, 2022 **What do you like best about Spark SQL?** Best thing is we can process our data in Spark with SQL code. SparkSQL has catalyst optimiser which come up with best execute plan and DAG to execute in spark. **What do you dislike about Spark SQL?** It uses much memory while processing the data which leads to out of memory issues. **Recommendations to others considering Spark SQL:** Definitely, you can go with this query engine. **What problems is Spark SQL solving and how is that benefiting you?** We used maintain our data warehouse in Hive and used to create data frames from existing hive tables in spark and processed the data with SparkSQL. Due to this our processing time optimised. ### 30. Spark SQL is very helpful for transformation on distributed data **Rating:** 3.5/5.0 stars **Reviewed by:** Pawan K. | Bigdata Platform Architect at HCL Technologies Limited, Enterprise (> 1000 emp.) **Reviewed Date:** February 16, 2022 **What do you like best about Spark SQL?** Spark SQL is faster, and additional type information makes more efficient **What do you dislike about Spark SQL?** As of now, no support for transaction tables **Recommendations to others considering Spark SQL:** If the company have a data analytics-based project and they have a business case for real-time and batch time data processing, they should use the spark framework **What problems is Spark SQL solving and how is that benefiting you?** In our project, we are using for batch processing of hive tables ### 31. Platform for Big data analytics **Rating:** 5.0/5.0 stars **Reviewed by:** Vishal A. | Head of Operations, Enterprise (> 1000 emp.) **Reviewed Date:** July 27, 2021 **What do you like best about Spark SQL?** Specifically talking the few best points I like about Spark SQL is as follows: - It is the best choice for big data analytics in collaboration with Hadoop. - It provides fast access to data in SQL workloads. - In Spark SQL, many types of data processing can be used together. - It is easy to pull in multiple data sources - from Spark RDD to external databases. - Spark SQL supports Map-reduce, SQL queries, Streaming data, Machine learning (ML), and Graph algorithms. **What do you dislike about Spark SQL?** My major dislike is Spark SQL's limitations, including Latency issues, minor files issues, and no real-time data processing. Apache has already resolved some with an alternative solution by Apache Apex. However, These issues need to be determined at Spark SQL as an alternative is okay, but some features that Spark SQL offers aren't available with Apex. **Recommendations to others considering Spark SQL:** My recommendations to others considering Spark SQL will be that if you want to work with Big data Analytics integrating with Hadoop. Then my suggestion would be Spark SQL straight away as It enables unmodified Hadoop Hive queries to run up to 100x faster. It also provides powerful integration with the rest of the Spark ecosystem. It can run SQL queries alongside complex analytic algorithms using tight integration. However, I still push to compare, analyze, then use your discretion to decide which Platform to choose. **What problems is Spark SQL solving and how is that benefiting you?** The problem we solved with the help of Spark SQL is based upon a feature that mixes SQL queries with Apache Spark programs. With the help of Spark SQL, we complied queries of structured data as a distributed dataset. We run SQL queries alongside complex analytic algorithms using tight integration, which solves complex problems more efficiently. It also made us realize that it could also act as a distributed SQL query engine and enabling unmodified Hadoop Hive queries to run faster on existing deployments and data. Spark SQL actually worked as a lifeline for us while working with Hadoop. ### 32. Analyse massive amounts of data **Rating:** 5.0/5.0 stars **Reviewed by:** Prabhjot S. | Lead Software Engineer, Machine Learning, Small-Business (50 or fewer emp.) **Reviewed Date:** April 15, 2022 **What do you like best about Spark SQL?** 1. Query big data with SQL syntax. 2. Perform aggregations. **What do you dislike about Spark SQL?** Setting up standalone spark cluster is difficult. **What problems is Spark SQL solving and how is that benefiting you?** Analysis of terabytes of data. ### 33. Data Processing at its best **Rating:** 4.5/5.0 stars **Reviewed by:** Nick M. | Graduate Research And Teaching Assistant, Mid-Market (51-1000 emp.) **Reviewed Date:** July 08, 2021 **What do you like best about Spark SQL?** One feature that I like very much as a software inclined person is that Spark is open source. This allows me to dig deeper into its value than other software that is out there that perform the same tasks. It is also extremely robust in functionality as software. Even more than that this framework and SQL version is extremely faster than its competitors because of the way that it processes data. Spark uses this idea called cluster mode and within that, it uses distributed processing and it allows for the computation in the frameworks to work entirely internally. Another great feature is the ability to source data from multiple different sources. The main use that I can see within this framework is machine learning and AI when you are using large amounts of data from multiple sources. It allows the best combinability and processing ability. **What do you dislike about Spark SQL?** One thing that I am not fond of with this software is that there is no dataset API support in the python version of this software. The reason this is an issue is that most machine learning and AI programs that I write are in the Python language, so without that support, it is a heavy downfall for me personally. It still works, but without that feature, it is a small letdown. In terms of other programming languages, it works great. Another potential issue is that there is an inability to support multiple concurrent users at a time which can provide potential roadblocks on big projects. **Recommendations to others considering Spark SQL:** A great tool in comparison to other software that is out there. Highly recommend trying it out. **What problems is Spark SQL solving and how is that benefiting you?** We are solving of the problem of using this for our AI and Machine learning algorithms, but also as a data storage and retrieval system in general. ### 34. Good tool with almost same syntax as prevalent SQL. **Rating:** 3.5/5.0 stars **Reviewed by:** Verified User in Information Technology and Services | Enterprise (> 1000 emp.) **Reviewed Date:** April 26, 2022 **What do you like best about Spark SQL?** Ability to handle big data and display the result in the least time. **What do you dislike about Spark SQL?** Noting. Faced no issue while using the same. **Recommendations to others considering Spark SQL:** It's nice tool. Give it a try. **What problems is Spark SQL solving and how is that benefiting you?** Querying the data with SPARK SQL and analyzing it. ### 35. Its Amazing, How convenient it is to use! **Rating:** 5.0/5.0 stars **Reviewed by:** Neeti T. | Automation Engineer, Enterprise (> 1000 emp.) **Reviewed Date:** August 16, 2021 **What do you like best about Spark SQL?** Some features that I like most about spark SQL are- Spark SQL is so smoothly integrated with big data tools that it makes it a go-to product. It can process a large amount of data in seconds, that the efficiency. Its query syntax are like regular SQL only which reduces efforts to learn a new language. **What do you dislike about Spark SQL?** Spark sql needs huge amount of RAM for processing data that maked too mamuch memory consumption. Also we can't create table in spark for union type. **Recommendations to others considering Spark SQL:** I would recommend it for sure. It definitely serves it's purpose. **What problems is Spark SQL solving and how is that benefiting you?** I'm using Spark SQL for Data analysis. We get a lot of data from various sources and mining them, and getting helpful information out of them using spark Programs. ### 36. Very usefull for complex spark jobs creation **Rating:** 4.5/5.0 stars **Reviewed by:** Nilanjan N. | Senior Associate Consultant, Small-Business (50 or fewer emp.) **Reviewed Date:** December 04, 2021 **What do you like best about Spark SQL?** Feasability smd easy understanding of the queries built inside sparksql **What do you dislike about Spark SQL?** Non friendly ui is very unsatisfying, it has to be improved **Recommendations to others considering Spark SQL:** Please understand your usecase first , this is for heavy jobs not streaming ot real time processing usefull **What problems is Spark SQL solving and how is that benefiting you?** Realtime data stream creaing becoming easy with spark sql ### 37. Spark sql **Rating:** 4.0/5.0 stars **Reviewed by:** Verified User in E-Learning | Small-Business (50 or fewer emp.) **Reviewed Date:** March 17, 2022 **What do you like best about Spark SQL?** Spark sql is a spark module for structured data processing **What do you dislike about Spark SQL?** No file management system,expensive,window criteria **What problems is Spark SQL solving and how is that benefiting you?** We can run sql queries alongside complex analytic algorithms using tight integration property of spark sql ### 38. Spark SQL **Rating:** 4.5/5.0 stars **Reviewed by:** Verified User in Information Technology and Services | Enterprise (> 1000 emp.) **Reviewed Date:** March 06, 2022 **What do you like best about Spark SQL?** Its support is to access different data sources like Hive, Avro, Parquet, ORC, JSON, and JDBC. **What do you dislike about Spark SQL?** It Doesn't support real-time processing. **What problems is Spark SQL solving and how is that benefiting you?** We are analyzing the critical business data coming in different formats more efficiently than earlier. ### 39. Best framework for dealing with extremely huge data **Rating:** 5.0/5.0 stars **Reviewed by:** Rama Chandra Rao N. | Senior Consultant, Enterprise (> 1000 emp.) **Reviewed Date:** January 11, 2021 **What do you like best about Spark SQL?** 1. We can write most of the SQL scripts to read the massive data 2. All kinds of SQL features are supported 3. The best feature I like is to use the SQL in python language and store the extract the data in python data frames 4. SQL views are very comfortable way to create access the data from data lakes 5. Since Spark SQL uses the Spark clusters it is very fast to run the data extraction related queries 6. Notebooks are more easy and comfortable way to write the code **What do you dislike about Spark SQL?** Only problem is create tables or views on top of the JSON data from data lake. And to setup the external meta store instead of upgrading the current metastore **What problems is Spark SQL solving and how is that benefiting you?** 1. Fast running jobs 2. Easier to deal with massive data 3. Easy for new users since it is mostly basic SQL 4. Integration with multiple third party software 5. Easy integration with different reporting tools ### 40. Spark SQL -Spark module for structured processing **Rating:** 5.0/5.0 stars **Reviewed by:** Deepika T. | Data Engineer, Small-Business (50 or fewer emp.) **Reviewed Date:** August 17, 2021 **What do you like best about Spark SQL?** It enables to run Hadoop-Hive queries 10x times faster than MR. It allows you to query data frames in python, scala, or java using queries in SQL. **What do you dislike about Spark SQL?** There is nothing I don't like about Spark SQL. It is best to query big data in a fast approach. **What problems is Spark SQL solving and how is that benefiting you?** We did some of the transformations using py spark and some using Spark SQL . Realized that spark SQL is much easier to write, and execution time is almost similar for both. ### 41. Runs with lighting speed **Rating:** 4.0/5.0 stars **Reviewed by:** Verified User in Information Technology and Services | Mid-Market (51-1000 emp.) **Reviewed Date:** December 07, 2021 **What do you like best about Spark SQL?** 1. Speed of the queries when executing on Databricks 2. It gets easily integrated with scala. 3. Very easy for distribution data. **What do you dislike about Spark SQL?** 1. Cost is very high 2. Maintenance is very high and complex **Recommendations to others considering Spark SQL:** Very recommended since it acts as distributed sql query engine **What problems is Spark SQL solving and how is that benefiting you?** 1. Using it for big data analysis 2. Using it to transform the data ### 42. My review on Spark SQL **Rating:** 5.0/5.0 stars **Reviewed by:** Verified User in Airlines/Aviation | Mid-Market (51-1000 emp.) **Reviewed Date:** June 22, 2021 **What do you like best about Spark SQL?** The speed and efficiency of SQL querying on distributed computing of Apache Spark platform is what I like the most about Spark SQL. **What do you dislike about Spark SQL?** The only disadvantage of Spark SQL that i dislike is, it lacks advanced security features **What problems is Spark SQL solving and how is that benefiting you?** I use Spark SQL daily on the organizational SQL-on-Hadoop tool. I find it useful as Spark SQL helps to fetch data for diverse machine learning tasks ### 43. Sparkling Fast SQL **Rating:** 5.0/5.0 stars **Reviewed by:** Verified User in Information Technology and Services | Enterprise (> 1000 emp.) **Reviewed Date:** August 19, 2021 **What do you like best about Spark SQL?** Spark SQL is incredibly fast when dealing with large amounts of data. And provides almost every function that is commonly used to transform and query data. **What do you dislike about Spark SQL?** It is not as intuitive as using exactly SQL. But I have found almost every function I need available but need to comb through the documentation. **What problems is Spark SQL solving and how is that benefiting you?** Massive queries dealing with millions of rows of data to enable machine learning models. ### 44. Nice distribution query and process large no of data **Rating:** 4.0/5.0 stars **Reviewed by:** Verified User in Electrical/Electronic Manufacturing | Mid-Market (51-1000 emp.) **Reviewed Date:** September 01, 2021 **What do you like best about Spark SQL?** Distribution and faster query process and aggregation **What do you dislike about Spark SQL?** Not specific one,sometime it took longer time for aggregation **What problems is Spark SQL solving and how is that benefiting you?** Process larger number of data in distribution mode. ### 45. Spark SQL is easy to learn and easy to write complex query in simple low code pattern. **Rating:** 5.0/5.0 stars **Reviewed by:** Verified User in Electrical/Electronic Manufacturing | Mid-Market (51-1000 emp.) **Reviewed Date:** June 16, 2021 **What do you like best about Spark SQL?** Easy to learn and easy to understand. Can write complex queries in just few lines compare with oracle SQL queries. **What do you dislike about Spark SQL?** While working with huge data sometimes get hang. **What problems is Spark SQL solving and how is that benefiting you?** While we reterive huge data from DB using Spark SQL quickly take remove duplicates easily massage data as per required formats. ## Spark SQL Discussions - [What type of SQL does Spark use?](https://www.g2.com/discussions/what-type-of-sql-does-spark-use) - 2 comments, 1 upvote - [What is the difference between SQL and Spark SQL?](https://www.g2.com/discussions/what-is-the-difference-between-sql-and-spark-sql) - 1 comment - [What is the functionality of Spark SQL?](https://www.g2.com/discussions/what-is-the-functionality-of-spark-sql) - 1 comment - [View Spark SQL pricing details and edition comparison](https://www.g2.com/products/spark-sql/reviews?section=pricing&secure%5Bexpires_at%5D=2026-06-01+15%3A26%3A32+-0500&secure%5Bsession_id%5D=e3074f6b-c102-4f58-8a8b-4ff92550790b&secure%5Btoken%5D=90a7fd4bec9ef667e15d22a0c269b595f69bff12fd212d45e4baac0b9a53e892&format=llm_user) ## Spark SQL Features **Management ** - Data Schema - Query Language - ACID - Complaint - Data Replication **Support ** - Text Search - Data Types - Languages - Operating Systems **Security** - Database Locking - Access Control - Encryption - Authentication **Performance ** - Disaster Recovery - Data Concurrency - Workload Management - Advanced Indexing - Query Optimizer **Database Features** - Storage - Availability - Stability - Scalability - Security - Data Manipulation - Query Language ## Top Spark SQL Alternatives - [Oracle Database](https://www.g2.com/products/oracle-database/reviews) - 4.3/5.0 (927 reviews) - [PostgreSQL](https://www.g2.com/products/postgresql/reviews) - 4.4/5.0 (646 reviews) - [ClickHouse](https://www.g2.com/products/clickhouse/reviews) - 4.5/5.0 (22 reviews)