If the company have a data analytics-based project and they have a business case for real-time and batch time data processing, they should use the spark framework Review collected by and hosted on G2.com.

What problems is Spark SQL solving and how is that benefiting you?

In our project, we are using for batch processing of hive tables Review collected by and hosted on G2.com.

Vishal A.

Head of Operations

Enterprise (> 1000 emp.)

7/27/2021

"Platform for Big data analytics"

5/5

What do you like best about Spark SQL?

Specifically talking the few best points I like about Spark SQL is as follows:

- It is the best choice for big data analytics in collaboration with Hadoop.

- It provides fast access to data in SQL workloads.

- In Spark SQL, many types of data processing can be used together.

- It is easy to pull in multiple data sources - from Spark RDD to external databases.

- Spark SQL supports Map-reduce, SQL queries, Streaming data, Machine learning (ML), and Graph algorithms. Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

My major dislike is Spark SQL's limitations, including Latency issues, minor files issues, and no real-time data processing. Apache has already resolved some with an alternative solution by Apache Apex. However, These issues need to be determined at Spark SQL as an alternative is okay, but some features that Spark SQL offers aren't available with Apex. Review collected by and hosted on G2.com.

Recommendations to others considering Spark SQL:

My recommendations to others considering Spark SQL will be that if you want to work with Big data Analytics integrating with Hadoop. Then my suggestion would be Spark SQL straight away as It enables unmodified Hadoop Hive queries to run up to 100x faster. It also provides powerful integration with the rest of the Spark ecosystem. It can run SQL queries alongside complex analytic algorithms using tight integration. However, I still push to compare, analyze, then use your discretion to decide which Platform to choose. Review collected by and hosted on G2.com.

What problems is Spark SQL solving and how is that benefiting you?

The problem we solved with the help of Spark SQL is based upon a feature that mixes SQL queries with Apache Spark programs. With the help of Spark SQL, we complied queries of structured data as a distributed dataset. We run SQL queries alongside complex analytic algorithms using tight integration, which solves complex problems more efficiently. It also made us realize that it could also act as a distributed SQL query engine and enabling unmodified Hadoop Hive queries to run faster on existing deployments and data. Spark SQL actually worked as a lifeline for us while working with Hadoop. Review collected by and hosted on G2.com.

Prabhjot S.

Lead Software Engineer, Machine Learning

Small-Business (50 or fewer emp.)

4/15/2022

"Analyse massive amounts of data"

5/5

What do you like best about Spark SQL?

1. Query big data with SQL syntax.

2. Perform aggregations. Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

Setting up standalone spark cluster is difficult. Review collected by and hosted on G2.com.

Nick M.

Graduate Research And Teaching Assistant

Mid-Market (51-1000 emp.)

7/8/2021

"Data Processing at its best"

4.5/5

What do you like best about Spark SQL?

One feature that I like very much as a software inclined person is that Spark is open source. This allows me to dig deeper into its value than other software that is out there that perform the same tasks. It is also extremely robust in functionality as software. Even more than that this framework and SQL version is extremely faster than its competitors because of the way that it processes data. Spark uses this idea called cluster mode and within that, it uses distributed processing and it allows for the computation in the frameworks to work entirely internally. Another great feature is the ability to source data from multiple different sources. The main use that I can see within this framework is machine learning and AI when you are using large amounts of data from multiple sources. It allows the best combinability and processing ability. Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

One thing that I am not fond of with this software is that there is no dataset API support in the python version of this software. The reason this is an issue is that most machine learning and AI programs that I write are in the Python language, so without that support, it is a heavy downfall for me personally. It still works, but without that feature, it is a small letdown. In terms of other programming languages, it works great. Another potential issue is that there is an inability to support multiple concurrent users at a time which can provide potential roadblocks on big projects. Review collected by and hosted on G2.com.

Recommendations to others considering Spark SQL:

A great tool in comparison to other software that is out there. Highly recommend trying it out. Review collected by and hosted on G2.com.

What problems is Spark SQL solving and how is that benefiting you?

We are solving the problem of using this for our AI and Machine learning algorithms, but also as a data storage and retrieval system in general. Review collected by and hosted on G2.com.

Verified User in Information Technology and Services

Enterprise (> 1000 emp.)

4/26/2022

"Good tool with almost same syntax as prevalent SQL."

3.5/5

What do you like best about Spark SQL?

Ability to handle big data and display the result in the least time. Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

Noting. Faced no issue while using the same. Review collected by and hosted on G2.com.

Neeti T.

Automation Engineer

Enterprise (> 1000 emp.)

8/16/2021

"Its Amazing, How convenient it is to use!"

5/5

What do you like best about Spark SQL?

Some features that I like most about spark SQL are- Spark SQL is so smoothly integrated with big data tools that it makes it a go-to product. It can process a large amount of data in seconds, that the efficiency. Its query syntax are like regular SQL only which reduces efforts to learn a new language. Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

Spark sql needs huge amount of RAM for processing data that maked too mamuch memory consumption. Also we can't create table in spark for union type. Review collected by and hosted on G2.com.

Recommendations to others considering Spark SQL:

I would recommend it for sure. It definitely serves it's purpose. Review collected by and hosted on G2.com.

What problems is Spark SQL solving and how is that benefiting you?

I'm using Spark SQL for Data analysis. We get a lot of data from various sources and mining them, and getting helpful information out of them using spark Programs. Review collected by and hosted on G2.com.

Nilanjan N.

Senior Associate Consultant

Small-Business (50 or fewer emp.)

12/4/2021

"Very useful for complex Spark jobs creation"

4.5/5

What do you like best about Spark SQL?

Feasibility and easy understanding of the queries built inside SparkSQL Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

Non-friendly UI is very unsatisfying, it has to be improved Review collected by and hosted on G2.com.

Verified User in E-Learning

Small-Business (50 or fewer emp.)

3/17/2022

"Spark SQL"

4/5

What do you like best about Spark SQL?

Spark SQL is a Spark module for structured data processing Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

No file management system, expensive, window criteria Review collected by and hosted on G2.com.

Verified User in Information Technology and Services

Enterprise (> 1000 emp.)

3/6/2022

"Spark SQL"

4.5/5

What do you like best about Spark SQL?

Its support is to access different data sources like Hive, Avro, Parquet, ORC, JSON, and JDBC. Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

It Doesn't support real-time processing. Review collected by and hosted on G2.com.

Rama Chandra Rao N.

Senior Consultant

Enterprise (> 1000 emp.)

1/11/2021

"Best framework for dealing with extremely huge data"

5/5

What do you like best about Spark SQL?

1. We can write most of the SQL scripts to read the massive data

2. All kinds of SQL features are supported

3. The best feature I like is to use the SQL in python language and store the extract the data in python data frames

4. SQL views are very comfortable way to create access the data from data lakes

5. Since Spark SQL uses the Spark clusters it is very fast to run the data extraction related queries

6. Notebooks are more easy and comfortable way to write the code Review collected by and hosted on G2.com.

What do you dislike about Spark SQL?

Only problem is create tables or views on top of the JSON data from data lake. And to setup the external meta store instead of upgrading the current metastore Review collected by and hosted on G2.com.