Spark SQL Reviews & Product Details

It's faster processing time and abstract over low level structures with dataframes. Review collected by and hosted on G2.com.

Spark SQL documentation needs to be improved. Review collected by and hosted on G2.com.

Explanation about rdd and how we perform all the operations in distributed worker node Review collected by and hosted on G2.com.

Very basic level not much was there about connecting spark withbother data source Review collected by and hosted on G2.com.

Caching

Temporary table usage

It can be used for structured databases as well Review collected by and hosted on G2.com.

No automatic optimisation techniques

High learning curve, better documentation needed Review collected by and hosted on G2.com.

I like the most about spark sql is it is easy to use and we can easily apply normal sql query for it and we can use it in btw pyspark notebook also using %sql by register dataframe as temp table. Review collected by and hosted on G2.com.

Not any particular dislike some time I faced difficulty in performance and run time of cell if we run any complex query that has lot join and other condition to reduce this I use cte and make the query in chunks. Review collected by and hosted on G2.com.

Spark SQL has a few capabilities that I particularly like:

Spark SQL is a go-to product because big data technologies interface with it so seamlessly.

It has a high efficiency and can quickly process plenty of data.

Because of how similar its query syntax is to standard SQL, learning a new language is made easier. Review collected by and hosted on G2.com.

My main gripe with Spark SQL is its limitations, which include latency issues, minor file issues, and the lack of real-time data processing. Apache Apex, an alternative solution, has already resolved some issues. However, these issues must be resolved at Spark SQL, as an alternative is acceptable, but some features provided by Spark SQL are not available with Apex. Review collected by and hosted on G2.com.

Spark sql is nothing but an query language which gives the developer for fetching and analysis the data in HDFS.spark sql support the indexing features as well as the partitions features so that the data is which is going to load is arranged in a manner so it can be easily fetch and good to go for analysis.spark core provide the platform or and terminal where we can write our queries and perform any operational task. So easy to learn and implement. Review collected by and hosted on G2.com.

Spark core doesn't support the cache features so it can't store any queries results or a query in cache so every time a developer perform a query its allways use the warehouse or full scan of data.so i feel it should give the cache features so the computation cost will be low when someone using the spark framework. Review collected by and hosted on G2.com.

As it helps to run a large number of data sets in a distribution manner.

Spark SQL provides a programming abstraction called Data Frames and can also act as a distributed SQL query engine. Review collected by and hosted on G2.com.

Nothing As such.

But joining in this manner is difficult if you are joining unbalance data sets.

It creates a large skewed data set that does not use for a suitable data model. Review collected by and hosted on G2.com.

The integration of sql queries with the spark programs is one the best features. Speed of processing the big queries as well as optimization of storage space is second to none. Review collected by and hosted on G2.com.

The cost of the tool is on the expensive side in comparison to what it offers. If you are on intranet, there could be a latency issue while executing the queries. Review collected by and hosted on G2.com.

Spark works more effectively compared to other databases like MySQL. It can load more data compared to other databases and is better when working with huge datasets. Review collected by and hosted on G2.com.

I wouldn't say it's a disadvantage, but the premier version of Spark is more costly compared to other databases like MySQL, SQLite, but Spark SQL provides a premier experience. Review collected by and hosted on G2.com.

Best thing is we can process our data in Spark with SQL code. SparkSQL has catalyst optimiser which come up with best execute plan and DAG to execute in spark. Review collected by and hosted on G2.com.

It uses much memory while processing the data which leads to out of memory issues. Review collected by and hosted on G2.com.