It's been two months since this profile received a new review

Pepperdata Capacity Optimizer Reviews & Product Details

Pepperdata Capacity Optimizer is the only real-time, automated cost optimization solution for Spark workloads that can immediately save you up to 47%. By instructing the scheduler to consider the actual resource utilization instead of allocated resources, Capacity Optimizer reclaims waste, maximizes resources utilization, and optimizes autoscaling in the cloud. It operates autonomously, continuously, and in real time to reclaim application waste.

Product Website

Seller

Pepperdata

Discussions

Pepperdata Capacity Optimizer Community

Languages Supported

English

Overview by

Tom Niermeyer

Pricing

Pricing provided by Pepperdata Capacity Optimizer.

Pay-As-You-Go

$0.02

1 Per hour of usage vCPU fee Per Month

View More Pricing Information

Pepperdata Capacity Optimizer Media

Pepperdata Capacity Optimizer Demo - Pepperdata Capacity Optimizer

Get recommendations for optimization, automatic tuning, and alerting. Pepperdata Capacity Optimizer visualizes real-time and historical information about your clusters including system demand, abusive users, and wasteful applications.

Pepperdata Capacity Optimizer Demo - Autoscaling Before and After

First chart shows cluster growing to 100 nodes for runtime duration. Second chart shows each node running a single task, and then zero tasks, for runtime duration.

Pepperdata Capacity Optimizer for Amazon EMR operates dynamically in real time to optimize performance without the need to model your workloads ahead of time, change application code, or platform settings.

In order for Autodesk to achieve their goals for application scaling and reducing their cloud costs, they installed Pepperdata Capacity Optimizer's autonomous, continuous, intelligent real-time optimization software.

Watch Ben Smith, VP Technical Operations at Extole, discuss how he discovered that there's around 30% of application waste within every running app, and how Extole went about saving that wasted capacity.

Through either our own website or the AWS Marketplace, you have multiple options and ways you can get Pepperdata Capacity Optimizer into your big data stack without any code changes or without manual tuning.

Official Downloads

(5)

edit

Pepperdata for Amazon EKS

Autodesk Reduces Amazon EMR Costs by 50% and Boosts Performance with Pepperdata

Pepperdata Reduces the Cost of Running Spark Workloads on Amazon EKS at Scale by 41.8%

Pepperdata Capacity Optimizer Reviews (39)

G2 reviews are authentic and verified.

Here's how.

Chiranjeevi E.

Consultant

Enterprise (> 1000 emp.)

8/26/2022

"Pepperdata helps us in optimizing our day to day tasks."

5/5

What do you like best about Pepperdata Capacity Optimizer?

Its easy to go through the UI and get the stats of the tasks and see the errors and optimize them accordingly. Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

Nothing to my knowledge. It helps us decoding the root cause while monitoring the application tasks very well. Review collected by and hosted on G2.com.

Response from Ashish Sahu of Pepperdata Capacity Optimizer

edit

Thank you for the review. We are pleased to hear that the UI is easy for you to use and that Pepperdata has been useful for your root cause analysis.

Trupal P.

Sr. Software Engineer , Cloud Infrastructure

Mid-Market (51-1000 emp.)

4/25/2022

"best for spark application monitor"

5/5

What do you like best about Pepperdata Capacity Optimizer?

easy to navigate for all metrics related to spark job, capture all yarn-related metrics. we can search by application id easily. multiple realm is also useful for EMR spark Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

while setting up pepper data autoscaling first time without pepper data support was made EMR autoscaling aggressive because we have to found working autoscaling config for our clusters Review collected by and hosted on G2.com.

Recommendations to others considering Pepperdata Capacity Optimizer:

none Review collected by and hosted on G2.com.

What problems is Pepperdata Capacity Optimizer solving and how is that benefiting you?

Pepperdata provides optimization suggestion on spark application, and also provide bottlenecks why jobs have not performed well. easy to find yarn level error on application Review collected by and hosted on G2.com.

Verified User in Retail

Enterprise (> 1000 emp.)

8/25/2022

"Pepperdata lets us see inside our ephemeral clusters even after they've been deleted."

5/5

What do you like best about Pepperdata Capacity Optimizer?

Being able to see the memory, cpu, io and other cluster metrics help us to appropriately size the clusters and tune our jobs. Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

The UI has a learning curve when dealing with large-scale ephemeral clusters, but after a while the filtering and drill-down become familiar. Review collected by and hosted on G2.com.

Response from Ashish Sahu of Pepperdata Capacity Optimizer

edit

Thank you for the review. We love that you're able to tune your ephemeral cluster workloads using Pepperdata. We understand that the UI can take some getting used to, but we're happy to hear that it has now become familiar. To add, we include user training for all our new enterprise customers if you’d like to learn more about the product’s capabilities.

Verified User in Financial Services

Enterprise (> 1000 emp.)

5/1/2022

"Great enterprise solution"

5/5

What do you like best about Pepperdata Capacity Optimizer?

Great enterprise solution for optimizing big data workloads. Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

I have not had negative experiences with Pepperdata. It's a great solution to meeting my needs. Review collected by and hosted on G2.com.

Uddipan M.

Engineering Manager

Enterprise (> 1000 emp.)

2/19/2021

"Pepperdata - Complete Spark Cluster performance monitoring"

4.5/5

What do you like best about Pepperdata Capacity Optimizer?

Pepperdata provides insightful details and helpful metrics to monitor cluster performance. The user-friendly interface helps to navigate several details like CPU, Memory, Storage usage, I/O etc.

Pepperdata also helps to optimize the cluster settings to ensure enhanced resource management.

App spotlight feature helps to figure out slow jobs or any data skew scenarios Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

Pepperdata can integrate automated alerting in case of any cluster parameter shows alarming behavior. It can also provide features like AI-based detection of resource containment and alert accordingly.

It will be good if Pepperdata can integrate workflow managers like Oozie in the monitored services and map Oozie coordinator/Job Ids to spark Application ids. This will help in easy mapping between interfaces and will result in quicker root cause analysis of issues. Review collected by and hosted on G2.com.

Recommendations to others considering Pepperdata Capacity Optimizer:

Pepperdata is a helpful spark cluster monitoring tool, which provides all relevant metrics of a cluster in a centralized portal. Review collected by and hosted on G2.com.

What problems is Pepperdata Capacity Optimizer solving and how is that benefiting you?

1) Automated Spark cluster optimization

2) Root cause analysis of any issues with Spark jobs

3) Detecting data skew scenarios Review collected by and hosted on G2.com.

Response from Andy Gremett of Pepperdata Capacity Optimizer

edit

Hi Uddipan! Thank you for your feedback. Regarding alerting, we have alarms and alerts like what you describe, and here’s the help documentation:

https://help.pepperdata.com/nav-pd-if/08-alarms-alerts/

Regarding workflows, we have a way of assigning workflow IDs that you might find helpful. There’s a way to assign IDs that show as workflows in the Pepperdata interface.

https://help.pepperdata.com/v6-4/admin-guide/reports/05-assigning-workflow-ids/

I have shared your feedback with our product team for review.

Pepper D.

Associate Director - Data Scientist

Enterprise (> 1000 emp.)

2/16/2021

"Pepperdata Helps Identify Cost Savings"

4/5

What do you like best about Pepperdata Capacity Optimizer?

I like that Pepperdata allows me to identify various jobs in my ETL architecture to isolate each job and provide a breakdown of: computational time used, memory required, wasted resources, estimated costs. Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

While Pepperdata provides a wealth of information which can be used to troubleshoot and improve the efficiency of ETL architectures; I have found that it is difficult to build and create a dashboard specific to my own needs. For example, the default dashboards made available to the Enterprise work well at an Enterprise level from the perspective of the person paying the bill, but there seems to be little to no instruction on how I can create a suite of dashboards to focus on my specific subset of the Enterprise jobs running. Review collected by and hosted on G2.com.

Recommendations to others considering Pepperdata Capacity Optimizer:

Use Pepperdata if you are paying the costs of compute. Review collected by and hosted on G2.com.

What problems is Pepperdata Capacity Optimizer solving and how is that benefiting you?

The problem that I have found most relevant to solve with Pepperdata is specifically in identifying bottlenecks and inefficiencies in the ETL architecture I have set up. Given that our ETL's as part of the Enterprise are written by a team of people, such monitoring allows me to not only identify jobs and ETL's which are inefficient and could be tightened, but also acts as a means of identifying members on the team who could use upskilling as a result of consistently running inefficient and poorly optimized jobs. Considering a portion of the Enterprise setup also takes into account forecasting of load demand to plan for capacity; we can also use Pepperdata to track and plan the amount of resources that will be needed by the cluster down the road. Review collected by and hosted on G2.com.

Response from Andy Gremett of Pepperdata Capacity Optimizer

edit

Hi,

Thank you for your feedback. We do offer custom dashboards and if you are looking for something different, we’d be curious to understand your request/requirements.

https://help.pepperdata.com/plat-spot/04-working-with-custom-dashboards/

We’d like to talk to you more about what you’d like to see.

Please contact Info@pepperdata.com and let us know how to reach you.

Surendar M.

Enterprise (> 1000 emp.)

2/16/2021

"Pepperdata - Perfect tool to monitor the production servers"

4.5/5

What do you like best about Pepperdata Capacity Optimizer?

What do you dislike about Pepperdata Capacity Optimizer?

Multiple metrics are available and it is bit difficult to identify the suitable metric. Also the log filters are not so efficient when it comes to analyzing huge logs with number of metrices involved. Initial configuration of the pepper data is challenging due to technical difficulties, but once the configurations are completed the handling of the tool is much simple. Review collected by and hosted on G2.com.

Recommendations to others considering Pepperdata Capacity Optimizer:

The visualization on the data collected from the server logs. With verities of options to create your own customized dashboards. Also can create customized alerts in case of any threshold breaches. It helped us a lot on resolving performance issues and analyze the server performance. Also to visualize the reports which can be presented to stakeholders. Create custom charts with available metrics to understand the data (or) performance trends for longer period of time. Multiple metrics are available and it is bit difficult to identify the suitable metric. Also the log filters are not so efficient when it comes to analyzing huge logs with number of metrices involved. Initial configuration of the pepper data is challenging due to technical difficulties, but once the configurations are completed the handling of the tool is much simple. Review collected by and hosted on G2.com.

What problems is Pepperdata Capacity Optimizer solving and how is that benefiting you?

It helped us a lot on resolving performance issues and analyze the server performance. Also to visualize the reports which can be presented to stakeholders. Create custom charts with available metrics to understand the data (or) performance trends for longer period of time. Review collected by and hosted on G2.com.

Response from Andy Gremett of Pepperdata Capacity Optimizer

edit

Hi Surendar,

We are glad to hear from you. Regarding metrics, we do offer many metrics, which can be challenging but gives you lots of flexibility and choices. Would it help to add a better description of these metrics?

We would like to better understand what “log filters” you are referring to. Please contact us at Info@pepperdata.com and let us know the best way to reach you if you are willing to discuss further.

We are at work on simplifying the initial installation of Pepperdata, particularly for cloud installations like EMR clusters.

I have shared your feedback with our product team for review.

Pervej A.

Senior cloud administrator

Enterprise (> 1000 emp.)

10/21/2021

"Big data insights using PD"

5/5

What do you like best about Pepperdata Capacity Optimizer?

Amount of metrics provided by PD along with recommendations to tune the job Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

The recommendations tab should also include ways to implement the suggestions ex. spark submit command Review collected by and hosted on G2.com.

Minhaj B.

Senior Technical Systems Analyst

Enterprise (> 1000 emp.)

2/18/2021

"Very good for hadoop cluster analysis"

4.5/5

What do you like best about Pepperdata Capacity Optimizer?

Pepper Data provides quite detailed analysis about your cluster health. It provides visualization of CPU Resources, memory utilization, Storage, RAM and other important details. For each visualization you can see more details for a specific range of time and for a specific user. You can switch between chart and table display modes. In addition to this it provides detailed information about Queues and Application Status. Apart from the cluster overall details, you can see the Resource Usage of a Spark App as well by using the yarn application id, which is very useful for improving performance and scaling the processes. Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

I wish that they provided more info, when hovered on column names, in tabular format/mode, so that it is more easily readable and understandable. Review collected by and hosted on G2.com.

Recommendations to others considering Pepperdata Capacity Optimizer:

Very good tool if you are into Hadoop or Spark and are going to use it extensively for Performance monitoring. Review collected by and hosted on G2.com.

What problems is Pepperdata Capacity Optimizer solving and how is that benefiting you?

We are using it to monitor our hadoop clusters in all our environments that is Dev, UAT and Prod. We use it to view and analyze our Spark Application by using the yarn application id. which is very useful for improving performance and scaling the processes. Review collected by and hosted on G2.com.

Response from Andy Gremett of Pepperdata Capacity Optimizer

edit

Minhaj,

Thank you for your question. Can you clarify which tables you’re referring to you which don’t have descriptive enough titles? We looked at several of our top tables and didn’t see this.

Please contact us at Info@pepperdata.com and let us know the best way to reach you.

Verified User in Financial Services

Enterprise (> 1000 emp.)

2/18/2021

"Peperdata is pretty handy to profile spark/hadoop job in terms of CPU, memory ussage and disk."

4/5

What do you like best about Pepperdata Capacity Optimizer?

it has YARN resource visualization, which is pretty intuitive. The UI looks good. Also it has some suggestion on resource usage. Review collected by and hosted on G2.com.

What do you dislike about Pepperdata Capacity Optimizer?

1. Sometimes it has some delay on data collection and I have to wait until the job is done.

2. Too expensive to purchase the service

3. Need some effort to adapt the vendor solution to existed system.

4. It lacks of data security options Review collected by and hosted on G2.com.

Recommendations to others considering Pepperdata Capacity Optimizer:

1. it can provide customized sdk , api or metrics I can use in my spark application, So I can gain more in sighted from application point of view.

2. It can support code level profiling, which can be used in different use cases.

3. it can support alarming config so I can get notification if the resource is out of quote. Review collected by and hosted on G2.com.

What problems is Pepperdata Capacity Optimizer solving and how is that benefiting you?

I use it to debug out of memory issue of Spark job and done some troubleshooting. Overall it's great.

It saves a lot of YARN resource and speed up my proformance tuning.

It provides my visualization to explain the actual question to management team to gain better understanding of how much spent in computation resource. Review collected by and hosted on G2.com.

Response from Andy Gremett of Pepperdata Capacity Optimizer

edit

Hi,

We’re sorry to hear you had problems adapting issues into Pepperdata. What specific issues did you encounter?

We see that you are a user in financial services, so we understand the need for security options. What data security options would you like to see?

Regarding your positive comment about debugging Spark applications, we’re glad the details help you debug your apps. We’d love to hear any feedback you might have on how we can continue to improve our Spark recommendations.

Please contact us at Info@pepperdata.com if you are willing to discuss further. Your input is much appreciated!