Amplitude Feature Experimentation Reviews & Product Details

I like Amplitude Feature Experimentation's ability to conduct A/B testing and try out different features to see what works best for our users. What I find most valuable is its depth and wide consideration of user needs, like defining the hypothesis and background, and setting primary and secondary success metrics. I also appreciate how it helps navigate to the users whom the experiment has been assigned. Additionally, creating multiple variants within the experiment cohort based on multiple logics is a feature I really enjoy. Review collected by and hosted on G2.com.

One thing I find difficult is triangulating the number of users to whom the experiment is assigned. For example, if the experiment is rolled out to 50% of the users, then we should have a clear definition of the base cohort from which the 50% users are rolled out. Also, I wish we had the option to directly download the cohort users for different variants from the experiment only. In the beginning, it was a little overwhelming, and we were afraid that one thing wrong could impact users, but now the team is used to it and we find it easy. Review collected by and hosted on G2.com.

I love that Amplitude Feature Experimentation gives full control over the rollout and lets me scale up or down or switch it off any time. It's also awesome for quickly building charts that dive deeper than primary and secondary metrics. The ability to control rollouts means product teams can make decisions and act on them quickly without needing engineering support straight away, allowing us to roll out successful experiences as soon as we conclude tests. Additionally, the setup was pretty quick and straightforward using their SDK. I also appreciate that Amplitude stood out during our assessment of different A/B testing tools as it's a world-class behavioral analytics tool with many add-ons like A/B testing, session replay, guides, and surveys, so it's like having one tool for end-to-end analytics. Review collected by and hosted on G2.com.

Our setup doesn't allow controlling filtering from the Amplitude platform; it's only available through the code, as the feature was disabled due to slowing down our backend. It would be great to have more self-serving features that don't impact internal systems performance. Review collected by and hosted on G2.com.

We like that Amplitude Feature Experimentation is integrated with the Amplitude product analytics platform. It allows us to iterate quickly, and since it is easy to use, I can recommend Amplitude Feature Experimentation especially to organizations that are new to experimentation. Review collected by and hosted on G2.com.

Looking at competitors, Amplitude Feature Experimentation lacks more advanced implementations of CUPED and Bonferroni correction, as well as from supporting a Bayesian inference approach.

In addition, it would be convenient if the automatically generated experiment summary could be exported as a .pdf or notify relevant Slack channels once a decision in an experiment is made. Review collected by and hosted on G2.com.

The best part is that when we have all our analytics data going into the amplitude, and from there how we can segment users into experiments and create different cohorts based on existing data and also perform analysis and breakdown which is tightly integrated into each other.

When we want to run simple experiments, create multiple different variants, and also not require engineering effort to run, start, stop and analyse experiment data is what makes it really good.

We can create specific conditions that only when those are matched we will be seeing assignment and also have a custom exposure when things are more tricky.

We use it for a very wide range of complex experimentation where we found custom exposure to be quite helpful, and the segmenting part of the experiment.

It is easy to use, easy-medium to implement (Engineering documentation needs to be improved), customer support is good. Review collected by and hosted on G2.com.

The downsides of amplitude expeirment is that we due to its anonymous id resolution which causes variant jumping. This causes the analytics data to be very inaccurate in some very specific scenarios where we are testing users who were anonymous going into the app. Rest of the times works perfectly Review collected by and hosted on G2.com.

Amplitude makes it easier for us to conduct feature experimentation. Whenever we encounter difficulties, we can reach out directly to customer support for assistance. Additionally, the platform allows us to estimate our minimum sample size and calculate the minimum period needed to run our experiments. Review collected by and hosted on G2.com.

There isn't an official plugin available for Flutter, which made things more complicated. When I reached out to customer support for help, the investigation into my issue took far too long. Review collected by and hosted on G2.com.

A/B experimentation is very easy to setup & without any manual management. We are currently using it for every new feature that we are releasing Review collected by and hosted on G2.com.

Till the date, I haven't faced any issues with this. Review collected by and hosted on G2.com.

- The integration with Analytics. It’s super useful to validate experiment results with real behavioral data, not just vanity metrics.

- The developer experience is great. The SDKs are flexible, and setting up feature flags or holdout groups. Review collected by and hosted on G2.com.

- The documentation could go deeper into real-world use cases, sometimes you need to piece things together from examples or community discussions.

- Managing multiple environments can get a bit confusing at first, especially when syncing flags between staging and production. Review collected by and hosted on G2.com.

I like that Amplitude Feature Experimentation directly connects experimentation with product analytics. It makes it really easy to not only run A/B tests, but also to understand user behavior before, during, and after the experiment, which speeds up decision-making and increases confidence in results." Review collected by and hosted on G2.com.

I was looking for a guide or a way to learn this quickly and a demo to get started on it . Took me some time to get used to it Review collected by and hosted on G2.com.

The automatic statistical diagnostics - to spot if anything funky is going on easily Review collected by and hosted on G2.com.

Would like it to surface interesting segment analysis automatically (AI) Review collected by and hosted on G2.com.

Amplitude is clearly built around a powerful analytics engine that allows very flexible segmentation, cohorts, and filtering. This makes tightly targeting experiments fairly straightforward and, coupled with flexible custom dashboards, allows deep analysis of the results right inside Amplitude. Feature Experimentation and Feature Flags being based on the same engine makes integrating with amplitude simpler. Amplitude has quickly become the cornerstone of the growth team's data-based decision making. We're in there pretty much every day. Review collected by and hosted on G2.com.

My biggest grievance is the UI can be slow to navigate. Secondly, I couldn't find anywhere in the documentation that explains clearly which combinations are treated as AND vs OR, for filters and segments (for analyzing after,) so had to figure it out by trial and error. It all makes sense and all the logical operators I could ask for are there, but filter/segment query semantics could be better documented IMHO. Review collected by and hosted on G2.com.