Confidence by Spotify Reviews & Product Details

- Ability to set up AB testing with pre-defined success/ guardrail metrics, calculate sample size, test period

- Add as many metrics as possible, can add back metrics to calculate even after the test ends

- Good documentation for AB testing 101, different methodologies and terms are well-explained for newbies who never conducted an A/B test before Review collected by and hosted on G2.com.

Is there a way to view results by segment? I ran an AB test in India with filters like language and app version. After seeing the results, I'd like to break down the success and guardrail metrics by manufacturer and OS version, ideally right in the results tab. Review collected by and hosted on G2.com.

The UX/UI is very clean, everything is understandable. Also the mobile SDK integration is straightforward, even with openfeature. Review collected by and hosted on G2.com.

I’m not sure it’s 100% accurate. For example, if I set up a flag so that basically 50% of users are A and 50% are B, with no third option, I still see a tiny percentage showing up as “no variant,” which shouldn’t be possible. We were also early adopters, and for months we had crashes that seemed related to (and caused by) the confidence integration, but those issues are gone now. Review collected by and hosted on G2.com.

The setup and dashboard management are very user-friendly. It’s fast and easy to set up and run experiments, and the Spotify team provides prompt support when needed. I also appreciate the real-time analysis and the overall data management. Review collected by and hosted on G2.com.

I ran into issues integrating it with our web app because the SDKs differed and the documentation wasn’t detailed enough. During the initial setup, the AI support and MCP didn’t help much, so getting everything configured took longer than expected. Review collected by and hosted on G2.com.

Confidence enables us to take better product decisions and protects us from biases by design. The bootcamp content has been very valuable as a resource to upskill both ourselves in the team, and the organisation as a whole, leading to a higher experimentation quality overall.

Having direct contact with the Confidence team for support and expert opinions have also contributing to a stronger experimentation practice, and help in a short amount of time when needed. Review collected by and hosted on G2.com.

The key benefit of Confidence is that it's built by genuine experimentation experts, however, that sometimes means for people in the organisation who are new to experimentation theory and key concepts, the copy in the interface can sometimes come across as inaccessible or intimidating. Review collected by and hosted on G2.com.

Spotify Confidence platform is well designed and easy to use, and it helps avoid a lot of the manual work that would otherwise be required to run an A/B test. Review collected by and hosted on G2.com.

The initial learning curve is quite steep, and the onboarding could be improved (I joined fairly early on, so I’m not sure what it’s like currently). Review collected by and hosted on G2.com.

UI is easy to use and navigate. MCP via claude makes analysis and report generation a sinch. Support team is super responsive to any questions Review collected by and hosted on G2.com.

I wish the platform was more critical on the data we provide it. Or provide suggestions. Like your data has X, if you added Y and Z, you could also make use of A,B and C Review collected by and hosted on G2.com.

I use Confidence by Spotify for mobile games, A/B tests, and remote control. It helps us manage the full experimentation workflow in one place. I appreciate the easy UI which makes the platform user-friendly, and the built-in A/B test analysis is really useful. It allows us to quickly understand whether a change has a meaningful impact on key metrics and consolidates assignment, experiment setup, and results, eliminating the need to pull data from multiple tools. This all-in-one A/B test assignment and analysis helps us make faster, more confident rollout decisions. Review collected by and hosted on G2.com.

One area for improvement would be making the platform easier for new users to learn. While Confidence by Spotify is very powerful, there is a learning curve to understanding the experimentation workflow and all of its capabilities. I think it would benefit from a more guided onboarding experience. Interactive tutorials or step-by-step templates for common workflows, such as creating your first A/B test, setting up player properties, segmentation, configuring remote config, and interpreting experiment results, or even providing an example would help new users become productive much faster. Review collected by and hosted on G2.com.

I find Confidence by Spotify to be a great product for A/B testing and experimentation on the platform. It helps us iterate really fast and develop new features efficiently, isolating them and shipping them quicker than before. The automation with feature flags over API is particularly impressive, allowing us to create tons of feature flags every day and experiment with ease. Additionally, the initial setup was smooth, thanks to the super helpful sales team which made the process seamless. Review collected by and hosted on G2.com.

/ Review collected by and hosted on G2.com.

I think it has a very intuitive UI, if you don't have much knowledge about statistics (like me) the learning curve can be a little bit steep, but overall, with a little bit of effort you can get things done easily Review collected by and hosted on G2.com.

More badges that explain what does thing mean, things like reach or exposure Review collected by and hosted on G2.com.

Not having a build a separate infrastructure to measure outcomes of experiments, the tool connects to data that we already produce. This can be behavioural data from frontend analytics tools but also transactional data from backend systems. Review collected by and hosted on G2.com.

It does have a quite high learning curve where I at times can be uncertain on the best ways of actually setting up and monitoring experiments - the tool is flexible but that also comes with a complexity. Review collected by and hosted on G2.com.