Automation Testing Tools Resources

If you are flagging features on production, we’ll test these features as we do A/B tests. If they are being introduced in staging, they may break our tests, at which point we will respond and use exploratory testing to understand what changed, and update the test. If they cause features to disappear completely, you’re likely to get a bug alert that you’ll know to disregard. If they introduce brand new behavior that does not break the tests, this looks more like a brand new feature—we’ll test it when it reaches production.

Show More

Show Less

If you’re A/B testing on production (such that different cohorts/customers see different features), we’ll detect these multiple pathways and test both. We will infer or force the conditions that trigger the A or B on staging.

Show More

Show Less

We break up different user groups and run analyses separately for them. For example: an education technology application may have students, teachers, school administrators, and super-admins. We’ll analyze each of these separately and deploy tests for all four groups, typically with different logins for each type of user.

Show More

Show Less