Automation Testing Tools Resources

When managing an end to end test suite, any engineering leader has two options. You can either: Pause development to go update your end to end tests in order to (hopefully) prevent them from ever breaking due to UI changes, or Develop code at full speed and override tests that break due to UI changes We use the second option in order to minimize the need for coordination and to maximize your deployment speed. If a ProdPerfect test fails after you make a UI change, you should still investigate the failed test, as there may be a bug. If there’s not, simply override and deploy through the test suite failure. ProdPerfect will comment out the test and fix it asynchronously.

Show More

Show Less

If a new feature affects a flow users are currently traversing, our tests are likely to break, and we will update them after checking to make sure there’s not a bug. This means we test most new features. Features that are unrelated to old flows (for example: you add a configuration workflow for a Slack integration) and introduce brand new behavior aren’t detected by default and won’t be tested. Ultimately ProdPerfect is a regression testing suite. There is an exception: if new features spend over 48 hours in staging or preprod, and are being used by internal users during that time, we can configure ProdPerfect to detect that behavior to bootstrap tests for new features. We have deployed to a Beta customer group our Chrome extension autogenerator, which allows them to tell us what brand new feature sets they want tested with a few clicks. This should become live to all customers in a few months.

Show More

Show Less

Our tests traverse your application in the same way that your users do, but in staging or other pre-prod environment. Our tests expect to be able to fully traverse the same flow, and receive the same application returns that users do in production. For all data that is entered into the application during the test, we expect that data or its transformations to be returned on the DOM consistently. We also expect the application to not throw Javascript or other common types of errors. If these conditions are not met, or the application throws an error during the traversal, our tests will fail. This kind of testing is often called User Acceptance Testing, Functional Testing, or (loosely) Selenium Testing.

Show More

Show Less