Reusing validation tests as monitoring

People seemed generally agreeable that we want to write tests once in dev and get reuse out of them in production.

• Biggest hurdle perceived as handling persistence/state
• Running passive (readonly tests) would be trivial to do. Tests that write become a problem.
• Multi-tenanted solutions might mitigate some of these issues, create a new customer/tenant in your software for each tests which could be deleted/ignored.
• We want to run these tests without and change to the tests, will need a standard approach/contract to environmental configuration which tests can consume.
• Pulling tests results out of production and making them available to all is important
• Tests need to be versioned with your application so a new deployment has the appropriate tests run.
  • Same repo for production code & tests = consistent. If you wanted to add new tests without changing the production code you could either publish the tests separately. Or if your releasing is cheap enough just release a new version and push it through the deployment pipeline in to keep a single consistent approach.
• 3rd party integrations... test data into these systems (bank, health records, etc) might be difficult. Talked briefly about the Simplicator pattern http://www.natpryce.com/articles/000785.html