Reusing validation tests as monitoring
People seemed generally agreeable that we want to write tests once in dev and get reuse out of them in production.
- Biggest hurdle perceived as handling persistence/state
- Running passive (readonly tests) would be trivial to do. Tests that write become a problem.
- Multi-tenanted solutions might mitigate some of these issues, create a new customer/tenant in your software for each tests which could be deleted/ignored.
- We want to run these tests without and change to the tests, will need a standard approach/contract to environmental configuration which tests can consume.
- Pulling tests results out of production and making them available to all is important
- Tests need to be versioned with your application so a new deployment has the appropriate tests run.
- Same repo for production code & tests = consistent. If you wanted to add new tests without changing the production code you could either publish the tests separately. Or if your releasing is cheap enough just release a new version and push it through the deployment pipeline in to keep a single consistent approach.
- 3rd party integrations... test data into these systems (bank, health records, etc) might be difficult. Talked briefly about the Simplicator pattern http://www.natpryce.com/articles/000785.html