Difference between revisions of "Fragile Test"
From CitconWiki
Jump to navigationJump to search| (2 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | = Less Fragile Tests = | + | == Less Fragile Tests == |
| − | == Tests == | + | === Tests === |
* Asserts | * Asserts | ||
| Line 14: | Line 14: | ||
* See ISO 9126 | * See ISO 9126 | ||
| − | == Good Tests == | + | === Good Tests === |
* Pass Repeatedly | * Pass Repeatedly | ||
| Line 28: | Line 28: | ||
*** Utility to build mocks | *** Utility to build mocks | ||
*** Easy switching mock to real | *** Easy switching mock to real | ||
| + | **** use same data, settings | ||
| − | == Bad Tests == | + | === Bad Tests === |
* Hidden cause of failure | * Hidden cause of failure | ||
| − | ** | + | ** use screen capture logger, e.g. TimeSnapper |
| − | * Visible Replay | + | * Visible Replay |
* Dump state data at failure | * Dump state data at failure | ||
| − | + | === Tool advice === | |
| − | == Tool advice == | ||
* Time Snapper | * Time Snapper | ||
* EasyMock | * EasyMock | ||
| Line 42: | Line 42: | ||
* DBUnit gets prod snapshot regularly (security issues: scrubbing) Table dependancies makes it a significant task | * DBUnit gets prod snapshot regularly (security issues: scrubbing) Table dependancies makes it a significant task | ||
| − | == Data == | + | === Data === |
* Domain (name, addr, etc class pulled from DB | * Domain (name, addr, etc class pulled from DB | ||
* (abstracted data from test) | * (abstracted data from test) | ||
| Line 51: | Line 51: | ||
| − | == Suggestion == | + | === Suggestion === |
* Use a virtual machine, setup - refresh test machine using VMWARe to simulate MSG based, 20 mins business transactions | * Use a virtual machine, setup - refresh test machine using VMWARe to simulate MSG based, 20 mins business transactions | ||
| − | == Test Data Repository == | + | === Test Data Repository === |
* Used keyword substitution to create the real data | * Used keyword substitution to create the real data | ||
| − | == Typical Data == | + | === Typical Data === |
* Edge / Border cases, Exception testing | * Edge / Border cases, Exception testing | ||
| − | == Broken windows == | + | === Broken windows === |
* Continually failing, low priority tests/defects cause a 'Broken windows' effect which degrades the value of CI. | * Continually failing, low priority tests/defects cause a 'Broken windows' effect which degrades the value of CI. | ||
Latest revision as of 02:46, 30 July 2007
Less Fragile Tests
Tests
- Asserts
- Component
- Integration
- Functional
- UAT
- Non-Functional
- Perf
- Security
- Usability
- See ISO 9126
Good Tests
- Pass Repeatedly
- Timing (e.g. Selenium)
- External Dependencies
- External Systems
- Reliability
- Availability
- Data
- Set Values
- Incorrect State
- External Systems
- Heavy Mocking
- Utility to build mocks
- Easy switching mock to real
- use same data, settings
Bad Tests
- Hidden cause of failure
- use screen capture logger, e.g. TimeSnapper
- Visible Replay
- Dump state data at failure
Tool advice
- Time Snapper
- EasyMock
- Pretend Classes
- DBUnit gets prod snapshot regularly (security issues: scrubbing) Table dependancies makes it a significant task
Data
- Domain (name, addr, etc class pulled from DB
- (abstracted data from test)
- Data Driven Approach
- Fitness Integration Testing
Strong correlation between importance of tests depending on the types of testing and what kind of thing you are doing (Product type development = high focus on test vs bespoke = low focus
Suggestion
- Use a virtual machine, setup - refresh test machine using VMWARe to simulate MSG based, 20 mins business transactions
Test Data Repository
- Used keyword substitution to create the real data
Typical Data
- Edge / Border cases, Exception testing
Broken windows
- Continually failing, low priority tests/defects cause a 'Broken windows' effect which degrades the value of CI.
