Difference between revisions of "Effective test automation"
Julianharty (talk | contribs) (Created the initial page and added a section on 'flakytests') |
(adding some valuable ideas mentioned in the discussion) |
||
| Line 4: | Line 4: | ||
Also Jonathan Kohl published an article on 'unreproduceable bugs' http://www.kohl.ca/articles/unrepeatablebug.pdf | Also Jonathan Kohl published an article on 'unreproduceable bugs' http://www.kohl.ca/articles/unrepeatablebug.pdf | ||
| + | |||
| + | There's nothing wrong about night-long running through-the-UI tests, but it's recommended to run some quick environment stability tests first so the unstable environment can be fixed before breaking hours of later tests. | ||
| + | |||
| + | Test code is code as well, so it should be treated like that: for example, decoupling tests from the test framework can lower the cost and the effects of a test framework upgrade. | ||
Latest revision as of 12:11, 23 October 2012
--- flakytests --- Flaky tests are common, but not inevitable. There are various ways to identify what's causing the tests to appear flaky. Simon Stewart - creator of WebDriver and lead for the Selenium opensource project has spoken and provided material on the topic on several occasions. Here's a link to a brief Q&A where he suggests 3 possible reasons and how to address them http://www.eurostarconferences.com/blog/2012/10/17/test-leaders-live-webinar-qa-with-simon-stewart and here's a longer article he wrote for the Google Testing blog http://googletesting.blogspot.hu/2009/06/my-selenium-tests-arent-stable.html
Also Jonathan Kohl published an article on 'unreproduceable bugs' http://www.kohl.ca/articles/unrepeatablebug.pdf
There's nothing wrong about night-long running through-the-UI tests, but it's recommended to run some quick environment stability tests first so the unstable environment can be fixed before breaking hours of later tests.
Test code is code as well, so it should be treated like that: for example, decoupling tests from the test framework can lower the cost and the effects of a test framework upgrade.
