View source for Questions on the right level and amount of testing

== Coverage ==

* measure on unit tests
* target levels are not important/helpful
* great negative metric (i.e.: if there is 0% coverage, we can say for a fact that that code is definitely NOT tested)

== Are we testing too much? ==

* manual assertions (checking) vs. human (exploratory testing)
* execution time increase
* fragile test, we spend more time fixing tests than writing code
* a lot of tests never fail
* solutions:
** intelligent test ordering + partial result feedback
*** based on historic probability of failure
*** coverage analysis of commit impact (take lines changed from diff & correlate past coverage data to unique tests)
*** manually maintain test suites (quick tests, smoke tests, full tests), smoke test can change as different features are being developed, and run them on different schedule
*** related past CITCON session - Josh Ram @ CITCON Australia 2008
*** tools
**** infitest
**** team city or bamboo
**** JTestMe, ProTest

== reused test step ordering ==

* tests can have overlapping steps, e.g.: scenario of "third attempt to log in with bad password locks account" has lot of overlap between steps
** => create a graph where the nodes are the steps (e.g.: login with correct_username, wrong_password), and the assertions hang off / are the values of the nodes
** => faster execution / more effective failfast (TODO: [[Peter Zsoldos]] to write it up in more detail