Functional tests take a long time

From CitconWiki
Jump to navigationJump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Long Running Functional Tests

These are the few notes I took from the "long running functional tests" discussions.

Problems

  • Complete features take 1 day
  • Functional test takes 15 hours

Alternatives, Risks, and Trade-offs

  • Parallel testing vs. pipelined testing
    • Compile
    • Fast unit tests
    • Slow unit tests
    • Functional tests
  • Incremental feedback during test runs
    • Show failures sooner, but
    • Does not typically lead to stopping the tests because we want to know all the results from that set of code


Alternate set of incomplete notes:

What do I do if my functional tests take 5 hours?

4 hour cycle vs. 5 minutes

(ci-machismo: "mine is smaller than yours")

some tests we run all the time and some that we don't

order of magnitude & build pipeline tradeoff: 5 minutes vs. 50 20 minutes vs. 30

long running test suite but reporting of status after each suite (separate emails?)

re-order decomposable steps

what tools are people using to do their functional tests?

new process: added qa, added release management. developers aren't responsive to breaks in the QA builds. often there is disagreement about what the customer really wants, what the problems are. mini-waterfall labeled as Agile. requirements sprint, design sprint, coding sprint, testing sprint.

Retrieved from "http://citconf.com/wiki/index.php?title=Functional_tests_take_a_long_time&oldid=5744"