Difference between revisions of "MutationTesting"
(Add links to tools - squirrel) |
m (Typo in link) |
||
| (7 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
| − | + | Mutation testing tools: Jester, Jumble & Test Police | |
| + | |||
| + | Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Martijn, Mark, David, Ivan, Kirsten | ||
| + | |||
| + | Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. | ||
| + | Problems: it is hard to integrate it with build scripts, and with multiple tests. | ||
| + | Promise of a new release! | ||
| + | It is too slow to run on an entire project. | ||
| + | |||
| + | Jumble: similar, but changes bytecode, not source. | ||
| + | * Problems: difficult to use, not clear how well maintained. Also, Class.getPackage() doesn't return the package name you expect. | ||
| + | some metrics: | ||
| + | * 5 seconds per compilation | ||
| + | * 20 mutations per class, each requiring compilation | ||
| + | * 1200 classes | ||
| + | * 120000 seconds to run = 33.3 hours | ||
| + | |||
| + | Ajester: similar to Jester, but with bytecode. The one interesting thing is that it is running on the classes that already have a good code coverage. | ||
| + | |||
| + | Martijn: it'd be nice if you could detect tests that always find out the same changes in the code, as tey would be redundant. | ||
| + | |||
| + | Ivan: an "MD5 test" is when a test is almost a rehash of the code, and it makes it harder to change the code. | ||
| + | |||
| + | David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests. | ||
| + | |||
| + | μJava: a nasty competitor to Jester! does mutation of classes | ||
| + | |||
| + | Iven: there is probably a tool integrated with Maven. Must find it again. | ||
| + | Must also look at Hansel (& Gretel) | ||
| + | |||
| + | Tuomas: I'd like to look into infinitest, and see if that could be used to show which lines are exercised by which tests: http://code.google.com/p/infinitest/ | ||
| + | also, I'm interested in seeing how mutation testing can properly identify which abstract superclasses should be mutated. (I wrote Test Police) | ||
== Links to tools == | == Links to tools == | ||
| − | Jumble: [http://jumble.sourceforge.net/] | + | * Jumble: [http://jumble.sourceforge.net/] |
| − | Jester: [http://jester.sourceforge.net/] | + | * Jester: [http://jester.sourceforge.net/] |
| − | MuJava: [http:// | + | * Test Police: [http://www.laughingpanda.org/mediawiki/index.php/Test_Police] |
| + | * MuJava: [http://cs.gmu.edu/~offutt/mujava/] | ||
| + | * More links on [http://ivan.truemesh.com/archives/000720.html] | ||
Latest revision as of 23:12, 24 October 2010
Mutation testing tools: Jester, Jumble & Test Police
Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Martijn, Mark, David, Ivan, Kirsten
Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. Problems: it is hard to integrate it with build scripts, and with multiple tests. Promise of a new release! It is too slow to run on an entire project.
Jumble: similar, but changes bytecode, not source.
- Problems: difficult to use, not clear how well maintained. Also, Class.getPackage() doesn't return the package name you expect.
some metrics:
- 5 seconds per compilation
- 20 mutations per class, each requiring compilation
- 1200 classes
- 120000 seconds to run = 33.3 hours
Ajester: similar to Jester, but with bytecode. The one interesting thing is that it is running on the classes that already have a good code coverage.
Martijn: it'd be nice if you could detect tests that always find out the same changes in the code, as tey would be redundant.
Ivan: an "MD5 test" is when a test is almost a rehash of the code, and it makes it harder to change the code.
David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests.
μJava: a nasty competitor to Jester! does mutation of classes
Iven: there is probably a tool integrated with Maven. Must find it again. Must also look at Hansel (& Gretel)
Tuomas: I'd like to look into infinitest, and see if that could be used to show which lines are exercised by which tests: http://code.google.com/p/infinitest/ also, I'm interested in seeing how mutation testing can properly identify which abstract superclasses should be mutated. (I wrote Test Police)
