- Near to release?
Adding code coverage to every build
- % of code
- % of branches
- watch the trend
- trend should grow
- Cobertura shows this
When transitioning to TDD, watch coverage
List of interesting metrics (dynamic or static properites of the product)
- line coverage
- branch coverage
- complexity metrics (NCSS, Callistic)
* links between metrics highlighted * defects added * defects corrected * densisty of defects for project
Metrics from CI system:
- time for build
- time for test
- commit frequency
- pairing matrix
- time for completion for user story
- identifies which unit test breaks the build the most
What is the real use of the metrics?
Use of metrics will affect behaviour of developers.
Experience of setting up Sonar. Often fails, developers start to ignore
Much easier if you start with new code, and don't let new defects in. Keep clean. Harder to apply to legacy code, because the number of warnings if often overwhelming.
Ivan: story of project that used static analysis tools from the start, but the code was a complete mess. This was probably as a direct result of the tools used for measurement
Interesting to correllate bugs to programmer or pair.
Book "Measuring and Managing Performance in Organisations": if you manage people by a metric, they will responding by gaming the system. That doesn't necessarily mean that you shouldn't use the metrics
Need to carefully choose which metrics to measure, and interpret intelligent conclusions from metrics. Can necessarily do this automatically.
Learning how to use the information presented. Collate data. What will the team do with the information? Will external groups monitor the metrics?
BI (Business Intelligence) - managers see similarities with code metrics. Can be helpful to distill code metrics to simplifed metrics based on threasholds. eg. "very maintainable". Perhaps produce a management metric view
Can be good idea to use metrics to identifiy areas that need documentation (e.g. high complexity)
- What is your problem?
- What metric can be used to identify problem?
- Montitor metric
Trends are important. It is often more important to the trend, rather than looking at a static snapshot.
It is important to plan how to use the metric. e.g. standing on a scale to see your weight is not sufficient to lose wieght. It is important that you have a plan to manage the metric (e.g. change diet, exercise)
Question: which metrics are most useful to help reduce defects and increase quality of code?
Metrics can give you numbers to help you feel confident about the state of the code, and if it is ready to release.
Metrics can help with
Tom deMarco: you need to measure before you can change (recanted later) http://www2.computer.org/portal/web/csdl/doi/10.1109/MS.2009.101
- Must be tested (use metrics, what is most complex, fan in)
- Should be tested
- Could be tested