Difference between revisions of "What's an agile tester?"

From CitconWiki
Jump to navigationJump to search
(Sometimes such perceptions are due to bad testing masquerading as intelligent ET)
 
(9 intermediate revisions by 4 users not shown)
Line 1: Line 1:
'''Making builds faster and more efficient'''
+
=== Problem ===
  
These notes are pretty much in the order in which we talked about the topics.
+
How does a developer on an agile project help testers on the same project who are new to agile understand their role?  More importantly what is the role of the tester on an agile project?
  
One concept is break the code into parts that build separately, each successful build goes to the main project to be built together with the other parts. I’m a little hazy on this but I get the idea.
+
=== Discussion ===
 +
==== How do we know we have a problem ====
 +
We started by exploring what indicates that a tester (or perhaps anyone) is having trouble on an agile project.
 +
* Not speaking in the daily standup
 +
* Not participating in the team
 +
* Not paying attention to what is going on in the project
  
Other ideas to speed up:
+
==== What should testers do in an agile project ====
 +
Next we transitioned to what do we want them to do?  One person volunteered that he wants the testers to help him solve the problem of failing user acceptance tests very quickly after launch.  Specifically he wants:
 +
# Testers to be part of analysis including task decomposition
 +
# Enable developers to think like testers
 +
# Find the problems with UAT
  
* If build uses app server, restart it every build.
+
Based on this feedbacke we began to drill into what testers do in the project?  Do they get involved early and try to figure out what to do with each story?  We came up with several concrete suggestions based on this line of questioning:
* Do cleanup tasks in the interim while the build is waiting to check for the next update and build again.
+
* Use FitNesse for defining the acceptance tests and for documentation of what the product is meant to do and what it does (once it has shipped).
* Use ant asynchronous execution.
+
* Write a high-level UAT as a team before the iteration starts
* Maven has concurrency built in – promote successful build to central area. Look at dependencies in project.
+
* Start with the happy path automation in FitNesse and wait for that to turn green befor starting more complex testing
 +
* Use automation to set up exploratory testing (e.g. set up an account and subscribers to enable exploration of particular features of the account).
 +
* Add inidcators to the product code (e.g. red border around controls) to indicate that it should not be tested.
 +
* Write the FitNesse tests first so that developers know what needs to be done before the story is complete.
 +
* Use a tool such as [http://www.sirius-sqa.com/ Test Explorer]  to document the exploratory tests as they are being done.
 +
* Use exploratory testing to collapse planning, design and execution into a single step.
  
Parallel execution, task distribution.
+
==== Testing Challenges ====
 +
Automated and exploratory testing clearly has value, but each present some challenges including:
 +
* Exploratory is seen as ad hoc testing without active exploration and thought of threat to the product.  For this reason it may not be seen as high value.  (Skilled exploratory testers can help dispel this perception.)
 +
* Automation can't find new bugs, it only serves to validate that what we think is working is still working.
 +
* FitNesse tests can be hard to create since testers may not have the skills to wire up the fixtures and developers see it as a tax.
 +
* Unit tests are not good functional automation since the unit tests show that the code works the way that the developer expects it to not necessarily the way the customer expects it to.
  
Attack slowest area first
+
==== Definition and Role of Testing ====
 +
A definition, from Cem Kaner,  of testing is: Software testing is [1] cognitively complex, [2] requires critical thinking, [3] requires effective communication,  and [4] requires rapid self-directed learning.
  
One participant described how they use a big pool of machines and split the unit tests across them to run concurrently. Sped up the build from 48 hours to 5 hours. Homegrown tool distributes the work to servers as they become available.
+
We identified a few pitfalls that testers should avoid:
 +
* Testers can not assume the role of gate keepers or quality assurers on the project. Instead their role is to examine and inform of risks
 +
* Definining the role of QA/Testers as finding bugs promotes an us v. them attitude
 +
* QA will fail in organizations that say they are agile, but do not manage/fully support agile principles.
  
Hardware on build server must be better or as good as what is on developers’ machines.
+
==== Final Thought ====
 
+
Agile testers are people who are passionate about doing their job better and continuously learning. They must be adaptive, fast learners, and good thinkers.
Tools help manage the process, gather results, add more resources as needed.
 
 
 
Multiple platforms are an added challenge.
 
 
 
Distributed tests with shared resources that developers can use are a good idea.
 
 
 
Hardware – solid state drive faster, if I/O is the limiting factor.
 
 
 
Is the problem only on the CI server, or will it show up on developer machines?
 
 
 
Look at the unit tests. Be smart about which ones need to run when – what value do they provide. Only run the tests that matter.
 
 
 
Lots of small incremental changes can add up to a difference to speed up build. For example, CVS was slow so team went to Perforce, much faster to check out code.
 
 
 
Clean builds vs. Dirty builds
 
Clean builds for production builds – reliable feedback
 
Dirty builds are incremental builds, some people said this was ok for a day, but you needed a clean build at least once a day. Other people said they had good success with incremental builds and even released code that was incrementally built. The latter was thought to be the exception to the rule.
 
 
 
“Sourdough” builds – I need a previous build for this build to work.
 
Dependencies on build-derived objects. Bad.
 
 
 
Tools:
 
Ant – more control than maven
 
Maven does more stuff for you
 
 
 
For continuous build:
 
Cruise control
 
Hudson
 
Electric commander
 
Buildbeat
 
CI Factory
 
Anthill Pro
 
Pulse
 
Team city
 
 
 
Metrics:
 
Build time on the CI server vs. local – this is useful if you look at WHY? Compile, checkout, tests, server restart, what is slowing it down?
 
 
 
Does every machine have enough resources?
 
 
 
Having someone responsible for the build is important.  Ownership.
 

Latest revision as of 15:05, 15 April 2008

Problem

How does a developer on an agile project help testers on the same project who are new to agile understand their role? More importantly what is the role of the tester on an agile project?

Discussion

How do we know we have a problem

We started by exploring what indicates that a tester (or perhaps anyone) is having trouble on an agile project.

  • Not speaking in the daily standup
  • Not participating in the team
  • Not paying attention to what is going on in the project

What should testers do in an agile project

Next we transitioned to what do we want them to do? One person volunteered that he wants the testers to help him solve the problem of failing user acceptance tests very quickly after launch. Specifically he wants:

  1. Testers to be part of analysis including task decomposition
  2. Enable developers to think like testers
  3. Find the problems with UAT

Based on this feedbacke we began to drill into what testers do in the project? Do they get involved early and try to figure out what to do with each story? We came up with several concrete suggestions based on this line of questioning:

  • Use FitNesse for defining the acceptance tests and for documentation of what the product is meant to do and what it does (once it has shipped).
  • Write a high-level UAT as a team before the iteration starts
  • Start with the happy path automation in FitNesse and wait for that to turn green befor starting more complex testing
  • Use automation to set up exploratory testing (e.g. set up an account and subscribers to enable exploration of particular features of the account).
  • Add inidcators to the product code (e.g. red border around controls) to indicate that it should not be tested.
  • Write the FitNesse tests first so that developers know what needs to be done before the story is complete.
  • Use a tool such as Test Explorer to document the exploratory tests as they are being done.
  • Use exploratory testing to collapse planning, design and execution into a single step.

Testing Challenges

Automated and exploratory testing clearly has value, but each present some challenges including:

  • Exploratory is seen as ad hoc testing without active exploration and thought of threat to the product. For this reason it may not be seen as high value. (Skilled exploratory testers can help dispel this perception.)
  • Automation can't find new bugs, it only serves to validate that what we think is working is still working.
  • FitNesse tests can be hard to create since testers may not have the skills to wire up the fixtures and developers see it as a tax.
  • Unit tests are not good functional automation since the unit tests show that the code works the way that the developer expects it to not necessarily the way the customer expects it to.

Definition and Role of Testing

A definition, from Cem Kaner, of testing is: Software testing is [1] cognitively complex, [2] requires critical thinking, [3] requires effective communication, and [4] requires rapid self-directed learning.

We identified a few pitfalls that testers should avoid:

  • Testers can not assume the role of gate keepers or quality assurers on the project. Instead their role is to examine and inform of risks
  • Definining the role of QA/Testers as finding bugs promotes an us v. them attitude
  • QA will fail in organizations that say they are agile, but do not manage/fully support agile principles.

Final Thought

Agile testers are people who are passionate about doing their job better and continuously learning. They must be adaptive, fast learners, and good thinkers.