CitconWiki - User contributions [en]

Mobile testing

2012-10-21T08:29:47Z

Julianharty: Created the initial notes for the Mobile Testing session. These are based on the notes made on flip chart paper.

* testing applications
* testing mobile browsers

* drive browsers in emulators
* saucelabs - cloud device testing. Apparently their 'cloud' is hosted in the USA. Response times were reported as 4x those from running the same tests on a local computer (for iOS testing)
* testing bot - a similar service. Apparently they have a 'cloud' in Europe. Response times were reported as 2x those from running the same tests as run locally - so faster than the service from saucelabs.

* calabash
* testdroid (who have not replied to an email request about their support for iOS testing)
* MonkeyTalk (from Gorilla Logic)
* AppCode

Using Virtual Machine APIs was suggested as a way to enable us to control the network conditions for tests running in the iOS simulator. Tools such as vagrant can reduce the burden of creating the virtual machine images.

Open question: is there a way to do a grid (conceptually equivalent of Selenium Grid) for native apps? There are some commercial services from http://www.lesspainful.com and http://www.testdroid.com

Crowd-sourced testers who can test mobile apps include:
* mob4hire
* utest

Participants who used these services to test their mobile apps were not very impressed by the results where only about 5 of 80 reported bugs were deemed real bugs worth fixing. The cost of the testing, for a month, was apparently $8,000 USD

Continuous rewriting

2012-10-21T08:06:15Z

Julianharty: Plea for contributions to the Continuous rewriting page.

To those who were in this session, I heard many positive things about it, please can you write some notes here?

Thanks

Julian who was part of another session at the time...

Effective test automation

2012-10-21T08:03:58Z

Julianharty: Created the initial page and added a section on 'flakytests'

--- flakytests ---
Flaky tests are common, but not inevitable. There are various ways to identify what's causing the tests to appear flaky. Simon Stewart - creator of WebDriver and lead for the Selenium opensource project has spoken and provided material on the topic on several occasions. Here's a link to a brief Q&A where he suggests 3 possible reasons and how to address them http://www.eurostarconferences.com/blog/2012/10/17/test-leaders-live-webinar-qa-with-simon-stewart and here's a longer article he wrote for the Google Testing blog http://googletesting.blogspot.hu/2009/06/my-selenium-tests-arent-stable.html

Also Jonathan Kohl published an article on 'unreproduceable bugs' http://www.kohl.ca/articles/unrepeatablebug.pdf

Integrated test tools and an IDE for Testers

2012-10-21T07:58:28Z

Julianharty: formatted content and added link for the HTML CodeSniffer utility

Suggested features / capabilities
* testcases
* scripted instructions
* coded instructions
* recording
* discovery/inspection
* reporting
* bug tracking
* test runs / test suite
* import existing (similar) test cases
* CI integration
* debugging
* backed by a version control system
* IDE autocompletion
* multiple ways to present stuff - visual, text, ...
* auto-suggest of test ideas e.g. for security tests taken from owasp.org

Who are the target users:
* Powerusers?
* Non-technical testers

Opensource, reducing barriers to entry

Hover over capability to see the UI of the app at each step of the test case
* See HTML Code Sniffer for an example of good UI that indicates where problems are visually displayed. http://squizlabs.github.com/HTML_CodeSniffer/

Visual Components - drag and drop to create test cases - see the Visual Studio Workflow tool

RubyMine recommended as a great IDE for Ruby code and writing Cucumber tests
Cubittest - front end for Selenium has visual presentation
Encrunch? a tool in the latest visual studio beta that runs tests all the time in memory without needing to save them

Integrated test tools and an IDE for Testers

2012-10-20T16:05:11Z

Julianharty: Notes for the session on creating an IDE for testers

Suggested features / capabilities
- testcases
- scripted instructions
- coded instructions
- recording
- discovery/inspection
- reporting
- bug tracking
- test runs / test suite
- import existing (similar) test cases
- CI integration
- debugging
- backed by a version control system
- IDE autocompletion
- multiple ways to present stuff - visual, text, ...
- auto-suggest of test ideas e.g. for security tests taken from owasp.org

Who are the target users:
- Powerusers?
- Non-technical testers

Opensource, reducing barriers to entry

Hover over capability to see the UI of the app at each step of the test case
- See HTML Code Sniffer for an example of good UI that indicates where problems are visually displayed.

Visual Components - drag and drop to create test cases - see the Visual Studio Workflow tool

RubyMine recommended as a great IDE for Ruby code and writing Cucumber tests
Cubittest - front end for Selenium has visual presentation
Encrunch? a tool in the latest visual studio beta that runs tests all the time in memory without needing to save them

CITCONEurope2012Sessions

2012-10-20T15:53:42Z

Julianharty: /* 4:30 Topics */

CITCON Europe 2012 Budapest Sessions

Back to the [[Main Page]]

== 10:00 Topics ==

# [[Helping people learn competences in test automation]]
# [[Browser automation in the cloud]]
# [[TDD2012]]
# [[Multi-platform build automation]]
# [[Avoiding rabbit holes]]

== 11:15 Topics ==

# [[What happens after the build is finished?]]
# [[Pain of testing legacy batch software]]
# [[Test automation engine features]]
# [[People come and go]]
# [[Learn another discipline without taking a pay cut]]

== 2:00 Topics ==

# [[Effective test automation]]
# [[Dealing with false positives]]
# [[Meaningful metrics]]
# [[Continuous rewriting]]
# [[Making devs do code reviews and unit tests]]

== 3:15 Topics ==

# [[Visualization for managers]]
# [[Do we need tests on planet retail?]]
# [[Mobile testing]]
# [[Test automation Q&A]]
# [[What tools?]]

== 4:30 Topics ==

# [[Risk management and voodoo charms]]
# [[How to implement CD in the cloud]]
# [[Tools to run parts of enormous suites]]
# [[Integrated test tools and an IDE for Testers]]
# [[Testing new feature in production]]

== Table View ==

{| class="wikitable"
|-
! #
! 10:00
! 11:15
! 2:00
! 3:15
! 4:30
|-
| CruiseControl (large)
|
|
|
|
|
|-
| AnthillPro (medium)
|
|
|
|
|
|-
| Jenkins (medium)
|
|
|
|
|
|-
| TeamCity (medium)
|
|
|
|
|
|-
| BuildBot (small)
|
|
|
|
|
|}