Mobile testing

• testing applications
• testing mobile browsers

• drive browsers in emulators
• saucelabs - cloud device testing. Apparently their 'cloud' is hosted in the USA. Response times were reported as 4x those from running the same tests on a local computer (for iOS testing)
• testing bot - a similar service. Apparently they have a 'cloud' in Europe. Response times were reported as 2x those from running the same tests as run locally - so faster than the service from saucelabs.

• calabash
• testdroid (who have not replied to an email request about their support for iOS testing)
• MonkeyTalk (from Gorilla Logic)
• AppCode

Using Virtual Machine APIs was suggested as a way to enable us to control the network conditions for tests running in the iOS simulator. Tools such as vagrant can reduce the burden of creating the virtual machine images.

Open question: is there a way to do a grid (conceptually equivalent of Selenium Grid) for native apps? There are some commercial services from http://www.lesspainful.com and http://www.testdroid.com

Crowd-sourced testers who can test mobile apps include:

• mob4hire
• utest

Participants who used these services to test their mobile apps were not very impressed by the results where only about 5 of 80 reported bugs were deemed real bugs worth fixing. The cost of the testing didn't justify the results.