You might be wondering why I would spend so much time on integrating things via an ANT or Gradle build tool. The whole idea was to be able to do full application testing from beginning to end similar to the way OEMs do framework testing. In other words the developer should be able to run multiple devices and emulators through suite of  tests that include TDD and BDD unit tests among other UI tests and should get back test reports and screen shots.

Now we all know about Monkey UI Exerciser which is a small java api to drive UI testing events. That is not what I am referring to as we need some type of framework.  So what do we need:

-run multiple devices or emulators

-run the application

-be able to interact with ADB

-generates UI events and screenshots

Something has popped up in that I have not noticed it before, its called Monkeyrunner. Its a jython/python api/framework for driving both the android emulators and android devices. Is has a plugin system for using java to extend it and we have an ANT jtyhon task via dbmstools.

With this amount of control to run full set of test suites on an application on multiple emulators/devices why would you choose robolectric over robotium? Sure you get speed of test execution bu than you are sacrificing being able to run a real full tests suite.

But what about not enough Mock objects?  Let me tel you a secret about android mock  objects. A lot of them are not fully documented yet in examples which is why you really should explore android source code as you can find all sorts of useful mock testing objects that the android framework developers use.

Basically this means that the individual tests are using the device or emulator to run along with android-mock and robotium to provide TDD and BDD gets implemented using Instinct during your coding sessions.  Than at the end of the day you are using Monkeyrunner to run a suite of tests as jtyhon/python script that runs a suite of tests on the application across multiple devices and or emulators.

There are enough emulator add-ons provided by SonyEricsson, Samsung, and Motorola that while it does not cover ever device it does allow you cover enough of the android device space among popular handsets to catch most of the bugs that may be device specific.

Evidently Monkeyrunner has been in the SDK since 2.0 but I did not notice it before. Okay, thus I now know why I should be using robotium over robolectric.

Enhanced by Zemanta
  1. Dario Laverde says:

    Hello Fred,

    So basically you’re saying that since Robotium uses Monkeyrunner under the covers and since instrumentation based testing can use third party emulator add-ons, choose test coverage with mocks over faster automated builds w/o mocks (Roboeletric)?

    At a recent google tech talk, Miško Hevery who works as an Engineer at Google where he is responsible for coaching Googlers to maintain the high level of automated testing culture mentioned Roboelectric but it was just a mention as he’s not part of the Android team. But I’d be interested to know which approach is more effective long term as apps get more complex and testing time becomes more of an issue. Perhaps we need to use both, with Robotium for end to end testing.

  2. Michael Zoech says:

    For me, robotium and robolectric are not mutually exclusive. In my dev setup, I have a test project where I use robotium to do functional tests on the device/emulator. Furthermore, i have a second test project that runs on the jvm, where i can use the full power of already existing test/mock frameworks (junit4, mockito, powermock, robolectric, …) to do more low-level stuff.

    The reason for the second test project is based on the fact that the tests execute faster (i.e. startup time) and i’ve the possibility to do mocking via an framework without the need to create mock objects by hand.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s