Device testing in the cloud – I’m now using Kobiton

Device testing in the cloud – I’m now using Kobiton

I’ve recently started using the Kobiton device testing cloud, to enhance the range of iOS and Android devices that I can test apps and websites on.

Over the last few years, I’ve looked into several other device testing cloud platforms from all sorts of companies and each time I haven’t been that impressed, as at least one (if not all) of the required features below have been missing or lacking.

Required Features

1) wide range of devices, including recent devices and OS versions
2) cost-effective pricing (for a freelance tester such as me)
3) access to the devices for both app testing and website testing
4) ease of use and effectiveness of the platform
5) platform allows for both manual and automated testing

Several device testing cloud platforms

Here’s what I’ve found over the years when researching the various device testing cloud platforms I’ve found:

  • Firebase Test Lab – when I checked the features, this was just for app testing and for automated testing only
  • Amazon AWS Device Farm – I gave this a trial, but found the device list available for manual testing/access was very limited
  • Perfecto – the pricing of this (Basic plan starts at $129/month) limits it to larger companies rather than freelance testers like me, plus I wasn’t able to find their device list after much browsing of their website
  • Sauce Labs – their device list does seem to be up-to-date but again their pricing (Live starts at $119/month, Automated starts at $349/month) limits it to larger companies
  • Bitbar Cloud – their device list does seem up-to-date but again pricing (Solo plan starts at $99/month) would be an issue for a freelance tester

So how does Kobiton measure up on the Required Features above?

1) wide range of devices – as of the end of April 2018, there were 44 iPhones, 17 iPads, 151 Android phones and 9 Android tablets available for testing and the range is up-to-date, including iPhone X, iPhone 8 / 8 Plus and Samsung Galaxy S8 / S8+
2) cost-effective pricing (for a freelance tester such as me) – the Indie plan costs $10/month and includes 100 minutes of testing and you can add further minutes as required, from $10 per 100 minutes
3) access to the devices for both app testing and website testing – this is available
4) ease of use and effectiveness of the platform – the platform is easy to use and features easy to access
5) platform allows for both manual and automated testing – the platform allows for both types

There’s been a few glitches along the way, but these have mostly been sorted by creating a support ticket and then leaving support to fix the glitch.

Website Testing on Kobiton devices

  • This is the easiest type of testing to perform, as you just select the required device and wait for it to start up, then you can startup the relevant browser and perform your website testing on the device

App Testing on Kobiton devices

  • App testing requires some further tasks, as you have to first upload your app to be tested, in .ipa or .apk file format (also, you need to make sure your app is setup properly for ad-hoc distribution)
  • The uploaded app is then scanned for any security issues such as viruses and malware – when it passes this, the app is then available to test with
  • To install your app, you startup the required device and then install the app to it

Common Features to using Kobiton devices

  •  On each device, you can manually set the location (using GPS co-ords) and timezone (from a dropdown list) you require, if that’s relevant for your testing
  • You can take a screenshot at any time and then download that locally, if required
  • Each device testing session is saved and you can see video of the session
  • Some device features such as making calls, sending SMS aren’t available

My thoughts on using Kobiton so far…

I was impressed enough with the free trial to move to the Kobiton Indie plan. Since then, I’ve used Kobiton devices in several test projects, along with my own set of test devices.

I think the best way to use a device testing cloud platform such as Kobiton is to use it to complement your own set of test devices, to allow you to test websites or do some smoke testing on a wider range of devices than you have physical access to.

For example, a Kobiton device would be ideal to smoke test an iOS app on a device and iOS version you don’t have access to, to check that the iOS app installs ok, starts up ok, and then do a quick run through the main features to check the stability, display and hopefully that the app is crash-free.