Mobile App Testing: Modern Practices Overview
A vast variety of mobile applications sees the market every week, differing in complexity and functionality and causing unique challenges for developers and testers. As a rule, mobile application testing makes up to 10% of the overall app development budget. It’s caused by the existence of diverse mobile devices, mobile browsers, operating systems, application use cases, infrastructures, and networks.
It is much easier to test common desktop applications mostly deployed on the dominant Windows OS. One of the biggest challenges is to keep pace with constant software and hardware development advance and rapid changes in customers’ demands. In this article, we will highlight some of the mobile applications testing practices used today.
Major mobile applications testing practices
Object recognition
In the process of writing test suites, it is vital to create reusable and maintainable mobile scripts for accomplishing thorough app coverage before its release. Object recognition is a key factor for creating reusable test suites. Apps with Graphical User Interface (GUI) layer consist of User Interface (UI) and a set of objects like text boxes, buttons, images and selection lists, etc. All the objects have a list of properties that can be used for identification, validation, or definition of the particular object.
There are three major approaches to object recognition: native, image, and optical character recognition (OCR).
- Native recognition allows to directly work with every aspect of the object and recognize it by its native ID. The advantage of native recognition: it provides robust, fast scripts and simplified debugging. Among the drawbacks is the necessity to keep automation tools updated with controls and components’ changes that usually come with new OS releases.
- Image recognition method is able to capture the screen of mobile device and identify a UI element as a unique object that has a certain position and behaves in a predefined way. Image recognition utilizes complex algorithms for the object recognition. If the testing framework is plugged into a different device we can still distinguish the UI element as a unique entity. If this object moves or acts in an inappropriate way, we get an alert. Image recognition allows testers not to depend so much on the updates of the mobile OS and use different devices and OS versions without major changes in test automation scripts.
- Optical Character recognition (OCR) also known as “screen scraping”, is used for test automation. It enables testers to extract a text from the screen of the device. Testers can utilize OCR to find a separate object, and this test doesn’t depend on the mobile provider’s OS updates. Although OCR is slower and more complex to maintain than other object recognition methods, it can employ many external OCR tools and libraries to improve the recognition results and make testing much more reliable.
Unit and functional testing
Unit testing means testing separate functions of the application, each part of it. This way the problematic parts of the codebase can be detected in the very early stages of QA. Functional testing is used to monitor an application’s behavior via checking the returns from every predefined type of input (usually some kind of user action — tap, swipe, input, etc.). Both unit and functional testing are usually performed with the help of special testing software suits that enable test automation, thorough testing process logging, deep integration with development tools, and many more features.
Physical device emulators
There currently exists an enormous amount of mobile devices and OS versions В (more than 100,000 combinations in total), and it’s simply impossible to test your application on all of them. For example, Open Signal reports that approximately 4,000 different Android devices are currently offered on the market. This diversity presents a real challenge for software developers and QA professionals. Software device emulators are used for the imitation of the partial or complete functionality of mobile devices or separate applications. Emulators can be used if the target mobile device is unavailable or you need to conduct tests that are dangerous for the device or can break the agreement terms with a mobile device producer.
Among the setbacks of using emulators is a high host machine resource consumption, and there is no guarantee that the app will run correctly on the device even if it works well on the emulator. In practice, it’s better to limit testing of your application to a small group of devices and platforms (between 8-12 mobile devices), select the largest and smallest screen sizes, and test your app on these phones.
Automated testing best practices
Below we listed best practices of creating robust and reliable automated tests for your mobile applications:
- Use test customization features provided by the testing tools;
- Conduct test cases analysis to find out which tests bring the highest ROI via automation;
- Validate the test automation value;
- Develop test automation framework to define the rules for software implementation and testing;
- Test your apps in conditions similar to the real-world ones;
- Use rapid defect resolution (via automatic creation of defect submissions);
- Reuse your test scripts.
Conclusions
Businesses and our everyday life is becoming increasingly reliant on mobile technology with more than 1 billion people using smartphones worldwide. Thus, mobile app testing is essential, and software developers and testers should take into consideration the following aspects of the app testing: object recognition methods (native, image, OCR, or even all three may be necessary), hardware and software base (which mobile devices/platforms is the app supposed to work on?), testing automation. Use best practices and your own experience to select the appropriate testing methodology, but don’t forget to test the methodology itself to find what suits you best.
