Testing without proper planning can cause major problems for an app release, as it can result in compromised software quality and an increase in total cost. Defining and following a suitable and thorough testing procedure is a very important part of the development process that should be considered from the very beginning. Time should be specifically allocated to the manual testing on devices and browsers, as this is a low cost strategy to significantly improve the quality of the app release. In this article, I will share some of the strategies we follow at MetaSys for real device and browser testing.
There are four points that we consider when defining our testing strategy.
- The first point is determining which devices and browsers will be used for testing. This is entirely dependent on the project requirements, and the development team analyses the application use cases to make the selection based on the following principles:
- For web applications, we usually test on the three most commonly used browsers (Chrome, Firefox and Safari). If time allows for more extensive testing, we will also test on other browsers like Internet Explorer and Microsoft Edge.
- For Device testing of web applications, we choose the devices based on the functional requirements and priorities of the applications. In other words, if a web application is supposed to run especially well on any particular device we focus the testing on the corresponding commonly used browsers with the appropriate resolution. For instance, for the Android platform we focus on Chrome and Firefox, whereas for the iOS platform we focus on Safari and Chrome.
- For Native applications we directly test the application on the devices themselves, rather than using an emulator. This provides the most accurate feedback in terms of functionality and application performance.
- There are instances where the project timeline and/or budget limits the amount of testing that we can do. It is very important to identify these situations, and to develop strategies in order to still deliver high quality software to the client. At MetaSys we handle these cases by focusing on high level general testing, which covers most of the UI and the functional part of the applications.
- For functional testing of web applications, we utilise automation as much as possible. For repetitive testing of browsers, we usually design automated test cases. Using automation not only helps save the time of the testers, it is also very useful for retesting resolved issues. We use the Selenium WebDriver tool for automation testing and the Microsoft Team Foundation Server 2019 and the Microsoft Test Management tools for bug reporting and test case management.
- For web applications, we put a strong emphasis on performance, in addition to the ‘look and feel’., The speed of the app is one of the most important factors that determines the user experience. For performance testing we use the Apache JMeter and New Relic tools which give very accurate results regarding the application performance. The New Relic tool also provides an analysis of database query level problems, and gives many more reports and real time graphs. This helps significantly with troubleshooting, and improving performance.