Amit Garg concludes his essential feature on multi-device testing in elearning.
The multi-device testing process
The sheer number of device-OS-browser combinations makes the testing process complex. Here’s a simplified process for testing multi-device/responsive projects:
Target environment and approach
For a precise, limited specification, i.e. a very specific and narrow target device range, defining a testing environment is relatively easy. However, for a wider target device range, an optimisation approach would be helpful. This involves analysing popularity, usage, and sales statistics for devices, browsers, and OSs to narrow down to a comprehensive, representative set.
Creating a testing matrix
Once an optimal set of devices, OSs, and browsers are identified, it is advisable to manually work out all the permutations and combinations to create a testing matrix. The lowest specifications form the parameters for minimum edge testing.
Availability of physical devices
The availability of actual devices is a huge plus. Though 'Synchronous Testing Tools' like GhostLab, Adobe Edge Inspect, Remote Preview help in performing the same interaction automatically across a number of devices and browsers at the same time, and 'Simulators' like BrowserStack, Crossbrowsertesting.com allow web-based Cross-browser testing, testing on real 'physical' devices allows to see and experience the learning resource exactly as the users will, resulting in true test integrity.
Hence, check if all the devices listed in the testing matrix are ‘physically’ available, and if required, evaluate if you can use tools/simulators to cover the missing ones.
Test, test, and re-test
It may be more efficient to first test on one desktop, one tablet in portrait view, and the smallest smartphone in portrait view. This allows validation of the content for all breakpoints as a first major stage of testing. It is advisable to perform this testing 100% manually on the initial three representative devices, one device at a time. Wait till the initial issues are fixed and re-verified before expanding to test on other combinations.
When testing the other combinations, it can be more effective to go device-type-wise – for example, first test on all tablets, covering all applicable browsers and OSs in succession on each tablet; then move on to smartphones.
Issue logging
Log the issues in an efficient manner, so as to make grouping easier and reduce duplication.
Tip: Make smart use of keywords while logging issues (e.g. device/ brand/ OS/ browser), either as a separate field or within the issue description.
Testing is the 'process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component', according to the coherent definition for testing from the IEEE Standard Glossary of Software Engineering Terminology. The testing for functionality, compatibility and accessibility done under specified conditions ensure that the eLearning course can be delivered seamlessly on multiple devices.
Amit Garg is director and co-founder of Upside Learning
