Testing for App Interactions with Device Software

Testing for App Interactions with Device Software

Testing for Notifications

There are various mechanisms used by the operating system to display notifications. Sometimes the operating system will either delay the display of the notifications or fail to display them at all in a bid to optimize power consumption. The following test conditions must be considered:

  • The correct handling of notifications received when the app is in the foreground or background, especially under low battery conditions.
  • If notifications allow direct interaction with the app content, (i.e., without opening the app itself), the user interaction must be provided by the app at a later time. If, for example, the user responds to a notification, then it must be possible to access that response from within the app at a later time.
  • If notifications allow access to the app then the corresponding page of the app must be opened instead of the home screen when the notification contains a deep link to that page.

Testing for Quick-access Links

Quick-access links such as app shortcuts in Android and Force-touch or 3d-touch for iOS may be provided by the software under test. These features perform a subset of the application functionality from the home screen without actually launching the entire app.

The following test conditions must be considered:

  • Where some of the features are only available on a particular version of the operating system, the system under test must behave correctly if it is installed on versions of the operating system which either offer or do not offer such features.
  • The actions performed in quick-access links are reflected correctly in the app when opened.

Testing for User Preferences Provided by the Operating System

Any preferences (settings) provided to users by the operating system must be tested. It creates a negative experience for users if a certain preference setting is not respected by the app. For example, if the device is set to mute, the app should not play sounds.

The following test conditions must be considered:

  • Users can amend typical preference options such as sound, brightness, network, power save mode, date and time, time zone, languages, access type and notifications.
  • The apps adhere to the set preferences by behaving accordingly.

Testing for Different Types of Apps

Specific tests may be performed depending on type of mobile app.  The following test conditions must be considered:

  • For native apps:
  • Device compatibility
  • Utilization of device features
  • For hybrid apps:
  • Interaction of the app with the device native features
  • Potential performance issues due to the abstraction layer
  • Usability (look and feel) compared to native apps on the platform in question
  • For web apps:
  • Testing to determine cross-browser compatibility of the app to various common
  • mobile browsers
  • Functionality is not impacted due to various JavaScript engines
  • Utilization of OS features (e.g., date picker and opening appropriate keyboard)
  • Usability (look and feel) compared to native apps on the platform in question

Testing for Interoperability with Multiple Platforms and Operating System Versions

Software companies often support apps on multiple operating systems. Each mobile operating system has its own limitations which need to be taken into account when testing apps. Testers must be aware of the specifics of each platform tested to ensure the app works as intended whilst still conforming to the look and feel of the platform.

The following test conditions must be considered:

  • Handling of interrupts, notifications and optimizations (e.g., for energy saving).
  • Correct functionality where multi-platform applications share some code or have been created using cross-platform development frameworks. Note that if the applications do not share code, then it is like testing two different applications and everything needs to be tested.
  • Testing for backward compatibility if a platform uses different operating system versions.
  • Testing new or changed features made to platforms. For example, in Android the introduction of the Doze framework required testing on the various versions of the operating system which support this framework and those that don’t.

Testing for Interoperability and Co-existence with other Apps on the Device

It is quite common for apps to interact with each other when installed on a device. Typical examples are the contact and email apps.

The following test conditions must be considered:

  • Data transfer between the system under test and the utilized app is correct.
  • There is no harm done to any user data stored within a utilized app.
  • Conflicting behaviors. For example, an app might turn off GPS to save energy, while another app turns on GPS automatically.

With millions of apps in the market, co-existence cannot realistically be tested for all of them.

Nevertheless, such potential issues should be considered and tested according to their risk.

Testing for Various Connectivity Methods

Mobile devices can use various methods to connect to networks. These include cellular networks such as 2G, 3G, 4G and 5G, as well as Wi-Fi and other wireless connection types such as NFC or Bluetooth.

The following alternatives should be considered when performing tests for connectivity:

  • Device emulators/simulators can simulate various network connections and some remote device access service providers include them within their features. Emulators/simulators are, however, of limited value.
  • Setting up your own mobile network to support various connection types and then applying bandwidth manipulation. This is a very costly alternative.
  • Field testing is potentially more cost-effective alternative, but is limited with regard to the reproduction of tests.

In real-world usage, connectivity methods differ. Users can be continuously connected using one particular mode, or they can switch between modes, such as from Wi-Fi to cellular (e.g., when a user leaves home while using the app). The user can switch between various Wi-Fi/cellular networks and versions, as well as between GSM cells. While on the move they may even hit dead spots with no network at all. Furthermore, the user can deliberately disconnect by, for example, switching to flight mode.

Connectivity testing must ensure that the following test conditions are considered:

  • Correct app functionality with different connectivity modes.
  • Switching between modes does not cause any unexpected behavior or data loss.
  • Clear information is given to the user if functionality is restricted due to limited or no network connection or if bandwidth is low. The message should state the limitations and their reasons.