What are the key challenges in wearable device testing?

The emergence of wearable devices brought with it a range of new challenges for companies and their quality assurance engineers. Because it is a relatively new space, companies need to ensure they can deliver new applications on wearables and provide a great experience for the end users or other businesses interacting with the devices.

In this series of blog posts, we will explore the current market, the development pace in the industry, the current leading manufacturers, as well as the main challenges that are likely to be encountered when testing software on wearable devices.

There are many different areas where wearables are becoming increasingly innovative and useful:

• Retail – beacons, indoor navigation, analytics, real-time client interactions, proposals, and marketing.
• Travel – tracking, navigation, viewpoints, weather and environmental conditions.
• Sport – health checks, status tracking, activity tracking, analytics and suggestions.
• Defense – security systems, home alarms, video cameras, and motion sensors.
• Communications – integrations, control, and communicating between devices.

Many of the challenges encountered in testing wearable devices are closely related to testing mobile applications on mobile devices. This is because the environment and main methods of use are similar:

• Interaction takes place in real time
• Use occurs without stops or pauses
• There is no need to think about the state of the network or connection with mobile internet
• Users are always in touch and want an immediate response from their device

As a result, many of the testing obstacles for wearables are similar to those seen across a range of mobile devices:

• Wide variety of device types
• Issues with operating systems
• Wide combination of parts
• Relatively high maintenance costs
• Working with cloud providers
• The device UX
• Integrations and dependencies

Now let’s explore these in more detail:

1. Device variety – there is a wide variety of devices on the market and here we aren’t even talking about manufacturers and versions of devices, simply the categories of wearables available. This includes the 5 main market segments: smartwatches, smart glasses, fitness bracelets, sensors, and smart clothes (including wearable cameras, ski and motorcycle helmets, etc.). Each of these types of devices requires its own specific testing types.

2. Operating systems – the similarity with mobile devices is also present in the operating systems that are installed on wearable devices. There are several market leaders: Apple, Google, and Samsung with their respective operating systems: WatchOS, Wear OS, and Tizen. There are also several other major players that make up the remaining roughly 20 percent of market share.

Here’s what we see in the distribution between wearable device vendors:

Apple is the market leader, while Samsung and Garmin are gradually gaining market share. There is strong overall growth in the total number of wearable devices purchased each year. As indicated in the graph below, wearable devices across all five categories have significantly increased their market size over the last 5 years, particularly in the fitness device and smart watch categories.

3. Combination of parts – different combinations of screens and sensors are used within applications. And we’re not talking about hardware here – difficulties are encountered as part of the communications processes between the various software products and the data coming from GPS, Bluetooth, Wi-fi, the cellular network, and motion sensors.

4. Maintenance costs – device costs and obsolescence need to be taken into consideration. Wearable device costs can be similar to or greater than other mobile devices and tend to have a similar length of life cycle. As a result, constant concern about the relevance of the device fleet is one of the difficulties of testing.

5. Cloud providers – cloud mobile device farms can be a solution that helps to mitigate the above outlined issues related to device costs. Unfortunately, they are not publicly available on the market, so all testing must be done on your own devices or emulators.

6. Device UX – many articles have been written on the UX of wearable devices. However, it is also important not to forget to test the device UX on targeted audiences as well as to be aware of the best practices offered by the device manufacturers themselves. Experience with existing applications that have already proven themselves in the market can also be a great help here.

7. Integrations and dependencies – there are always potential complexities to consider when interacting with the huge variety of mobile devices that are available on the market. So for example when talking about the software installed on a smartwatch, it is not always natively supported by a mobile device, as in Apple products, and often requires third-party applications that also need to be tested in conjunction with different wearable devices. This is an often overlooked element that must be considered in the testing stage.

Here are some examples of applications that are installed on a mobile device, but their main task is to interact with a wearable device: Garmin Connect, Mi Fit, ‎Fitbit Health & Fitness, and Samsung Galaxy Watch.

In summary, let’s emphasize the importance of testing wearable devices. The variety of dependencies and use cases forces us to conduct not only functional testing of applications but also to think about how the non-functional characteristics of wearable devices can affect the behavior of the application. It’s also necessary to check the interaction with already familiar mobile applications, from which you can control the wearable device.

This article highlights the difficulties that any developers of high-quality software for wearable devices will face. It’s important to perform functional testing and apply test design techniques and testing principles. But it’s also necessary to cover the features of testing wearable devices, consider the variety of operating systems and manufacturers on the market, as well as integrations with the software on wearable devices. By focusing efforts on each of these components, it’s possible to ensure the release of software that will satisfy the end users as well as enable the business to more quickly achieve its goals.

In the next article, we will shine the spotlight on solutions and explore how Grid Dynamics approaches solving the major challenges of the wearable devices testing industry. We look forward to hearing your thoughts and feedback so please let us know if you have any questions related to wearable device testing.