From the exquisite penmanship of Radu Tasca our gifted QA engineer
Over the years, Bluetooth has become ubiquitous in the realm of IoT due to its versatility, allowing users to connect to smartwatches, car stereos, video cameras, printers, other computers, and even electric toothbrushes. Multiple developments have immensely refined this technology since its nascent days, however power requirements have dictated the need to develop a more efficient means of connectivity. Thus, Bluetooth Low Energy was born.
In this article we will explore the basics of Bluetooth Low Energy, some of its most popular use cases, and the ramifications it has when testing mobile apps powered by BLE devices.
What is Bluetooth Low Energy (BLE) and how did we get here?
As mentioned before, standard Bluetooth has high rates of energy consumption as its main downside. This, of course, ripples throughout major design aspects and requires several compromises – sacrificing uptime in favor of reduced bulk, increasing bulk in order to maintain longer-lasting connections, or increased costs when using continuous power supplies. As such and as required by the drive for technological miniaturisation, Bluetooth Low Energy started being developed by Nokia in 2006 (known as Wibree at the time) and was introduced globally in 2009 as part of Bluetooth 4.0 as a more energy efficient alternative.
While most current devices natively support both standard Bluetooth and BLE connections and both function within mostly the same 2.4GHz frequency band, these two protocols are fundamentally incompatible with each other, as they function based on different principles. The main difference between these two is the connection dynamic, with Bluetooth being an always-awake protocol, and BLE utilising an “as-needed” principle of communication – this means that, while Bluetooth represents a constantly open channel between connected devices, BLE facilitates communication only when required and enters sleep mode when no data transfer is being executed, thus conserving energy.
Other differences include lower latencies when compared to standard Bluetooth, reduced optimal range, slower overall transfer rates, etc. As such, while BLE addresses the power consumption issue successfully, this does limit its use to cases where a constant connection is not necessary and the amounts of data being transmitted are relatively small – as an example, BLE cannot currently be used for audio streaming. However, it is the perfect solution for devices such as health monitoring devices, fitness trackers, smartwatches, smart home appliances, and many others.
An outline of testing dynamics for BLE devices
The testing of applications which communicate with BLE devices is based on scenarios which would be applicable to Bluetooth in general (eg. range, interference, stability), but each scenario will have its own specificities and dynamics as a result of BLE’s differences. Some of the main points of focus will intuitively be long-term connection stability and viability, correct transfer of data packets in range- and interference-focused scenarios, as well as reliability in cases of power supply fluctuations.
Next, we’ll take a look at some outlines for the major areas which are vital when testing BLE-enabled devices and applications.
1.Connection
As a result of BLE’s intermittent communication method, connectivity tests are the cornerstone of any test suite which covers BLE-enabled devices. These tests are oftentimes mostly focused on the Client (the device or application which generates requests to and receives responses from the BLE device), as each environment has its own specificities, but several Server-focused (the device which receives requests from and sends responses to the Client) tests should always be included as well.
As a concrete example, neither Android, nor iOS support maintained connections for background applications as a result of both operating systems disabling old and passive processes in order to free up resources for applications which are actively being used. This means that such processes must be constantly hooked to the foreground in order for them to maintain their priority. This reveals the first scenario, namely running and idling applications for long periods of time (8h+) and monitoring connection persistence for the duration – this is especially important for wearable devices, as these are expected to function around the clock and deliver up-to-date information to the user at all times.
Another important aspect related to connectivity is device detection. These tests should include repeated Pairing scenarios in various contexts (eg. unpairing the Server from the Client device mid-use must not prohibit the reconnection of said Server to the Client and must not affect its correct functioning afterwards), as well as ensuring that Paired Servers successfully reconnect to their Clients in multiple interruption situations – killing the Client’s process and restarting it, switching the Server from one Client to another and ensuring that it performs within acceptable parameters with every supported Client, verifying successful reconnection when the Client and Server are brought back within range of one another, testing connectivity under low battery conditions, etc.
2.Range
BLE’s range limitations must also be verified through comprehensive tests, with one of the most relevant aspects being the difference between Out Of Range and At The Border – this is due to the fact that, unlike standard Bluetooth, BLE’s range limit is not a hard value (it is rated at less than 100m, while Bluetooth has a hard 100m-limit), which means that signal variability becomes a factor even without interference.
As touched upon within the previous section, when a BLE Server goes fully Out Of Range of its Client, the connection between the two is completely interrupted, requiring the user to return the Server within range of its Client in order to reestablish connection. These tests can be performed relatively easily, simply requiring the user to increase the distance between the two communicating devices until the threshold is reached. Extra layers of complexity can be added in the shape of long-term OOR disconnects and verifying that connectivity is successfully restored, repeated OOR disconnects in a short amount of time, verifying functionality when both the Server and the Client are OOR of each other when initiated (launching the app, turning on the wearable, etc.), and many others.
At The Border scenarios, however, are somewhat trickier to perform successfully. During our tests, we’ve noticed that, while the average range thresholds are respected consistently across applications, several undesired behaviours may appear when the Client and Server are At The Border of their connection range and the signal between the two becomes unstable – due to BLE’s on-demand connection, this may lead to Advertising/Data packets being partially lost and thus generating issues in both connectivity as well as response processing. An example of such a scenario that we’ve discovered during our time working with BLE devices is that if Advertising packets are lost due to signal instability, security validation may fail and the Server becomes forcefully and permanently disconnected from the Client, requiring the user to reset the connection and re-Pair the two devices.
3.Interference
Just like Range-based tests, Interference tests are essential in ensuring that the connection between the Server and Client is stable and functional. While many of the behaviours encountered during Interference testing resemble Range-based ones, the added signal instability plays a major role in undermining said connection.
Examples of interference tests are ensuring that the Server and Client can, within reason, still communicate through dense objects (eg. concrete walls, drywall, wood, etc.), monitoring the connection when either the Server, the Client, or both are in the vicinity of a running microwave oven, attempting to generate interference by using other Bluetooth or BLE-enabled devices (mobile phones, other wearables, speakers, printers, car sets, etc.), and many devices utilising WiFi technology which also support communication via the 2.4GHz band.
One of our favourite scenarios is verifying connection behaviour when introducing variations of a Faraday cage into the mix, as complete cessation of connectivity is expected and standard reconnection/rediscovery checks are necessary. There is a surprising amount of Faraday cages present within a standard household and, depending on how it was built, even the household itself may perform as one under certain circumstances. As such, many tests including metal ovens, metal pans, washing machines, and reinforced concrete are performed, as these are simply elements of daily life and must not prevent overall functionality.
4.Data
Last but not least of the major focus areas is data transfer and validation. Given that BLE devices are mainly meant to periodically transmit packets of data, one must ensure said data’s transfer rates and integrity are maintained in contexts of variable volume and complexity.
In terms of transfer rates, these are monitored both throughout normal usage, as well as during situations where Range and Interference play a role in modulating the connection between Server and Client, in order to ensure consistency and timeliness in data transfers and to identify any potential vulnerabilities should transfer rates be negatively affected.
Data integrity is just as vital, as exemplified within the previous sections, and is monitored alongside transfer rates. Tests include not only happy path scenarios which allow the Server and Client to communicate under optimal conditions, but also verifying how invalid/incomplete data is handled both Server- and Client-side.
Another aspect which should always be considered is data volume. While most BLE devices send relatively small packets when in active use, some devices, such as wearables, also offer the option of synching large packets of recorded data after long periods of disconnect between the Server and the Client in order to ensure that a full timeline is available to the user the instant connection is reestablished. As such, tests include repeated historic data synchronisation in short periods of time in order to ensure that no element is overwhelmed, and generating large amounts of historic data by keeping the Server and the Client disconnected from one another for long periods of time in order to ensure that pipelines can process large volumes, that connection between the Server and Client is not affected by said transfer, and that all data is correctly transmitted.
Data complexity varies according to the Server’s purpose, as a thermometer will generate relatively simple data when compared to a full sensor bank, such as a smartwatch which incorporates GPS capabilities, health monitoring suites, thermometers, WiFi capabilities, etc. In cases where the Server generates complex sets of data, it is vital to verify that said complexity is correctly accommodated by the connection between the Server and Client.
Conclusion
In conclusion, while BLE is a technology which is meant to simplify connectivity between various devices and to utilise power more efficiently, it is precisely its specifics which generate unexpected complexity in its interactions. We believe that in-depth tests at every level of interaction and variability are vital in ensuring a smooth user experience no matter the application. In addition, incorporating as many real-life scenarios and elements as possible and going as outside the box as possible when formulating test scenarios for any BLE-enabled application ensures that even the most unorthodox situations are covered and handled successfully, as we all know that it is precisely the unorthodox which becomes commonplace in day-to-day use.
Recent Comments