Measuring consumer experience of using mobile services

16 June 2017

In 2016 Ofcom piloted a new method of measuring consumers’ experiences of using voice and data services on their mobile phones.

An app was developed which, when downloaded by users, ran a series of ‘passive’ tests on their phones measuring network availability and performance.

The app was downloaded by 6,632 people between September and December 2016. Of these, 4,288 retained the app on their phones for at least a week, enabling us to measure their experiences over this time.

Users’ experiences were affected by a number of factors in addition to network performance, including handset settings and potentially the tariff they were on. This means the findings are different to those from other pieces of research, which focus mainly on measuring mobile network performance.

Key findings were divided into three areas: data service availability; data performance; and voice performance. They included:

Data service availability

  • More than nine in ten mobile data downloads are successful for both 4G (95.6%) and 3G-only (92.4%) users.
  • Almost seven in ten users (69%) are happy with their overall service, with 4G users more satisfied than 3G-only users (71% vs 60%).
  • When using apps, 4G consumers are connected to Wi-Fi 69% of the time. When 4G users are connected to a cellular network, 65% of time is spent on a 4G network, 30% on 3G and 5% on 2G.

Data performance

  • Connection speeds when using YouTube and Chrome are faster over 4G than 3G, with Wi-Fi providing higher average speeds than both mobile technologies.
  • 4G networks are more responsive than 3G (48ms vs 64ms response time). Wi-Fi is even more responsive at 27ms.

Voice performance

  • Once initiated, less than 1% of all calls are dropped due to loss of service.
  • Nine in ten (90%) panellists say they are happy with the performance of their network when making a call.

When it came to overall satisfaction with their mobile service, over two-thirds (69%) of users said they were ‘fairly’ or ‘very’ satisfied, while those whose tariff and handset enabled them to use 4G services were more satisfied than 3G-only users (71% vs 60%).

Meanwhile, people in urban areas were more likely to say they were satisfied with their service than those in rural areas (72% vs 56%). This was true for both 4G users (73% vs 57%) and 3G-only users (63% vs 49%), and may be a reflection of better network coverage in urban areas.

The research findings also showed WiFi is a fundamental element for mobile phone use - 4G users connected to WiFi rather than mobile networks more than two-thirds of the time when using apps.

In most cases (95.6% of occasions), 4G users were able to access a mobile network (either 2G, 3G or 4G) and successfully download data. However, this was lower for 3G-only users (92.4%).

Looking throughout the day, the findings showed the highest levels of mobile use are between 10am and 8pm, and that during these times average recorded speeds are at their lowest. This indicates that network congestion results in lower throughput speeds at busy times.

The research also found that satisfaction with voice call performance was high, with 90% of users saying they were happy with their network performance when making a phone call. However, satisfaction was higher among those in urban areas than those in rural areas (90% vs 86%).