Home > TV > Ofcom PSB Review > Ofcom reports > Public Service Broadcasting > Measurement of delivery

Once we know what we are looking for, we can set about putting in place effective systems for measuring its delivery, with a focus in particular on those organisations charged with special responsibilities. Indeed, this is a fundamental part of the regulatory process: the Communications Act requires Ofcom to conduct annual statistical reviews of the whole broadcasting market, and periodic major reviews of the main PSBs and the extent to which they are meeting the purposes of public service broadcasting. These reviews are intended to be considerably more fact-based than has previously been typical of regulatory studies in the broadcasting sector, such as the annual ITC reports on the commercial PSBs.

A number of concerns are raised when we come to developing a framework to measure delivery:

• Inadequacy of available indicators - how can harder-to-define aspects of PSB such as quality and innovation be measured? (This leads to the aphorism that 'if you could measure quality it would just be quantity'.)
• Distortions caused by the use of indicators - once measures have been identified, broadcasters might overly focus on those indicators at the expense of the underlying performance that the indicators are attempting to measure. This leads to fears about the dangers of a return to 'box-ticking'.
• Use of too many indicators - there is a danger that by specifying indicators for everything that could conceivably be measured, the real story will get lost under the mass of indicators. This would be exacerbated if attention were to focus primarily on those indicators that are easiest to measure rather than more difficult ones that might be more revealing.

Notwithstanding the challenges that these issues raise, we believe that it is possible to conduct a useful and systematic review of the broadcasting sector to assess how effectively different aspects of PSB are being delivered. Key to a successful analysis is the recognition that indicators are only useful if they contribute to the overall assessment or narrative. So the indicators used must be clearly linked to key questions or aspects of broadcasting which are important PSB aims or objectives. They should be thought of as clues or pieces of evidence in constructing the story. These indicators will include measures of output (how much is being produced), impact (how many people are watching, and how much they are appreciating the programmes) and efficiency (how cost-effective it is to reach the audiences).

The next diagram shows one possible way of thinking about the different objectives of PSB, as set out in the Communications Act. For each objective, it is possible to devise measures that can then be tracked over time to build up a view of how effectively they are being delivered, and to allow comparisons between broadcasters and over time.

By way of example, consider the provision of news programmes. To consider the contribution of this genre towards range and balance, we might track the number of hours of output on each channel, in total and in peak-time, and expenditure on the provision of news. We might also measure the size of the audience for news programmes on different channels by socio-demographic group, to determine the extent to which the needs of different audiences are being met, as part of the analysis of diversity. Cost-per-hour data for news programmes will give a proxy for quality. We might also want to conduct 'content analysis' to measure the relative prominence given to regional, national and international stories; or to the focus on, say, political or show business stories (different broadcasters' news agendas may be seen to have an impact on range, diversity and quality). Finally, to look at 'informed democracy', we might use market research to monitor audiences' views on the extent to which news programmes promote fair and well-informed debate, are comprehensive and authoritative, and are regarded as being accurate and impartial.

An important additional dimension in the measurement of PSB must be the efficiency with which it is delivered. We are concerned not just with what is produced and consumed, but the resources used to produce and distribute it. Measures such as amount spent per viewer for each hour of broadcast (across different genres) can be useful here. This might enable us to monitor, amongst other things, the comparative effectiveness of different broadcasters in producing comparable type of programming.

The Ofcom review will mark the first instance such a comprehensive, evidence-based analysis of PSB will be conducted across the entire sector, covering both the BBC and the commercial players. This is a potentially powerful exercise, and should significantly increase the extent to which broadcasters become accountable for fulfilling the PSB purposes set out in the Act (and elaborated in the review).

Back to top