Access key 0 - Accessibility, Access key 2 - Jump to content, Access key 7 - Jump to navigation
Skip To Content | Skip To Navigation
 

Home > About Ofcom > Accountability > Annual Reports and Plans > Ofcom Annual Report 2008 - 09 > Performance and evaluation


Performance and evaluation

We are committed to reviewing and evaluating our performance, and then applying what we have learnt. We discuss below the various ways in which we measure our effectiveness and what the results tell us.

Unlike a private company that seeks to maximise profits on behalf of its shareholders, Ofcom does not have a single objective that lends itself to easy measurement. There are three main reasons for this. First, although our primary duty is to further the interests of citizens and consumers, including businesses, we have a wide range of specific duties and a diverse range of stakeholders. Second, it is often hard to differentiate between our impact on market developments and the impact of a wide range of other factors. And third, deciding whether we are fulfilling our duties and are applying our regulatory principles is often subjective. This means that assessing our performance in a measurable way is not straightforward.

It is therefore appropriate to use a number of methods to review our effectiveness. We measure:

Measuring market developments

Ultimately, we believe that our performance should be judged by reference to market developments and the outcomes that are delivered for citizens and consumers. We measure these outcomes in a number of ways and publish the results during the course of the year. We describe a selection of the relevant reports below.

Progress against our Annual Plan is recorded on our website. We publish a table showing all our projects and planned outputs: http://www.ofcom.org.uk/about/accoun/reports_plans/annual_plan0910/projects/. This table is updated every quarter so that stakeholders can check when we will be publishing consultations they are interested in. The table also shows the extent to which we deliver on our plans.

Every year we publish our report, The Communications Market (http://www.ofcom.org.uk/research/cm/cmr08/), which provides a comprehensive picture of developments in the communications sector. This is supplemented by a number of reports on specific aspects of the sectors we regulate, such as The International Communications Market (November 2008 - http://www.ofcom.org.uk/research/cm/icmr08/). We also publish reports on specific issues, such as the take-up of digital TV and broadband. In particular, we published a report called The Consumer Experience (November 2008 - http://www.ofcom.org.uk/research/tce/ce08/), which provided a picture of the positive benefits that competition has delivered for consumers, and a range of Media Literacy Audit publications, which analysed the abilities of UK citizens and consumers to access, understand and create communications in a variety of contexts. All these reports build on our extensive programme of market research.

We also examine the impact of particular policy initiatives. For example, in December 2008 we published our interim review of restrictions on the advertising of food and drink that is high in fat or salt or sugar (HFSS). Our statement, Changes in the nature and balance of television food advertising to children , looked to assess whether or not the restrictions we phased in from April 2007 were having the expected effects in terms of:

Measuring compliance with our objectives and regulatory principles

Ofcoms approach to regulation reflects the fast-moving nature of the communications sector. The speed with which the communications sector is changing makes it especially important for us to have clear guiding principles. We are required by the Communications Act 2003 to have regard to the principles of better regulation, namely that regulation should be transparent, proportionate, consistent, accountable and targeted. When Ofcom was established, we built on these principles by developing a more specific set of principles to inform our day-to-day work. These are set out in Figure 1. In 2008 we also identified a set of principles for analysing the opportunity to introduce self- and co-regulation: http://www.ofcom.org.uk/consult/condocs/coregulation/statement/

In line with our regulatory principles, we have focused on evaluating Ofcoms performance in four key areas:

Reducing regulation

Figure 1 : A summary of how and where we regulate, and how we support regulation

A key element of better regulation is ensuring that regulation is properly targeted and does not impose undue burdens on our stakeholders. In December 2008, we published our annual Simplification Plan, which sets out all the initiatives we are taking to remove or reduce regulation. This shows how we are fulfilling our duty under Section 6 of the Communications Act to minimise the regulatory burden on our stakeholders.

Figure 2 summarises Ofcoms policy statements published during 2008/9. In this figure:

The Figure provides an overview of Ofcoms regulatory decisions. The last year was a time of big and important regulatory decisions. Some decisions allowed us to continue our trend of reducing regulation. Other decisions required additional or increased regulation to achieve optimal citizen and consumer outcomes.

Specifically, Figure 2 shows that we are prepared to take decisive and effective action to protect citizens and consumers where necessary, for example:

It also demonstrates that we are committed to reducing regulation by, for example:

There has also been an increase in the number of streamlined or co-regulatory solutions. In 2007/8 there was one instance of this, in 2008/9 there were 6 instances.

Figure 2: Ofcom's key decisions - direction of regulatory change List of statements

Consultation periods and responses

Consultations are one of the key ways in which we engage with stakeholders and enable them to influence our policy decisions.

If a consultation is too short, some of those with important views to share may not have enough time to prepare their responses. If it is too long, the market concerned may have changed dramatically. This could affect our ability to deal with an issue as quickly as the stakeholders involved would like. When we decide how long a consultation should last, we need to strike the right balance between these two considerations. There are generally three categories of consultation:

Figure 3 sets out an analysis of the length of our policy consultations by sector. A longer consultation period gives stakeholders greater opportunity to respond and become involved in policy decisions. This is particularly important when the consultation issue has great significance and a high profile. Figure 3 shows that in 2008/9 most of our consultation periods lasted at least ten weeks (52% of consultations), an increase on our performance in 2007/8 (42%).

Figure 3: analysis of consultation duration

Ofcoms consultation guidelines are published on its website: http://www.ofcom.org.uk/consult/consult_method/ofcom_consult_guide

*The majority of spectrum consultations were statutory consultations on regulations for which the minimum period for consultation is set out in statute, and is one month

Ofcom also collects statistics on the number of responses to consultations. This is shown in Figure 4. The chart illustrates the large number of responses we have had over the year, based on the 63 consultations where we subsequently published a statement. A number of consultations have attracted a very wide range of responses, many more than in 2007/8, while most of the issues we consulted on were narrower in scope and of interest to a smaller constituency of stakeholders.

Figure 4: Responses to consultations

Timeliness of decision-making

We have also analysed the time that we take to publish a policy statement following the close of a consultation period.

There are a number of factors that affect the length of the period from the close of the consultation period until publication of the subsequent statement, including:

An analysis of our decision-making time is shown in Figure 5. The majority of our statements are now published within ten weeks of the end of the consultation period. This data compares favourably with our performance in 2007/8.

Figure 5: Analysis of the time taken to publish an Ofcom policy decision from the close of the consultation period

The five statements that were published more than 30 weeks after the consultation period closed (and the reasons for the delay) were:

Managing our work

Ofcom uses a standardised project management methodology for all its project work, which was developed in 2007/8. This methodology divides a project lifecycle into five key stages, with formal review points at the end of each stage. It provides us with more robust project information such as cost and headcount information for the lifetime of a project, a more formal and controlled approach to starting and closing projects, defining project scopes and assessing project performance.

During 2008/9, we made various enhancements to our existing project reporting system, Artemis, to incorporate the new five-stage project management methodology. The modified system was rolled out in early April 2009. We will continue to develop the systems reporting and project planning capabilities this year, in order to help us manage our work more effectively and efficiently.

Impact Assessments

Impact Assessments (IAs) are an important part of the policymaking process. They ensure that in relation to our policy decisions:

Ofcom has a statutory duty to publish a list of the impact assessments carried out during the year. This list is set out in Figure 6.

Figure 6: Impact assessments carried out during 2008/9

In July 2005, we published guidelines Best-practice policy-making: Ofcoms approach to impact assessment that emphasised Ofcoms commitment to conducting assessments as an integral part of the policy-making process and stated that we expected to carry out IAs in the great majority of our policy decisions. We plan to publish revised guidelines over the course of 2009 to improve the effectiveness of IAs and reflect changes in best practice. Figure 7 sets out an analysis of the extent to which Ofcoms consultation documents contained a clearly labelled IA, i.e. an IA that was set out in a specific section or annex of the consultation document.

Figure 7: Analysis of impact assessments carried out

The table shows that 76 per cent of consultation documents contained a clearly-labelled impact assessment. This analysis shows that we are meeting the commitment that we made in our guidelines, and we will continue to ensure that impact assessments are carried out and properly presented in all relevant cases.

Measuring stakeholder perceptions

We also measure stakeholders perceptions of our performance, seeking their views on a wide range of issues, including the quality of our outputs and how well we have consulted. In addition, we conduct regular surveys of those stakeholders for whom we provide a service, such as spectrum users and individuals who contact the Ofcom Advisory Team (OAT), and the main findings are presented below alongside our key performance indicators (KPIs).



Back to top Back to top

 Accessibility tools


Further information

Help

Keep up-to-date