Home > About Ofcom > Accountability > Annual Reports and Plans > Ofcom Annual Report 2007 - 08 > Performance and evaluation

We are committed to reviewing and evaluating our performance, and then applying what we have learnt. We discuss below the various ways in which we measure our effectiveness and what the results tell us.

Unlike a private company that seeks to maximise profits on behalf of its shareholders, Ofcom does not have a single objective that lends itself to easy measurement. There are three main reasons for this. First, although our primary duty is to further the interests of citizens and consumers, including businesses, we have a wide range of specific duties and a diverse range of stakeholders. Second, it is often hard to differentiate between our impact on market developments and the impact of a wide range of other factors. And third, deciding whether we are fulfilling our duties and applying our regulatory principles is often subjective, meaning that assessing our performance in a measurable way is not straightforward.

This means that we need to use a number of methods to review our effectiveness. We measure:

• whether we are achieving our objectives and applying our regulatory principles;
• our internal efficiency and effectiveness;
• market developments; and
• stakeholder perceptions.

In the rest of this section we focus on the first two of these, the extent to which we are achieving our objectives and applying our regulatory principles, and our internal efficiency and effectiveness, including service delivery and enforcement.

Ultimately, however, we believe that our performance should be judged by reference to market developments and the outcomes that are delivered for citizens and consumers. We measure these outcomes in a number of ways and publish the results during the course of the year.

Progress against our Annual Plan is recorded on our website. We publish a table showing all our projects and planned outputs: http://www.ofcom.org.uk/about/accoun/reports_plans/annual_plan0809/projects/. This table is updated every quarter so that stakeholders can check when we will be publishing consultations they are interested in. The table also shows the extent to which we deliver on our plans.

Every year we publish our report, The Communications Market, which provides a comprehensive picture of developments in the communications sector. This is supplemented by a number of special communications market reports, such as Ethnic Minority Groups and Communications Services (June 2007) and the Nations and Regions Communications Market Report (May 2007). We also publish reports on specific issues, such as the take-up of digital TV and broadband, and have an extensive programme of market research. In particular, we published a report called The Consumer Experience (November 2007), which provided a picture of the positive benefits that competition has delivered for consumers.

We also examine the impact of particular policy initiatives. For example, in December 2007 we published our second report evaluating The Impact of the Telecoms Review. This sought to assess the effectiveness of the action we took to stimulate competition in the telecoms sector and thereby deliver increased benefits for consumers.

We also measure stakeholders’ perceptions of our performance, seeking their views on a wide range of issues, including the quality of our outputs and how well we have consulted. In addition, we conduct regular surveys of those stakeholders for whom we provide a service, such as spectrum users and individuals who contact the Ofcom Advisory Team (OAT), and the main findings are presented below alongside our key performance indicators (KPIs).

Ofcom’s approach to regulation reflects the fast-moving nature of the communications sector. The speed with which the communications sector is changing makes it especially important for us to have clear guiding principles. We are required by the Communications Act 2003 to have regard to the principles of better regulation, namely that regulation should be transparent, proportionate, consistent, accountable and targeted. When Ofcom was established, we built on these principles by developing a more specific set of principles to inform our day-to-day work. These are set out in Figure 1.

Figure 1: Ofcom’s regulatory principles

In line with these principles, we have focused on evaluating Ofcom’s performance in four key areas:

• our success in reducing regulation, where that has been appropriate;
• the effectiveness of our consultation with stakeholders;
• the timeliness of our decision-making following consultation; and
• our performance in carrying out impact assessments to inform our policy decisions.

Reducing regulation

A key element of better regulation is ensuring that regulation is properly targeted and does not impose undue burdens on our stakeholders. In December 2007, we published our Simplification Plan, which sets out all the initiatives we are taking to remove or reduce regulation. This shows how we are fulfilling our duty under Section 6 of the Communications Act to minimise the regulatory burden on our stakeholders.

Figure 2 summarises Ofcom’s policy statements published during 2007/8. In this figure:

• we have assessed how each decision affected the direction of regulation and have reflected this in the colour of the circles; and
• we have assessed the impact of the decision by reference to the size of the market affected by the impact of the regulation and this is reflected in the size of the circles.

The figure provides a useful overview of Ofcom’s regulatory decisions. From it we conclude that:

• in terms of the number of decisions, we reduced regulation in more cases than we increased it; and
• in terms of impact, the overall net effect was a significant reduction in regulation.

Figure 2 shows that we are reducing regulation in a range of areas, including:

• liberalising the use of the radio spectrum by, for example, removing the need for some spectrum users to buy licences and allowing mobile phones to be used on aircraft;
• extending spectrum trading;
• releasing more spectrum to the market;
• reducing the regulation on commercial radio stations; and
• reducing the regulation of retail telecoms markets to enable both consumers and businesses to benefit from increased competition.

It also demonstrates that Ofcom is prepared to take firm action to protect citizens and consumers where necessary. For example:

• we have acted to ensure that users of voice over IP services will have access to the emergency services; and
• we have introduced more stringent regulation to protect consumers from the mis-selling of telecoms services.

Figure 2: Ofcom’s key decisions – direction of regulatory change

1. Wireless Telegraphy Exemption: Radio Frequency Identification Equipment
2. Geographic number conservation measures
3. Protecting consumers from mis-selling of telecommunications services
4. Replicability: the regulation of BT's retail business exchange line services
5. Changes to BT’s regulatory financial reporting and audit requirements
6. Raising confidence in telephone numbers (Amending General Condition 17)
7. Public Call Boxes Consent for non-uniform charging for geographic tariffing trials
8. WRC-07 agenda item 1.4
9. Self-help TV relays and Digital Switchover
10. Future pricing of spectrum used for terrestrial broadcasting
11. BT OSS Separation
12. Guidance for PSBs commissioning from independent producers
13. Change to BT’s regulatory financial reporting obligations 2006/7
14. Spectrum award: 10, 28, 32 and 40GHz Bands
15. Wireless Telegraphy Licence Charges - Amendment
16. Wireless Telegraphy Exemption: UltraWideband Equipment
17. BT exemptions re Undertakings
18. Number Availability: Supplying numbers for 09 premium rate services
19. UK Broadband application for licence variation
20. The Future of Radio: the next phase
21. Telephone number portability for consumers switching suppliers
22. Regulations in connection with the award of 10, 28, 32 and 40GHz Bands
23. Signing on television
24. Licence-Exemption Framework Review
25. Regulation of VoIP Services: Access to the Emergency Services
26. Award of available spectrum: 1452 – 1492 MHz
27. Digital Dividend Review
28. Spectrum Usage Rights
29. Monitoring Compliance with Charge Controls
30. Statement on the Outline procedures for sanctions in cases relating to broadcasting
31. Decision to make Wireless Telegraphy Licence Charges Regulations
32. Spectrum Framework Review: the Public Sector
33. The Future of Radio: Localness on analogue commercial radio and stereo and mono broadcasting on DAB
34. Decision to make the Wireless Telegraphy (Exemption) (Amendment) Regulations 2008
35. Decision to make the Wireless Telegraphy (Automotive Short Range Radar) (Exemption) (No.2) (Amendment) Regulations 2008
36. Participation TV Part 1: protecting viewers and consumers
37. Statement on the making of regulations in connection with the award of 1452-1492 MHz
38. Service level guarantees: incentivising performance
39. The future use of the 169MHz Ex-ERMES Band
40. Amending the Wireless Telegraphy (Exemption) Regulations 2003
41. Mobile Communications onboard Aircraft (MCA)
42. Approval of amendments to the PhonepayPlus Code of Practice (11th Edition)
Source: Ofcom’s regulatory log

Consultation periods and responses

Consultations are one of the key ways in which we engage with stakeholders and enable them to influence our policy decisions. We have updated the guidance on our approach to consultation: http://www.ofcom.org.uk/consult/consult_method/ofcom_consult_guide

If a consultation is too short, some of those with important views to share may not have enough time to prepare their responses. If it is too long, the market involved may have changed dramatically. This could affect our ability to deal with an issue as quickly as the organisations involved would like. When we decide how long a consultation should last, we need to strike the right balance between these two considerations. There are generally three categories of consultation:

• Category 1: consultations which contain major policy initiatives and/or are of interest to a wide range of stakeholders (especially those who may need a longer time to response); we will consult for ten weeks.
• Category 2: consultations which, while containing important policy proposals, will be of interest to a limited number of stakeholders who will be aware of the issues; we will consult for 6 weeks.
• Category 3: consultations which fall within one or more of the following categories, where the time period for consultations is one month:
  • detailed technical issues;
  • where there is a need to complete the project in a specified timetable because of market developments or other factors which require the project to be concluded within a short period;
  • the issue has already been the subject of a consultation;
  • a proposal will have a limited effect on a market; or
  • a proposal is only a limited amendment to existing policy or regulation.

Figure 3 sets out an analysis of the length of our policy consultations by sector.

Figure 3: Analysis of consultation duration

Ofcom also collects statistics on the number of responses to consultations. This is shown in Figure 4. The chart illustrates the large number of responses we have had over the year, based on the 62 consultations where we subsequently published a statement. A small number of consultations have attracted a very wide range of responses, while most of the issues we consulted on were narrower in scope and of interest to a smaller constituency of stakeholders.

Figure 4: Responses to consultations

Timeliness of decision-making

We have also analysed the time that we take to publish a policy statement following the close of a consultation period.

There are a number of factors that affect the length of the period from the close of the consultation period until publication of the subsequent statement, including:

• the number and type of responses we get;
• whether our final decision is dependent on external factors; and
• the need for us to prioritise the use of our resources.

An analysis of our decision-making time is shown in figure 5. Most of our statements are now published within ten weeks of the consultation period closing. This is a significant improvement on the performance that we reported last year and reflects a concerted effort to improve our efficiency.

Figure 5: Analysis of the time taken to publish an Ofcom policy decision from the close of the consultation period

Impact Assessments

Impact Assessments (IAs) are an important part of the policy-making process. They ensure that in relation to our policy decisions:

• a wide range of options is considered, including the option of not regulating;
• these options are clearly presented;
• the impacts that would flow from each option are analysed carefully; and
• the costs associated with the chosen option are outweighed by the benefits.

Ofcom has a statutory duty to publish a list of the impact assessments carried out during the year. This list is set out in figure 6.

Figure 6: Impact Assessments carried out during 2007/8

In July 2005, we published guidelines Best-practice policy-making: Ofcom’s approach to impact assessment that emphasised Ofcom’s commitment to conducting assessments as an integral part of the policy-making process and stated that we expected to carry out IAs in the great majority of our policy decisions. Figure 7 sets out an analysis of the extent to which Ofcom’s consultation documents contained a clearly-labelled IA, i.e. an IA that was set out in a specific section or annex of the consultation document.

The table shows that 75 per cent of consultation documents contained a clearly-labelled impact assessment. This analysis shows that we are meeting the commitment that we made in our guidelines, and we will continue to ensure that impact assessments are carried out and properly presented in all relevant cases.

Figure 7: Analysis of impact assessments carried out

• Ofcom Annual Report 2007/08 - Section C [pdf]
• Adroddiad Blynyddol 2007-08 Ofcom - Adran C [pdf]

Back to top