This experimental study focuses on evaluating the effectiveness of various behavioural interventions designed to help users assess the reliability, accuracy, and authenticity of online content. It employs a novel methodology known as Adaptive Randomised Control Trials to explore this area.
Methodology
In our study, 6,000 adults representative of the UK population viewed a feed consisting of 15 posts, 10 of which were true and 5 false. Participants were asked to assess the accuracy of each post and report their confidence in their assessments. We employed an Adaptive Randomised Controlled Trial (ARCT) methodology, which allows for real-time updates in treatment allocation.
The interventions included user-focused tools, such as an educational quiz and reminder prompts, delivered either before or during feed exposure. Additionally, post-focused tools, including fact-checker labels, AI, and crowdsourced note labels, were directly applied to the false posts. This comprehensive approach allowed us to test a wide range of strategies and their combinations within a unified experimental framework.
We found that
- A combination of user-focused tools and labels were most effective at improving participants’ ability to identify false content.
- When tested in isolation, user-focused tools generally outperformed labels.
- Interventions did not significantly reduce trust in verifiably true information.
- Crowdsourced notes were marginally less effective than AI and fact checking labels.
While these findings provide novel evidence on the effect of different types of media-literacy tools and their complementarities, they should be interpreted in light of the study’s limitations. Therefore, the findings should be viewed as indicative evidence in a controlled setting rather than direct estimates of real-world impacts.
If you would like to find more about this research, see the Helping Users to Assess Content Online - Discussion Paper.
For more details about the methodology, trial design and data analysis, see the Helping Users to Assess Content Online - Technical Paper.
If you would like to comment on this Insights paper, please email behavioural.insights@ofcom.org.uk.