TRL | Monitoring and Evaluation

TRL’s Monitoring and Evaluation (M&E) team are experts in the design and delivery of robust evaluations, applying relevant evaluation guidance (for example, the HM Treasury Magenta book) during complex trials, fast-moving programmes and portfolio delivery. Our M&E team work in cross-discipline project teams with our transportation subject matter experts in a range of areas such as environment and decarbonisation, transport safety, new mobility, automation and road user behaviour. This in-depth understanding of the transport context, supplemented by our knowledge of the wide range of possible data collection methods and published sources, for each evaluation means that we can adapt the evaluation design and delivery throughout the course of the project, without compromising the validity of the results. We understand how the findings from the evaluation can be applied to future policy or programmes.

Our M&E team can help you answer the following questions:

What difference has the intervention/programme made?
What can be learned from how the intervention/programme was delivered?
Was this a good use of resources?
What can be learnt from this intervention that can applied to future similar initiatives?

Experts in experimental impact evaluations

Our team are experts in the design of experimental and quasi-experimental impact evaluations including a large-scale Randomised Control Trial; RCTs are the gold standard in evaluation design but are rarely implemented in transport studies due to their complexity. TRL are the lead organisation designing and delivering the Department for Transport (DfT) Driver2020 trial, which reports in 2023. This seeks to evaluate the effectiveness of five interventions in improving the safety record of young novice drivers in their first year of post-test driving using an RCT design with over 28,000 participants. This complex project has involved significant stakeholder and intervention delivery partner management; the development of an evaluation framework which had to be adapted due to the impact of Covid-19; statistical modelling work to understand the impact of the interventions; as well as process evaluation elements using qualitative methods to investigate barriers to engagement and lessons for delivery.

Flexible, proportionate approach

We design evaluation approaches that are proportionate and appropriate for each individual situation; our strength is in our deep knowledge of the transport context and how to balance the needs for robust evaluation with the practicalities of time and cost, and the restrictions imposed by real-world implementation.

Our portfolio includes national and international evaluations of regulation changes, monitoring and evaluation for demonstrators of innovative technologies and evaluation support for large-scale research programmes (e.g. Future Transport Zones). We have also delivered smaller-scale evaluations at a local or fleet level to investigate the impact of infrastructure changes, technologies and behavioural interventions.

We have extensive experience in the design, monitoring, and evaluation of both on and off road trials, often using quasi-experimental approaches or theory-based methods to understand the impact of the technology or scenario being tested. The monitoring is supported by the safety case to ensure that the risks have been identified and controlled.

For example, TRL led the HelmUK consortium delivering the first on-road trial of connected and automated Heavy Goods Vehicles on UK roads. We conducted a comprehensive evaluation of the platooning technology, using information gathered from the real-world trials. This involved a quasi-experimental impact evaluation, process evaluation and economic modelling of the potential costs and benefits to feed into the business case. The outputs and recommendations are enabling National Highways, the UK government and the freight industry to set their approach to future HGV platooning on UK roads with confidence that these decisions were backed by real-world evidence.

Assessing the cost-benefit of future scenarios

In addition to post-implementation evaluation, our team are also experienced in transport appraisal and cost-benefit analysis. For example, we have carried out complex cost-benefit analysis of vehicle safety technologies for Europe and internationally, the results of which have supported the introduction of regulations for these technologies.

Developing tools to support appraisal and evaluation activities

Decarbonisation of transport is a key challenge in the UK and internationally. TRL have developed VAME (the Vehicle Analytics Monitoring and Evaluation) platform to support decarbonisation commitments by making it quicker, cheaper and easier to collect and analyse data gathered in large scale trials. These tools help vehicle operators participate in robust trials without compromising their operational activities, and lead to rapid validation of commercial claims related to technical performance, safety, greenhouse gas (GHG) emissions, and operational costs of different types of vehicles. This tool has been successfully piloted during the evaluation of the Teesside Hydrogen Transport Hub demonstrations.

In 2017, TRL were also involved in a European project to develop a cost-benefit tool to compare the costs and monetised benefits of Cooperative Intelligent Transport Systems in various contexts; this tool aims to support National Road Authorities with investment decisions for different deployment scenarios. TRL are currently carrying out similar projects to develop appraisal frameworks for investment decisions for other clients.

Ethical evaluations

All our evaluation work is governed by TRL’s ethics policy, which sets out guidelines for research involving human participants. These guidelines are informed by EPSRC’s Guide to Good Practice in Science and Engineering Research and satisfy the ethical standards of professional bodies such as the British Psychological Society, the Market Research Society, and funding bodies such as the EPSRC and ESRC.

Notable benefits from TRL evaluations

There are many examples of our work having resulted in significant benefits to clients and society:

Evaluation activity for Driver and Vehicle Standards Agency (DVSA) aimed to understand the impact on safety and performance for a revised practical driving test. The findings from the study informed development of the driving test in GB ensuring it was fit-for-purpose.
Between 2016 and 2020, TRL carried out a substantial programme of work for National Highways to evaluate the safety and customer satisfaction of changing the speed limit through roadworks in certain conditions. Ultimately this has resulted in changes to guidance which will have notable benefits for traffic flow and journey times.
The CVEI project was a multi-million-pound consortium project led by TRL. It carried out trials of Ultra Low Emission Vehicles (ULEVs) with mass-market consumers to inform policy and industry strategies to enhance energy integration between consumers, vehicles and energy systems.
Our evaluation of innovative cycling infrastructure informed Design Standards and regulations.
In the first trial of its kind, the Low Emission Freight and Logistics Trial (LEFT), TRL demonstrated how low emission technologies could be monitored and evaluated. The data collection and analysis methodologies developed will inform future trials; for example, the government’s Zero Emission Road Freight Demonstrators (ZERFD).