Children and young people (CYP) who come into contact with the criminal justice system (CJS) share many enduring early-life risk factors, including increased likelihood of vulnerable childhoods at home or in care, school exclusion and interrupted education, and more often than not reside in heavily deprived areas. They may experience multiple adversities at once, and many become both victims and perpetrators of crime before they even reach adulthood.

News items with tabloid headlines remind us regularly of the rising threat of knife and other violent crime in our neighbourhoods and often call for increased punitive powers for criminal justice system bodies from police to prisons. However, it is unclear the extent to which this system can appropriately prevent offending or re-offending in CYP. An increasing consensus instead points to the need for better research and evaluation to understand the potential impacts of alternative strategies, including therapy-based approaches, diversionary schemes, and improved policing methods.

In response, the UK Government contributed £200 million to set-up the Youth Endowment Fund (YEF). The YEF operates as a ‘What Works’ Centre with a 10-year mandate from the Home Office to lead and fund crucial research to address evidence gaps and identify potential strategies to reduce violence affecting children and young people in England and Wales. 

Randomised controlled trials (RCTs) to reduce violence affecting children and young people

With a goal to provide the most effective evidence for stakeholders, the YEF have rightly prioritised the design and funding of evaluations using randomised controlled designs across a range of target areas including mentoring, music and sport programmes, therapy, and school-based interventions, and have funded some excellent schemes in the first half of the endowment period.

However, some potential strategies with research gaps are more difficult to randomize. A key example is police stop and search (SAS), which is highly valued by forces who argue for their deterrent effects, however there is little or low-quality evidence for their effectiveness in reducing violent crime in CYP. A key issue is the difficulty in designing robust evaluations of SAS; randomised trials are rarely possible, given issues with the removal of what are perceived as core police powers. Clearly other approaches are needed for both SAS, and many other interventions for which randomisation may not be straightforward to implement. 

What are the alternatives when RCTS aren’t possible?

The use of ‘quasi experimental’ and observational designs (QEDs), i.e. designs that assess the causal impact of an exposure on an outcome, usually of programmes applied in non-research contexts and without explicit randomisation, are common in both epidemiology and economics research. However, navigating the relative merits of different QEDs is often complex even for practitioners. Bias due to participant selection, measurement and confounding are difficult pitfalls to avoid, and not all QEDs are equal. 

The YEF invited organisations to bid for grant funding to design a package of training sessions on QEDs for its own staff, in order to better equip them to critically appraise the increasing number of projects proposing QEDs in the next phase of their endowment period.

LSHTM awarded funding to design QED training package for the YEF

The LSHTM has a long tradition of robust research and teaching on QEDs, and significant methodological expertise and research in complementary areas including domestic partner violence and school-based interventions. Our bid, led by the Centre for Evaluation (CfE) was successful. In choosing to award the grant to the Centre, the YEF praised our commitment to capacity building and our values-led proposal. Dr Claryn Kung, Senior Evaluation Manager at the YEF, said: “We particularly valued that the LSHTM CfE core mission is to drive capacity building in the understanding, use and support of appropriate applications of evaluation methods.”

We took a flexible approach - working directly with the YEF to decide on the level and pitch of the training material, as YEF evaluation staff are from mixed backgrounds with a significant proportion of social scientists. We also incorporated relevant, motivating examples in key YEF areas. The foundation of the training expanded upon the School’s Evaluation of Public Health Interventions MSc module. The final offering covered causal inference, statistics, and a range of QEDs including interrupted time series, synthetic controls, regression discontinuity, propensity scores and matching, instrument variables, target trial emulation, and difference-in difference designs. 

LSHTM Staff Stepped Up

We put out a call to our network for staff to design and deliver training sessions either in person at the YEF headquarters in Shoreditch or remotely. We received a very positive response from all the staff we invited to contribute, resulting in a 10-person team of: Dr Melissa Neuman, Dr Daniel Carter (pictured below), Dr William Rudgard (University of Oxford) Dr Julia Pescarini, Professor Ruth Keogh, Dr Stephen O’Neill, Dr Thiago Cerqueira Silva, and Professor Tim Powell-Jackson. Professor Mitzy Gafos (CfE co-Director) gave the introductory talk, and Dr Shay Soremekun (CfE co-Deputy Director who led the bid) led the opening and closing sessions on applications of QED theory, and the relative merits of different designs.

The training was delivered over the autumn term of 2024 in a series of half-day workshops, in which the YEF rated the complexity level and trainers themselves very highly in their final feedback to the Centre (both 9/10). The LSHTM trainers also fed back their positive experiences of delivering the training. Professor Powell-Jackson said: “It was a hugely rewarding experience to deliver training to such an engaged group of people, responsible for commissioning real-world evaluations of anti-youth violence programmes.”

Dr Carter, who runs the LSHTM social epidemiology MSC module and works in a similar field was equally enthusiastic: “I delivered one of the first sessions on fundamental concepts for inferring causality from observational data – as a researcher working in violence prevention, being able to share, discuss, and think through these methods together with colleagues outside of academia working in a similar field was a fantastic opportunity.”

To recognise the significant time and effort of the trainers, the CfE allocated a portion of the grant award to each team member either to use for research-related expenses or to transfer to other LSHTM Centres of their choice – resulting in some of the award being transferred to the Centre for Data and Statistical Science for Health and the Global Health Economics Centre.

Our takeaway? This award was an opportunity for Centre members to lead the development of policy-relevant evaluation training materials suitable for stakeholders beyond our usual academic partnerships.

Professor Keogh noted this from her experience: “I delivered a training session on the topic of ‘target trial emulation’, a framework for designing studies to assess the effects of interventions using observational data. The team at YEF were very welcoming and engaged in the session, asking thought-provoking questions about how the trial emulation framework could be useful to them in evaluating evidence on interventions and policies of interest in their context. It was interesting to hear about the YEF’s important work and the types of evaluations they carry out, which was something previously quite unfamiliar to me.”

Beyond our involvement, the benefits of this partnership with the YEF will encourage the future use of alternative approaches by UK-based researchers to assess the merits of government and grassroots programmes. Professor Gafos summarised this role: “The YEF training programme was a great example of how we can promote the use of evaluation designs that are able to robustly and pragmatically assess the impact of complex public health interventions to directly inform policy and practice.”

Hopefully this work will result in the generation of better, evidence-informed strategies to protect children and young people from involvement in violence.

Please note: All CfE QED training materials will be stored in a permanent repository, which will be accessible to staff and research degree students for future use in training and development. 

Blog written by Dr Shay Soremekun (co-Director for the Centre for Evaluation and Assistant Professor of Epidemiology at LSHTM)