London School of Hygiene & Tropical Medicine

A new framework for evaluating tools for national health planning has been published in BMJ Open.

The framework was developed by a team of researchers from the University of Strathclyde, the Center for Global Development and the London School of Hygiene & Tropical Medicine (LSHTM).

The framework is intended to aid the assessment of tools to support decision making in national health planning within low- and middle-income countries (LMICs).

The framework does not rank tools but instead assesses their fitness for purpose by providing information on tools’ characteristics and an understanding of their ability to produce analyses or processes that adhere to good practices of economic analyses, as outlined by the International Decision Support Initiative (iDSI) Reference Case.

The framework also factors in the time, data and technical expertise requirements of each tool to help determine their suitability for a given context.

Until now, it has been difficult for policymakers to decide which tool should be to aid priority setting. This framework simplifies the process of priority setting by providing tables which allow for the quick comparison of tools.

Francis Ruiz, Senior Policy Fellow at LSHTM and GHECO member, who contributed to the framework’s development, said “Our research focused on information available in the public domain – it revealed that some important information relating to how the tools can and should be used remains opaque.”

The research team define national health planning tools as a software package that requires input and produces outputs that are designed to support country-level decision making.

The tools included in the framework were selected based on the following factors: if they provide quantitative analysis; are applicable for multiple countries; can be adapted to use new data; and are available in the public domain during the period of evaluation. Seven tools were evaluated using the framework: CAPACITI, PriorityVax, HIPtool, Optima, OneHealth (spectrum suite), LiST and EQUIST.

The tools are grouped into three categories: evidence-generating, evidence-synthesising and process. Evidence-generating tools, including LiST and Optima, produce outputs use in the prioritisation process. Evidence-synthesising tools, including the HIPtool, provide a comprehensive picture for users by combining evidence from different sources. Process tools, including PriorityVax and CAPACITI, help capture and balance views to support prioritisation processes more broadly.

Itamar Megiddo, Senior Lecturer at the University of Strathclyde in Glasgow who led the study, said: “Understanding the strengths and limitations of priority setting tools not only aids in selecting the most suitable tool for specific questions and contexts but also provides valuable insights for their future enhancement. We identified gaps in these tools’ ability to compare interventions across disease and programme areas for strengthening health systems. This gap limits analysts, and thus decision-makers’ ability to make such comparisons, which are important for informing investment decisions. Engaging with stakeholders further will enhance our understanding and guide future directions for improvement.”

The research team note that no tool is optimal in every case and emphasise the context-specific nature of the tools included in the framework.

The researchers note that the study was limited to desk-based research and in-depth interviews with a range of stakeholders, including tool developers and users, could provide additional information on the tools’ fitness for purpose and future direction.

Read the paper.