London School of Hygiene & Tropical Medicine

In fact, the results show that the introduction of public reporting of individual surgeons’ outcomes coincided with a substantial reduction in mortality for patients having non-emergency (‘elective’ or ‘scheduled’) bowel cancer surgery.

The team of UK researchers, led by the London School of Hygiene & Tropical Medicine and the Royal College of Surgeons, also found similar improvements were not found for emergency surgery, suggesting that improvements in care before, during and after major bowel surgery - only possible for elective procedures - may explain the findings.

Professor Jan van der Meulen from the London School of Hygiene & Tropical Medicine’s Department of Health Services Research and Policy, and a senior author on the paper, said: “This study shows that monitoring and reporting outcomes of major surgical procedures by the National Bowel Cancer Audit is an important step towards improving the quality of surgery, with the surgeons themselves closely involved in the audit. The improvement in surgical outcomes that we saw immediately after the public reporting of outcomes began, demonstrates the importance of the principle ‘no audit about me without me’.

“Collaboration between surgeons working in the NHS and data analysts based at the London School of Hygiene & Tropical Medicine underpins a system that can feedback surgical outcomes that are clinically relevant and methodologically robust.”

When patient death rates for individual surgeons were first published in June 2013, the move was hailed as a major breakthrough in transparency that would drive up standards of care in England. However, critics argue that public reporting of outcomes encourages risk averse behaviour, whereby surgeons are less likely to offer surgery to patients at higher risk, and manipulation of data to increase patients’ predicted risk or to make patients ineligible for public reporting, often referred to as ‘gaming’.

So far, the evidence that public reporting leads to improvements in the quality of patient care is surprisingly weak, and its effect has been studied only in cardiac surgery and almost exclusively in the US. Therefore the research team decided to look for evidence of risk averse behaviour, manipulation of data, and change in death rates immediately before and after the introduction of surgeon specific outcome reporting in colorectal cancer surgery across the NHS in England.

They analysed data for more than 111,000 patients included in the National Bowel Cancer Audit (NBOCA) diagnosed with colorectal cancer from April 2011 to March 2015. To investigate risk averse behaviour, they compared the proportion of patients who had elective surgery, predicted 90 day mortality, and observed 90 day mortality, before and after the introduction of public outcome reporting.

After factors that could have affected the results were taken into account, such as patient characteristics and tumour grade, the researchers found that the proportion of patients with colorectal cancer who had major surgery did not change after the introduction of public outcome reporting (63.3% before compared with 63.2% after). The proportion of urgent or emergency procedures - and therefore ineligible for public reporting - also did not change after the introduction of public reporting (15.5% before compared with 15.6% after).

The predicted 90 day mortality remained the same (2.7%), but the observed 90 day mortality fell from 2.8% before to 2.1% after. Further analysis showed that this reduction was over and above the existing downward trend in mortality before the introduction of public reporting.

Professor Derek Alderson, President of the Royal College of Surgeons, said: “The RCS has supported the publication of individual surgeons’ results as a way of improving the quality and safety for patients undergoing surgery. It is important that patients know that as a profession we are robustly scrutinising our surgery and seeking to understand our results to improve the care we provide.

“There has however been concern in some quarters that reporting the outcomes of individual surgeons could discourage some from offering surgery to high-risk patients. Today’s study is reassuring as it did not find any evidence of risk-averse patient selection following the introduction of public reporting of outcomes after bowel cancer surgery.

Professor James Hill, President of the Association of Coloproctology of Great Britain and Ireland and lead clinician on the study, said:

“Providing safe and effective surgical care requires team work. However, the publication of surgical outcomes turns the spotlight on individual surgeons. The improvements that we have seen following the introduction of public reporting of outcomes of planned bowel cancer surgery demonstrate that surgeons have an important role in galvanising the entire team involved in managing patients before, and after this major surgical procedure.”

The authors acknowledge limitations of their study, including that it was observational, so no firm conclusions can be drawn about cause and effect - and say that using a ‘before-after’ design is a potential weakness as changes may occur in the quality of data over time.

Publication

Abigail E Vallance, Nicola S Fearnhead, Angela Kuryba, James Hill, Charles Maxwell-Armstrong, Michael Braun, Jan van der Meulen, Kate Walker. Effect of public reporting of surgeons’ outcomes on patient selection, “gaming,” and mortality in colorectal cancer surgery in England: population based cohort study. The BMJ. DOI: 10.1136/bmj.k1581.