The role of clinical audits in driving performance in radiotherapy

By the Radiation Oncology Safety and Quality Committee

ESTRO 2026 Congress Report, Interdisciplinary Track Symposium

The highlight of ESTRO 2026 for me from a quality perspective was the excellent symposium on the role of clinical audits in improving the quality of radiotherapy . This session covered: the comprehensive peer-review audits of radiotherapy centres in Belgium; national dosimetric audits in Australia and New Zealand; Danish contouring audits; and national audits of adverse events in patients receiving prostate radiotherapy in the UK National Health Service (NHS)

1. Aude Vanderling from Belgium presented on “Clinical Audits Driving Quality Improvements: the Belgian experience”

These audits have been conducted in Belgium since 2011. The driver for them was the implementation of a law in Belgium mandating the conduct of audit in radiotherapy. The National Cancer Control Programme then allocated an annual budget for employment of a quality manager at each centre on the basis that an audit took place. Nationally they initially adopted the audit methodology set up by the International Atomic Energy Agency’s Quality Assurance Team for Radiation Oncology (QUATRO), i.e. a peer-review audit with a pre-audit and onsite evaluation. These were comprehensive; they assessed infrastructure, patient- and equipment-related procedures, quality and safety management and education and training programmes. Teams of auditors were trained, and the first cycle of audits took place between 2011 and 2015. The role of the auditors (a Radiation Oncologist(RO), a Medical Physicist(MP) and a Radiation Therapist (RTT)) was to identify gaps in the radiotherapy process from referral to follow-up. After each audit, a final report with commendations and recommendations was produced and given to each department. The focus of the audit was on quality improvement rather than regulation. After the first cycle of audits was conducted, they modified the methodology in recognition of technical advances in radiotherapy, and the term (B)-QUATRO was coined to reflect this Belgian modification of QUATRO.

The third phase of the audit cycle is ongoing (2024-2029). The audits have enabled the identification of gaps in services. These have included issues related to elderly patients, implementation of image-guided radiotherapy (IGRT), RTT training and patient follow-up, and several changes have taken place nationally to address these deficiencies. Feedback that followed the first audit cycle demonstrated that 54% of departments found the audits very useful, 80% felt they were relevant, and 54% felt the audits had an impact on improvements. The feedback from the current cycle of audits is that seven of the eight departments that have been audited consider the audit recommendations impactful. The auditors have found that most gaps that were identified in the first audit have been corrected by the second or third audit. The audits have strengthened relationships between radiotherapy departments as the auditors have been chosen from all over Belgium.

The B-QUATRO audits demonstrate that audits drive change and have shown the importance of high-level governmental support with the allocation of a budget for these audits. A strength would seem to be that the audits are formative rather than regulatory and that peers meet face-to-face and develop those relationships that endure. The fact that the audits are conducted on a cyclical basis with re-audit and modifications of the audit methodology as technology evolves makes them very effective.

Kate Francis from Australia presented on “The impact of ACD dosimetry audits”

The Australian Clinical Dosimetry Service (ACDS) was set up in 2011 by the Australian government in response to two high-profile incidents that had occurred in the preceding years, in which several patients had been under- or overdosed. ACDS is a multiprofessional group that independently checks the accuracy of dose delivery in radiotherapy to ensure that radiation treatment across Australia and New Zealand is accurate and safe. Participation in audits is mandatory to receive funding. The auditors are trained and audits vary in complexity from remote audits of beam outputs using optically stimulated luminescent dosimeters to end-to-end testing onsite that includes volumetric modulated arc therapy(VMAT), image-guided(IGRT) and stereotactic ablative radiotherapy(SABR), and motion management. The outcome of audits is to facilitate improvement; the ACDS is not a regulatory body. During these audits errors in commissioning, treatment planning systems (TPSs), treatment delivery, and IGRThave been discovered . Some illustrative cases were discussed.

The outcomes of the audits are that a result is either (a)out of tolerance (OOT), (b)improvement is required or (c) it is satisfactory. Reports are produced following audits. A clinical advisory group(CAG) guides the ACDS and ensures that audits are kept up-to-date with technology. They also advise departments that have OOT results. ACDS has to date carried out 1400 audits and reports an average OOT of 10%. Interestingly, the OOT rate has not changed with time. This point is very important , demonstrating that as technology evolves there are new potential pitfalls for us in radiotherapy and we can’t afford to be complacent.

These ACDS audits have led to the discovery of systematic discrepancies in treatment planning algorithms, e.g., the AAA algorithm during upgrades of Eclipse versions. This and other TPS discrepancies have been widely shared within the international radiotherapy community as ACDS has many peer-reviewed publications and an active website.

As with the Belgian experience , it appears that high-level government support to ensure good-quality radiotherapy was crucial to the development of these audits. The audits are reviewed so that what is measured adapts to technological advances, and the focus is on facilitation and collegiality rather than regulation .

Christian Ronn Hansen from Denmark presented on “Contouring Audits”

This presentation showed how the Danish Head and Neck Cancer Study Group (DAHANCA) have worked to reduce interobserver variation in head and neck organ at risk (OAR) contouring and how this improved consistency has allowed them to implement artificial intelligence(AI) based autocontouring of OARs. Interobserver variation in contouring of both targets and OARs is well described as a major issue in radiotherapy. The negative impact of major deviations in contouring targets on clinical outcome was first described by Peters et al in TROG 0202.

DAHANCA have held several workshops to improve the quality and consistency of contouring in head and neck cancer. They knew that AI based auto-contouring was imminent and that interobserver variation in OAR contouring would be a barrier to its implementation given that high-quality consistent data is required to train AI algorithms.

Prior to the workshops, participants met to discuss how OAR contouring should be done; at the workshops, they each contoured multiple cases. This data was collected, and interobserver variations were analysed using concordance metrics such as the Dice similarity coefficient(DSC), the Hausdorff distance(HD) and the mean surface distance(MSD). The greatest interobserver variation was noted in contouring of parotids, constrictor muscles and buccal mucosa, and this finding has led to educational efforts to clarify contouring guidelines for these structures. From this data, DAHANCA has created a harmonised set of OAR contours for head-and-neck radiotherapy; these can be used for educational purposes and a web-based resource is openly available: https://github.com/Oncology-OUH/OAR_contouring_variation_in_HN. This data has also been used to train autosegmentation algorithm used to contour OARs. It was seen from the DAHANCA 35 trial, that AI-based auto-contouring of OARs was more consistent that of humans,in addition to having the advantage of saving time and effort. This workshop-based approach to measuring variation in OAR contouring and reducing it has been replicated across Denmark for multiple tumour sites.

Additionally a national radiotherapy dicom data-sharing database, DcmCollab which contains anonymised dicom data on >140,000 patients has been used to improve contouring. National contouring audits have been conducted which have led to a reduction in interobserver OAR contouring variation. DcmCollab has been used to help to implement AI based autocontouring nationally in head and neck and other tumour sites as it contains vast amounts of data that can be used to train AI algorithms.

As with the previous audits, a huge amount of effort has gone into this quality improvement initiative. It seems to have been driven by DAHANCA rather than the government. For me, it is inspiring to see how Danish colleagues have addressed this contouring variation with such a determined national approach We are all aware of the issue of interobserver variation in contouring but few of us actually address it effectively.

Joanna Dodkins from the UK presented on “National Clinical Audits Driving Improvements in Cancer Outcomes”

The presenter described the National Cancer Audit Collaborating Centre (NATCAN), which co-ordinates audits of cancer outcomes in the NHS in England and Wales. Unlike the audits described above, these audits cover diagnosis and other treatments in addition to radiotherapy. NATCAN uses routinely collected data from national linked datasets including the National Cancer Registry the National Radiotherapy Dataset (RTDS), Hospital Episode Statistics (HES) and Patient-Reported Outcome Measures(PROMS) for these audits.

The focus of this presentation was on one of the national prostate cancer audits, which had measured radiotherapy-induced adverse effects in patients treated for prostate cancer across the 50 centres in England and Wales. The audit data was obtained from the RTDS, HES and PROMs. They looked at meaningful levels of toxicity (grade 2 and above) that affected the quality of patients’ lives. They adjusted outcomes to account for competing factors that affected toxicity rates, i.e., age, sex, comorbidities and socioeconomic status.

National differences were observed in rates of significant toxicity. Both negative outliers and positive exemplars were identified. Key drivers of these differences were contouring practices (particularly margin size), image guidance protocols, dosimetry and OAR constraints in addition to bowel and bladder preparation protocols. All centres received feedback, and a central governance structure ensured that performance was managed promptly and effectively. Structured reviews were conducted of centres that were negative outliers and recommendations provided. These centres were required to develop action plans and were supported to draw up quality improvement initiatives. Positive exemplars were used to provide advice.

These audits were seen to be effective: centres that were identified as negative outliers improved over time. An example was given of one centre where G2+ toxicity fell from 17% to 7%. This audit-driven improvement was also seen in a systematic review of quality improvement interventions in radiotherapy which this presenter authored which demonstrated evidence of reduced waiting times, improved guideline adherence, reduced contouring variation, and better toxicity management following audits.

Again we see the power of national audits, in this case reducing the rate of significant complications following prostate radiotherapy in the NHS. High-level support from the Department of Health to set up NATCAN and allocate funding for audits were key to the success of these audits. It would be interesting to know the oncological outcomes of these patients , which are likely to be published in time. Do the centres who have fewer complications also have better disease outcomes?

Reflections and Key Takeaways

One of the most striking aspects of these audit programmes is that they were led nationally by professionals who recognised issues in their services and were committed to improving them. The success of the initiatives reflects not only strong governmental support but also the dedication of individuals who invested considerable time and effort in designing robust audit methodologies and ensuring that findings were translated into meaningful service improvements.

The closing comment from Dr Dodkins particularly resonated with me: “The future of oncology is not just innovation in treatment, but innovation in measuring and improving care delivery.” As radiotherapy professionals, there can be a tendency to focus on exciting technological advances in treatment, which may deliver only marginal improvements in outcomes, while we overlook the importance of developing the tools needed to measure and improve outcomes.

Aileen Flavin
Radiation Oncologist
Medical Imaging and Radiation Therapy, University College Cork
Ireland

Resources

Aggarwal, A., Nossiter, J., Parry, M., Sujenthiran, A., Zietman, A., Clarke, N. et al 2021. Public reporting of outcomes in radiation oncology: the National Prostate Cancer Audit. The Lancet Oncology 2021, 22(5), e207-e215.

B-QUATRO–comprehensive audits of radiotherapy practices: a tool for quality improvement adapted to the Belgian context. Collège de Radiothérapie, 2024.

Dodkins J, Zachou G, Rashid A, van der Meulen J, Nossiter J, Tree A et al. Improving performance in radiation oncology: an international systematic review of quality improvement interventions. Radiother Oncol. 2025 206.110798

Dunn L, Lehmann J, Lye J, Kenny J, Kron T, Alves A, et al. National dosimetric audit network finds discrepancies in AAA lung inhomogeneity corrections. Physica Medica. 2015. 31(5):435-41.

Hansen CR, Samsøe E, Smulders B, Sand HMB, Holm AIS, Furre T et al. Radiotherapy quality assurance of patients with squamous cell carcinoma of the head and neck included in the DAHANCA 19 randomised phase III trial. Radiother Oncol. 2025. 211:111096

https://github.com/Oncology-OUH/OAR_contouring_variation_in_HN

Krogh, S.L., Brink, C., Lorenzen, E.L., Samsøe, E., Vogelius, I.R., Zukauskaite, R, et al. A national repository of complete radiotherapy plans: design, results, and experiences. Acta Oncol. 2023. 62(10), 1161-1168.

Lorenzen, E.L., Kallehauge, J.F., Byskov, C.S., Dahlrot, R.H., Haslund, C.A., Guldberg, T.L et al. A national study on the inter-observer variability in the delineation of organs at risk in the brain. Acta Oncologica. 2021, 60(11), 1548-1554

Nielsen CP, Lorenzen EL, Jensen K, Eriksen JG, Johansen J, Gyldenkerne et al. Interobserver variation in organs at risk contouring in head and neck cancer according to the DAHANCA guidelines. Radiother Oncol. 2024. 197:110337.

Nielsen CP, Lorenzen EL, Jensen K, Sarup N, Brink C, Smulders B, et al. Consistency in contouring of organs at risk by artificial intelligence vs oncologists in head and neck cancer patients. Acta Oncol. 2023. 62(11):1418-1425.

Nielsen CP, Samsøe E, Offersen BV, Lorenzen EL, Persson G, Mortensen HR et al. Recommendations for radiotherapy quality assurance in clinical trials. Radiother Oncol. 2025 Aug;209, 110950.

Peters LJ, O'Sullivan B, Giralt J, Fitzgerald TJ, Trotti A, Bernier J, et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol. 2010. 28(18):2996-3001.

Roques TW. Patient selection and radiotherapy volume definition: can we improve the weakest links in the treatment chain? Clinical Oncology 2014.26(6) 353-355.