A Novel Tool for Evaluating Patient Safety: Patient Safety in German Radiation Oncology Study (PASAGERO)
Principal investigator: Dr. Andrea Baehr, with Professor Dr. Isabelle Scholl
The need for patient safety indicators in radiation oncology
There is currently no structured instrument by which to evaluate patient safety in radiation therapy facilities, despite its clear relevance. The collection of adverse-events data is a powerful way to identify safety gaps, but we must assume that many cases are not reported. Certain process areas may be inadequately covered.
One approach to assessing patient safety involves the use of patient safety indicators (PSIs), which define specific measures, thresholds, or circumstances that are associated with safe care. In contrast to the quality indicators that are widely implemented in oncology, radiation therapy-specific PSIs have not yet been defined in Europe. Current PSIs that are derived from other fields, such as surgery, are unsuitable for evaluation of radiotherapy processes for several reasons. For instance, they do not specifically account for outpatient care, even though approximately 80% of radiotherapy patients are treated on an outpatient basis. Some indicator sets for radiation oncology have been defined for specific conditions, such as cervical cancer [1], or for radiation therapy processes, such as the safety profile assessment by the American Association of Physicists in Medicine [2]. In these examples, indicators are presented in a structured manner, but they require supplementation to ensure holistic care. Critical aspects such as continuity of care, therapeutic drug assessment, and continuous staff education are underrepresented.
Currently, the selection and evaluation of safety-related measures is the responsibility of individual facilities. Due to the absence of centralised standards, institutions must invest considerable resources in the development of individual solutions. Furthermore, this non-standardised approach carries the risk that essential elements for ensuring patient safety are insufficiently addressed within established processes.
The PASAGERO approach
To bridge this gap strategically with a high degree of evidence, we launched the PASAGERO project, a multi-institutional study spanning three years. As part of this initiative, we developed a specialised questionnaire through which we could evaluate safety specifically within the field of radiation therapy. This questionnaire is freely available to all institutions. Although initially developed and validated in German, the instrument is available in both German and English under the name patient safety in radiation oncology (PASARO) [3].
Our interdisciplinary research group at the University Medical Centre Hamburg-Eppendorf, Germany, comprises radiation oncologists, medical physicists and psycho-oncologists. Throughout various study phases, the project involved radiation oncology staff and patients from more than 20 centres across Germany. We consider that a robust assessment instrument for patient safety must be developed by experts across all professional groups in order to guarantee that all safety dimensions are comprehensively evaluated. This instrument is based on specific PSIs that were developed in conjunction with the study, with a primary focus on evaluating institutional processes and structures. The final questionnaire is available in a digital format. As a self-assessment tool, it identifies specific areas for improvement and provides institutions with actionable insights to enhance their safety protocols.
Our results
Over a three-year period, this study established content-validated PSIs using a mixed-methods approach. A comprehensive literature review served as the foundation, through which we identified approximately 160 indicators from previously proposed quality and safety sets, incident reports, expert opinions, and clinical guidelines [1].
To refine this set, focus-group discussions and interviews were conducted with patients and professionals. This process resulted in a preliminary framework that comprised over 200 PSIs across four core categories: quality and risk management, human resources, institutional culture, and patient-specific processes.
These indicators were further evaluated and refined through a two-phase Delphi study, in which experts nationwide were invited to rate each PSI based on its relevance to patient safety. Participants were also given the opportunity to propose new indicators or to suggest wording modifications. In total, more than 200 individuals participated across the various developmental stages. Consensus was reached for over 150 PSIs, all of which were rated as highly relevant. Finally, all PSIs underwent cognitive interviews to ensure comprehensibility. Figure 2 provides a comprehensive overview of the Delphi study results.
All 158 PSIs were incorporated into the PASARO questionnaire. This document enables professionals in radiation oncology to assess the implementation of PSIs in their departments using a six-point Likert scale that ranges from 1 (completely disagree) to 6 (completely agree).
The instrument was evaluated for its psychometric properties within a large, multicentre cohort, assessing metrics such as test-retest reliability and interrater reliability. Additionally, the digital version of the final questionnaire, comprising 155 PSIs and hosted within a web-based environment, underwent usability testing by partner sites, with feedback gathered through direct interviews.
A closing workshop was held, in which stakeholders discussed the future implications of the study findings. The resulting improvements to the web app were integrated into the final user interface to ensure optimal usability.
The PASAGERO Webapp is now freely available through the website of the German Society for Radiation Oncology (DEGRO) at https://pasagero.degro.org/en. It contains the self-assessment tool as well as a comprehensive set of PSIs and further details on the project.
The project teams thank all participating centres and participants, as well as the patient advisory board, the University Hospital Hamburg-Eppendorf and the societies the German Society for Radiotherapy (DEGRO), the working group for radiation oncology in the German Cancer Society (ARO) and ESTRO, for their continuous support.
The study is funded by German Cancer Aid (Deutsche Krebshilfe), and our webapp is hosted through DEGRO.
Authors
Andrea Baehr, Department of Radiation Oncology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
Maximilian Grohmann, Department of Radiation Oncology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany, and Department of Radiooncology, University of Leipzig Medical Centre, Leipzig, Germany
Isabelle Scholl and Eva Christalle, Department of Medical Psychology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
References
[1] Chargari C, Tanderup K, Planchamp F, Chiva L, Humphrey P, Sturdza A, Tan LT, van der Steen-Banasik E, Zapardiel I, Nout RA, Fotopoulou C. ESGO/ESTRO quality indicators for radiation therapy of cervical cancer. Radiother Oncol. 2023 Jun;183:109589. doi: 10.1016/j.radonc.2023.109589. PMID: 37268359.
[2] Dunscombe, Peter et al., Safety Profile Assessment: An online tool to gauge safety-critical performance in radiation oncology. Practical Radiation Oncology, Volume 5, Issue 2, 127 - 134.
[3] Baehr A, Christalle E, Grohmann M, Scholl I. Development and psychometric validation of a patient safety assessment tool in German Radiation Oncology: the PaSaGeRO Study protocol. BMJ Open. 2024 Aug 17;14(8):e086214. doi: 10.1136/bmjopen-2024-086214.