Online

ESTRO 2020

Session Item

Saturday
November 28
10:30 - 11:30
Physics Stream 2
Proffered papers 6: Novel treatment planning strategies
1204
Proffered Papers
Physics
11:00 - 11:10
Inter-observer variability in quality scores of Pareto optimal plans
elisabetta cagni, Italy
OC-0105

Abstract

Inter-observer variability in quality scores of Pareto optimal plans
Authors: Andrea Botti.(Azienda USL-IRCCS di Reggio Emilia, Medical Physics Unit, Reggio Emilia, Italy), Elisabetta Cagni.(Azienda USL-IRCCS di Reggio Emilia, Medical Physics Unit, Reggio Emilia, Italy), Salvatore Cozzi.(Azienda USL-IRCCS di Reggio Emilia, Radiotherapy Unit, Reggio Emilia, Italy), Marco Galaverni.(Azienda USL-IRCCS di Reggio Emilia, Radiotherapy Unit, Reggio Emilia, Italy), Ben J.M. Heijmen.(Erasmus MC Cancer Institute, Radiation Oncology, Rotterdam, The Netherlands), Mauro Iori.(Azienda USL-IRCCS di Reggio Emilia, Medical Physics Unit, Reggio Emilia, Italy), Cinzia Iotti.(Azienda USL-IRCCS di Reggio Emilia, Radiotherapy Unit, Reggio Emilia, Italy), Matteo Orlandi.(Azienda USL-IRCCS di Reggio Emilia, Medical Physics Unit, Reggio Emilia, Italy), Ala Rosca.(Azienda USL-IRCCS di Reggio Emilia, Radiotherapy Unit, Reggio Emilia, Italy), Linda Rossi.(Erasmus MC Cancer Institute, Radiation Oncology, Rotterdam, The Netherlands), Roberto Sghedoni.(Azienda USL-IRCCS di Reggio Emilia, Medical Physics Unit, Reggio Emilia, Italy), Emiliano Spezi.(Cardiff University, School of Engineering, Cardiff, United Kingdom), Giorgia Timon.(Azienda USL-IRCCS di Reggio Emilia, Radiotherapy Unit, Reggio Emilia, Italy)
Show Affiliations
Purpose or Objective

A treatment plan that is not Pareto-optimal is never clinically optimal, explaining the current interest in Pareto-optimal plans. On the other hand, of all Pareto-optimal plans only few are clinically favourable or even acceptable. In clinical practice, medical physicists (MPs) perform a first plan quality assessments during planning and then radiation oncologists (ROs) perform it for plan approval. Here we investigate inter-observer variability in scoring of Pareto-optimal plans, including differences between MPs and ROs.     

Material and Methods

15 head and neck cancer patients, treated with a three-level SIB technique, were selected for the study. Three to five plans were evaluated per patient by 5 ROs and 4 MPs from the same department. Each observer independently evaluated all plans available for a patient in a single session, i.e. all 3-5 plans were available in the clinical TPS. Scoring was blinded, i.e. observers did not know how the plans were generated. Plan scores were given from 1-7 (1-2: unacceptable, 3-5: acceptable, if failure to improve quality in further planning, 6-7: acceptable, no further planning needed). For all patients, the plan database contained the clinically delivered plan (CLIN) and 2-4 Pareto-optimal plans, generated automatically with an in-house algorithm for automated a priori Multi-Criteria Optimization (MCO). Together with the clinical team, this algorithm was first configured to generate clinically favourable Pareto-optimal plans according to the department’s protocol and tradition. For each patient, the plan generated with that configuration (MCOa) was added to the database. Next, 20 sub-optimal configurations (x=b,c,d, …) were then performed and for each patient, the database was complemented with 1 or 3 of such MCOx plans.  



Results

Overall, for 60%, 27%, and 13% of the patients, MCOa, CLIN and MCOx plans where ranked as best plan. In 20%, 27% and 40% of the patients, all observers, all ROs and all MPs choose the same plan as the best for treatment. Table1 shows the scores for all investigated 65 plans and all 9 observers, including mean scores. Mean scores of ROs and MPs were 4.7±1.3 and 5.5±1.0 for MCOa plans, 3.9±1.3 and 4.6±1.4 for CLIN plans and 3.5±1.4 and 3.8±1.4 for MCOx plans. Differences between the optimal autoplanning configuration MCOa plans and corresponding CLIN plans and between MCOa and MCOx plans were statistically significant (p<0.05). Interobserver variations (1 SD) in absolute scores were 1.1, 1.0, 0.9 (All, ROs, MPs). Correlation between ROs’ and MPs’ scores was moderate with R2=0.67 (Fig. 1). 

Conclusion

On average, MCOa plans resulted better than CLIN plans and MCOx plans. However, large inter-observer differences in plan scores were observed with moderate correlation between scores of medical physicists and radiation oncologists. This can result in suboptimal plan quality in the usual workflow of physicists who plan for clinicians, and in selecting a Pareto-optimal plan using Pareto Navigation.