Session Item

A treatment plan that is not Pareto-optimal is never clinically optimal, explaining the current interest in Pareto-optimal plans. On the other hand, of all Pareto-optimal plans only few are clinically favourable or even acceptable. In clinical practice, medical physicists (MPs) perform a first plan quality assessments during planning and then radiation oncologists (ROs) perform it for plan approval. Here we investigate inter-observer variability in scoring of Pareto-optimal plans, including differences between MPs and ROs.     

15 head and neck cancer patients, treated with a three-level SIB technique, were selected for the study. Three to five plans were evaluated per patient by 5 ROs and 4 MPs from the same department. Each observer independently evaluated all plans available for a patient in a single session, i.e. all 3-5 plans were available in the clinical TPS. Scoring was blinded, i.e. observers did not know how the plans were generated. Plan scores were given from 1-7 (1-2: unacceptable, 3-5: acceptable, if failure to improve quality in further planning, 6-7: acceptable, no further planning needed). For all patients, the plan database contained the clinically delivered plan (CLIN) and 2-4 Pareto-optimal plans, generated automatically with an in-house algorithm for automated a priori Multi-Criteria Optimization (MCO). Together with the clinical team, this algorithm was first configured to generate clinically favourable Pareto-optimal plans according to the department’s protocol and tradition. For each patient, the plan generated with that configuration (MCOa) was added to the database. Next, 20 sub-optimal configurations (x=b,c,d, …) were then performed and for each patient, the database was complemented with 1 or 3 of such MCOx plans.  

Overall, for 60%, 27%, and 13% of the patients, MCOa, CLIN and MCOx plans where ranked as best plan. In 20%, 27% and 40% of the patients, all observers, all ROs and all MPs choose the same plan as the best for treatment. Table1 shows the scores for all investigated 65 plans and all 9 observers, including mean scores. Mean scores of ROs and MPs were 4.7±1.3 and 5.5±1.0 for MCOa plans, 3.9±1.3 and 4.6±1.4 for CLIN plans and 3.5±1.4 and 3.8±1.4 for MCOx plans. Differences between the optimal autoplanning configuration MCOa plans and corresponding CLIN plans and between MCOa and MCOx plans were statistically significant (p<0.05). Interobserver variations (1 SD) in absolute scores were 1.1, 1.0, 0.9 (All, ROs, MPs). Correlation between ROs’ and MPs’ scores was moderate with R²=0.67 (Fig. 1). 

On average, MCOa plans resulted better than CLIN plans and MCOx plans. However, large inter-observer differences in plan scores were observed with moderate correlation between scores of medical physicists and radiation oncologists. This can result in suboptimal plan quality in the usual workflow of physicists who plan for clinicians, and in selecting a Pareto-optimal plan using Pareto Navigation.