Vienna, Austria

ESTRO 2023

Session Item

May 15
11:40 - 12:40
Plenary Hall
Highlights of Proffered Papers - Latest clinical trials
Anna Kirby, United Kingdom;
Matthias Guckenberger, Switzerland
Proffered Papers
11:40 - 11:50
Benefit of AI-assisted organ-at-risk contouring in head-and-neck cancer: A global randomized study
Mathis Ersted Rasmussen, Denmark


Benefit of AI-assisted organ-at-risk contouring in head-and-neck cancer: A global randomized study

Mathis Ersted Rasmussen1, Kamal Akbarov2, Egor Titowich2, Katherine Wakeham2, Jasper Albertus Nijkamp3, Wouter Van Elmpt4, A.F.M. Kamal Uddin, Ahmed Mohamed5, Ben Prajogi, Brohet Kartika Erida5, Catherine Nyongesa, Darejan Lomidze5, Gisupnikha Prasiko, Gustavo Ferraris5, Humera Mahmood, Igor Stojkovski5, Isa Isayev, Issa Mohamad5, Leivon Shirley, Lotfi Kochbati5, Ludmila Eftodiev, Maksim Piatkevich5, Maria Matilde Bonilla Jara, Orges Spahiu5, Rakhat Aralbayev, Raushan Zakirova6, Sandya Subramaniam, Solomon Kibudde, Uranchimeg Tsegmed5, Stine Sofia Korreman7, Jesper Grau Eriksen7

1Principal Investigator, The ELAISA Consortium, Aarhus, Denmark; 2International Atomic Energy Agency, The ELAISA Consortium, Vienna, Austria; 3Consultant, The ELAISA Consortium, Aarhus, Denmark; 4Consultant, The ELAISA Consortium, Maastricht, The Netherlands; 5Chief Scientific Investigators, The ELAISA Consortium, Vienna, Austria; 6Chief Scientific Investigators, The ELAISA Consortium , Vienna, Austria; 7Principal Investigator and shared last author, The ELAISA Consortium, Aarhus, Denmark

Show Affiliations
Purpose or Objective

Global roll-out of artificial intelligence-assisted (AI-assisted) contouring has immense potential for the access and quality of care in radiotherapy. However, the current evidence is primarily from high-income countries.

The purpose of this study was to compare inter-observer variation (IOV) between radiation oncologists (ROs) in low- and middle-income countries (LMICs) using either AI-assisted or manual contouring of organs-at-risk in head-and-neck cancer.

Material and Methods

97 ROs from 23 institutions in 22 nations across 5 continents were invited to the study. The institutions were randomized to either manual contouring (control) or AI-assisted contouring (intervention) of 8 common organs-at-risk of head-neck cancer. Randomization was balanced on (1) the annual number of head-neck cases treated at institutions and (2) the availability of auto-contouring at institutions.

Four head-neck cases were randomly assigned to ROs institution-wise resulting in 2-3 institutions (8-15 ROs) per case in each group. Deep learning-based auto-contours were made with MVision AI Oy, Helsinki, Finland. Contouring was performed online in EduCase™ (RadOnc eLearning Center, Inc).

ROs were informed about the contouring guidelines used in the study and that their task was to “generate clinically acceptable contours”. ROs were also asked to complete in a short survey on their professional backgrounds.

IOV was quantified as medians of Dice coefficients and Hausdorff distances 95th percentile (HD95) between contours of the ROs and a median contour within groups. Median contours were made by summing structures and thresholding where the median number of ROs or more had included a given voxel. High Dice and low HD95 indicate low IOV.

Confidence intervals were estimated with bootstrap and groups were compared with the Mann-Whitney U test.


89 ROs handed in the contours and 74 completed the survey. The four cases were contoured respectively by 18 (7/11), 24 (12/12), 23 (11/12) and 24 (12/12) ROs (Manual/AI-assisted) (See Table 1 for details).

Table 1

AI-assisted contouring as compared to manual contouring significantly reduced IOV across all organs-at-risk [Figure 1]. Below is an overview of the differences in medians for Dice and HD95 with manual contouring as reference value.

Absolute (relative) differences.
Control as reference

Dice (%)

HD95 [mm] (%)


0.07 (8.0)

-0.82 (-29.2)


0.11 (13.5)

-0.93 (-39.7)


0.10 (11.5)

-1.47 (-42.5)


0.06 (6.2)

-0.60 (-26.7)


0.08 (9.6)

-0.53 (-23.9)


0.09 (10.2)

-5.60 (-59.2)


0.16 (21.7)

-1.41 (-47.1)


0.06 (7.0)

-0.66 (-35.9)

Figure 1:


AI-assisted contouring significantly reduced IOV across multiple institutions located in LMICs globally. The results add a significant piece of evidence favoring further adoption of AI-assisted contouring worldwide. To the best of our knowledge, this is the largest study ever performed on inter-observer variation and auto-contouring, and it is the first time LMICs has been the primary focus.