In a double blind evaluation of 60 digital dermatoscopic images by 4 "junior", 4 "senior" and 4 "expert" dermatologists (dermatoscopy training respectively less than 1 year, between 1 and 5 years, and more than 5 years), a significant inter-operator variability was observed in melanocytic lesion border identification (with a disagreement of the order of 10 - 20% of the area of the lesions). Expert dermatologists showed greater agreement among themselves than with senior and junior dermatologists, and a slight tendency towards "tighter" segmentations. The human inter-operator variability was then used to evaluate the segmentation accuracy of 4 algorithms, representative of the 3 fundamental state-of-the-art automated segmentation techniques and of a fourth, novel, technique. Our evaluation methodology addresses a number of crucial difficulties encountered in previous studies and may be of independent interest. 3 of the 4 algorithms showed considerably less agreement with expert dermatologists than even senior and junior dermatologists did (with a disagreement of the order of 30% of the area of the lesions); the remaining algorithm, however, showed agreement with expert dermatologists comparable to that of other expert dermatologists.

Variability in human and automatic localization of melanocytic lesions

SILLETTI, ALBERTO;PESERICO STECCHINI NEGRI DE SALVI, ENOCH;ZATTRA, EDOARDO;PESERICO STECCHINI NEGRI DE SALVI, ANDREA;BELLONI, FORTINA ANNA
2009

Abstract

In a double blind evaluation of 60 digital dermatoscopic images by 4 "junior", 4 "senior" and 4 "expert" dermatologists (dermatoscopy training respectively less than 1 year, between 1 and 5 years, and more than 5 years), a significant inter-operator variability was observed in melanocytic lesion border identification (with a disagreement of the order of 10 - 20% of the area of the lesions). Expert dermatologists showed greater agreement among themselves than with senior and junior dermatologists, and a slight tendency towards "tighter" segmentations. The human inter-operator variability was then used to evaluate the segmentation accuracy of 4 algorithms, representative of the 3 fundamental state-of-the-art automated segmentation techniques and of a fourth, novel, technique. Our evaluation methodology addresses a number of crucial difficulties encountered in previous studies and may be of independent interest. 3 of the 4 algorithms showed considerably less agreement with expert dermatologists than even senior and junior dermatologists did (with a disagreement of the order of 30% of the area of the lesions); the remaining algorithm, however, showed agreement with expert dermatologists comparable to that of other expert dermatologists.
2009
Proc. of IEEE EMBC 2009
9781424432967
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11577/2474144
Citazioni
  • ???jsp.display-item.citation.pmc??? 0
  • Scopus 8
  • ???jsp.display-item.citation.isi??? 3
social impact