TY - GEN
T1 - XAI-CBIR
T2 - 16th IEEE International Symposium on Biomedical Imaging, ISBI 2019
AU - Chittajallu, Deepak Roy
AU - Dong, Bo
AU - Tunison, Paul
AU - Collins, Roddy
AU - Wells, Katerina
AU - Fleshman, James
AU - Sankaranarayanan, Ganesh
AU - Schwaitzberg, Steven
AU - Cavuoto, Lora
AU - Enquobahrie, Andinet
N1 - Publisher Copyright: © 2019 IEEE.
PY - 2019/4
Y1 - 2019/4
N2 - In this paper, we present a human-in-the-loop explainable AI (XAI) system for content based image retrieval (CBIR) of video frames similar to a query image from minimally invasive surgery (MIS) videos for surgical education. It extracts semantic descriptors from MIS video frames using a self-supervised deep learning model. It then employs an iterative query refinement strategy where in a binary classifier trained online based on relevance feedback from the user is used to iteratively refine the search results. Lastly, it uses an XAI technique to generate a saliency map that provides a visual explanation of why the system considers a retrieved image to be similar to the query image. We evaluated the proposed XAI-CBIR system on the public Cholec80 dataset containing 80 videos of minimally invasive cholecystectomy surgeries with encouraging results.
AB - In this paper, we present a human-in-the-loop explainable AI (XAI) system for content based image retrieval (CBIR) of video frames similar to a query image from minimally invasive surgery (MIS) videos for surgical education. It extracts semantic descriptors from MIS video frames using a self-supervised deep learning model. It then employs an iterative query refinement strategy where in a binary classifier trained online based on relevance feedback from the user is used to iteratively refine the search results. Lastly, it uses an XAI technique to generate a saliency map that provides a visual explanation of why the system considers a retrieved image to be similar to the query image. We evaluated the proposed XAI-CBIR system on the public Cholec80 dataset containing 80 videos of minimally invasive cholecystectomy surgeries with encouraging results.
KW - Content based video retrieval
KW - Deep learning
KW - Explainable AI
KW - Laparoscopy
KW - Minimally invasive surgery
KW - Self-supervised learning
KW - Surgical data science
UR - https://www.scopus.com/pages/publications/85073896747
U2 - 10.1109/ISBI.2019.8759428
DO - 10.1109/ISBI.2019.8759428
M3 - Conference contribution
T3 - Proceedings - International Symposium on Biomedical Imaging
SP - 66
EP - 69
BT - ISBI 2019 - 2019 IEEE International Symposium on Biomedical Imaging
PB - IEEE Computer Society
Y2 - 8 April 2019 through 11 April 2019
ER -