Deep morphological recognition of kidney stones using intra-operative endoscopic digital videos

Vincent Estrade (1) , Michel Daudon (2) , Emmanuel Richard (3) , Jean-Christophe Bernhard (1) , Franck Bladou (1) , Gregoire Robert (1) , Laurent Facq (4) , Baudouin Denis de Senneville (4, 5)

1 CHU de Bordeaux Pellegrin [Bordeaux]
2 CoRaKID - Maladies rénales fréquentes et rares : des mécanismes moléculaires à la médecine personnalisée
3 Biothérapies des maladies génétiques et cancers
4 IMB - Institut de Mathématiques de Bordeaux
5 MONC - Modélisation Mathématique pour l'Oncologie

Abstract Objective. To assess the performance and added value of processing complete digital endoscopic video sequences for the automatic recognition of stone morphological features during a standard-of-care intra-operative session. Approach. A computer-aided video classifier was developed to predict in-situ the morphology of stone using an intra-operative digital endoscopic video acquired in a clinical setting. Using dedicated artificial intelligence (AI) networks, the proposed pipeline selects adequate frames in steady sequences of the video, ensures the presence of (potentially fragmented) stones and predicts the stone morphologies on a frame-by-frame basis. The automatic endoscopic stone recognition (A-ESR) is subsequently carried out by mixing all collected morphological observations. Main results. The proposed technique was evaluated on pure (i.e. include one morphology) and mixed (i.e. include at least two morphologies) stones involving ‘Ia/Calcium Oxalate Monohydrate’ (COM), ‘IIb/Calcium Oxalate Dihydrate’ (COD) and ‘IIIb/Uric Acid’ (UA) morphologies. The gold standard ESR was provided by a trained endo-urologist and confirmed by microscopy and infra-red spectroscopy. For the AI-training, 585 static images were collected (349 and 236 observations of stone surface and section, respectively) and used. Using the proposed video classifier, 71 digital endoscopic videos were analyzed: 50 exhibited only one morphological type and 21 displayed two. Taken together, both pure and mixed stone types yielded a mean diagnostic performances as follows: balanced accuracy = [88 ± 6] (min = 81)%, sensitivity = [80 ± 13] (min = 69)%, specificity = [95 ± 2] (min = 92)%, precision = [78 ± 12] (min = 62)% and F1-score = [78 ± 7] (min = 69)%. Significance. These results demonstrate that AI applied on digital endoscopic video sequences is a promising tool for collecting morphological information during the time-course of the stone fragmentation process without resorting to any human intervention for stone delineation or the selection of adequate steady frames.

Mots clés: Morpho-constitutional analysis of urinary stones; endoscopic diagnosis; automatic recognition; artificial intelligence; deep learning; aetiological lithiasis

Domaines: Traitement du signal et de l'image [eess.SP]

Vincent Estrade, Michel Daudon, Emmanuel Richard, Jean-Christophe Bernhard, Franck Bladou, et al.. Deep morphological recognition of kidney stones using intra-operative endoscopic digital videos. Physics in Medicine and Biology, 2022, 67 (16), pp.165006. ⟨10.1088/1361-6560/ac8592⟩. ⟨hal-03772477⟩