On Predictive Explanation of Data Anomalies - ETIS, équipe MIDI Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2022

On Predictive Explanation of Data Anomalies

Nikolaos Myrtakis
  • Fonction : Auteur
  • PersonId : 1130317
Ioannis Tsamardinos
  • Fonction : Auteur

Résumé

Numerous algorithms have been proposed for detecting anomalies (outliers, novelties) in an unsupervised manner. Unfortunately, it is not trivial, in general, to understand why a given sample (record) is labelled as an anomaly and thus diagnose its root causes. We propose the following reduced-dimensionality, surrogate model approach to explain detector decisions: approximate the detection model with another one that employs only a small subset of features. Subsequently, samples can be visualized in this low-dimensionality space for human understanding. To this end, we develop PROTEUS, an AutoML pipeline to produce the surrogate model, specifically designed for feature selection on imbalanced datasets. The PROTEUS surrogate model can not only explain the training data, but also the out-of-sample (unseen) data. In other words, PROTEUS produces predictive explanations by approximating the decision surface of an unsupervised detector. PROTEUS is designed to return an accurate estimate of out-of-sample predictive performance to serve as a metric of the quality of the approximation. Computational experiments confirm the efficacy of PROTEUS to produce predictive explanations for different families of detectors and to reliably estimate their predictive performance in unseen data. Unlike several ad-hoc feature importance methods, PROTEUS is robust to high-dimensional data.
Fichier principal
Vignette du fichier
On Predictive Explanation of Data Anomalies.pdf (2.49 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03608625 , version 1 (15-03-2022)

Identifiants

  • HAL Id : hal-03608625 , version 1

Citer

Nikolaos Myrtakis, Ioannis Tsamardinos, Vassilis Christophides. On Predictive Explanation of Data Anomalies. 2022. ⟨hal-03608625⟩
25 Consultations
94 Téléchargements

Partager

Gmail Facebook X LinkedIn More