Ifme: Influence function based model explanation for black box decision systems

Benbo Zha, Hong Shen

Research output: Contribution to journalConference articlepeer-review


Due to the high precision and the huge prediction needs, machine learning models based decision systems has been widely adopted in all works of life. They were usually constructed as a black box based on sophisticated, opaque learning models. The lack of human understandable explanation to the inner logic of these models and the reason behind the predictions from such systems causes a serious trust issue. Interpretable Machine Learning methods can be used to relieve this problem by providing an explanation for the models or predictions. In this work, we focus on the model explanation problem, which study how to explain the black box prediction model globally through human understandable explanation. We propose the Influence Function based Model Explanation (IFME) method to provide interpretable model explanation based on key training points selected through influence function. First, our method introduces a novel local prediction interpreter, which also utilizes the key training points for local prediction. Then it finds the key training points to the learning models via influence function globally. Finally, we provide the influence function based model agnostic explanation to the model used. We also show the efficiency of our method through both theoretical analysis and simulated experiments.

Original languageEnglish
Pages (from-to)299-310
Number of pages12
JournalCommunications in Computer and Information Science
Publication statusPublished - 2020
Externally publishedYes
Event10th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2019 - Guangzhou, China
Duration: 12 Dec 201914 Dec 2019


  • Explaining the black box
  • Influence function
  • Interpretable Machine Learning
  • Model explanatory


Dive into the research topics of 'Ifme: Influence function based model explanation for black box decision systems'. Together they form a unique fingerprint.

Cite this