Due to the ubiquity of AI systems in our society, awareness has been raised for the need of neural networks and their predictions to be transparent and explainable.
This has increased the interest in eXplainable AI (XAI) in the machine learning (ML) research community. In recent years, a plethora of XAI methods has been developed. Examples include Saliency, Guided Backprop, Excitation Backprop and Layer-wise Relevance Propagation (LRP). Given an input sample and a ML model, XAI methods compute relevance scores for all input values, e.g., pixels (or voxels) for image classification tasks, which can be presented to the user in the form of heatmaps.
Throughout this post, we use the XAI methods lists above to explain predictions for a VGG-16 model which was trained to classify samples from the ILSVRC2017 dataset.