Unveiling AI Decisions: A Unified Approach Through Explainable Learning for Optimizing Counterfactual Explanations

Understanding the decisions made by AI-based image classification systems is crucial for high-stakes applications such as medical diagnostics and automated surveillance. In these contexts, interpretable explanations and targeted modifications of classification decisions can significantly improve end-user trust and acceptance of AI. This paper proposes an integrated approach that combines the slot attention-based classifier, SCOUTER, with the counterfactual explanation algorithm, SEDC-T, to provide detailed explanations and controlled modifications of image classification decisions. Additionally, this work evaluates a variant of SEDC-T, which operates by retaining a segment and eliminating all other segments, offering a different perspective on counterfactual explanations. The authors tested the proposed methodology using the “G-Litter” dataset, comprising 2054 images of marine litter, and a reduced version of the CUB-200 dataset, which includes 200 bird species. Experimental results demonstrate that the integrated approach not only reduces execution time but also enhances the detail and relevance of explanations compared to using SEDC-T alone. This unified approach significantly improves the transparency and reliability of AI-based image classification systems, with potential applications in environmental monitoring and other high-stakes domains.