A model that denies a loan without explanation is hard to trust and harder to challenge. Explainable AI develops methods that make model decisions understandable to humans, whether through simpler surrogate models, feature attributions, or visual explanations.

Approaches split into two camps. Local methods explain individual predictions, such as SHAP values or LIME. Global methods describe overall model behaviour, such as feature importance rankings or decision rules extracted from complex models.

Common techniques

• SHAP values for feature attribution
• LIME for local surrogate models
• Saliency maps for image models
• Attention visualization for text
• Counterfactual explanations

Explanations can mislead. A saliency map may highlight pixels that correlate with a prediction without revealing the model's actual reasoning. Regulators increasingly require explainability in high-stakes domains, but the field has no universal standard for what counts as a good explanation.