Explainable AI, Feature Attribution

[EXAMPTOPIC] Explainable AI, Feature Attribution methods 를 정리합니다.

Explainable artificial intelligence (XAI)

XAI - Tools and frameworks to understand and interpret your machine learning models

• Understand AI output, models' behavior and build trust
• Debug and improve model performance
• AI Explanations in AutoML Tables, Vertex AI Predictions, and Notebooks
• Generate Feature attributions for model predictions : AutoML Tables , Vertex AI
• Visually investigate model behavior : What-If Tool to investigate model behavior at a glance
• BigQuery Explainable AI

Meaning & Scope of Feature Attribution *

Feature attribution

XAI에서 how each feature in a row of data contributed to the predicted result 각 피쳐가 예측 결과에 기여하는 방식(정도) 를 정의한 것을 의미한다.

• Local vs. Global Explainability : Explainability 설명가능성은 다음 2가지 유형을 포함한다.

  1. Local explainability : Each attribution only shows how much the feature affected the prediction for that particular example. A single attribution might not reflect the overall behavior of the model.
  2. Global explainability : To understand approximate model behavior on an entire dataset, aggregate attributions over the entire dataset.

• Feature attribution 정보 활용해서 하기 내용들을 확인해볼 수 있다.

  • Model is behaving as expected 모델이 예상한대로 작동하는 지
  • Recognize biases in your models 모델의 편향
  • Inform ways to improve your model and training data 모델/훈련 데이터 개선 방법

주의해야할 점

1. not a relationship between that feature and the target but the presence or absence of a strong attribution to a certain feature
   • Attribution 기여분석은 전적으로 학습 데이터, 모델에 의존하고, 그 안에 존재하는 패턴을 찾을 수는 있지만, 근본적인 관계를 감지하지는 못한다. 즉, Feature Attribution은 특정 피쳐가 강한 기여도가 존재하는 지/모델이 특정 피쳐를 모델이 예측에 사용하는 지 여부를 나타낼 뿐, 해당 피쳐와 타겟간의 관계 여부를 나타내는 것이 아니다.
2. Attributions alone cannot tell if your model is fair, unbiased, or of sound quality. Carefully evaluate your training dataset, procedure, and evaluation metrics in addition to the attributions.

3 Feature Attribution methods

AI Explanations 기능은 특성 기여 분석 방법 3가지 sampled Shapley, integrated gradients, XRAI 를 제공한다.

Methods	Integrated gradients	XRAI (eXplanation with Ranked Area Integrals)	Sampled Shapley
Basic explanation	A gradients-based method to efficiently compute feature attributions with the same axiomatic properties as the Shapley value.	Based on the integrated gradients method, XRAI assesses overlapping regions of the image to determine which regions of the image contribute the most to a given class prediction. (highlights relevant regions of the image rather than pixels.)	Assigns credit for the outcome to each feature, and considers different permutations of the features. This method provides a sampling approximation of exact Shapley values.
Recommended model types	Differentiable models, such as neural networks. Recommended especially for models with large feature spaces. Recommended for low-contrast images, such as X-rays.	Models that accept image inputs. Recommended especially for natural images, which are any real-world scenes that contain multiple objects.	Non-differentiable models, such as ensembles of trees and neural networks
USE CASES	Classification and regression on tabular data, Classification on image data	Classification on image data	Classification and regression on tabular data.

• Integrated Gradients : Gradients-based method to compute feature importance.

  • Use in classification and regression of tabular data or image data. (Image, tabular 데이터 모두 지원하는 특성기여분석 방법은 Integrated Gradients이다.)

• XRAI : Explain feature importance for images (saliency map) - highlights relevant regions.

  • Use in image classification : Pixel-level attribution, Oversegmentation, Region selection
    

• Sampled Shapley : Explain feature importance (sampling approximation of Shapley values).

  • Use in classification and regression of tabular data.

How to Improve Attribution Methods

AI Explanations - attribution methods approximate the Shapley value. 기여분석은 Shapley 계산 비용이 매우 높기 때문에, Shapley값의 근사치를 계산하고, 근사치 오차도 결과로 출력된다. 근사치 오차가 0.05를 초과하는 경우, 이 근사치의 precision을 높이기 위해

1. Increasing the number of integral steps for the integrated gradients or XRAI methods.
2. Increasing the number of integral paths for the sampled Shapley method.

Resources

AI Explainability Whitepaper

• Why you need to explain machine learning models | Google Cloud Blog
• Visualizing explanations for image data - Integrated gradients 실습 코드/결과

Q 1. (Official Sample Questions)

You work for a textile manufacturer and have been asked to build a model to detect and classify fabric defects. You trained a machine learning model with high recall based on high resolution images taken at the end of the production line. You want quality control inspectors to gain trust in your model. Which technique should you use to understand the rationale of your classifier?

• A. Use K-fold cross validation to understand how the model performs on different test datasets.
• B. Use the Integrated Gradients method to efficiently compute feature attributions for each predicted image.
• C. Use PCA (Principal Component Analysis) to reduce the original feature set to a smaller set of easily understood features.
• D. Use k-means clustering to group similar images together, and calculate the Davies-Bouldin index to evaluate the separation between clusters.

Scenario

• to **_understand the rationale of model_**, to gain _trust in your model_ ; XAI(Explainable AI) 의 목적
• Image classification

2 attribution methods to support "image" data : Integrated gradients and XRAI

Integrated gradients

ai-explanations/overview#ig

• pixel-based attribution method
• highlights important areas in the image regardless of contrast 이미지의 중요한 영역 강조
• ⭕ Work well best on non-natural images : X-rays
• Granular output : difficult to assess the relative importance of areas. 세밀한 출력으로 각 영역의 상대적 중요성 평가 어려울 수 있다.
  • The default output highlights areas in the image that have high positive attributions by drawing outlines, but these outlines are not ranked and may span across objects.
• USE CASES : Classification and regression on tabular data, Classification on image data

XRAI (eXplanation with Ranked Area Integrals)

ai-explanations/overview#xrai

• Region-based attributions
• Smoother, more human-readable heatmap of regions that are most salient for a given image classification.
• ⭕ Work well best on natural, higher-contrast images containing multiple objects
• ❌ Not work well on
  • Low-contrast images that are all one shade : X-rays
  • Very tall or very wide images : panoramas
  • Very large images (may slow down overall runtime.)
• USE CASES : Classification on image data

---

Source : Professional Machine Learning Engineer Sample Questions