A Visual Introduction to Shapley Values

UCLouvain Applied Statistics Workshop

\[ \newcommand{\bs}[1]{\mathbf{#1}} \newcommand{\reals}{\mathbb{R}} \newcommand{\widebar}[1]{\overline{#1}} \newcommand{\E}{\mathbb{E}} \newcommand{\Earg}[1]{\mathbb{E}\left[{#1}\right]} \newcommand{\Esubarg}[2]{\mathbb{E}_{#1}\left[{#2}\right]} \]

Readings: 1, 2, 3, Code

Setup

Goal. Given a model \(f\) and a sample \(x \in \reals^{D}\), return a local feature attribution \(\varphi_d\left(x\right)\) that quantifies the contribution of feature \(d\) to the prediction \(f\left(x\right)\).

Requirements.

• Local feature attributions. Unlike global variable importances, \(\varphi_d\left(x\right)\) are specific to sample \(x\). This matters a stakeholder cares specifically about particular prediction \(f(x)\) and wants an explanation for it.

• Model agnostic. Attributions should be computable by querying \(f\) alone, without any assumptions about what kind of model it is – it could be a black box.

• Principled. The attribution measure should be derivable from a clear set of mathematical axioms.

Approach. SHAP values satisfy all three requirements. This handout focuses on their conceptual foundation (Section 1) and practical computation (Section 2).

Conceptual Foundation

Local Feature Attributions

1. High-stakes decisions. When an individual has a medical diagnosis made or an insurance claim denied, knowing the globally most important features isn’t enough. They deserve an explanation specific to their case.

   • A related use case is algorithmic recourse. What could a stakeholder change to reverse a decision? (e.g., which change to their resume would have gotten them a job interview?)

2. Model debugging. A model might classify \(x\) as a husky because it had snow in the background (large \(\varphi_d(x)\) on pixels \(d\) in the snow region) rather than the dog itself. This means the model has learned a “shortcut” and won’t generalize well – a wolf in the snow might get misclassified as a husky (Ribeiro et al. 2016).

   

3. Scientific discovery. In heterogeneous populations (e.g., different disease subtypes), a model might rely on different sets of features for each subpopulation. Local attributions can highlight these differences – e.g., identifying which features drive drug effectiveness in one subgroup vs. another.

   Exercise: Give one example (hypothetical, or from your own experience) where local feature attribution would be useful. How would it differ from global variable importance?

Game Theory Definitions

1. SHAP is motivated by the credit assignment problem from game theory. Imagine agents \(\mathcal{D} = \{1, \dots, D\}\) where any subset \(S\) earns profit \(v(S)\). How much of the total profit \(v(\mathcal{D})\) should agent \(d\) receive? This share is the Shapley value \(\varphi_{d}(v)\).

2. Intuitively, agent \(d\)’s contribution depends on how much they add to each team \(S\). Define the marginal contributions of agent \(d\) to team \(S\) as, \[C\left(d \vert S\right) = v(S \cup \{d\}) - v(S)\]

   

3. The Shapley value is a weighted average of these marginal contributions across all subsets \(S \subseteq{\mathcal{D} - \{d\}}\) excluding agent \(d\),

   \[ \varphi_d(v) = \sum_{S \subseteq \mathcal{D} - \{d\}} \frac{1}{D {D - 1 \choose \left|S\right|}} C(d \vert S) \tag{1}\] The summation is over all subsets that don’t include agent \(\{d\}\) (if it had included agent \(d\), then the definition of the contribution \(C(d \vert S)\) of \(d\) to \(S\) wouldn’t make sense).

   Exercise: Express the following in terms of \(v\). Which of these could appear in the definition of the Shapley value?

   • \(C(1 \vert 2)\)
   • \(C(1 \vert \emptyset)\)
   • \(C(1 \vert 1)\)
   • \(C(3 \vert \{1, 2\})\)
   • \(C(\{1, 2\} \vert 3)\)

4. The weights \(1/(D {D - 1 \choose \left|S\right|})\) ensure that the weights sum to 1, making \(\varphi_{d}(v)\) a proper weighted average of marginal contributions. For any coalition of size \(s\), there are \({D - 1 \choose s}\) subsets with that size, so the total weight is,

   \[ \sum_{s=0}^{D-1} \binom{D-1}{s} \frac{1}{D\binom{D-1}{s}} = \sum_{s=0}^{D-1} \frac{1}{D} = 1. \]