ABSTRACT: 

Shapley values and similar methods are widely used to explain the importance of features in model predictions. Clarity in the semantics of these feature importances is subtle, but crucial: What do these explanations actually mean? And how are they useful? We illustrate using explanations of predictions in three domains: (a) medical outcomes, (b) image content, and (c) first impressions of people—specifically their warmth and competence—derived from video recordings and transcripts. In each scenario, the presence of intermediate-level features enhances the clarity and usefulness of
explanations.