When Shapley Wins by Construction: Auditing Proxy-Target Attribution Benchmarks
Dipankar Sarkar
Abstract
Offline attribution benchmarks can fail for two distinct reasons: the reconstructed journeys may not support the multi-channel problem being evaluated, and the evaluation target may be mathematically coupled to the candidate method. We audit both failure modes in multi-touch attribution. Reconstructing Criteo Attribution by its documented event key (user, conversion\_id, conversion\_timestamp) yields 438,730 conversions, each containing a single raw campaign. For CriteoPrivateAds, shard provenance and display-order continuity certify 388,848 of 760,280 observed sequences as complete; among 2,619 converting complete journeys, 2,594 (99.05%) contain one raw publisher and none contain more than two. Across nine eligible attribution outputs, the maximum pairwise total variation (TV) distance is 0.00881, below a declared 0.01 aggregate-credit margin. Separately, we use the known coverage-game identity to show that its Shapley vector equals count-weighted equal split exactly; evaluating it against that constructed target therefore yields zero error by design rather than empirical validation. Together these results show how insufficient journey support and target dependence can make offline attribution comparisons appear more informative than the underlying benchmark permits.