Statisticians often face the choice between using probability models or a paradigm defined by minimising a loss function. Both approaches are useful and, if the loss can be re-cast into a proper probability model, there are many tools to decide which model or loss is more appropriate for the observed data, in the sense of explaining the data's nature. However, when the loss leads to an improper model, there are no principled ways to guide this choice. We address this task by combining the Hyvärinen score, which naturally targets infinitesimal relative probabilities, and general Bayesian updating, which provides a unifying framework for inference on losses and models. Specifically we propose the $\mathscr{H}$-score, a general Bayesian selection criterion and prove that it consistently selects the (possibly improper) model closest to the data-generating truth in Fisher's divergence. We also prove that an associated $\mathscr{H}$-posterior consistently learns optimal hyper-parameters featuring in loss functions, including a challenging tempering parameter in generalised Bayesian inference. As salient examples, we consider robust regression and non-parametric density estimation where popular loss functions define improper models for the data and hence cannot be dealt with using standard model selection tools. These examples illustrate advantages in robustness-efficiency trade-offs and enable Bayesian inference for kernel density estimation, opening a new avenue for Bayesian non-parametrics.

中文翻译：

---

一般贝叶斯损失函数的选择和不当模型的使用

统计学家经常面临是使用概率模型还是通过最小化损失函数定义的范式之间的选择。这两种方法都很有用，如果可以将损失重新转换为适当的概率模型，则有许多工具可以决定哪种模型或损失更适合观察到的数据，从解释数据的性质的意义上讲。然而，当损失导致模型不合适时，没有原则性的方法来指导这种选择。我们通过结合自然地针对无穷小相对概率的 Hyvärinen 分数和一般贝叶斯更新来解决这个任务，后者为损失和模型的推理提供了一个统一的框架。具体来说，我们提出$\mathscr{H}$-score，一个通用的贝叶斯选择标准，并证明它始终选择最接近 Fisher 散度中数据生成真相的（可能不正确的）模型。我们还证明了相关的$\mathscr{H}$-posterior 始终学习损失函数中的最佳超参数，包括广义贝叶斯推理中具有挑战性的回火参数。作为突出的例子，我们考虑稳健回归和非参数密度估计，其中流行的损失函数为数据定义了不正确的模型，因此无法使用标准模型选择工具进行处理。这些例子说明了鲁棒性-效率权衡的优势，并使贝叶斯推理能够用于核密度估计，为贝叶斯非参数学开辟了一条新途径。