We provide a uniform, general, and complete formal account of evaluation metrics for ranking, classification, clustering, and other information access problems. We leverage concepts from measurement theory, such as scale types and permissible transformation functions, and we capture the nature of evaluation metrics in many tasks by two formal definitions, which lead to a distinction of two metric/tasks families, and provide a comprehensive classification of the tasks that have been proposed so far. We derive some theorems to analyze the suitability (or otherwise) of some common metrics. Within our model we can derive and explain the theoretical properties and drawbacks of the state of the art metrics for multiple tasks. The main contributions of this paper are that, differently from previous studies, the formalization is well grounded on a solid discipline, it is general as it can take into account most effectiveness metrics as well as most existing tasks, and it allows to derive important consequences on metrics and their limitations.

中文翻译：

---

关于信息访问评估指标的性质：统一框架

对于排名，分类，聚类和其他信息访问问题，我们提供了评估指标的统一，通用和完整的正式说明。我们利用度量理论中的概念，例如量表类型和允许的转换函数，并通过两个正式定义来捕获许多任务中评估指标的性质，这导致了两个度量/任务族的区别，并提供了对到目前为止已经提出的任务。我们得出一些定理，以分析某些通用指标的适用性（或其他适用性）。在我们的模型中，我们可以推导和解释用于多个任务的最新技术指标的理论特性和缺点。与以前的研究不同，本文的主要贡献是，