Abstract

Using knowledge graph embedding models (KGEMs) is a popular approach for predicting links in knowledge graphs (KGs). Traditionally, the performance of KGEMs for link prediction is assessed using rank-based metrics, which evaluate their ability to give high scores to ground-truth entities. However, the literature claims that the KGEM evaluation procedure would benefit from adding supplementary dimensions to assess. That is why, in this paper, we extend our previously introduced metric Sem@K that measures the capability of models to predict valid entities w.r.t. domain and range constraints. In particular, we consider a broad range of KGs and take their respective characteristics into account to propose different versions of Sem@K. We also perform an extensive study to qualify the abilities of KGEMs as measured by our metric. Our experiments show that Sem@K provides a new perspective on KGEM quality. Its joint analysis with rank-based metrics offers different conclusions on the predictive power of models. Regarding Sem@K, some KGEMs are inherently better than others, but this semantic superiority is not indicative of their performance w.r.t. rank-based metrics. In this work, we generalize conclusions about the relative performance of KGEMs w.r.t. rank-based and semantic-oriented metrics at the level of families of models. The joint analysis of the aforementioned metrics gives more insight into the peculiarities of each model. This work paves the way for a more comprehensive evaluation of KGEM adequacy for specific downstream tasks.

中文翻译：

---

Sem@ K：我的知识图嵌入模型具有语义感知能力吗？

 抽象的

使用知识图嵌入模型（KGEM）是预测知识图（KG）中链接的流行方法。传统上，KGEM 的链接预测性能是使用基于排名的指标来评估的，该指标评估它们为真实实体提供高分的能力。然而，文献声称 KGEM 评估程序将受益于添加补充维度进行评估。这就是为什么在本文中，我们扩展了之前引入的指标 Sem@K，该指标衡量模型预测有效实体的能力。域和范围的限制。特别是，我们考虑了广泛的知识图谱并考虑了它们各自的特点，提出了不同版本的 Sem@K。我们还进行了广泛的研究，以确定 KGEM 的能力（按照我们的指标衡量）。我们的实验表明 Sem@K 为 KGEM 质量提供了新的视角。它与基于排名的指标的联合分析提供了关于模型预测能力的不同结论。关于 Sem@K，一些 KGEM 本质上比其他 KGEM 更好，但这种语义优势并不表明它们的性能。基于排名的指标。在这项工作中，我们概括了 KGEM 相对性能的结论。模型系列级别的基于排名和面向语义的度量。对上述指标的联合分析可以更深入地了解每个模型的特性。这项工作为更全面地评估 KGEM 对特定下游任务的充分性铺平了道路。