This study explores the use of Large Language Models (LLMs) for automatic evaluation of knowledge graph (KG) completion models. Historically, validating information in KGs has been a challenging task, requiring large-scale human annotation at prohibitive cost. With the emergence of general-purpose generative AI and LLMs, it is now plausible that human-in-the-loop validation could be replaced by a generative agent. We introduce a framework for consistency and validation when using generative models to validate knowledge graphs. Our framework is based upon recent open-source developments for structural and semantic validation of LLM outputs, and upon flexible approaches to fact checking and verification, supported by the capacity to reference external knowledge sources of any kind. The design is easy to adapt and extend, and can be used to verify any kind of graph-structured data through a combination of model-intrinsic knowledge, user-supplied context, and agents capable of external knowledge retrieval.

中文翻译：

---

KGValidator：知识图谱构建自动验证框架

本研究探讨了使用大型语言模型（LLM）来自动评估知识图（KG）完成模型。从历史上看，验证知识图谱中的信息一直是一项具有挑战性的任务，需要以高昂的成本进行大规模的人工注释。随着通用生成人工智能和法学硕士的出现，人机交互验证现在有可能被生成代理所取代。我们引入了使用生成模型验证知识图时的一致性和验证框架。我们的框架基于最近用于法学硕士输出的结构和语义验证的开源开发，以及事实检查和验证的灵活方法，并得到引用任何类型的外部知识源的能力的支持。该设计易于调整和扩展，并且可以通过模型内在知识、用户提供的上下文和能够进行外部知识检索的代理的组合来验证任何类型的图结构数据。