Conversational agents, or chatbots, have become popular to access all kind of software services. They provide an intuitive natural language interface for interaction, available from a wide range of channels including social networks, web pages, intelligent speakers or cars. In response to this demand, many chatbot development platforms and tools have emerged. However, they typically lack support to statically measure properties of the chatbots being built, as indicators of their size, complexity, quality or usability. Similarly, there are hardly any mechanisms to compare and cluster chatbots developed with heterogeneous technologies.

To overcome this limitation, we propose a suite of 21 metrics for chatbot designs, as well as two clustering methods that help in grouping chatbots along their conversation topics and design features. Both the metrics and the clustering methods are defined on a neutral chatbot design language, becoming independent of the implementation platform. We provide automatic translations of chatbots defined on some major platforms into this neutral notation to perform the measurement and clustering. The approach is supported by our tool Asymob, which we have used to evaluate the metrics and the clustering methods over a set of 259 Dialogflow and Rasa chatbots from open-source repositories. The results open the door to incorporating the metrics within chatbot development processes for the early detection of quality issues, and to exploit clustering to organise large collections of chatbots into significant groups to ease chatbot comprehension, search and comparison.

会话代理或聊天机器人已变得流行用于访问各种软件服务。它们提供直观的自然语言交互界面，可通过社交网络、网页、智能扬声器或汽车等多种渠道进行交互。针对这种需求，出现了许多聊天机器人开发平台和工具。然而，它们通常缺乏对静态测量正在构建的聊天机器人属性的支持，作为其大小、复杂性、质量或可用性的指标。同样，几乎没有任何机制可以对使用异构技术开发的聊天机器人进行比较和聚类。

为了克服这一限制，我们提出了一套包含 21 个聊天机器人设计指标的套件，以及两种有助于根据对话主题和设计功能对聊天机器人进行分组的聚类方法。指标和聚类方法都是在中立的聊天机器人设计语言上定义的，独立于实施平台。我们将一些主要平台上定义的聊天机器人自动翻译成这种中性符号，以执行测量和聚类。该方法得到我们的工具Asymob的支持，我们用它来评估来自开源存储库的 259 个 Dialogflow 和 Rasa 聊天机器人的指标和聚类方法。结果为将指标纳入聊天机器人开发流程中以尽早发现质量问题打开了大门，并利用集群将大量聊天机器人组织成重要的组，以简化聊天机器人的理解、搜索和比较。