Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere IoT datasets both capture IoT-related attacks, but the deep neural network performed the best when tested using the BoT_IoT dataset while HELAD performed the best when tested using the Stratosphere IoT dataset. So although we found that a deep neural network solution had the highest average F1 scores on tested datasets, it is not always the best-performing one. We further discuss difficulties in using IDS from literature and project repositories, which complicated drawing definitive conclusions regarding IDS selection.

中文翻译：

---

期望与现实：在实践中评估入侵检测系统

我们的论文提供了最新 IDS 之间的实证比较，以提供它们之间的客观比较，以帮助用户根据自己的需求选择最合适的解决方案。我们的结果表明，没有一种解决方案是最好的，而是取决于外部变量，例如数据集中的攻击类型、复杂性和网络环境。例如，BoT_IoT 和 Stratosphere IoT 数据集都捕获与 IoT 相关的攻击，但使用 BoT_IoT 数据集测试时深度神经网络表现最佳，而使用 Stratosphere IoT 数据集测试时 HELAD 表现最佳。因此，尽管我们发现深度神经网络解决方案在测试数据集上具有最高的平均 F1 分数，但它并不总是性能最好的解决方案。我们进一步讨论了从文献和项目存储库中使用 IDS 的困难，这使得就 IDS 选择做出明确的结论变得复杂。