Assessing the impact of variants of unknown significance on splicing has become a critical issue and a bottleneck, especially with the widespread implementation of whole-genome or exome sequencing. Although multiple in silico tools are available, the interpretation and application of these tools are difficult and practical guidelines are still lacking. A streamlined decision-making process can facilitate the downstream RNA analysis in a more efficient manner. Therefore, we evaluated the performance of 8 in silico tools (Splice Site Finder, MaxEntScan, Splice-site prediction by neural network, GeneSplicer, Human Splicing Finder, SpliceAI, Splicing Predictions in Consensus Elements, and SpliceRover) using 114 NF1 spliceogenic variants, experimentally validated at the mRNA level. The change in the predicted score incurred by the variant of the nearest wild-type splice site was analyzed, and for type II, III, and IV splice variants, the change in the prediction score of de novo or cryptic splice site was also analyzed. SpliceAI and SpliceRover, tools based on deep learning, outperformed all other tools, with AUCs of 0.972 and 0.924, respectively. For de novo and cryptic splice sites, SpliceAI outperformed all other tools and showed a sensitivity of 95.7% at an optimal cut-off of 0.02 score change. Our results show that deep learning algorithms, especially those of SpliceAI, are validated at a significantly higher rate than other in silico tools for clinically relevant NF1 variants. This suggests that deep learning algorithms outperform traditional probabilistic approaches and classical machine learning tools in predicting the de novo and cryptic splice sites.

中文翻译：

---

使用已建立的剪接变异体预测剪接改变变异体的计算机工具比较：最终用户的观点

评估未知意义的变异对剪接的影响已成为一个关键问题和瓶颈，特别是随着全基因组或外显子组测序的广泛实施。尽管有多种计算机工具可用，但这些工具的解释和应用很困难，并且仍然缺乏实用的指导方针。简化的决策过程可以以更有效的方式促进下游 RNA 分析。因此，我们使用 114 NF1评估了 8 个计算机工具（Splice Site Finder、MaxEntScan、Splice-site prediction by neural network、GeneSplicer、Human Splicing Finder、SpliceAI、Splicing Predictions in Consensus Elements 和 SpliceRover）的性能剪接变体，在 mRNA 水平上经过实验验证。分析了最近的野生型剪接位点变异引起的预测分数变化，对于II、III和IV型剪接变异，还分析了从头或隐蔽剪接位点预测分数的变化。基于深度学习的工具 SpliceAI 和 SpliceRover 优于所有其他工具，AUC 分别为 0.972 和 0.924。对于从头和隐蔽的剪接位点，SpliceAI 优于所有其他工具，并且在 0.02 分数变化的最佳截止值下显示出 95.7% 的灵敏度。我们的结果表明，深度学习算法，尤其是 SpliceAI 的算法，在临床相关方面的验证率明显高于其他计算机工具NF1变体。这表明深度学习算法在预测从头和隐秘剪接位点方面优于传统的概率方法和经典机器学习工具。