Pre-trained deep learning (DL) models are increasingly accessible in public repositories, i.e., model zoos. Given a new prediction task, finding the best model to fine-tune can be computationally intensive and costly, especially when the number of pre-trained models is large. Selecting the right pre-trained models is crucial, yet complicated by the diversity of models from various model families (like ResNet, Vit, Swin) and the hidden relationships between models and datasets. Existing methods, which utilize basic information from models and datasets to compute scores indicating model performance on target datasets, overlook the intrinsic relationships, limiting their effectiveness in model selection. In this study, we introduce TransferGraph, a novel framework that reformulates model selection as a graph learning problem. TransferGraph constructs a graph using extensive metadata extracted from models and datasets, while capturing their inherent relationships. Through comprehensive experiments across 16 real datasets, both images and texts, we demonstrate TransferGraph's effectiveness in capturing essential model-dataset relationships, yielding up to a 32% improvement in correlation between predicted performance and the actual fine-tuning results compared to the state-of-the-art methods.

中文翻译：

---

通过图学习使用 Model Zoo 进行模型选择

预训练的深度学习 (DL) 模型越来越多地可以在公共存储库（即模型动物园）中访问。给定新的预测任务，找到最佳的微调模型可能需要大量计算且成本高昂，特别是当预训练模型数量很大时。选择正确的预训练模型至关重要，但由于不同模型系列（如 ResNet、Vit、Swin）的模型多样性以及模型和数据集之间的隐藏关系而变得复杂。现有的方法利用模型和数据集的基本信息来计算表明模型在目标数据集上的性能的分数，忽略了内在关系，限制了它们在模型选择中的有效性。在本研究中，我们引入了 TransferGraph，这是一种新颖的框架，它将模型选择重新表述为图学习问题。 TransferGraph 使用从模型和数据集中提取的大量元数据构建图表，同时捕获它们的内在关系。通过对 16 个真实数据集（包括图像和文本）的综合实验，我们证明了 TransferGraph 在捕获基本模型-数据集关系方面的有效性，与状态相比，预测性能和实际微调结果之间的相关性提高了 32% - 最先进的方法。