Most of the existing works on fine-grained image categorization and retrieval focus on finding similar images from the same species and often give little importance to inter-species similarities. However, these similarities may carry species correlations such as the same ancestors or similar habits, which are helpful in taxonomy and understanding biological traits. In this paper, we devise a new fine-grained retrieval task that searches for similar instances from different species based on body parts. To this end, we propose a two-step strategy. In the first step, we search for visually similar parts to a query image using a deep convolutional neural network (CNN). To improve the quality of the retrieved candidates, structural cues are introduced into the CNN using a novel part-pooling layer, in which the receptive field of each part is adjusted automatically. In the second step, we re-rank the retrieved candidates to improve the species diversity. We achieve this by formulating a novel ranking function that balances between the similarity of the candidates to the queried parts, while decreasing the similarity to the query species. We provide experiments on the benchmark CUB200 dataset and Columbia Dogs dataset, and demonstrate clear benefits of our schemes.

大多数现有的细粒度图像分类和检索工作都侧重于从同一物种中寻找相似图像，而往往不太重视物种间的相似性。然而，这些相似性可能带有物种相关性，例如相同的祖先或相似的习性，这有助于分类学和理解生物特征。在这篇论文中，我们设计了一个新的细粒度检索任务，根据身体部位从不同物种中搜索相似的实例。为此，我们提出了一个两步走的策略。在第一步中，我们使用深度卷积神经网络 (CNN) 搜索与查询图像视觉相似的部分。为了提高检索到的候选对象的质量，使用新的部分池化层将结构线索引入 CNN，其中每个部分的感受野是自动调整的。在第二步中，我们对检索到的候选者进行重新排序以提高物种多样性。我们通过制定一个新颖的排名函数来实现这一点，该函数在候选者与查询部分的相似性之间取得平衡，同时降低与查询物种的相似性。我们提供了基准 CUB200 数据集和 Columbia Dogs 数据集的实验，并展示了我们方案的明显优势。