Modern transformer-based information retrieval models achieve state-of-the-art performance across various benchmarks. The self-attention of the transformer models is a powerful mechanism to contextualize terms over the whole input but quickly becomes prohibitively expensive for long input as required in document retrieval. Instead of focusing on the model itself to improve efficiency, this paper explores different bag of words document representations that encode full documents by only a fraction of their characteristic terms, allowing us to control and reduce the input length. We experiment with various models for document retrieval on MS MARCO data, as well as zero-shot document retrieval on Robust04, and show large gains in efficiency while retaining reasonable effectiveness. Inference time efficiency gains are both lowering the time and memory complexity in a controllable way, allowing for further trading off memory footprint and query latency. More generally, this line of research connects traditional IR models with neural “NLP” models and offers novel ways to explore the space between (efficient, but less effective) traditional rankers and (effective, but less efficient) neural rankers elegantly.

现代基于变压器的信息检索模型在各种基准测试中实现了最先进的性能。变压器模型的自注意力是一种强大的机制，可以将整个输入的术语置于上下文中，但对于文档检索所需的长输入来说，很快就会变得非常昂贵。本文没有关注模型本身来提高效率，而是探索了不同的词袋文档表示形式，这些表示形式仅通过部分特征术语对完整文档进行编码，从而使我们能够控制和减少输入长度。我们在 MS MARCO 数据上尝试了各种文档检索模型，并在 Robust04 上进行了零样本文档检索，结果表明，在保持合理有效性的同时，效率大幅提高。推理时间效率的提高以可控的方式降低了时间和内存复杂性，从而可以进一步权衡内存占用和查询延迟。更一般地说，这一系列研究将传统的 IR 模型与神经“NLP”模型联系起来，并提供了新颖的方法来优雅地探索（高效但效率较低）传统排序器和（有效但效率较低）神经排序器之间的空间。