当前位置: X-MOL 学术ACM Trans. Inf. Syst. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Decoupled Progressive Distillation for Sequential Prediction with Interaction Dynamics
ACM Transactions on Information Systems ( IF 5.6 ) Pub Date : 2023-12-29 , DOI: 10.1145/3632403
Kaixi Hu 1 , Lin Li 2 , Qing Xie 2 , Jianquan Liu 3 , Xiaohui Tao 4 , Guandong Xu 5
Affiliation  

Sequential prediction has great value for resource allocation due to its capability in analyzing intents for next prediction. A fundamental challenge arises from real-world interaction dynamics where similar sequences involving multiple intents may exhibit different next items. More importantly, the character of volume candidate items in sequential prediction may amplify such dynamics, making deep networks hard to capture comprehensive intents. This article presents a sequential prediction framework with Decoupled Progressive Distillation (DePoD), drawing on the progressive nature of human cognition. We redefine target and non-target item distillation according to their different effects in the decoupled formulation. This can be achieved through two aspects: (1) Regarding how to learn, our target item distillation with progressive difficulty increases the contribution of low-confidence samples in the later training phase while keeping high-confidence samples in the earlier phase. And, the non-target item distillation starts from a small subset of non-target items from which size increases according to the item frequency. (2) Regarding whom to learn from, a difference evaluator is utilized to progressively select an expert that provides informative knowledge among items from the cohort of peers. Extensive experiments on four public datasets show DePoD outperforms state-of-the-art methods in terms of accuracy-based metrics.



中文翻译:

用于交互动力学顺序预测的解耦渐进蒸馏

顺序预测对于资源分配具有巨大的价值,因为它能够分析下一次预测的意图。现实世界的交互动态产生了一个基本挑战,其中涉及多个意图的相似序列可能会表现出不同的下一个项目。更重要的是,顺序预测中体积候选项的特征可能会放大这种动态,使深层网络难以捕获全面的意图。本文利用人类认知的渐进本质,提出了一种采用解耦渐进蒸馏 (DePoD) 的顺序预测框架。我们根据目标项和非目标项在解耦公式中的不同效果重新定义它们。这可以通过两个方面来实现:(1)关于如何学习,我们的目标项蒸馏具有渐进的难度,增加了低置信度样本在后期训练阶段的贡献,同时保留了高置信度样本在早期阶段的贡献。并且,非目标项目蒸馏从非目标项目的一个小子集开始,该子集的大小根据项目频率而增加。(2)关于向谁学习,利用差异评估器逐步选择在同行群体中的项目中提供信息知识的专家。对四个公共数据集的广泛实验表明,DePoD 在基于准确性的指标方面优于最先进的方法。

更新日期:2023-12-31
down
wechat
bug