Derun Song
Skip Abstract Section
Abstract
Multi-scenario and multi-task recommendation can use various feedback behaviors of users in different scenarios to learn users’ preferences and then make recommendations, which has attracted attention. However, the existing work ignores feature interactions and the fact that a pair of feature interactions will have differing levels of importance under different scenario-task pairs, leading to sub-optimal user preference learning. In this article, we propose a Multi-scenario and Multi-task aware Feature Interaction model, dubbed MMFI, to explicitly model feature interactions and learn the importance of feature interaction pairs in different scenarios and tasks. Specifically, MMFI first incorporates a pairwise feature interaction unit and a scenario-task interaction unit to effectively capture the interaction of feature pairs and scenario-task pairs. Then MMFI designs a scenario-task aware attention layer for learning the importance of feature interactions from coarse-grained to fine-grained, improving the model’s performance on various scenario-task pairs. More specifically, this attention layer consists of three modules: a fully shared bottom module, a partially shared middle module, and a specific output module. Finally, MMFI adapts two sparsity-aware functions to remove some useless feature interactions. Extensive experiments on two public datasets demonstrate the superiority of the proposed method over the existing multi-task recommendation, multi-scenario recommendation, and multi-scenario & multi-task recommendation models.
- [1] . 2009. Curriculum learning. In Proceedings of the ICML. 41–48.Google Scholar
Digital Library
- [2] Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. 2016. Higher-order factorization machines. 566 In Proceedings of the NeurIPS.Google Scholar
- [3] . 1993. Multitask learning: A knowledge-based source of inductive bias1. In Proceedings of the ICML. Citeseer, 41–48.Google ScholarCross Ref
- [4] Jianxin Chang, Chenbin Zhang, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, and Kun Gai. 2023. Pepnet: Parameter and embedding personalized network for infusing with personalized prior information. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3795–3804.Google Scholar
- [5] . 2020. Scenario-aware and Mutual-based approach for multi-scenario recommendation in e-commerce. In Proceedings of the ICDMW. IEEE, 127–135.Google ScholarCross Ref
- [6] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. 7–10.Google Scholar
- [7] . 2022. Bi-directional contrastive distillation for multi-behavior recommendation. In Proceedings of the ECML-PKDD. Springer, 491–507.Google Scholar
- [8] . 2016. Deep neural networks for Youtube recommendations. In Proceedings of the RecSys. 191–198.Google ScholarDigital Library
- [9] . 2023. Uniform sequence better: Time interval aware data augmentation for sequential recommendation. In Proceedings of the AAAI, Vol. 37. 4225–4232.Google ScholarDigital Library
- [10] . 2023. A unified multi-task learning framework for multi-goal conversational recommender systems. TOIS 41, 3 (2023), 1–25.Google ScholarDigital Library
- [11] Ke Ding, Xin Dong, Yong He, Lei Cheng, Chilin Fu, Zhaoxin Huan, Hai Li, Tan Yan, Liang Zhang, Xiaolu Zhang, and Linjian Mo. 2021. MSSM: a multiple-level sparse sharing model for efficient multi-task learning. In SIGIR. 2237–2241.Google Scholar
- [12] . 2011. Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12, 7 (2011), 2121–2159.Google ScholarDigital Library
- [13] . 2006. An introduction to ROC analysis. Pattern Recognition Letters 27, 8 (2006), 861–874.Google ScholarDigital Library
- [14] . 2023. Enhanced multi-task learning and knowledge graph-based recommender system. TKDE 35, 7 (2023), 10281–10294.Google Scholar
- [15] . 2017. DeepFM: A factorization-machine based neural network for CTR prediction. In Proceedings of the IJCAI. 1725–1731.Google ScholarCross Ref
- [16] . 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the SIGIR. 355–364.Google ScholarDigital Library
- [17] . 2017. Neural collaborative filtering. In Proceedings of the WWW. 173–182.Google ScholarDigital Library
- [18] . 2022. MetaBalance: Improving multi-task recommendations via adapting gradient magnitudes of auxiliary tasks. In Proceedings of the WWW. 2205–2215.Google Scholar
- [19] . 2023. Revisiting scalarization in multi-task learning: A theoretical perspective. In Proceedings of the NeurIPS.Google Scholar
- [20] Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Dapeng Liu, Jie Jiang, and Mingsheng Long. 2023. ForkMerge: Mitigating negative transfer in auxiliary-task learning. In NeurIPS.Google Scholar
- [21] . 2016. Field-aware factorization machines for CTR prediction. In Proceedings of the RecSys. 43–50.Google ScholarDigital Library
- [22] . 2015. Adam: A method for stochastic optimization. In Proceedings of the ICLR.Google Scholar
- [23] . 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.Google ScholarDigital Library
- [24] . 2020. Improving multi-scenario learning to rank in e-commerce by exploiting task relationships in the label space. In Proceedings of the CIKM. 2605–2612.Google ScholarDigital Library
- [25] . 2018. xDeepFM: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the KDD. 1754–1763.Google ScholarDigital Library
- [26] . 2020. Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proceedings of the SIGKDD. 2636–2645.Google ScholarDigital Library
- [27] . 2019. End-to-end multi-task learning with attention. In Proceedings of the CVPR. 1871–1880.Google ScholarCross Ref
- [28] . 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of the SIGKDD. 1930–1939.Google ScholarDigital Library
- [29] . 2018. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In Proceedings of the SIGIR. ACM, 1137–1140.Google ScholarDigital Library
- [30] . 2016. Cross-stitch networks for multi-task learning. In Proceedings of the CVPR. 3994–4003.Google ScholarCross Ref
- [31] . 2019. Predicting different types of conversions with multi-task learning in online advertising. In Proceedings of the SIGKDD. 2689–2697.Google ScholarDigital Library
- [32] . 2018. Field-weighted factorization machines for click-through rate prediction in display advertising. In Proceedings of the WWW. 1349–1357.Google ScholarDigital Library
- [33] . 2018. On the Convergence of Adam and Beyond. In Proceedings of the ICLR.Google Scholar
- [34] . 2010. Factorization machines. In Proceedings of the ICDM. IEEE, 995–1000.Google ScholarDigital Library
- [35] . 2019. Latent multi-task architecture learning. In Proceedings of the AAAI. AAAI Press, 4822–4829.Google ScholarDigital Library
- [36] . 2018. Multi-task learning as multi-objective optimization. InProceedings of the NeurIPS.Google Scholar
- [37] . 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In Proceedings of the ICLR.Google Scholar
- [38] . 2017. A review on imbalanced data handling using undersampling and oversampling technique. IJRTER 3, 4 (2017), 444–449.Google ScholarCross Ref
- [39] . 2021. SAR-Net: A scenario-aware ranking network for personalized fair recommendation in hundreds of travel scenarios. In Proceedings of the CIKM. 4094–4103.Google ScholarDigital Library
- [40] Xiang-Rong Sheng, Liqin Zhao, Guorui Zhou, Xinyao Ding, Binding Dai, Qiang Luo, Siran Yang, Jingshan Lv, Chi Zhang, Hongbo Deng, and Xiaoqiang Zhu. 2021. One model to serve all: Star topology adaptive recommender for multi-domain ctr prediction. In CIKM. 4104–4113.Google Scholar
- [41] . 2020. Learning sparse sharing architectures for multiple tasks. In Proceedings of the AAAI, Vol. 34. 8936–8943.Google ScholarCross Ref
- [42] . 2021. FM2: Field-matrixed factorization machines for recommender systems. In Proceedings of the WWW. 2828–2837.Google ScholarDigital Library
- [43] . 2020. Progressive layered extraction (PLE): A novel multi-task learning (MTL) model for personalized recommendations. In Proceedings of the RecSys. 269–278.Google ScholarDigital Library
- [44] . 2013. Deep content-based music recommendation. In Proceedings of the NeurIPS.Google Scholar
- [45] . 2017. Attention is all you need. In Proceedings of the NeurIPS.Google Scholar
- [46] . 2023. Multi-view enhanced graph attention network for session-based music recommendation. TOIS 42, 10 (2023), 1–30.Google Scholar
- [47] . 2017. Deep & cross network for ad click predictions. In Proceedings of the ADKDD’17. 1–7.Google ScholarDigital Library
- [48] . 2021. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In Proceedings of the WWW. 1785–1797.Google ScholarDigital Library
- [49] . 2019. Neural graph collaborative filtering. In Proceedings of the SIGIR. 165–174.Google ScholarDigital Library
- [50] . 2020. Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction. In Proceedings of the SIGIR. ACM, 2377–2386.Google ScholarDigital Library
- [51] . 2021. Modeling the sequential dependence among audience multi-step conversions with multi-task learning in targeted display advertising. In Proceedings of the KDD.Google Scholar
- [52] . 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In Proceedings of the IJCAI. 3119–3125.Google ScholarCross Ref
- [53] . 2021. Personalized approximate pareto-efficient recommendation. In Proceedings of the WWW. ACM / IW3C2, 3839–3849.Google ScholarDigital Library
- [54] . 2021. Core interest network for click-through rate prediction. TKDD 15, 2 (2021), 1–16.Google ScholarDigital Library
- [55] . 2023. AdaTask: A task-aware adaptive learning rate approach to multi-task learning. In Proceedings of the AAAI, Vol. 37. 10745–10753.Google ScholarDigital Library
- [56] . 2023. MHANER: A multi-source heterogeneous graph attention network for explainable recommendation in online games. TIST (2023).Google ScholarDigital Library
- [57] . 2022. Tenrec: A large-scale multipurpose benchmark dataset for recommender systems. In Proceedings of the NeurIPS Datasets and Benchmarks Track.Google Scholar
- [58] . 2022. Leaving no one behind: A multi-scenario multi-task meta learning approach for advertiser modeling. In Proceedings of the WSDM. 1368–1376.Google ScholarDigital Library
- [59] . 2018. Deep interest network for click-through rate prediction. In Proceedings of the SIGKDD. 1059–1068.Google ScholarDigital Library
- [60] Jie Zhou, Xianshuai Cao, Wenhao Li, Lin Bo, Kun Zhang, Chuan Luo, and Qian Yu. 2023. Hinet: novel multi-scenario & multi-task learning with hierarchical information extraction. In 2023 IEEE 39th International Conference on Data Engineering (ICDE). IEEE, 2969–2975.Google Scholar
- [61] . 2021. Open benchmarking for click-through rate prediction. In Proceedings of the CIKM. ACM, 2759–2769.Google ScholarDigital Library
- [62] . 2019. A sufficient condition for convergences of Adam and RMSProp. In Proceedings of the CVPR. 11127–11135.Google ScholarCross Ref
- [63] . 2022. Automatic expert selection for multi-scenario and multi-task search. In Proceedings of the SIGIR. 1535–1544.Google ScholarDigital Library
Index Terms
- Multi-Scenario and Multi-Task Aware Feature Interaction for Recommendation System
Recommendations
Using a trust network to improve top-N recommendation
RecSys '09: Proceedings of the third ACM conference on Recommender systemsTop-N item recommendation is one of the important tasks of recommenders. Collaborative filtering is the most popular approach to building recommender systems which can predict ratings for a given user and item. Collaborative filtering can be extended ...
An Effective Implicit Multi-interest Interaction Network for Recommendation
Neural Information ProcessingAbstractData features in real industrial recommendation scenarios are high-dimensional, diverse and sparse. Rich feature interaction can improve the model effect and bring practical benefits. Factorization machines (FMs) can perform explicit second-order ...
Typicality-Based Collaborative Filtering Recommendation
Collaborative filtering (CF) is an important and popular technology for recommender systems. However, current CF methods suffer from such problems as data sparsity, recommendation inaccuracy, and big-error in predictions. In this paper, we borrow ideas ...
Comments