This paper proposes a novel multi-modal probabilistic topic model (LSTIT) to infer consumer preferences by jointly leveraging textual and visual data. Specifically, we use the title and image of the items purchased by consumers. Considering that the titles of items are relatively short text, we thus restrict the topic assignment for these titles. Meanwhile, we employ the same topic distribution to model the relationship between the title and the image of the item. To learn consumer preferences, the proposed model extracts several important dimensions based on textual words in titles and visual features in images. Experiments on the Amazon dataset show that the proposed model outperforms other baseline models for the task of learning consumer preferences. Our findings provide significant implications for managers to understand users’ personalized interests behind purchase behavior from a fine-grained level and a multi-modal perspective.

本文提出了一种新颖的多模态概率主题模型（LSTIT），通过联合利用文本和视觉数据来推断消费者偏好。具体来说，我们使用消费者购买的商品的标题和图像。考虑到项目的标题是相对较短的文本，因此我们限制了这些标题的主题分配。同时，我们采用相同的主题分布来建模项目的标题和图像之间的关系。为了了解消费者的偏好，所提出的模型根据标题中的文本单词和图像中的视觉特征提取几个重要的维度。对亚马逊数据集的实验表明，所提出的模型在学习消费者偏好的任务上优于其他基线模型。我们的研究结果对于管理者从细粒度和多模式的角度理解用户购买行为背后的个性化兴趣具有重要意义。