Skip to main content
SpringerLink
Log in

SEHP: stacking-based ensemble learning on novel features for review helpfulness prediction

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

The review’s helpfulness and its impact on purchase decisions are well established. This study presents a robust helpfulness prediction model for customer reviews. To this end, significant review textual features and newly defined reviewer characteristics are explored with a stacking-based ensemble model. More specifically, stylistic, time complexity, summary language, psychological, and linguistics features are introduced. According to our knowledge, these features are not explored earlier with the stacking-based ensemble model for review helpfulness prediction. The proposed predictive model is evaluated on three benchmark Amazon review datasets, consisting of 200,979 reviews in total. Two algorithms are proposed to help readers for understanding the methodology and researchers to regenerate the results. We compared several machine-learning, stacking-based ensemble, and 1-dimenional convolutional neural network (1D CNN) models. The stacking-based ensemble model shows benchmark performance by obtaining 0.009 mean square error with a hybrid combination of the proposed (reviewer and textual) features. Moreover, the proposed model outperformed five baselines including the fine-tuned BERT (Bidirectional Encoder Representations from Transformers) model by reducing mean square error by 40%. The results show that review textual features are better predictors than reviewer features as a standalone model. The findings of this article have significant implications for the researchers and the business owners.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Mudambi SM and Schuff D (2010) Research note: what makes a helpful online review? A study of customer reviews on Amazon. com. MIS quarterly, pp 185–200.

  2. Zhang Y, Lin Z (2018) Predicting the helpfulness of online product reviews: A multilingual approach. Electron Commer Res Appl 27:1–10

    Article  Google Scholar 

  3. Younas MZ, Malik MSI, Ignatov DI (2023) Automated defect identification for cell phones using language context, linguistic and smoke-word models. Expert Syst Appl 227:120236

    Article  Google Scholar 

  4. Malik M, Hussain A (2017) Helpfulness of product reviews as a function of discrete positive and negative emotions. Comput Hum Behav 73:290–302

    Article  Google Scholar 

  5. Singh JP et al (2017) Predicting the “helpfulness” of online consumer reviews. J Bus Res 70:346–355

    Article  Google Scholar 

  6. Krishnamoorthy S (2015) Linguistic features for review helpfulness prediction. Expert Syst Appl 42(7):3751–3759

    Article  Google Scholar 

  7. Malik M, Iqbal K (2018) Review helpfulness as a function of Linguistic Indicators. Int J Comput Sci Netw Secur 18(1):234–240

    Google Scholar 

  8. Malik M, Hussain A (2020) Exploring the influential reviewer, review and product determinants for review helpfulness. Artif Intell Rev 53(1):407–427

    Article  MathSciNet  Google Scholar 

  9. Malik M, Hussain A (2018) An analysis of review content and reviewer variables that contribute to review helpfulness. Inf Process Manage 54(1):88–104

    Article  Google Scholar 

  10. Saumya S, Singh JP, Dwivedi YK (2019) Predicting the helpfulness score of online reviews using convolutional neural network. Soft Comput 63:1–17

    Google Scholar 

  11. Xiao Y et al. (2022) Modeling review helpfulness with augmented transformer neural networks. In: 2022 IEEE 16th international conference on semantic computing (ICSC). IEEE.

  12. Spencer M et al (2019) Understanding the influence of text complexity and question type on reading outcomes. Read Writ 32(3):603–637

    Article  Google Scholar 

  13. Verma PK et al (2021) WELFake: word embedding over linguistic features for fake news detection. IEEE Trans Comput Social Syst 8(4):881–893

    Article  Google Scholar 

  14. Malik MSI, Imran T, Mamdouh JM (2023) How to detect propaganda from social media? exploitation of semantic and fine-tuned language models. Peer J Comput Sci 9:e1248

    Article  Google Scholar 

  15. Moon S, Kim M-Y, Iacobucci D (2021) Content analysis of fake consumer reviews by survey-based text categorization. Int J Res Mark 38(2):343–364

    Article  Google Scholar 

  16. Holtgraves T (2011) Text messaging, personality, and the social context. J Res Pers 45(1):92–99

    Article  Google Scholar 

  17. Saeed NM et al (2020) An enhanced feature-based sentiment analysis approach. Wiley Interdiscip Rev Data Min Knowl Discov 10(2):e1347

    Article  Google Scholar 

  18. Bilal M, Almazroi AA (2022) Effectiveness of fine-tuned BERT model in classification of helpful and unhelpful online customer reviews. Electr Commer Res 45:1–21

    Google Scholar 

  19. Cao Q, Duan W, Gan Q (2011) Exploring determinants of voting for the “helpfulness” of online user reviews: a text mining approach. Decis Support Syst 50(2):511–521

    Article  Google Scholar 

  20. Lee S, Choeh JY (2014) Predicting the helpfulness of online reviews using multilayer perceptron neural networks. Expert Syst Appl 41(6):3041–3046

    Article  Google Scholar 

  21. Hu Y-H, Chen K (2016) Predicting hotel review helpfulness: The impact of review visibility, and interaction between hotel stars and review ratings. Int J Inf Manage 36(6):929–944

    Article  Google Scholar 

  22. Liu Y et al (2008) Modeling and predicting the helpfulness of online reviews. In: Liu Y (ed) 2008 Eighth IEEE international conference on data mining. IEEE, Singapore

    Google Scholar 

  23. Luo Y, Xu X (2019) Predicting the helpfulness of online restaurant reviews using different machine learning algorithms: a case study of yelp. Sustainability 11(19):5254

    Article  Google Scholar 

  24. Ullah R, Zeb A, Kim W (2015) The impact of emotions on the helpfulness of movie reviews. J Appl Res Technol 13(3):359–363

    Article  Google Scholar 

  25. Felbermayr A, Nanopoulos A (2016) The role of emotions for the perceived usefulness in online customer reviews. J Interact Mark 36:60–76

    Article  Google Scholar 

  26. Xu D et al (2022) Emotions for attention in online consumer reviews: the moderated mediating role of review helpfulness. Indust Manag Data Syst 122(3):729–751

    Article  MathSciNet  Google Scholar 

  27. Xu C, Zheng X, Yang F (2023) Examining the effects of negative emotions on review helpfulness: the moderating role of product price. Comput Hum Behav 139:107501

    Article  Google Scholar 

  28. Qazi A et al (2016) A concept-level approach to the analysis of online review helpfulness. Comput Hum Behav 58:75–81

    Article  MathSciNet  Google Scholar 

  29. Yang S, Yao J, Qazi A (2020) Does the review deserve more helpfulness when its title resembles the content? locating helpful reviews by text mining. Inf Process Manage 57(2):102179

    Article  Google Scholar 

  30. Dong H et al (2020) Method for ranking the helpfulness of online reviews based on so-iles todim. IEEE Access 9:1723–1736

    Article  Google Scholar 

  31. Du J et al (2020) An interactive network for end-to-end review helpfulness modeling. Data Sci Eng 5(3):261–279

    Article  Google Scholar 

  32. Malik MSI (2020) Predicting users’ review helpfulness: the role of significant review and reviewer characteristics. Soft Comput 24(18):13913–13928

    Article  Google Scholar 

  33. Olmedilla M, Martínez-Torres MR, Toral S (2022) Prediction and modelling online reviews helpfulness using 1D Convolutional Neural Networks. Expert Syst Appl 198:116787

    Article  Google Scholar 

  34. Boluki A, Sharami JPR and Shterionov D (2023) Evaluating the effectiveness of pre-trained language models in predicting the helpfulness of online product reviews. arXiv preprint arXiv:2302.10199,.

  35. Mauro N, Ardissono L, Petrone G (2021) User and item-aware estimation of review helpfulness. Inf Process Manage 58(1):102434

    Article  Google Scholar 

  36. Lee S, Lee S, Baek H (2021) Does the dispersion of online review ratings affect review helpfulness? Comput Hum Behav 117:106670

    Article  Google Scholar 

  37. Filieri R, Galati F, Raguseo E (2021) The impact of service attributes and category on eWOM helpfulness: an investigation of extremely negative and positive ratings using latent semantic analytics and regression analysis. Comput Hum Behav 114:106527

    Article  Google Scholar 

  38. Lee M, Kwon W, Back K-J (2021) Artificial intelligence for hospitality big data analytics: developing a prediction model of restaurant review helpfulness for customer decision-making. Int J Contemp Hosp Manag 33(6):2117–2136

    Article  Google Scholar 

  39. Bilal M et al (2021) Profiling reviewers’ social network strength and predicting the “Helpfulness” of online customer reviews. Electron Commer Res Appl 45:101026

    Article  Google Scholar 

  40. Du J et al (2021) Neighbor-aware review helpfulness prediction. Decis Support Syst 148:113581

    Article  Google Scholar 

  41. Han MM (2022) How does mobile device usage influence review helpfulness through consumer evaluation? Evid Trip Advisor Decision Support Syst 153:113682

    Article  Google Scholar 

  42. Lutz B, Pröllochs N, Neumann D (2022) Are longer reviews always more helpful? Disentangling the interplay between review length and line of argumentation. J Bus Res 144:888–901

    Article  Google Scholar 

  43. Kashyap R, Kesharwani A, Ponnam A (2022) Measurement of online review helpfulness: A formative measure development and validation. Electr Commer Res 56:1–34

    Google Scholar 

  44. Chou Y-C, Chuang HH-C, Liang T-P (2022) Elaboration likelihood model, endogenous quality indicators, and online review helpfulness. Decis Support Syst 153:113683

    Article  Google Scholar 

  45. Wu R et al (2022) The influence of emoji meaning multipleness on perceived online review helpfulness: the mediating role of processing fluency. J Bus Res 141:299–307

    Article  Google Scholar 

  46. Yang Y, Wang Y, Zhao J (2023) Effect of user-generated image on review helpfulness: perspectives from object detection. Electron Commer Res Appl 57:101232

    Article  Google Scholar 

  47. Pajola L et al (2023) A novel review helpfulness measure based on the user-review-item paradigm. ACM Transactions on the Web.

  48. Malik MSI (2020) Predicting users’ review helpfulness: the role of significant review and reviewer characteristics. Soft Comput 12:1–16

    Google Scholar 

  49. Pennebaker JW, Booth RJ, and Francis ME (2007) LIWC2007: Linguistic inquiry and word count. Austin, Texas: liwc. net.

  50. Devlin J, et al. (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

  51. Daud A et al (2015) Using machine learning techniques for rising star prediction in co-author network. Scientometrics 102(2):1687–1711

    Article  Google Scholar 

  52. Amjad M et al (2021) Threatening language detection and target identification in Urdu tweets. IEEE Access 9:128302–128313

    Article  Google Scholar 

  53. Nawaz A, Malik M (2022) Rising stars prediction in reviewer network. Electron Commer Res 22(1):53–75

    Article  Google Scholar 

  54. Mehboob A, Malik M (2021) Smart fraud detection framework for job recruitments. Arab J Sci Eng 46(4):3067–3078

    Article  Google Scholar 

  55. Abbas Y, Malik M (2021) Defective products identification framework using online reviews. Electr. Commerce Res 45:1–22

    Google Scholar 

  56. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    Article  Google Scholar 

  57. Freund Y, Schapire R, Abe N (1999) A short introduction to boosting. J Jpn Soc Artif Intell 14(771–780):1612

    Google Scholar 

  58. Wolpert DH (1992) Stacked generalization. Neural Netw 5(2):241–259

    Article  Google Scholar 

  59. Saeed RM, Rady S, Gharib TF (2022) An ensemble approach for spam detection in Arabic opinion texts. J King Saud Univ Comput Inf Sci 34(1):1407–1416

    Google Scholar 

  60. Kim SJ, Maslowska E, Malthouse EC (2018) Understanding the effects of different review features on purchase probability. Int J Advert 37(1):29–53

    Article  Google Scholar 

Download references

Acknowledgements

This article is an output of a research project implemented as part of the Basic Research Program at the National Research University Higher School of Economics (HSE University). Moreover, this research was supported in part by computational resources of HPC facilities at HSE University.

Author information

Authors and Affiliations

  1. Department of Computer Science, National Research University Higher School of Economics, 11 Pokrovskiy Boulevard, Moscow, Russia, 109028

    Muhammad Shahid Iqbal Malik

  2. Department of Computer Science, Comsats University Islamabad, Attock Campus, Attock, Pakistan

    Aftab Nawaz

Authors
  1. Muhammad Shahid Iqbal Malik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aftab Nawaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.S.I.M. was involved in conceptualization, data curation, investigation, methodology, software, supervision; M.S.I.M. and A.N. helped in validation; visualization was done by M.S.I.M.; A.N. contributed to writing—original draft; M.S.I.M contributed to writing—review and editing.

Corresponding author

Correspondence to Muhammad Shahid Iqbal Malik.

Ethics declarations

Conflict of interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malik, M.S.I., Nawaz, A. SEHP: stacking-based ensemble learning on novel features for review helpfulness prediction. Knowl Inf Syst 66, 653–679 (2024). https://doi.org/10.1007/s10115-023-02020-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-02020-3

Keywords

Search

Navigation