Skip to main content
SpringerLink
Log in

Silently killing your panelists one email at a time: The true cost of email solicitations

  • Original Empirical Research
  • Published:
Journal of the Academy of Marketing Science Aims and scope Submit manuscript

Abstract

Marketing firms routinely interact with their panelists via email. While sending an invitation to respond to a survey may seem virtually costless, over-solicitation could lead to panelists unsubscribing or ignoring future emails. Since online panels are a crucial resource for a marketing research firm, such attrition is a major issue. We account for the unobserved cost of solicitations in a joint model of response and attrition propensities. Using a data set of more than 150,000 email solicitations sent over three years, we demonstrate that additional solicitations not only temporarily decrease the likelihood of future participation but also increase the attrition rate, likely due to wearout. The model where solicitations “kill” panelists outperforms out of sample a benchmark model that assumes dropout is caused by the passage of time instead. Since the impact of solicitations is both transient (on the response model) and permanent (on the dropout process), managers should wait for the temporary impact to dissipate before risking to “kill” their panelists with another solicitation. We illustrate the economic importance of this finding using a differential evolution method that optimizes the firm’s solicitation strategy under different scenarios and show a 30.7% improvement. In the long term, a greedy strategy (targeting the best-responding panelists) performs worse than a random policy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The data set was acquired from a European marketing research company specialized in managing an online panel, who wishes to remain anonymous. The company has not given permission for researchers to publicly share the data. 

Notes

  1. Note that none of the interviewees cited attention checks as effective mechanisms to detect fraud, since simple attention checks are easy to spot, and complex ones may cause false positives.

  2. Philippe Guilbert, member of the ESOMAR Executive Committee and consultant in market research methodologies.

  3. In our interviews, when asked about respondents’ motivation to participate in an online panel, respondents’ “good will” was cited as often as financial rewards. As anecdotal evidence, recruitment landing pages use words such as “share your voice, inspire change, influence your world” (Toluna), “voice your opinion” (Kantar), or “Share your opinion, shape the news” (YouGov), while financial rewards are mentioned but not as a primary motivation.

  4. Our setting contrasts with the classic CLV setting. Continuous-time BTYD model, such as the Pareto/NBD model, assumes that purchases can occur at any point in time, e.g., buying a CD as in the original application. This assumption is not valid in our setup. Panelists can only respond to a survey if they are invited by the company to do so. In discrete-time BTYD models such as the BG/BB model, solicitations are received at evenly-spaced time intervals. This is not the case either in our setup, as the firm can send solicitations at any point in time.

  5. Web Appendix WA1 provides further details on the specification of priors and model estimation.

  6. The model specification is: \(logit(y)={b}_{0}+{b}_{1}log(({t}_{So{l}_{it}} - {t}_{So{l}_{i,0}})+1)+{b}_{2}log(Nso{l}_{it})+{b}_{3} log(({t}_{So{l}_{it}}-{t}_{So{l}_{i,0}})+1)*log(Nso{l}_{it})+{\epsilon }_{it}.\) The parameter estimates of the main effects are \(b_1=.135;\;\left(p<.01\right);\;b_2=.246;\;(p<.01).\)

  7. The model specification is: \(logit(y)={b}_{0}+{b}_{1}log(({t}_{So{l}_{it}} - {t}_{So{l}_{i,t-1}})+1)+{b}_{2}log(Nso{l}_{it})+{b}_{3} log(({t}_{So{l}_{it}}-{t}_{So{l}_{i,t-1}})+1)*log(Nso{l}_{it})+{\epsilon }_{it}\). We remove waiting times of 50 weeks or more from the data set when running the regression. Results are robust when including these observations in the regression. The parameter estimates of the main effects are \(b_1=-.127;\;\left(p<.01\right);\;b_2=-.554;\;(p<.01)\).

  8. The three exceptions are the parameters assessing the standard deviation of the dropout rate, and the correlations between the dropout rate and the impact of solicitations and responses in Model 3. The Gelman-Rubin statistic is slightly above 1.1 for these parameters.

  9. We also tested a Markov model with three hidden states (a high-response state, a low-response state, and an inactive state). The model does not converge and is not identifiable.

  10. This time horizon is important because any algorithm needs a stopping point which implicitly decides the solicit patterns. While infinite time horizons are not implementable computationally, shorter time horizons tend to be myopic. Any implementation needs to recognize this fundamental trade-off between implementability and potential myopia.

References

  • Abe, M. (2009). “Counting your customers” one by one: A hierarchical Bayes extension to the Pareto/NBD model. Marketing Science, 28(3), 541–553.

    Article  Google Scholar 

  • Ahler, D., Roush, C., & Sood, G. (2019). The micro-task market for lemons: Data quality on Amazon’s Mechanical Turk. Political Science Research and Methods, 1–20.

  • Ascarza, E., & Hardie, B. G. (2013). A joint model of usage and churn in contractual settings. Marketing Science, 32(4), 570–590.

    Article  Google Scholar 

  • Ascarza, E., Netzer, O., & Hardie, B. G. (2018). Some customers would rather leave without saying goodbye. Marketing Science, 37(1), 54–77.

    Article  Google Scholar 

  • Bachmann, P., Meierer, M., & Näf, J. (2021). The role of time-varying contextual factors in latent attrition models for customer base analysis. Marketing Science, 40(4), 783–809.

    Article  Google Scholar 

  • Balakrishnan, P. V., & Jacob, V. S. (1996). Genetic algorithms for product design. Management Science, 42(8), 1105–1117.

    Article  Google Scholar 

  • Bauer, E. (2018). 15 outrageous spam statistics that still ring true in 2018. https://www.propellercrm.com/blog/email-spam-statistics, last accessed on December 11, 2023.

  • Bemmaor, A. C., & Glady, N. (2012). Modeling purchasing behavior with sudden “death”: A flexible customer lifetime model. Management Science, 58(5), 1012–1021.

    Article  Google Scholar 

  • Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. arXiv preprint arXiv:1701.02434.

  • Callegaro, M., & DiSogra, C. (2008). Computing response metrics for online panels. Public Opinion Quarterly, 72(5), 1008–1032.

    Article  Google Scholar 

  • Campbell, D., Erdahl, R., Johnson, D., Bibelnieks, E., Haydock, M., Bullock, M., & Crowder, H. (2001). Optimizing customer mail streams at Fingerhut. Interfaces, 31(1), 77–90.

    Article  Google Scholar 

  • Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M. A., Guo, J., Li, P., & Riddell, A. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76(1).

  • Chittenden, L., & Rettie, R. (2003). An evaluation of e-mail marketing and factors affecting response. Journal of Targeting, Measurement and Analysis for Marketing, 11, 203–217.

    Article  Google Scholar 

  • Das, S., & Suganthan, P. N. (2010). Differential evolution: A survey of the state-of-the-art. IEEE Transactions on Evolutionary Computation, 15(1), 4–31.

    Article  Google Scholar 

  • Dennis, S. A., Goodson, B. M., & Pearson, C. A. (2020). Online worker fraud and evolving threats to the integrity of MTurk data: A discussion of virtual private servers and the limitations of IPbased screening procedures. Behavioral Research in Accounting, 32(1), 119–134.

    Article  Google Scholar 

  • Elliott, M. T., & Speck, P. S. (1998). Consumer perceptions of advertising clutter and its impact across various media. Journal of Advertising Research, 38(1), 29–30.

    Google Scholar 

  • Fader, P. S., Hardie, B. G., & Lee, K. L. (2005). “Counting your customers” the easy way: An alternative to the Pareto/NBD model. Marketing Science, 24(2), 275–284.

    Article  Google Scholar 

  • Fader, P. S., Hardie, B. G., Liu, Y., Davin, J., & Steenburgh, T. (2018). “How to project customer retention” revisited: The role of duration dependence. Journal of Interactive Marketing, 43(1), 1–16.

    Article  Google Scholar 

  • Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7(4), 457–472.

    Article  Google Scholar 

  • Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis. CRC Press.

    Book  Google Scholar 

  • Gopalakrishnan, A., Bradlow, E. T., & Fader, P. S. (2017). A cross-cohort changepoint model for customer-base analysis. Marketing Science, 36(2), 195–213.

    Article  Google Scholar 

  • Göritz, A. S., et al. (2012). Using Online Panels in Psychological Research. In A. Joinson (Ed.), Oxford Handbook of Internet Psychology. Oxford Library of Psychology.

    Google Scholar 

  • Hill, R. P., & Watkins, A. (2009). The profit implications of altruistic versus egoistic orientations for business-to-business exchanges. International Journal of Research in Marketing, 26(1), 52–59.

    Article  Google Scholar 

  • Hogg, A. (2003). Web efforts energize customer research. Electric Perspectives, 28(5), 81–83.

    Google Scholar 

  • Holland, J. H. (1992). Genetic algorithms. Scientific American, 267(1), 66–73.

    Article  Google Scholar 

  • Huang, J., Leng, M., & Liang, L. (2012). Recent developments in dynamic advertising research. European Journal of Operational Research, 220(3), 591–609.

    Article  Google Scholar 

  • Kanich, C., Kreibich, C., Levchenko, K., Enright, B., Voelker, G. M., Paxson, V., & Savage, S. (2008). Spamalytics: An empirical analysis of spam marketing conversion. In Proceedings of the 15th ACM conference on Computer and communications security (pp. 3–14)

    Google Scholar 

  • Kennedy, Ryan, Clifford, Scott, Burleigh, Tyler, Waggoner, Philip D., Jewell, Ryan, & Winter, Nicholas J. G. (2020). The shape of and solutions to the MTurk quality crisis. Political Science Research and Methods, 8, 614–629.

    Article  Google Scholar 

  • Liu, Q., Otter, T., & Allenby, G. M. (2007). Investigating endogeneity bias in marketing. Marketing Science, 26(5), 642–650.

    Article  Google Scholar 

  • Mapp. (2016). Consumer views of email marketing. http://www.kateycharles.com/wp-content/uploads/2016/11/Mapp_Consumer_Views_Survey_2016.pdf, Last accessed on December 11, 2023.

    Google Scholar 

  • Naik, P., & Piersma, N. (2002). Understanding the role of marketing communications in direct marketing, No. EI 2002-13.

  • Neslin, S. A., Novak, T. P., Baker, K. R., & Hoffman, D. L. (2009). An optimal contact model for maximizing online panel response rates. Management Science, 55(5), 727–737.

    Article  Google Scholar 

  • Netzer, O., Lattin, J. M., & Srinivasan, V. (2008). A hidden Markov model of customer relationship dynamics. Marketing Science, 27(2), 185–204.

    Article  Google Scholar 

  • Omnisend. (2018). The best email frequency for small business marketing. In Ecommerce Email Marketing and Statistics.

    Google Scholar 

  • Platzer, M., & Reutterer, T. (2016). Ticking away the moments: Timing regularity helps to better predict customer activity. Marketing Science, 35(5), 779–799.

    Article  Google Scholar 

  • Porter, C. O. L. H., et al. (2019). The Use of Online Panel Data in Management Research: A Review and Recommendations. Journal of management, 45(1), 319–344.

    Article  Google Scholar 

  • Postoaca, A. (2006). The anonymous elect: Market research through online access panels. Springer Science & Business Media.

    Google Scholar 

  • Price, K., Storn, R. M., & Lampinen, J. A. (2006). Differential evolution: a practical approach to global optimization. Springer Science & Business Media.

    Google Scholar 

  • Prolific. (2021). https://www.prolific.co/blog/we-recently-went-viral-on-tiktok-heres-what-we-learned. Last accessed on December 11, 2023.

  • Reutterer, T., Platzer, M., & Schröder, N. (2021). Leveraging purchase regularity for predicting customer behavior the easy way. International Journal of Research in Marketing, 38(1), 194–215.

    Article  Google Scholar 

  • Romero, J., Van der Lans, R., & Wierenga, B. (2013). A partially hidden Markov model of customer dynamics for CLV measurement. Journal of Interactive Marketing, 27(3), 185–208.

    Article  Google Scholar 

  • Schmittlein, D. C., Morrison, D. G., & Colombo, R. (1987). Counting your customers: Who-are they and what will they do next? Management Science, 33(1), 1–24.

    Article  Google Scholar 

  • Schröder, N., & Hruschka, H. (2016). Investigating the effects of mailing variables and endogeneity on mailing decisions. European Journal of Operational Research, 250(2), 579–589.

    Article  Google Scholar 

  • Schwartz, E. M., Bradlow, E. T., & Fader, P. S. (2014). Model selection using database characteristics: Developing a classification tree for longitudinal incidence data. Marketing Science, 33(2), 188–205.

    Article  Google Scholar 

  • Schweidel, D. A., & Knox, G. (2013). Incorporating direct marketing activity into latent attrition models. Marketing Science, 32(3), 471–487.

    Article  Google Scholar 

  • Shirley, K. E. (2007). Hidden Markov models for alcoholism treatment trial data. University of Pennsylvania.

    Google Scholar 

  • Shirley, K. E., Small, D. S., Lynch, K. G., Maisto, S. A., & Oslin, D. W. (2010). Hidden Markov Models for alcoholism treatment trial data. The Annals of Applied Statistics, 4(1), 366–395.

    Article  Google Scholar 

  • Singh, S. S., Borle, S., & Jain, D. C. (2009). A generalized framework for estimating customer lifetime value when customer lifetimes are not observed. Quantitative Marketing and Economics, 7, 181–205.

    Article  Google Scholar 

  • Smith, S. M., et al. (2016). A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples. Journal of business research, 69(8), 3139–3148.

    Article  Google Scholar 

  • Statista. (2023). https://www.statista.com/topics/1293/market-research/#topicOverview (last accessed: 6 Dec 2023).

  • Storn, R., & Price, K. (1996). Minimizing the real functions of the ICEC’96 contest by differential evolution. Proceedings of IEEE international conference on evolutionary computation (pp. 842–844). IEEE.

    Chapter  Google Scholar 

  • Storn, R., & Price, K. (1997). Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341.

    Article  Google Scholar 

  • SurveyMonkey. (2023). https://www.surveymonkey.com/mp/legal/esomar-37/. Last accessed on December 11, 2023

  • Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.

    Google Scholar 

  • The Radicati Group. (2019). Email statistics report 2015-2019. https://www.radicati.com/wp/wp-content/uploads/2015/02/Email-Statistics-Report-2015-2019-Executive-Summary.pdf, last accessed on December 11, 2023.

  • Tsafarakis, S., Marinakis, Y., & Matsatsinis, N. (2011). Particle swarm optimization for optimal product line design. International Journal of Research in Marketing, 28(1), 13–22.

    Article  Google Scholar 

  • Tsafarakis, S., Saridakis, C., Baltas, G., & Matsatsinis, N. (2013). Hybrid particle swarm optimization with mutation for optimizing industrial product lines: An application to a mixed solution space considering both discrete and continuous design variables. Industrial Marketing Management, 42(4), 496–506.

    Article  Google Scholar 

  • Vehtari, Aki, Gelman, Andrew, & Gabry, Jonah. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27, 1413–1432.

    Article  Google Scholar 

  • Venkatesan, R., Krishnan, T. V., & Kumar, V. (2004). Evolutionary estimation of macro-level diffusion models using genetic algorithms: An alternative to nonlinear least squares. Marketing Science, 23(3), 451–464.

    Article  Google Scholar 

  • Verhoef, P. C., Spring, P. N., Hoekstra, J. C., & Leeflang, P. S. (2003). The commercial use of segmentation and predictive modeling techniques for database marketing in the Netherlands. Decision Support Systems, 34(4), 471–481.

    Article  Google Scholar 

  • Veselovsky, V., Ribeiro, M. H., & West, R. (2023). Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks. arXiv preprint arXiv:2306.07899.

  • Wansink, B., & Sudman, S. (2002). Predicting the future of consumer panels. Journal of Database Marketing & Customer Strategy Management, 9, 301–311.

    Article  Google Scholar 

  • Watkins, C. (1989). Learning from delayed rewards, Ph. D. Thesis, King's College, Cambridge.

  • Zantedeschi, D., Feit, E. M., & Bradlow, E. T. (2017). Measuring multichannel advertising response. Management Science, 63(8), 2706–2728.

    Article  Google Scholar 

  • Zettasphere. (2016). Email frequency send sweet spot is 6.21 emails per week. https://www.zettasphere.com/email-frequency-send-sweet-spot-is-6-emails-per-week/, last accessed on December 11, 2023.

  • Zhang, X., Kumar, V., & Cosguner, K. (2017). Dynamically managing a profitable email marketing program. Journal of Marketing Research, 54(6), 851–866.

    Article  Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge the helpful comments of Bruce Hardie, Elisabeth Honka, Martina Pocchiari, Jason Roos, and participants at seminars at Rotterdam School of Management, Erasmus University, HEC-ESSEC-INSEAD conference, the ISMS Marketing Science Conference 2018, and the Interactive Marketing Research Conference 2020. We also thank the company who wishes to remain anonymous for sharing the data.

Author information

Authors and Affiliations

  1. Department of Marketing Management, Rotterdam School of Management, Erasmus University, Rotterdam, 3062 PA, Burgemeester Oudlaan 50, Netherlands

    Alina Ferecatu

  2. Department of Marketing, ESSEC Business School, Cergy, 95000, Avenue Bernard Hirsch, France

    Arnaud De Bruyn

  3. Department of Marketing, Amrut Mody School of Management, Ahmedabad University, Naranpark Society, Navarangapura, Ahmedabad, 380009, India

    Prithwiraj Mukherjee

Authors
  1. Alina Ferecatu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arnaud De Bruyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Prithwiraj Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alina Ferecatu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

V Kumar served as Area Editor for this article.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 708 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ferecatu, A., De Bruyn, A. & Mukherjee, P. Silently killing your panelists one email at a time: The true cost of email solicitations. J. of the Acad. Mark. Sci. (2024). https://doi.org/10.1007/s11747-023-00992-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11747-023-00992-w

Keywords

Search

Navigation