Sustaining long-term smoking cessation is difficult because many quitters resume smoking due to mood problems, strong nicotine dependence, low self-efficacy, and lack of social support (Jaén et al. 2008). A recent modelling study on abstinence prevalence in 43 randomized controlled trials (RCT) showed that 12-week pharmaceutical interventions, including varenicline (quit rate: 32%), bupropion (20%), and nicotine replacement therapy (NRT) (16%) resulted in no more than half of smokers maintaining abstinence at 6-month follow-up, and about 40% of those who quit at the end of the pharmaceutical treatment relapsed within 12 months (Jackson et al. 2019).

Psychosocial support via mobile health (m-health) has shown potential for smoking relapse prevention by enhancing immediate help-seeking and delivering to quitters who encounter smoking cues or experience withdrawal symptoms in out-of-clinic environment (Selby et al. 2010). As of April 2023, we identified four pilot randomized controlled trials (RCTs) in PubMed on m-health psychosocial interventions using smartphone app (Hicks et al. 2017) or WhatsApp (Cheung et al. 2015; Durmaz et al. 2019; Luk et al. 2022) for smoking relapse prevention in smokers (Hicks et al. 2017; Durmaz et al. 2019) and quitters (Cheung et al. 2015; Luk et al. 2022). Two of these four were our trials in Hong Kong (Cheung et al. 2015; Luk et al. 2022). Two studies showed feasibility and acceptability in smokers (Hicks et al. 2017) and quitters (Luk et al. 2022). Chat-based psychosocial support using WhatsApp yielded higher self-reported 14-day point abstinence at 6-month follow-up (40.9% versus 22.7%, odds ratio (OR) = 2.31, 95% CI = 1.03 to 5.16, P < 0.05) compared to conventional counselling (Durmaz et al. 2019). Our pilot RCTs showed preliminary evidence that personalized text messaging via WhatsApp (versus generic text messaging, validated abstinence at 6 months: 31% vs. 22%, adjusted risk ratio = 1.72, 95%CI = 0.91 to 3.23, P = 0.09) (Luk et al. 2022) and group discussions (versus self-help booklet on smoking cessation, validated abstinence at 6 months: 26.2% vs. 14.8%, OR = 2.04, 95%CI = 0.74 to 5.65, P > 0.05) (Cheung et al. 2015) was effective to increase abstinence in quitters. Peer sharing through group discussion of smoking and quitting experiences could be more persuasive than expert advice (Coley et al. 2013), facilitate mutual support in smokers (Phua 2013), and strengthen quitting motivation and communication to quit (Cheung et al. 2015). However, evidence from these pilot trials (Hicks et al. 2017; Cheung et al. 2015; Durmaz et al. 2019; Luk et al. 2022) for the effectiveness of online group support for relapse prevention is only preliminary, and sample sizes were small (n = 11 to 136).

This RCT aimed to examine the effectiveness of the instant messaging-based online group support versus conventional text messaging for smoking relapse prevention in quitters who recently quit (abstinence for 3 to 30 days). We also examined if greater engagement in the participants who joined the WhatsApp groups would increase tobacco abstinence.

Methods

Study Design

We conducted a two-arm open-labelled pragmatic RCT in 11 smoking cessation (SC) clinics under Hospital Authority (N = 4), Tung Wah Group of Hospitals (N = 6), and Pok Oi Hospital (N = 1), which are the major SC service providers that provide free or low-cost interventions. The trial was designed to be pragmatic (Loudon et al. 2015), with an average score of a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) of 4.1 (Appendix Fig. 2). The trial protocol has been published (Cheung et al. 2020). The RCT was approved by the University of Hong Kong and the Hong Kong West Cluster of Hospital Authority (HKU/HA HKW) Institutional Review Board (UW 18–018) and was registered in the ClinicalTrials.gov registry (NCT03760224). Results were reported according to the Consolidated Standards of Reporting Trials (CONSORT).

Participants

All adult smokers who attended a usual SC counselling session were approached by the clinic counsellor or a research assistant. The inclusion criteria were (1) aged 18 years or above, (2) daily tobacco user before the service intake, (3) enrolling in the SC treatment for no more than 8 weeks, (4) not using any tobacco products in the past 3- to 30-day, (5) able to communicate in Cantonese/Putonghua and read Chinese, and (6) own a smartphone with the local network connection. Exclusion criteria included (1) not using WhatsApp as a communication tool and showing no interest in using WhatsApp, (2) having an unstable physical or psychological condition (i.e., depression, anxiety, and schizophrenia) as advised by doctors or counsellors, or (3) having become pregnant in the past 2 months.

Recruitment Procedures

The clinic counsellor or research assistant briefly introduced the importance of relapse prevention and our phone-based relapse prevention intervention to smokers. If the smokers agreed to join, the recruitment staff member would then assess their eligibility. Participants who met all the eligibility criteria were (1) introduced the RCT, that they would be randomized to either an intervention or control group; (2) invited to sign the consent form; (3) asked to complete a baseline questionnaire; and (4) offered a leaflet about the RCT, an 8-page self-help booklet about relapse prevention, and a small souvenir worthy of HK$20 (\(\approx\) US$2.6). Afterwards, the participants were informed their group allocation via telephone 1 week later.

Randomization, Allocation Concealment, and Blinding

Participants were randomly assigned individually to one of the two trial groups. The principal investigator generated a list of random numbers to create a list of random group allocations (either intervention or control; allocation ratio 1:1), using the rand function of Excel. We used sequentially numbered, opaque, sealed envelopes (SNOSE) to maintain concealment of the allocation sequence for both recruitment personnel and study participants prior to group assignment. The principal investigator prepared about 1100 uniform SNOSE, each assigned a distinct 4-digit serial number. Half of the envelopes contained cards designating the intervention group, while the remaining envelopes designated the control group. Upon enrolling a participant, the research staff member opened a single envelope and allocated the individual according to the enclosed card. Participants and group moderators were not blinded to the behavioral intervention. Outcome assessors, who were not involved in the recruitment process or intervention delivery, remained blinded to the group status of all participants.

Interventions

Our moderator created a WhatsApp discussion group for every 10 intervention group participants. Each group had an 8-week real-time discussion moderated by a trained nurse or counsellor, with receiving 3 weekly standardized messages (texts, videos, or pictures) addressing common smoking relapse issues in “Treatments for the Recent Quitter” of the US Clinical Practice Guidelines on Treating Tobacco Use and Dependence (A clinical practice guideline for treating tobacco use and dependence 2008). We chose this messaging frequency because these participants were receiving conventional SC treatment and too frequent messaging may be overwhelming. The groups followed four moderation principles (active listening, genuine appreciation, mindful group dynamics, and empowering individuals) and referred participants to SC services when needed (see our protocol for details (Cheung et al. 2020)).

The control group received 3 weekly text messages without videos and pictures for 8 weeks. The content was similar to the intervention group regarding the intervention dosage and time, but participants could not interact with each other. The study did not interfere with the existing cessation services in the SC clinics including counselling and pharmacotherapy.

Follow-Up of the Participants

All participants were interviewed via telephone by an allocation-blinded interviewer at 3-, 6-, and 12-month. They were offered a HK$50 (US$6.4) supermarket voucher upon for each follow-up assessment. Those who self-reported abstinence in the past 7 days at 6- and 12-month follow-up provided exhaled breath and/or saliva samples for validation, receiving HK$100 (US$12.8) cash for travel expenses. The validation incentive was unknown at enrolment.

Outcomes

In the original protocol, the primary outcome was tobacco abstinence validated by exhaled carbon monoxide (CO) (< 4 parts per million) and saliva cotinine (< 10 ng/mL), measured by both Bedfont Smokerlylzer and NicAlert strips (Nymox Pharmaceutical, St. Laurent, QC, Canada), respectively, at 12-month follow-up (Cropsey et al. 2014; Cooke et al. 2008). Due to the production shortage of the NicAlert strips since early 2020 amid the COVID-19 pandemic, we conducted the saliva cotinine validation with iScreen OFD Cotinine Saliva Test Kit, which defined saliva cotinine < 30 ng/mL as abstinence (Tourangeau et al. 2019). Amid the pandemic, exhaled breath collection by Bedfont Smokerlyzers was not feasible due to the increased risk of virus transmission. Therefore, only iScreen OFD Cotinine Saliva Test kit was used for validation. In the final analysis, validated abstinence included a pass in either the CO test or the salvia cotinine test, or both.

The secondary outcomes included (1) validated tobacco abstinence at 6-month follow-up; (2) self-reported 7-day and continuous abstinence at each follow-up, and (3) relapse rate, which was defined as the proportion of quitters who smoked at least 5 cigarettes in 3 consecutive days at each follow-up. The ancillary outcomes included (1) time to relapse (i.e., number of days from their enrolment date or the first day of abstinence to the date they relapsed); (2) number of posts made by participants in WhatsApp groups; (3) change in frequency and intensity of smoking urge; (4) change in the Minnesota Nicotine Withdrawal Scale (MNWS); and (5) health-related quality of life by EuroQoL 5-dimension 5-level (EQ-5D-5L) (Wong et al. 2018).

Data Analysis

Our previous RCT (Cheung et al. 2015) indicated an odds ratio (OR) of 2.04 (95%Cl 0.74–5.65) (intervention: 26.0%, control: 15.0%) for CO validated quit rates between the intervention and control group. The current RCT, focusing on recent quitters, estimated a conservative OR of 1.70 (23.0% vs. 15.0%). To detect a significant difference with 90% power and 5% significant level using G*power 3.1, we required 1008 participants (504 per group).

All data were analyzed with STATA 14.0 (StataCorp LLC). A 2-sided P < 0.05 indicated statistical significance. The primary and secondary outcomes were analyzed by intention-to-treat. Participants with missing outcomes due to loss of contact or refusal were treated as smokers with unchanged daily cigarette consumption from the baseline. Multiple imputation (MI) procedure assuming missing at random was conducted to impute the missing outcomes as sensitivity analysis. The assumption of homogeneity was confirmed by showing the clinic-by-intervention interaction was negligible (P > 0.1) with clinic as a fixed effect in the logistic model. We planned to calculate the odds ratio (OR) in the original protocol, but we switched to risk ratio (RR) by log-binomial regression because RR is more easily understood and appropriate when the study prevalence of outcome is higher than 10% (Cummings 2009). The number needed to treat (NNT), which shows the number of subjects needed to be treated to have one additional quitter at 12-month compared to those who were not treated (the control group), was computed by taking the reciprocal of the risk difference between the 2 trial groups. Linear mixed model (LMM) was used to determine the group-by-time interaction effect on the ancillary outcomes. The number of messages posted by each participant was analyzed as a measure of WhatsApp group engagement and then included in log-binominal regression and generalized estimating equations to assess their association with the cessation outcomes. The heterogeneity of smoking cessation outcome due to group moderator was assessed by tabulating tobacco abstinence at 6- and 12-month follow-up and moderator, and ANOVA F-test.

The intervention’s total cost included recruitment materials, manpower, and text messages delivery. We planned a within-trial and life-time cost-effectiveness analysis in the original protocol. As the intervention effect was not significant and the cost for the intervention delivery was greater than that of the control group, the life-time cost-effectiveness analysis was not done.

A within-trial cost effectiveness analysis was conducted, comparing costs per validated quitter in both groups. The incremental cost effectiveness ratio (ICER) in terms of cost per additional tobacco abstinence of 12-month gained and the quality of life years (QALYs) gained for the intervention group compared to the control group was calculated, i.e., ICER = (cost intervention – cost control)/(QALYs intervention – QALYs control). The intervention was determined as cost-effective if the ICER was less than 3 times the gross domestic product (GPD) per capita in Hong Kong, as recommended by WHO.

Results

Characteristics of Participants

From Oct 4, 2018, to January 17, 2020, 1260 participants were eligible at the first screening. After excluding 332 participants who did not quit for 3–30 days or were lost to further contact in the further screening, 928 participants were randomly allocated to the intervention (n = 469) and control (n = 459) group (Fig. 1). We only recruited 92% of 1008 as stated in the original protocol because the recruitment stopped since 17th January 2020 due to the COVID-19 pandemic and closure of the SC clinics. Baseline socio-demographic characteristics, smoking, and quitting history were similar between the two groups (Table 1). The mean age of the participants was 43.0 (SD 11.7) years, and about 79.7% were men. The retention rate at 3-, 6- and 12-month follow-up was 74.2%, 72.6%, and 72.5%, respectively, with no between group difference (P from 0.84–0.96). About 31.2% (144/461) and 26.4% (110/416) of participants who self-reported abstinence at 6 and 12 months, respectively, participated in the biochemical validation.

Fig. 1
figure 1

CONSORT diagram of the randomized controlled trial

Full size image
Table 1 Socio-demographic and smoking characteristics at baseline in the intervention and control groups
Full size table

Primary and Secondary Outcomes

Table 2 shows no significant between-group difference in biochemically validated tobacco abstinence at 12-month follow-up by intention-to-treat (11.9% vs. 11.7%, RR = 1.01, 95%CI, 0.71–1.44, P = 0.93, NNT = 270.1), and after adjustment for baseline socio-demographic factors. Appendix Table 4 shows similar results from the MI analysis.

Table 2 Primary and secondary outcomes, by intention-to-treat analysis in the intervention and control groups
Full size table

Table 2 shows no significant between-group differences in all secondary outcomes at the 3 follow-ups. Both groups had similar time to relapse (97.9 days [SD = 80.61] vs. 106.7 days [SD = 81.12], P = 0.48). In both groups, frequency (P < 0.001) and intensity (P < 0.001) of smoking urge, and MNWS (P < 0.001) significantly decreased during the follow-up period (Appendix Fig. 3a–c). In both groups, EQ-5D-5L showed an upward trend from baseline to 3-month follow-up (P < 0.001), which was sustained at 6- and 12-month follow-up (Appendix Fig. 3d). No significant group by time interaction effects in these four outcomes were found.

Intervention Group Participants’ Engagement in WhatsApp Group

Four hundred thirty-four participants (92.5%) in the intervention group joined the 42 WhatsApp groups. Thirty-five participants did not join because they refused (n = 24, i.e., being not interested or having no time to join) or their abstinence had been over 30 days when being allocated to the WhatsApp groups (n = 11). Four hundred thirty-four participants sent 7082 posts (median = 3, interquartile range = 12) in total. Three hundred twenty-four of 434 participants (74.7%) sent at least one post (including an emoji or a single word only). The proportion of 469 participants who sent zero (including the 35 participants who did not join the WhatsApp groups), 1–9, 10–19, 20, or above posts was 30.9% (n = 145), 39.0% (n = 183), 16.2% (n = 76), and 13.9% (n = 65), respectively. Moderators responded on average 20.4 posts (858 posts/42 groups) in each WhatsApp group.

Table 3 shows that in the intervention group, participants posting 20 or more posts had significantly greater validated abstinence than those who posted nothing (6 months: 27.7% versus 7.6%, adjusted risk ratio (ARR) 3.38, 95%CI 1.60 to 7.14, P = 0.001; 12 months: 23.1% versus 9.0%, ARR 2.32, 95% CI 1.11 to 4.89, P = 0.03), and also greater than the control group (6 months: 16.3%, ARR 1.67, 95%CI 1.01 to 2.75, P = 0.046; 12 months: 11.8%, ARR 2.26, 95%CI 1.24 to 4.12, P = 0.01) at both 6 and 12 months, respectively. Participants in the intervention group who did not send any posts had significantly lower validated abstinence than the control group (ARR: 0.52, 95%CI: 0.28–0.98, P = 0.043) at 6-month but not 12-month follow-up. No significant heterogeneity in the validated abstinence at 6- and 12-month across the subgroups of the 3 moderators (Appendix Table 5).

Table 3 Association between validated tobacco abstinence at 6- and 12-month follow-up by intention-to-treat and number of posts posted by participants in WhatsApp group discussion in the intervention and control groups
Full size table

Cost-Effectiveness Analysis

Appendix Table 6 shows that the cost of a validated quitter in the intervention group (USD 449) was 2.68 times greater than in the control group (USD 122) at 12-month follow-up. This was mainly due to the manpower cost as the total manpower cost in the intervention group (USD 23,028) was 3.44 times greater than in the control group (USD 5182). The ICER of cost per additional tobacco abstinence at 12 months gained and the QALY gained was USD 92,700 and USD 9,270,000, respectively. The ICER of cost per additional tobacco abstinence at 12-month gained was more than 0.86 times the GDP per capita in Hong Kong (USD 49,800.5) in 2021. The incremental cost per participant in intervention group amounted to USD39.53 compared to control group (total cost intervention group − total cost control group)/total number of participant intervention group). Based on the ED-5D-5L, the estimated QALY gained was 0.002 (intervention group 0.957, control group 0.955). The resulting ICER, USD 9,270,000, exceeded Hong Kong’s GDP per capita (USD 49,800.5) by 185 times in 2021.

Discussion

We have first shown by a pragmatic RCT that 8-week counsellor-moderated IM-based online discussion group intervention for relapse prevention was very costly but did not significant increase smoking abstinence compared to simple text messages (SMS) delivering the same contents in smoking cessation clinic users who had quit smoking within 3 to 30 days. The results suggest that more interactive online group interventions which cost a lot more do not necessarily increase smoking abstinence than simple and low-cost SMS.

Four factors might account for the lack of the anticipated intervention effectiveness. First, both the present RCT and our previous pilot RCT tested the same intervention messages, but the present RCT involved quitters who had quit within 3 to 30 days, which was a shorter duration than the quitters in our previous pilot RCT (Cheung et al. 2015) who had quit for 8 weeks. At early stage of quitting, some of self-reported quitters might not perceive themselves as truly successful and hence refrained from sharing their experiences with other smokers or quitters in the online group discussion. Moreover, some quitters, due to reasons such as being introverts or having social anxiety, might not be proactive or willing to share experiences with strangers online. Thus, the online group discussion, designed to encourage participants to share their cessation experiences, which could be positive or negative, may offer limited benefits in supporting quitters or overcoming relapse during the early stage of abstinence. Those who had abstinence for many weeks might be more willing to share their successes and to help others. Second, unlike the control group in our previous pilot RCT which received no additional intervention (Cheung et al. 2015), the control group received text messages delivering similar contents on relapse prevention which might be beneficial for smoking cessation. The 2019 Cochrane review concluded that automated text messaging interventions were more effective than minimal smoking cessation support (n = 14,133 participants, RR = 1.54, 96%CI = 1.19 to 2.00, P = 0.001) (Whittaker et al. 2019). Third, although quit rates from different trials cannot be directly compared, the quit rates in the present trial appeared to be quite high, as validated quit rate in the SMS group in our previous trial was 8% at 6-month follow-up (Wang et al. 2019). Our null findings suggest that text messaging could have reached the ceiling effect. As we did not have a control group with no intervention at all, our results should not be interpreted as evidence for ineffectiveness of IM-based online group intervention. Instead, the present and other evidence suggest that simple SMS might still be considered as effective (Whittaker et al. 2019). Fourth, some WhatsApp group discussions, 6-month and 12-month follow-up surveys were done during COVID-19 pandemic, which might have reduced abstinence (Bommelé et al. 2020; Grogan et al. 2022) and engagement in peer support strategies (Adams et al. 2022). A 2020 online survey (n = 957) (Bommelé et al. 2020) on Dutch current smokers found a dose–response effect of self-reported stress caused by the pandemic on increased smoking (n = 957 current smokers, OR = 2.37, 95% CI: 1.49 to 3.78, P < 0.001). Smokers who reported severe stress were more likely to increase their smoking compared to those with no stress (OR = 3.75, 95% CI: 1.84 to 7.64, P < 0.001). In addition, a thematic analysis of 132 UK current smokers also found that increased smoking was used as a coping mechanism to deal with stress in pandemic (Grogan et al. 2022). Moreover, a cross-sectional online survey on mental health peer supporters (n = 1280) in the USA found that a lower engagement in individual support provision (mean [SD], post vs. pre: 3.33 [1.42] vs. 4.06 [1.12], P < 0.001) and group facilitation (post vs. pre: 2.35 [1.47] vs. 3.07 [1.45], P < 0.001) during COVID-19 pandemic compared to pre-COVID-19 pandemic levels (Adams et al. 2022). This decline in peer support might make it more challenging for individuals to maintain abstinence during the pandemic.

We observed in the intervention group that those with very high group engagement (having sent 20 posts or more within 8 weeks) had a higher validated abstinence at 6-month follow-up than no-post participants and the control group. Such differences were also observed at 12-month follow-up, probably because the intervention effect from the 8-week intervention sustain after 6 months. We also found that the intervention group participants who posted nothing had a significantly lower quit rate than the control group at 6 months but not at 12 months. This small group had agreed to join but failed to interact could have felt guilty when they had given up their quit attempts after relapse and became unwilling to share their failure experiences. Hence, the effectiveness of engagement on quitting is not confirmed, as highly engaged participants were probably more motivated to quit and share their successful experiences with others. On the other hand, failure to quit may lead to disengagement of the group intervention. Nevertheless, some people who are highly sociable, prefer to engage with peers for support, and have attempted quitting without success may benefit from IM-based group support through sharing experience and receiving peer support. This approach may be particularly effective for individuals who possess better communication skills, enjoy interacting with others, and are more prepared to seek peer support in their cessation journey. Smoking cessation service providers should consider that many smokers are not proactive in online groups even after they have joined, but RCTs on such people are warranted. Future research on IM-based group support needs to consider, firstly, how to screen out those who are likely or unlikely to be benefited, what would be the optimal duration and efforts to increase engagement, whether positive or negative content sent by the participants influence abstinence, and assess whether the intervention could be cost effective. Intervention with additional interactive online group chat and discussion, moderated by trained nurses or counsellors, are quite costly, and robust evidence of cost-effectiveness from an RCT is needed before they are routinely implemented in smoking cessation services.

Limitations

This RCT recruited a large number of recent quitters from multiple SC clinics to have greater generalizability. As the participants were recent quitters receiving SC intervention from community-based SC clinics in Hong Kong, our results might not be applicable to unassisted quitters or those who do not use simple cessation service. We did not have a control group with no intervention at all because doing so might seem unethical. Hence, the effectiveness of the IM-based experimental intervention and the SMS control intervention could not be assessed separately. Future RCTs may consider adding a control group with no or minimal intervention. Furthermore, during the COVID pandemic, the participation rates of biochemical validation in the self-reported quitters were suboptimal because most quitters were worried of the infection risk or the social distance measures.

Conclusions

Our study showed that both WhatsApp group discussions and SMS text messages yielded similar abstinence outcomes in recent quitters in smoking cessation clinics. The results suggest that more intensive and hence costly interventions, such as WhatsApp group discussions moderated by trained professionals, do not necessarily lead to better outcomes compared to simpler and more affordable approaches like text messages delivering similar contents.