Models with random effects, such as generalised linear mixed models (GLMMs), are often used for analysing clustered data. Parameter inference with these models is difficult because of the presence of cluster-specific random effects, which must be integrated out when evaluating the likelihood function. Here, we propose a sequential variational Bayes algorithm, called Recursive Variational Gaussian Approximation for Latent variable models (R-VGAL), for estimating parameters in GLMMs. The R-VGAL algorithm operates on the data sequentially, requires only a single pass through the data, and can provide parameter updates as new data are collected without the need of re-processing the previous data. At each update, the R-VGAL algorithm requires the gradient and Hessian of a “partial” log-likelihood function evaluated at the new observation, which are generally not available in closed form for GLMMs. To circumvent this issue, we propose using an importance-sampling-based approach for estimating the gradient and Hessian via Fisher’s and Louis’ identities. We find that R-VGAL can be unstable when traversing the first few data points, but that this issue can be mitigated by introducing a damping factor in the initial steps of the algorithm. Through illustrations on both simulated and real datasets, we show that R-VGAL provides good approximations to posterior distributions, that it can be made robust through damping, and that it is computationally efficient.

具有随机效应的模型，例如广义线性混合模型（GLMM），通常用于分析聚类数据。由于存在特定于簇的随机效应，这些模型的参数推断很困难，在评估似然函数时必须将其积分出来。在这里，我们提出了一种顺序变分贝叶斯算法，称为潜变量模型的递归变分高斯近似（R-VGAL），用于估计 GLMM 中的参数。 R-VGAL 算法按顺序对数据进行操作，仅需要一次遍历数据，并且可以在收集新数据时提供参数更新，而无需重新处理先前的数据。每次更新时，R-VGAL 算法都需要在新观察中评估的“部分”对数似然函数的梯度和 Hessian 函数，而 GLMM 通常无法以封闭形式提供这些函数。为了解决这个问题，我们建议使用基于重要性采样的方法通过 Fisher 和 Louis 恒等式来估计梯度和 Hessian 矩阵。我们发现 R-VGAL 在遍历前几个数据点时可能不稳定，但可以通过在算法的初始步骤中引入阻尼因子来缓解这个问题。通过模拟和真实数据集的插图，我们表明 R-VGAL 提供了对后验分布的良好近似，它可以通过阻尼变得鲁棒，并且计算效率高。