Federated Learning (FL) is a distributed learning approach that allows each client to retain its original data locally and share only the parameters of the local updates with the server. While FL can mitigate the problem of “data islands”, the training process involving non-independent and identically distributed (non-IID) data still faces the formidable challenge of model performance degradation due to “client drift” in practical applications. To address this challenge, in this paper, we design a novel approach termed “Two-fold Knowledge Distillation for non-IID Federated Learning” (FedTweet), meticulously designed for the personalized training of both local and global models within various heterogeneous data contexts. Specifically, the server employs global pseudo-data for fine-tuning the initial aggregated model through knowledge distillation and adopts dynamic aggregation weights for local generators based on model similarity to ensure diversity in global pseudo-data. Clients freeze the received global model as a teacher model and conduct adversarial training between the local model and local generator, thus preserving the personalized information in the local updates while correcting their directions. FedTweet enables both global and local models to serve as teacher models for each other, ensuring bidirectional guarantees for personalization and generalization. Finally, extensive experiments conducted on benchmark datasets demonstrate that FedTweet outperforms several previous FL methods on heterogeneous datasets.

联邦学习（FL）是一种分布式学习方法，允许每个客户端在本地保留其原始数据，并仅与服务器共享本地更新的参数。虽然FL可以缓解“数据孤岛”问题，但涉及非独立同分布（non-IID）数据的训练过程在实际应用中仍然面临着由于“客户端漂移”而导致模型性能下降的巨大挑战。为了应对这一挑战，在本文中，我们设计了一种新颖的方法，称为“非独立同分布联合学习的双重知识蒸馏”（FedTweet），专为本地和全球的个性化培训而设计各种异构数据上下文中的模型。具体来说，服务器利用全局伪数据通过知识蒸馏对初始聚合模型进行微调，并根据模型相似性对本地生成器采用动态聚合权重，以确保全局伪数据的多样性。客户端将收到的全局模型冻结为教师模型，并在本地模型和本地生成器之间进行对抗性训练，从而在纠正其方向的同时保留本地更新中的个性化信息。FedTweet 使全球和本地模型能够相互充当教师模型，从而确保个性化和泛化的双向保证。最后，在基准数据集上进行的大量实验表明，FedTweet 在异构数据集上优于之前的几种 FL 方法。