Privacy-preserving federated learning AdaBoost via knowledge distillation

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

There are many scenarios where different entities would like to share their knowledge through building a joint machine learning model for a common goal. Federated learning arises to solve such problems when the data privacy of each entity is imposed as an essential requirement. Although much work has been done in applying deep learning networks to federated learning scenarios, scarce effort has been made to adapt other classic and lighter machine learning algorithms to the aforementioned problem. This paper presents an adaptation of the AdaBoost ensemble method to federated scenarios, using ensemble knowledge distillation. To do so, in each iteration, the clients build a model using their local data and send the predictions over a small unlabeled public dataset to the server, thus keeping their sensitive information private. Then, the server aggregates these predictions and creates a model using the public data, sending it back to all the clients. The paper also presents and analyzes several versions of such an algorithm. The performance of the proposed method has been tested using 14 datasets, three evaluation metrics, and four data distribution scenarios. It demonstrates significantly better performance than other state-of-the-art methods, better privacy protection, and robustness against adversarial attacks on the network.