Machine Learning I

B2L-C: Machine Learning I

Session Type: Lecture
Session Code: B2L-C
Location: Room 3
Date & Time: Thursday March 23, 2023 (10:20-11:20)
Chair: Anqi Liu
Track: 5
Paper No. Paper NameAuthorsAbstract
3045Federated Learning with Server Learning for Non-IID DataVan Sy Mai{1}, Richard La{2}, Tao Zhang{1}, Yuxuan Huang{1}, Abdella Battou{1}Federated Learning (FL) has gained popularity as a means of distributed learning using local data samples at clients. However, recent studies showed that FL may experience slow learning and poor performance when client samples have different distributions. In this paper, we consider a server with access to a small dataset, on which it can perform its own learning. This approach is complementary to and can be combined with other approaches, e.g., sample sharing among clients. We study and demonstrate the benefits of proposed approach via experimental results obtained using two datasets - EMNIST and CIFAR10.
3112Personalized Decentralized Multi-Task Learning Over Dynamic Communication GraphsMatin Mortaheb, Sennur UlukusDecentralized and federated learning algorithms face data heterogeneity as one of the biggest challenges, especially when users want to learn a specific task. Even when personalized headers are used concatenated to a shared network (PF-MTL), aggregating all the networks with a decentralized algorithm can result in performance degradation as a result of heterogeneity in the data. Our algorithm uses exchanged gradients to calculate the correlations among tasks automatically, and dynamically adjusts the communication graph to connect mutually beneficial tasks and isolate those that may negatively impact each other. This algorithm improves the learning performance and leads to faster convergence compared to the case where all clients are connected to each other regardless of their correlations. We conduct experiments on a synthetic Gaussian dataset and a large-scale celebrity attributes (CelebA) dataset. The experiment with the synthetic data illustrates that our proposed method is capable of detecting tasks that are positively and negatively correlated. Moreover, the results of the experiments with CelebA demonstrate that the proposed method may produce significantly faster training results than fully-connected networks.
3155Greedy Centroid Initialization for Federated K-MeansKun Yang{2}, Mohammad Mohammadi Amiri{1}, Sanjeev R. Kulkarni{2}K-means is a widely-adapted data clustering algorithm which aims to partition the set of data points to $K$ clusters through finding the best $K$ centroids representing the data points. Initialization plays a vital role in the traditional centralized K-means clustering algorithm where the clustering is carried out at a central node accessing the entire data points. In this paper, we focus on K-means in a federated setting, where the clients store data locally, and the raw data never leaves the devices. Given the importance of initialization on the federated K-means algorithm, we aim to find better initial centroids by leveraging the local data on each client. To this end, we start the centroid initialization at the clients rather than at the server, which has no information about the clients\' data initially. The clients first select their local initial clusters, and they share their clustering information (cluster centroids and sizes) with the server. The server then uses a greedy algorithm to choose the global initial centroids based on the information received from the clients. Numerical results on synthetic and public datasets show that our proposed method can achieve better and more stable performance than three federated K-means variants, and similar performance to the centralized K-means algorithm.