Federated Learning

Assumptions and Their Meaning in Optimization Problems

by ComputingNotes
1. Lipschitz Continuity: The “Speed Limit”This assumption prevents the function from changing too rapidly over a certain distance. 2. 𝐿-Smoothness: The Curvature CeilingSmoothness ensures the gradient (slope) of the function doesn’t change abruptly. A function is 𝐿-smooth if its gradient is Lipschitz continuous. 3. 𝜇-Strong Convexity : The Curvature Floor – Bowl ShapeStrong convexity guarantees that ... Read More

FL Algorithms FedAvg

by ComputingNotes
FedSGD – FedAVG H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” Jan. 26, 2023, arXiv: arXiv:1602.05629. doi: 10.48550/arXiv.1602.05629. In FedSGD, server sends current model $w_t$ to all devices and each device calculates the “slope” or gradient $g_k$ of its local data. ... Read More

Federated Learning Algorithm Averaging Methods FedAvgM, FedProx, FedAvg

by ComputingNotes
FedAvg: https://arxiv.org/abs/1602.05629 __________________________________________________________________________________________________________ FedAvgM: https://arxiv.org/pdf/1909.06335 stands for Federated Averaging with Server Momentum. It is an upgrade to the original FedAvg algorithm designed specifically to solve the “Client Drift” problemwhere the model gets confused because different users have very different data (Non-IID). The Formula: How it works mathematically The FedAvgM update happens in three steps on ... Read More
error: