Notes

Assumptions and Their Meaning in Optimization Problems

Federated Learning, Machine Learning
1. Lipschitz Continuity: The “Speed Limit”This assumption prevents the function from changing too rapidly over a certain distance. 2. 𝐿-Smoothness: The Curvature CeilingSmoothness ensures the gradient (slope) of the function doesn’t change abruptly. A function is 𝐿-smooth if its gradient is Lipschitz continuous. 3. 𝜇-Strong Convexity : The Curvature Floor – Bowl ShapeStrong convexity guarantees that ... Read More

Calculus – Linearization and Differentials

Calculus, Mathematics
Book Calculus by Thomas/Finney, 9th Edition, Page 248 Linearization: Approximating functions https://math.stackexchange.com/questions/1385936/approximating-sqrt1-frac1n-by-1-frac12n#:~:text=The%20function%20f(%20x)=%20%E2%88%9A%201+%20x,%E2%88%9A%201+%20x%20for%20x%20%E2%89%A5%200. Differentials Differential Estimate of Change:In calculus, estimating change with differentials is a method from calculus used to approximate how much a function’s output changes when the input changes by a small amount. Instead of calculating the exact change (which can be difficult and ... Read More

Quantum Information – Quantum Noise and Error

Quantum Computing
Book Quantum Computation and Quantum Information, Nielson and Chuang. Cambridge, Page 353 and Page426 Systems can be either closed or open sytems. All systems somehow have to interact with outside environment. These unwanted interactions from outside world can introduce noie to the information processing system.

WordNet?

Uncategorized
https://wordnet.princeton.edu https://direct.mit.edu/books/edited-volume/1928/WordNetAn-Electronic-Lexical-Database Wordnet is a lexical database for English.

Calculus – Application of Derivatives

Calculus, Mathematics
Book calculus by Thomas/Finney, 9th Edition, Page 189 Derivatives can be used to analyze the behaviour of functions and solve practical optimization problems. Max-Min Theorem relates to existence of max and min value for a function $f4$ that is continous at every point of closed interval I. Absoluted maxima and minima on a closed interval ... Read More

FL Algorithms FedAvg

Federated Learning
FedSGD – FedAVG H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” Jan. 26, 2023, arXiv: arXiv:1602.05629. doi: 10.48550/arXiv.1602.05629. In FedSGD, server sends current model $w_t$ to all devices and each device calculates the “slope” or gradient $g_k$ of its local data. ... Read More
error: