Machine Learning

Types of Reinforcement Learning

by ComputingNotes
First, we need to understand the key nomenclature in reinforcement learning, which includes: Agent, Environment, Reward, Action, State, Policy Function, and Value Function. Terms Agent It takes action; e.g., if you are training a drone, then the drone is the agent. Environment Where the agent exists and moves. Actions What the agent performs on the ... Read More

Multi-Objective Optimization

by ComputingNotes
Multi-Objective Optimization is an interesting approach where instead of focusing on one objective we simultaneously try to minimize or maximize more than one objective function. The objective functions are conflicting in multi-objective optimization problem because a single solution doesn’t optimize each objective simultaneously. Thus, the solution in multi-objective function is called nondominated or pareto optimal ... Read More

Linear Algebra, Optimization Problem

by ComputingNotes
Linear algebra is used to solve various optimization problems. Basic concepts in Linear algebra include vectors, vector space, linear transformations, systems of linear equations. While optimization problems can be linear or non linear. Vectors: Reference:C. C. Aggarwal, Linear Algebra and Optimization for Machine Learning: A Textbook. Cham: Springer International Publishing, 2020. doi: 10.1007/978-3-030-40344-7. Y. Geophys, ... Read More

Bayesian Decision Theory

by ComputingNotes
Beginner’s Guide to Bayesian Decision Theory Making Sense of the Unknown The term “Bayesian Decision Theory” sounds like something you’d only hear in a graduate-level statistics lecture. It can definitely feel intimidating at first glance! But at its core, it’s actually a highly intuitive concept. In fact, it’s a mathematical formalization of how we naturally ... Read More

Deep Reinforcement Learning

by ComputingNotes
Resources: 🛒 The Supermarket Analogy [Plaat et al. Page 25] Imagine you have just moved to a new city, you are hungry, and you want to buy some groceries. There is an unrealistic catch: you have no map and no smartphone. After some random exploration, you find a supermarket. You carefully note the route in ... Read More

Machine Learning Applications – Learning Associations

by ComputingNotes
References: Learning Associations Association Rule Learning is a rule-based method used to discover interesting relations or “hidden patterns” between variables in large databases. It is primarily an unsupervised learning technique because it doesn’t require pre-labeled data; it simply looks for items that frequently occur together. The most famous application is Market Basket Analysis, which retailers ... Read More

Various Bounds in Optimization

by ComputingNotes
Regret Bound:https://www.emergentmind.com/topics/cumulative-regret-analysishttp://sbubeck.com/LecturesALL_Bubeck.pdfhttps://www.jmlr.org/papers/volume11/jaksch10a/jaksch10a.pdfhttps://datascience.stackexchange.com/questions/62141/what-are-regret-boundshttps://arxiv.org/pdf/1403.5556 Lecture 1:Introduction to regret analysisS´ebastien BubeckMachine Learning and Optimization group, MSR AI

Assumptions and Their Meaning in Optimization Problems

by ComputingNotes
1. Lipschitz Continuity: The “Speed Limit”This assumption prevents the function from changing too rapidly over a certain distance. 2. 𝐿-Smoothness: The Curvature CeilingSmoothness ensures the gradient (slope) of the function doesn’t change abruptly. A function is 𝐿-smooth if its gradient is Lipschitz continuous. 3. 𝜇-Strong Convexity : The Curvature Floor – Bowl ShapeStrong convexity guarantees that ... Read More

Federated Learning Algorithm Averaging Methods FedAvgM, FedProx, FedAvg

by ComputingNotes
FedAvg: https://arxiv.org/abs/1602.05629 __________________________________________________________________________________________________________ FedAvgM: https://arxiv.org/pdf/1909.06335 stands for Federated Averaging with Server Momentum. It is an upgrade to the original FedAvg algorithm designed specifically to solve the “Client Drift” problemwhere the model gets confused because different users have very different data (Non-IID). The Formula: How it works mathematically The FedAvgM update happens in three steps on ... Read More
error: