Notes - Computing Notes

Matrices and Matrix Operations – Linear Algebra

1 & 2. Matrix Notation, Terminology & Equality A Matrix is simply a rectangular array of numbers, symbols, or expressions, arranged in rows and columns. Matrices are considered Equal only if they have the exact same dimensions AND every corresponding element is identical. 2 4 -1 9 = 2 4 -1 9 Conversely, here is ... Read More

Bayesian Decision Theory

Machine LearningLeave a Comment

Beginner’s Guide to Bayesian Decision Theory Making Sense of the Unknown The term “Bayesian Decision Theory” sounds like something you’d only hear in a graduate-level statistics lecture. It can definitely feel intimidating at first glance! But at its core, it’s actually a highly intuitive concept. In fact, it’s a mathematical formalization of how we naturally ... Read More

Calculus – $a^x$ and $log_a x$

Calculus, MathematicsLeave a Comment

In calculus, exponential functions with a base other than e (like ax) and logarithms with base a (like loga x) are handled by converting them into their “natural” forms. Since the natural exponential ex and natural log ln(x) have very simple derivatives, we use them as the bridge to understand ax and loga x. Interactive ... Read More

Q-Learning Agent – Using DRL Toolbox

Deep Reinforcement LearningLeave a Comment

Q Learning algorithm is: The figure below shows how a Q-learning agent trains on Q-Value function critic in order to estimate the optimal policy value. It follows epsilon greedy policy based on the value that is determined by the critic. The Internal Flow of a Q-Learning Agent Unlike basic algorithms, a Q-Learning agent splits its ... Read More

Types of RL Algorithms

Deep Reinforcement LearningLeave a Comment

The Two Paths: Model-Free vs. Model-Based In Reinforcement Learning, the term “Model” is used to describe the rules of the environment. Specifically, a model consists of the Transition Dynamics (where an action takes you) and the Reward Function (what score that action earns). The biggest divide in algorithm design is whether the agent attempts to ... Read More

Key Concepts and Terminology in Reinforcement Learning

Deep Reinforcement LearningLeave a Comment

The Agent-Environment Interaction In Reinforcement Learning, the two core components are the Agent and the Environment. The environment serves as the simulated or physical world where the agent operates. 🤖 Agent Action (a_t) Observation & Reward 🌍 Environment In a continuous loop, the agent receives an observation detailing the current condition of this world. Based ... Read More

Deep Reinforcement Learning

Deep Reinforcement LearningLeave a Comment

What is Deep Reinforcement Learning? Demystifying Deep Reinforcement Learning How AI learns to make decisions through trial and error. Let’s break down deep reinforcement learning (DRL). It is a type of artificial intelligence where a computer program learns to make a sequence of decisions through trial and error, guided by a system of rewards. The ... Read More

Reinforcement Learning – Gymnasium

Deep Reinforcement LearningLeave a Comment

If you want to teach an Artificial Intelligence to play a video game, control a robot, or optimize a trading strategy, you need an environment for it to practice in. Enter Gymnasium (formerly OpenAI Gym), the standard API for single-agent Reinforcement Learning (RL). 1. The Agent-Environment Loop Reinforcement learning is fundamentally about trial and error. We don’t ... Read More

CDMA – Code Division Multiple Access

Wireless CommunicationLeave a Comment

Multiple Access is done using TDMA, FDMA and CDMA. CDMA is also multiple access technology. Example below show two users User 0 and User 1. a0 and a1 are data to be sent to users 0 and 1 respectively. Codes C0 is for User 0 data and C1 is for User 1 data. The key ... Read More

CDMA, FDMA, TDMA

Wireless CommunicationLeave a Comment

What is Multiple Access? Managing the digital traffic of our wireless world. Think of Multiple Access as the traffic controller for radio waves. It is the protocol that prevents millions of signals from crashing into each other on a shared communication channel. By organizing data from multiple transmitters into one structured data link, it prevents ... Read More