Markov Chains – Linear Algebra

Understanding Markov Chains: The Memoryless Math

In probability and statistics, some systems look backward to determine the future. Whereas Markov Chains look only at the present moment. It was named after the Russian mathematician Andrey Markov. These mathematical models describe systems that jump between different states over time.

The Markov Property: The future state of the system depends only upon its current state, not on the sequence of events that preceded it. In other words, tt has “no memory” of how it got there.

How It Works: The Two Main Components

1. States

States are the set of all possible conditions the system can occupy. For instance, if you are modeling a weather pattern, your states might simply be Sunny and Rainy.

2. Transition Probabilities

The mathematical likelihood of moving or occuring from one state to another. These probabilities are organized into a matrix where every row adds up to 100% or 1.

Example: The Weather Predictor

Imagine a simple climate model:

If it is Sunny today, there is an 80% chance it stays sunny tomorrow, and a 20% chance it rains.
If it is Rainy today, there is a 40% chance it clears up to sunny tomorrow, and a 60% chance it stays rainy.

To predict the weather two days from now, we don’t need last week’s data. We only need today’s forecast to step forward.

Where Are They Used?

Markov chains are foundational across many modern technologies such as:

Artificial Intelligence & NLP: Early predictive text systems and language models use them to guess the next word in a sequence.
Search Engines: Google’s original PageRank algorithm modeled a “random surfer” clicking through links as a massive Markov chain.
Finance: Modeling shifting credit ratings, stock market volatility, and algorithmic trading strategies.

We can represent a Markov chain as a Markov matrix.

References:

Markov chains and Google’s PageRank Algorithm https://math.libretexts.org/Bookshelves/Linear_Algebra/Understanding_Linear_Algebra_(Austin)/04%3A_Eigenvalues_and_eigenvectors/4.05%3A_Markov_chains_and_Google’s_PageRank_algorithm

Other resources:

References:

[1] Libretexts, “10.1: Introduction to Markov Chains,” Mathematics LibreTexts, Mar. 22, 2020. https://math.libretexts.org/Bookshelves/Applied_Mathematics/Applied_Finite_Mathematics_(Sekhon_and_Bloom)/10%3A_Markov_Chains/10.01%3A_Introduction_to_Markov_Chains

Disclaimer: If you find any issues with the blog please contact – info at computingnotes dot com

‌