Probability

Book Probability by Jim Pitman, Springer

The main concepts of probability theory include:

An outcome space
Events
Probability

These three concepts form the fundamental framework of probability theory. They describe the structure of any random process, from a simple coin toss to complex risk modeling.

Outcome Space (Sample Space)

The outcome space (often denoted by the Greek letter Omega, Ω, or S) is the set of all possible outcomes of a random experiment. It represents everything that could happen. nothing outside this set can occur.

Definition: The universal set containing every distinct, elementary result of the trial.
Example (Rolling a Die): When you roll a standard die, the only possible results are the numbers 1 through 6. Ω={1,2,3,4,5,6}

Events

An event is a specific subset of the outcome space. It is a collection of outcomes that we are interested in observing. An event is said to “occur” if the actual result of the experiment is one of the outcomes in that subset.

Definition: Any combination of outcomes from the sample space.
Example: Let’s say we are interested in the event E of rolling an even number.
E={2,4,6}
If we roll a 4, the event E has occurred. If we roll a 1, it has not.

Probability is a measure (a number) assigned to an event that tells us how likely it is to occur. This number must be between 0 (impossible) and 1 (certain).

Definition: A function P that maps an event to a real number between 0 and 1.
Formula: For an experiment where all outcomes are equally likely (like a fair die), the probability of an event E is the size of the event subset divided by the size of the total outcome space: $P(E)=\frac{Number of favorable outcomes}{Total number of possible outcomes}$
Example: To find the probability of rolling an even number (E):
$P(E)= \frac{∣\{2,4,6\}∣}{∣\{1,2,3,4,5,6\}∣}$
$=63$
$=0.5$

Equally Likely Outcomes

Equally likely outcomes occur when every single result in the sample space has the exact same chance of happening. In other words, there is no reason to expect one outcome over another.

This assumption is the foundation of “Classical Probability.”

Examples:

Equally likely – fair coin, fai die, lottery balls etc.
Not Equally likely – Loaded or weighted die, weather, shooting a basketball

Odds

Odds are an alternative way of expressing the likelihood of an event occurring. While probability compares the number of favorable outcomes to the total possible outcomes, odds compare the number of favorable outcomes to the unfavorable outcomes.

In short: Probability looks at the “whole pie,” while Odds compare the “slices.”

Odds are typically expressed as a ratio of two numbers, written as A : B (read as “A to B”).

There are two ways to state odds:

Odds in Favor (A:B): Comparing the chances of the event happening vs. not happening. Odds in Favor = Number of Favorable Outcomes : Number of Unfavorable Outcomes
Odds Against (B:A): Comparing the chances of the event not happening vs. happening (Common in gambling).
Odds Against = Number of Unfavorable Outcomes : Number of Favorable Outcomes

Example of odds: Let’s look at the event of rolling a 6 on a standard die.

Unfavorable Outcomes: 5 (rolling a 1, 2, 3, 4, or 5)

Favorable Outcomes: 1 (rolling a 6)

Interpretations

How probability should be interpreted in applications.

“Interpretations of probability” refers to the philosophical debate over what probability actually means. When we say, “There is a 50% chance of X,” what are we actually describing?

Classical Interpretation (Theoretical) : This is the “Equally Likely” concept we just discussed. It is based on physical symmetry and logic, not experimentation.
Example: “The probability of heads is 0.5 because the coin has two symmetrical sides.”
Frequency Interpretation (Empirical) : This view defines probability as the relative frequency of an event occurring over the long run. It relies on data and observation.
Example: “I flipped this specific coin 10,000 times, and it landed on heads 5,023 times. Therefore, the probability is approximately 0.5023.”
Subjective Interpretation (Bayesian | Opinion) : This view defines probability as a degree of belief. It is a measure of how confident an individual is that an event will happen, based on the information they currently have.
Example: “I am 90% sure I locked the door.” (You cannot repeat this experiment 10,000 times; it’s a measure of your confidence).

Probability – Introduction