SyntaxPilot
Courses/Data Analytics with Python/Lesson 39
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Data Analytics with Python · Lesson 39 of 90

Probability 1: events, rules and conditional probability

experimentoutcomesample spaceeventprobabilitycomplementaddition rulemutually exclusive eventsindependent eventsmultiplication rulejoint probabilitymarginal probabilityconditional probabilitycontingency table

These are the notes for Data Analytics with Python, episode 39. In episode 38 you found that "38% of late orders go unrated": a probability that depends on a condition. Today you build probability from the ground up: experiment, outcome, sample space, event, probability, the complement, the addition rule, mutually exclusive events, joint and marginal probability in a contingency table, independent events and the multiplication rule, and conditional probability. Every rule appears because something breaks without it, and every rule is checked with simulations and data. Every term is defined the first time it appears and collected in the Glossary.

Every program below was run for real (Python 3.13, NumPy 2.5.3, pandas 3.0.5). Outputs are from those runs. All data is made up for teaching. Programs start with sys.tracebacklimit = 0 so an error shows only its message, as in the video.

The problem: how many samosas to fry?

Ramesh's chai stall opens at 7 a.m. Samosas take 20 minutes to fry, so he must decide now how many to make. Will the next customer want a snack? He can't know for sure. His gut says "about half", and "more when it rains". A gut feeling can't be compared, added or checked. He needs the chance as a number: a probability.

Four words: experiment, outcome, sample space, event

Term Meaning Die example Chai stall example
Experiment an action whose result is uncertain rolling a die serving the next customer
Outcome one possible result rolling a 4 "tea and samosa"
Sample space the set of all possible outcomes {1, 2, 3, 4, 5, 6} every possible order
Event a set of outcomes you care about "even" = {2, 4, 6} "buys a snack"

If all outcomes are equally likely, the probability of an event is

P(event) = number of favourable outcomes ÷ total number of outcomes

so P(even) = 3 ÷ 6 = 0.5. A probability is always between 0 (impossible) and 1 (certain). P(A) is read "the probability of A".

Probability by simulation

Check P(even) = 0.5 by rolling a virtual die with NumPy's random generator (episode 10):

import sys; sys.tracebacklimit = 0
import numpy as np, pandas as pd
rng = np.random.default_rng(39)
rolls = rng.integers(1, 6, size=100_000)
print(pd.Series(rolls).value_counts().sort_index())
1    20227
2    19665
3    20002
4    20028
5    20078
Name: count, dtype: int64

Faces 1 to 5 only: the 6 never appears, with no error. In rng.integers(low, high) the upper end is exclusive, like range in Python, so integers(1, 6) gives 1 to 5. A die is integers(1, 7) (or integers(1, 6, endpoint=True)).

for n in [10, 100, 10_000, 1_000_000]:
    rolls = rng.integers(1, 7, size=n)
    print(f"{n:>9} rolls: P(even) = {np.mean(rolls % 2 == 0):.3f}")
       10 rolls: P(even) = 0.700
      100 rolls: P(even) = 0.530
    10000 rolls: P(even) = 0.490
  1000000 rolls: P(even) = 0.499

10 rolls say 0.7, far from the truth; a million rolls say 0.499. The more times you repeat an experiment, the closer the share of times an event happens gets to its probability (the law of large numbers, episode 44). That gives a second, practical meaning of probability: the long-run share. It is how you get probabilities from data: past orders are repeated experiments. Small samples can mislead badly.

Probabilities from data, and the complement

The made-up chai stall data: 1,000 orders with the weather, whether the customer bought tea, and whether they bought a snack.

import sys; sys.tracebacklimit = 0
import numpy as np, pandas as pd
rng = np.random.default_rng(39); n = 1000          # made-up chai stall orders
rain = rng.random(n) < 0.3
df = pd.DataFrame({"weather": np.where(rain, "rain", "dry"),
                   "tea": rng.random(n) < 0.8,
                   "snack": rng.random(n) < np.where(rain, 0.7, 0.4)})
print(df.head(3))
print("P(tea)   =", df["tea"].mean())
print("P(snack) =", df["snack"].mean())
print("P(no snack) =", round(1 - df["snack"].mean(), 3))
  weather    tea  snack
0     dry   True   True
1     dry  False  False
2    rain   True  False
P(tea)   = 0.806
P(snack) = 0.497
P(no snack) = 0.503

The probability of an event is the mean of a True/False column, because True counts as 1 and False as 0. P(tea) = 0.806, P(snack) = 0.497: the gut feeling "about half" was right.

"No snack" is the complement of "snack": every outcome in the sample space that is not in the event. One of the two always happens, so they add up to 1:

P(not A) = 1 − P(A) → P(no snack) = 1 − 0.497 = 0.503

The complement is often the shortcut: P(at least one) = 1 − P(none). If three customers arrive independently and each buys a snack with probability 0.497:

p_none = (1 - 0.497) ** 3
print("P(no snack in 3 customers)       =", round(p_none, 3))
print("P(at least one snack in 3)       =", round(1 - p_none, 3))
P(no snack in 3 customers)       = 0.127
P(at least one snack in 3)       = 0.873

The addition rule

What share of customers buy tea or a snack (at least one of the two)? In English you'd write "tea and snack", so you try Python's and:

both = df[df["tea"] and df["snack"]]
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

ValueError: The truth value of a Series is ambiguous. Python's and/or need one True or False; a Series holds 1,000. For columns, use & (and), | (or) and ~ (not), with parentheses around comparisons (episode 15).

Now the obvious maths, P(tea or snack) = P(tea) + P(snack):

p_tea, p_snack = df["tea"].mean(), df["snack"].mean()
print("P(tea) + P(snack) =", round(p_tea + p_snack, 3))
p_both = (df["tea"] & df["snack"]).mean()
print("P(tea and snack)  =", p_both)
print("P(tea or snack)   =", (df["tea"] | df["snack"]).mean())
print("rule              =", round(p_tea + p_snack - p_both, 3))
P(tea) + P(snack) = 1.303
P(tea and snack)  = 0.408
P(tea or snack)   = 0.895
rule              = 0.895

Plain adding gives 1.303, above 1: impossible. The real share is 0.895.

Real-life example: hands up in class. "Who likes cricket?" 30 hands. "Football?" 20 hands. That's not 50 students: 12 raised their hand both times and were counted twice. 30 + 20 − 12 = 38.

Addition rule: P(A or B) = P(A) + P(B) − P(A and B)

Check: 0.806 + 0.497 − 0.408 = 0.895, exactly the real share. "Or" in probability means at least one (A, B, or both).

Mutually exclusive events

Two events are mutually exclusive (disjoint) when they cannot both happen: one roll can't be a 1 and a 6; one order can't be in dry weather and in rain. Then P(A and B) = 0 and the addition rule shrinks to plain adding:

P(1 or 6) = 1/6 + 1/6 = 1/3

import numpy as np
rng = np.random.default_rng(1)
rolls = rng.integers(1, 7, size=1_000_000)
print("P(1 or 6) simulated =", round(np.mean((rolls == 1) | (rolls == 6)), 4), " exact =", round(1/3, 4))
P(1 or 6) simulated = 0.3331  exact = 0.3333

Tea and snack are not mutually exclusive (40.8% of orders have both), which is why plain adding failed. Don't confuse mutually exclusive (can't happen together) with independent (don't affect each other; below). Mutually exclusive events with non-zero probabilities are never independent: if one happens, the other's chance drops to 0.

Joint and marginal probability: the contingency table

A contingency table is a crosstab (episode 27) of two categorical variables. With normalize="all", every cell becomes a share of all rows; margins=True adds totals:

t = pd.crosstab(df["weather"], df["snack"], margins=True, normalize="all")
print(t.round(3))
snack    False   True    All
weather                     
dry      0.411  0.286  0.697
rain     0.092  0.211  0.303
All      0.503  0.497  1.000
  • Each inside cell is a joint probability: the chance of two things together, P(A and B). P(rain and snack) = 0.211; the four cells add up to 1.
  • The totals in the margins are marginal probabilities: one variable alone, ignoring the other. P(rain) = 0.303, P(snack) = 0.497. They are named after the margins of the table where they're written; each is the sum of its row or column.

Independence and the multiplication rule

Two dice: what is the chance both show a 6?

a = rng.integers(1, 7, size=1_000_000)
b = rng.integers(1, 7, size=1_000_000)
print("P(a=6)        =", round(np.mean(a == 6), 4))
print("P(a=6, b=6)   =", round(np.mean((a == 6) & (b == 6)), 4))
print("1/6 * 1/6     =", round(1/36, 4))
P(a=6)        = 0.166
P(a=6, b=6)   = 0.0276
1/6 * 1/6     = 0.0278

Simulated 0.0276 vs 1/6 × 1/6 = 0.0278. Multiplying works because the first die has no effect on the second. Events like that are independent: knowing one happened doesn't change the chance of the other.

Multiplication rule for independent events: P(A and B) = P(A) × P(B)

Try the shortcut on Ramesh's data:

p_rain = (df["weather"] == "rain").mean()
for col in ["tea", "snack"]:
    joint = ((df["weather"] == "rain") & df[col]).mean()
    print(f"{col:5}: P(rain)*P({col}) = {p_rain * df[col].mean():.3f}"
          f"   P(rain and {col}) = {joint:.3f}")
tea  : P(rain)*P(tea) = 0.244   P(rain and tea) = 0.251
snack: P(rain)*P(snack) = 0.151   P(rain and snack) = 0.211

For tea, 0.244 vs 0.251: close (the small difference is random noise). For snack, multiplying says 0.151 but reality says 0.211: the shortcut fails, because rain changes snack buying (people want something hot and fried in the rain). Rain and snack are dependent events. You need a probability that knows it's raining.

Conditional probability

P(A | B), read "the probability of A given B", is a conditional probability: the chance of A once you know B happened. Keep only the rows where B happened and take the share where A happened. As a formula:

P(A | B) = P(A and B) ÷ P(B)

rain = df["weather"] == "rain"
print("P(snack | rain) =", round((rain & df["snack"]).mean() / rain.mean(), 3))
print(pd.crosstab(df["weather"], df["snack"], normalize="index").round(3))
print(pd.crosstab(df["weather"], df["tea"], normalize="index").round(3))
P(snack | rain) = 0.696
snack    False  True 
weather              
dry      0.590  0.410
rain     0.304  0.696
tea      False  True 
weather              
dry      0.204  0.796
rain     0.172  0.828

P(snack | rain) = 0.211 ÷ 0.303 = 0.696. crosstab(..., normalize="index") gives P(column | row) for every row at once: snack is 0.41 when dry and 0.70 in rain; tea is 0.80 dry and 0.83 in rain (barely different).

General multiplication rule (always true): P(A and B) = P(B) × P(A | B). The independent version P(A) × P(B) is the special case where P(A | B) = P(A).

Independence, properly defined: A and B are independent when P(A | B) = P(A), i.e. knowing B doesn't change the chance of A. Tea: 0.83 given rain vs 0.81 overall → roughly independent. Snack: 0.70 vs 0.50 → dependent.

The order matters: P(snack | rain) = 0.70 is not P(rain | snack) = 0.211 ÷ 0.497 = 0.42. Confusing the two is one of the most common mistakes in statistics (episode 40, Bayes' theorem).

Ramesh's plan: with 100 customers expected, a dry morning means about 41 snack buyers, a rainy one about 70. Tea stays about 80% either way.

Common mistakes

Code / habit Real error / symptom Why Fix
df[df["tea"] and df["snack"]] ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all(). and/or need one True/False df["tea"] & df["snack"]; the pipe for or
rng.integers(1, 6) for a die no error; never rolls a 6 the upper end is exclusive rng.integers(1, 7)
P(A or B) = P(A) + P(B) no error; 1.303 > 1 the overlap is counted twice subtract P(A and B)
P(A and B) = P(A) × P(B) no error; 0.151 vs 0.211 events are not independent P(B) × P(A | B)
estimating from 10 trials no error; 0.7 instead of 0.5 small samples vary a lot more data; report the count
swapping P(A | B) and P(B | A) no error; wrong conclusion conditional probability isn't symmetric say "given" out loud; see episode 40
calling disjoint events "independent" no error; wrong reasoning disjoint events depend on each other exclusive ≠ independent

Mini project: two dice, every rule

import numpy as np, pandas as pd
rng = np.random.default_rng(39)
a = rng.integers(1, 7, 200_000); b = rng.integers(1, 7, 200_000)
d = pd.DataFrame({"a6": a == 6, "sum7": a + b == 7, "double": a == b})
p = d.mean()
print("P(sum = 7)       =", round(p["sum7"], 4), " exact 6/36 =", round(6/36, 4))
print("P(not double)    =", round(1 - p["double"], 4), " exact 30/36 =", round(30/36, 4))
both = (d["sum7"] & d["double"]).mean()
print("P(sum7 and double) =", both, "-> mutually exclusive")
print("P(sum7 or double)  =", round((d["sum7"] | d["double"]).mean(), 4), " exact 12/36 =", round(12/36, 4))
print("P(sum7 | a = 6)    =", round(d.loc[d["a6"], "sum7"].mean(), 4), " vs P(sum7) =", round(p["sum7"], 4))
P(sum = 7)       = 0.1681  exact 6/36 = 0.1667
P(not double)    = 0.8338  exact 30/36 = 0.8333
P(sum7 and double) = 0.0 -> mutually exclusive
P(sum7 or double)  = 0.3343  exact 12/36 = 0.3333
P(sum7 | a = 6)    = 0.1685  vs P(sum7) = 0.1681

A sum of 7 and a double can't happen together (7 is odd, a double is even): mutually exclusive, so P(or) is plain adding. And P(sum 7 | first die is 6) ≈ P(sum 7) ≈ 1/6: knowing the first die is a 6 doesn't change the chance of a 7 (only b = 1 works, probability 1/6), so those two events are independent.

Practice

1. A bag has 5 red, 3 blue and 2 green balls. You draw one. P(red)? P(not green)? P(red or blue)?

Solution

P(red) = 5/10 = 0.5. P(not green) = 1 − 2/10 = 0.8 (complement). Red and blue are mutually exclusive for one draw: P(red or blue) = 0.5 + 0.3 = 0.8.

2. 60% of app users use UPI, 50% use cards, 30% use both. What share uses at least one?

Solution

Addition rule: 0.6 + 0.5 − 0.3 = 0.8.

3. Simulate: what is the probability that two dice sum to 7? (Homework from the video.)

Solution
import numpy as np
rng = np.random.default_rng(7)
a = rng.integers(1, 7, 1_000_000); b = rng.integers(1, 7, 1_000_000)
print(round(np.mean(a + b == 7), 4), "exact:", round(6 / 36, 4))
0.1665 exact: 0.1667

4. From this table of 200 customers, find P(app), P(app and repeat), P(repeat | app) and decide whether "app" and "repeat" are independent.

import pandas as pd
df = pd.DataFrame({"channel": ["app"] * 120 + ["walk-in"] * 80,
                   "repeat": [True] * 84 + [False] * 36 + [True] * 32 + [False] * 48})
print(pd.crosstab(df["channel"], df["repeat"], margins=True))
repeat   False  True  All
channel                  
app         36    84  120
walk-in     48    32   80
All         84   116  200
Solution
import pandas as pd
df = pd.DataFrame({"channel": ["app"] * 120 + ["walk-in"] * 80,
                   "repeat": [True] * 84 + [False] * 36 + [True] * 32 + [False] * 48})
app = df["channel"] == "app"
print("P(app) =", app.mean())
print("P(app and repeat) =", (app & df["repeat"]).mean())
print("P(repeat | app) =", (app & df["repeat"]).mean() / app.mean())
print("P(repeat) =", df["repeat"].mean())
P(app) = 0.6
P(app and repeat) = 0.42
P(repeat | app) = 0.7
P(repeat) = 0.58

P(repeat | app) = 0.7 but P(repeat) = 0.58: knowing the channel changes the chance, so they are dependent.

Quick quiz

P(late) = 0.25
P(rain) = 0.30
P(late and rain) = 0.15

What is P(late or rain)? A. 0.55 B. 0.40 C. 0.075 D. 0.15

Answer

B. 0.40 = 0.25 + 0.30 − 0.15. Bonus: P(late | rain) = 0.15 ÷ 0.30 = 0.5, double the overall 0.25, so late and rain are not independent.

Interview questions

  1. Define sample space and event. The set of all possible outcomes of an experiment; an event is a subset of it.
  2. State the addition rule. When can you just add? P(A ∪ B) = P(A) + P(B) − P(A ∩ B); just add when A and B are mutually exclusive.
  3. Mutually exclusive vs independent? Exclusive: can't both happen (P(A and B) = 0). Independent: one doesn't change the other's chance (P(A | B) = P(A)). Exclusive events with positive probabilities are dependent.
  4. What is conditional probability? P(A | B) = P(A and B) / P(B), the probability of A given B happened.
  5. Joint vs marginal probability? Joint: two events together (a cell of a normalised contingency table); marginal: one variable alone (row or column total).
  6. How do you estimate P(A | B) in pandas? df.loc[B, "A"].mean() or pd.crosstab(B, A, normalize="index").
  7. Is P(A | B) equal to P(B | A)? No, in general; they're linked by Bayes' theorem.

Glossary

Term Meaning Example
Experiment An action with an uncertain result rolling a die
Outcome One possible result of an experiment a 4
Sample space The set of all possible outcomes {1, 2, 3, 4, 5, 6}
Event A set of outcomes you care about even = {2, 4, 6}
Probability A number from 0 to 1 for how likely an event is; the long-run share P(even) = 0.5
Equally likely Every outcome has the same chance a fair die
Simulation Imitating an experiment many times with random numbers rng.integers(1, 7, 10**6)
Complement Everything in the sample space not in the event no snack; P = 1 − P(snack)
Addition rule P(A or B) = P(A) + P(B) − P(A and B) 0.806 + 0.497 − 0.408
Mutually exclusive (disjoint) Events that can't happen together a 1 and a 6 on one roll
Joint probability Probability of two events together, P(A and B) P(rain and snack) = 0.211
Marginal probability Probability of one variable alone, from the margins P(rain) = 0.303
Contingency table A table of counts or shares for every combination of two categorical variables weather × snack
Independent events One happening doesn't change the other's chance two dice
Dependent events One happening changes the other's chance rain and snack
Multiplication rule P(A and B) = P(B) × P(A | B); = P(A) × P(B) if independent 1/6 × 1/6
Conditional probability P(A | B) = P(A and B) ÷ P(B), "A given B" P(snack | rain) = 0.696
&, pipe, ~ Element-wise and, or, not for pandas/NumPy df["tea"] & df["snack"]

Summary

  • An experiment has outcomes; the sample space holds them all; an event is a set of outcomes.
  • A probability is between 0 and 1: favourable ÷ total for equally likely outcomes, or the long-run share, estimated by simulation or from data as the mean of a True/False column.
  • Complement: P(not A) = 1 − P(A). Addition rule: subtract the overlap; it's zero for mutually exclusive events.
  • A contingency table shows joint probabilities in its cells and marginal probabilities in its margins.
  • Independent events multiply; otherwise use conditional probability: P(A | B) = P(A and B) ÷ P(B).
  • Next episode: Bayes' theorem and base rates: what happens when you flip the condition.