SyntaxPilot
Courses/Python/Lesson 29

Python · Lesson 29 of 40 · 10 min

Testing with unittest and pytest

queued

These are the notes for Python episode 29. Raju's offer board says bills of Rs 100 or more get 10% off. One evening he tidies his code and, without noticing, >= becomes >. He "checks" by printing a few bills, every number looks believable, and a week later a customer with a bill of exactly Rs 100 complains. In this episode you invent automated tests: small programs that check your program for you, every time it changes. You start with the assert keyword, move to Python's built-in unittest module, and finish with pytest, which most Python teams use. Every compiler example below runs as-is in the SyntaxPilot compiler, with its real output from Python 3.13; pytest output is from pytest 9 run in a terminal.

The problem: checking by eye

def final_bill(amount):
    if amount > 100:
        return amount - amount // 10
    return amount

print(final_bill(50))
print(final_bill(100))
print(final_bill(200))

Output:

50
100
180

Does that look right? At a glance, yes. But Rs 100 should give 90. Nothing on the screen says "wrong".

Why it fails: printing shows what the code does, not what it should do. You have to remember the right answers and compare by eye, after every change, forever. Nobody keeps doing that.

assert: write the right answer down

A teacher doesn't re-solve every question for every paper; she has an answer key. A test is the same idea: next to each call, write the answer it should give, and let Python compare.

assert condition does nothing if the condition is true, and stops the program with an AssertionError if it is false. An optional message goes after a comma.

def final_bill(amount):
    if amount > 100:
        return amount - amount // 10
    return amount

assert final_bill(50) == 50
assert final_bill(200) == 180
assert final_bill(100) == 90, "Rs 100 should get 10% off"
print("all checks passed")

Output:

Traceback (most recent call last):
  File "main.py", line 8, in <module>
    assert final_bill(100) == 90, "Rs 100 should get 10% off"
           ^^^^^^^^^^^^^^^^^^^^^
AssertionError: Rs 100 should get 10% off

Caught in a second, not a week. But: the first failing assert stops everything (with three bugs you'd see one at a time), and there's no report of what passed.

assert is for tests and for "this can never happen" checks while developing. Don't use it to validate user input in real programs: Python skips every assert when run with python -O.

unittest: Python's built-in tester

Piece What it does
import unittest the testing module that comes with Python
class TestX(unittest.TestCase): a group of tests (inheritance, episode 22)
a method named test_... one test; only methods starting with test are run
self.assertEqual(actual, expected) the check: what the code gave vs what it should give
unittest.main() finds every test, runs each one even if others fail, prints a report

In the report: . = passed, F = failed (an assert was false), E = error (the test crashed with an unexpected exception).

import unittest

def final_bill(amount):
    if amount > 100:
        return amount - amount // 10
    return amount

class TestFinalBill(unittest.TestCase):
    def test_small_bill(self):
        self.assertEqual(final_bill(50), 50)

    def test_exactly_100(self):
        self.assertEqual(final_bill(100), 90)

    def test_big_bill(self):
        self.assertEqual(final_bill(200), 180)

unittest.main()

Output:

.F.
======================================================================
FAIL: test_exactly_100 (__main__.TestFinalBill.test_exactly_100)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "main.py", line 13, in test_exactly_100
    self.assertEqual(final_bill(100), 90)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
AssertionError: 100 != 90

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

It names the test, the line, what we got (100) and what we wanted (90). Fix the bug (>=) and add a test that a negative bill is refused. with self.assertRaises(ValueError): means "the code inside this block must raise ValueError, or the test fails".

import unittest

def final_bill(amount):
    if amount < 0:
        raise ValueError("amount cannot be negative")
    if amount >= 100:
        return amount - amount // 10
    return amount

class TestFinalBill(unittest.TestCase):
    def test_small_bill(self):
        self.assertEqual(final_bill(50), 50)

    def test_exactly_100(self):
        self.assertEqual(final_bill(100), 90)

    def test_big_bill(self):
        self.assertEqual(final_bill(200), 180)

    def test_negative(self):
        with self.assertRaises(ValueError):
            final_bill(-20)

unittest.main()

Output:

....
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

More assert methods

Method Passes when
assertEqual(a, b) / assertNotEqual(a, b) a == b / a != b
assertTrue(x) / assertFalse(x) x is truthy / falsy
assertIn(a, b) / assertNotIn(a, b) a in b / a not in b
assertIsNone(x) / assertIsNotNone(x) x is None / x is not None
assertAlmostEqual(a, b) a and b are equal to 7 decimal places
assertRaises(Error) the block raises Error

Decimals: assertAlmostEqual

import unittest

class TestMath(unittest.TestCase):
    def test_total(self):
        self.assertEqual(0.1 + 0.2, 0.3)

unittest.main()

Output (key lines):

AssertionError: 0.30000000000000004 != 0.3
FAILED (failures=1)

Floats are stored in binary and are slightly off (episode 4). Use self.assertAlmostEqual(0.1 + 0.2, 0.3), which prints OK.

setUp: a fresh start for every test

setUp runs before each test, so every test gets fresh data and tests can't affect each other. verbosity=2 prints one line per test.

import unittest
class TestBill(unittest.TestCase):
    def setUp(self):
        self.menu = {"chai": 10, "samosa": 15}
    def test_chai(self):
        self.assertEqual(self.menu["chai"], 10)
    def test_add_item(self):
        self.menu["poha"] = 25
        self.assertEqual(len(self.menu), 3)
    def test_menu_fresh(self):
        self.assertEqual(len(self.menu), 2)
unittest.main(verbosity=2)

Output:

test_add_item (__main__.TestBill.test_add_item) ... ok
test_chai (__main__.TestBill.test_chai) ... ok
test_menu_fresh (__main__.TestBill.test_menu_fresh) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

pytest: less ceremony

pytest is a third-party package (install it with pip install pytest, episode 30). Tests are plain functions whose names start with test, using the plain assert keyword. Run pytest in the project folder: it finds files named test_*.py (or *_test.py) and runs every test function in them.

Project layout:

shop/
├── bill.py
└── test_bill.py

bill.py (still with the > bug):

def final_bill(amount):
    if amount < 0:
        raise ValueError("amount cannot be negative")
    if amount > 100:
        return amount - amount // 10
    return amount

test_bill.py:

import pytest
from bill import final_bill

def test_small_bill():
    assert final_bill(50) == 50

def test_exactly_100():
    assert final_bill(100) == 90

def test_negative():
    with pytest.raises(ValueError):
        final_bill(-20)
$ pytest
collected 3 items

test_bill.py .F.                                                         [100%]

=================================== FAILURES ===================================
_______________________________ test_exactly_100 _______________________________

    def test_exactly_100():
>       assert final_bill(100) == 90
E       assert 100 == 90
E        +  where 100 = final_bill(100)

test_bill.py:8: AssertionError
=========================== short test summary info ============================
FAILED test_bill.py::test_exactly_100 - assert 100 == 90
========================= 1 failed, 2 passed in 0.03s ==========================

pytest shows the actual values inside a plain assert (where 100 = final_bill(100)). pytest.raises does what assertRaises does.

Many cases at once: @pytest.mark.parametrize

Bugs love edges. Instead of one function per case, give one test a list of inputs and expected answers:

import pytest
from bill import final_bill

@pytest.mark.parametrize("amount, expected", [
    (50, 50), (99, 99), (100, 90), (200, 180), (0, 0),
])
def test_final_bill(amount, expected):
    assert final_bill(amount) == expected

def test_negative():
    with pytest.raises(ValueError):
        final_bill(-20)

After fixing the bug (>=), pytest -v (verbose):

test_bill.py::test_final_bill[50-50] PASSED                              [ 16%]
test_bill.py::test_final_bill[99-99] PASSED                              [ 33%]
test_bill.py::test_final_bill[100-90] PASSED                             [ 50%]
test_bill.py::test_final_bill[200-180] PASSED                            [ 66%]
test_bill.py::test_final_bill[0-0] PASSED                                [ 83%]
test_bill.py::test_negative PASSED                                       [100%]

============================== 6 passed in 0.02s ===============================

Boundary values (99 and 100, right on either side of the rule, and 0) are where bugs hide. Always test them. For floats in pytest: assert 0.1 + 0.2 == pytest.approx(0.3).

unittest vs pytest

unittest pytest
comes with Python yes no: pip install pytest
a test is a test_ method in a TestCase class a test_ function
checking self.assertEqual, assertTrue, ... plain assert
expected errors with self.assertRaises(E): with pytest.raises(E):
many cases a loop with self.subTest @pytest.mark.parametrize
run unittest.main() or python -m unittest pytest

pytest also runs unittest tests, so you can switch any time.

Common mistakes

Mistake What you see Why Fix
a test method named check_bill Ran 0 tests / NO TESTS RAN only test... names are run def test_bill(self):
assertEqual(final_bill(100), 90) fails AssertionError: 100 != 90 the code (or the expected value) is wrong read got vs wanted, fix the code
assertEqual(0.1 + 0.2, 0.3) AssertionError: 0.30000000000000004 != 0.3 floats are inexact assertAlmostEqual / pytest.approx
calling final_bill(-20) in a test without assertRaises E and ERROR: ... ValueError: amount cannot be negative an exception the test didn't expect with self.assertRaises(ValueError):
only testing 50 and 200 no error: the > bug slips through edges not tested test 99, 100, 0
tests that depend on each other's data a test passes alone, fails with others shared state build fresh data in setUp

Mini project: testing last episode's phone-number checker

Two numbers must pass; four must fail (too short, starts with 5, eleven digits, contains a letter). with self.subTest(n=n): runs each number as its own small test, so the report names the one that failed. This version has a planted bug: re.search instead of re.fullmatch.

import re
import unittest

def is_mobile(text):
    return re.search(r"[6-9]\d{9}", text) is not None

class TestIsMobile(unittest.TestCase):
    def test_valid(self):
        for n in ["9876543210", "6000000000"]:
            with self.subTest(n=n):
                self.assertTrue(is_mobile(n))

    def test_invalid(self):
        for n in ["12345", "5876543210", "98765432101", "98765 4321a"]:
            with self.subTest(n=n):
                self.assertFalse(is_mobile(n))

unittest.main()

Output:

F.
======================================================================
FAIL: test_invalid (__main__.TestIsMobile.test_invalid) (n='98765432101')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "main.py", line 16, in test_invalid
    self.assertFalse(is_mobile(n))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
AssertionError: True is not false

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

search found ten good digits inside the eleven-digit number. Change re.search to re.fullmatch and run again: Ran 2 tests ... OK.

The habit: when you find a bug, first write a test that fails because of it, then fix the code until the test passes. Writing tests before the code, on purpose, is called test-driven development (TDD).

Test first: red, green, refactor

Raju's friends want to split a bill so that no rupee is lost. Write the tests before the function exists:

import unittest

class TestSplitBill(unittest.TestCase):
    def test_even(self):
        self.assertEqual(split_bill(300, 3), [100, 100, 100])

    def test_uneven(self):
        self.assertEqual(split_bill(100, 3), [34, 33, 33])

unittest.main()

Output (key lines): EE, then twice NameError: name 'split_bill' is not defined, and FAILED (errors=2). This is the red step: the tests say what we want, and nothing passes yet.

The simplest code that could work:

def split_bill(total, people):
    return [total // people] * people

Output (key lines):

.F
FAIL: test_uneven (__main__.TestSplitBill.test_uneven)
AssertionError: Lists differ: [33, 33, 33] != [34, 33, 33]

First differing element 0:
33
34

- [33, 33, 33]
+ [34, 33, 33]
FAILED (failures=1)

One rupee went missing. divmod(a, b) divides and returns both the quotient and the remainder, so the first extra people pay one rupee more:

def split_bill(total, people):
    share, extra = divmod(total, people)
    return [share + 1] * extra + [share] * (people - extra)

Output: .., Ran 2 tests in 0.000s, OK. This is green. Now you can tidy the code (refactor) and the tests tell you at once if you broke anything. Red, green, refactor is the cycle of test-driven development.

Practice

1. Write five tests for a grade(marks) function, including the edges and an out-of-range value.

Solution
import unittest

def grade(marks):
    if marks < 0 or marks > 100:
        raise ValueError("marks must be 0 to 100")
    if marks >= 90:
        return "A"
    if marks >= 60:
        return "B"
    return "C"

class TestGrade(unittest.TestCase):
    def test_top(self):
        self.assertEqual(grade(95), "A")

    def test_edge_90(self):
        self.assertEqual(grade(90), "A")

    def test_edge_89(self):
        self.assertEqual(grade(89), "B")

    def test_low(self):
        self.assertEqual(grade(10), "C")

    def test_out_of_range(self):
        with self.assertRaises(ValueError):
            grade(101)

unittest.main()

Output:

.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK

2. Test that int("abc") raises ValueError, and that the message mentions "invalid literal".

Solution
import unittest

class TestInt(unittest.TestCase):
    def test_bad_text(self):
        with self.assertRaises(ValueError) as cm:
            int("abc")
        self.assertIn("invalid literal", str(cm.exception))

unittest.main()

Output:

.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

as cm keeps the caught exception in cm.exception.

3. Write parametrized pytest tests for the fixed is_mobile.

Solution

mobile.py:

import re

def is_mobile(text):
    return re.fullmatch(r"[6-9]\d{9}", text) is not None

test_mobile.py:

import pytest
from mobile import is_mobile

@pytest.mark.parametrize("number", ["9876543210", "6000000000"])
def test_valid(number):
    assert is_mobile(number)

@pytest.mark.parametrize("number", ["12345", "5876543210", "98765432101", "98765 4321a"])
def test_invalid(number):
    assert not is_mobile(number)
$ pytest -q
......                                                                   [100%]
6 passed in 0.01s

Quick quiz

import unittest

class T(unittest.TestCase):
    def test_a(self): self.assertEqual(1, 1)
    def check_b(self): self.assertEqual(1, 2)
    def test_c(self): self.assertTrue(0)

unittest.main()

How many tests run, and what is the result? A. 3 tests, OK B. 2 tests, OK C. 2 tests, 1 failure D. 3 tests, 2 failures

Answer

C. check_b doesn't start with test, so it never runs. test_c fails with AssertionError: 0 is not true. The report ends with Ran 2 tests and FAILED (failures=1).

Interview questions

  1. What is a unit test? A small automated check of one unit (a function or class) in isolation: given this input, expect this output.
  2. Failure vs error in unittest? A failure (F) is an assertion that was false; an error (E) is an unexpected exception inside the test.
  3. Why test boundary values? Off-by-one mistakes (> vs >=, range(n) vs range(n + 1)) only show at the edges.
  4. What does setUp do? Runs before each test so every test starts from fresh data; tearDown runs after each test.
  5. Why is pytest popular? Plain assert with detailed failure messages, less boilerplate, parametrize, fixtures, and plugins; it also runs unittest tests.
  6. What is TDD? Test-driven development: write a failing test, write just enough code to pass it, then clean up (red, green, refactor).
  7. Why not use assert for input validation in production code? python -O removes asserts; raise a real exception (ValueError) instead.

Glossary

Term Meaning
test code that checks other code by comparing actual results with expected ones
automated test a test the computer runs for you, as often as you like
unit test a test of one small piece (a function or class)
assert a keyword: do nothing if the condition is true, raise AssertionError if false
AssertionError the exception raised when an assertion is false
expected vs actual what the code should give vs what it gave
unittest Python's built-in testing module
TestCase the unittest class your test classes inherit from
test method a method whose name starts with test; one test
test runner the tool that finds and runs tests and prints a report (unittest.main(), pytest)
failure (F) a test whose assertion was false
error (E) a test that crashed with an unexpected exception
assertRaises / pytest.raises checks that a block raises a given exception
assertAlmostEqual / pytest.approx compares floats allowing a tiny difference
subTest runs one case of a loop as its own reported sub-test
setUp / tearDown methods run before / after each test
pytest a popular third-party test runner using plain functions and assert
parametrize running one test function with many input/expected pairs
edge case / boundary value an input at the limit of a rule (99, 100, 0, empty list)
regression a bug that comes back after a change; tests catch regressions
red, green, refactor the TDD cycle: a failing test, code that passes it, then tidy-up
divmod(a, b) returns (a // b, a % b): quotient and remainder
test-driven development (TDD) writing a failing test first, then the code that makes it pass

Summary

  • Printing shows what code does; a test states what it should do.
  • assert is silent when true, raises AssertionError when false.
  • unittest: a TestCase class, test_ methods, self.assertEqual, unittest.main(); . pass, F fail, E error.
  • assertRaises for expected errors, assertAlmostEqual for floats, subTest for loops, setUp for fresh data.
  • pytest: plain functions and assert, great failure messages, @pytest.mark.parametrize, pytest.raises.
  • Test the edges, and write a failing test for every bug before fixing it.
  • Next episode: two projects need two versions of the same package. You'll invent virtual environments, and learn pip and project structure.