These are the notes for Data Analytics with Python, episode 3. You will learn why analysts work in notebooks instead of plain scripts, what cells, code cells, markdown cells and the kernel are, the hidden-state trap and Restart and Run All, the four places notebooks run (Jupyter Notebook, JupyterLab, Google Colab, VS Code), what import, modules, libraries and packages are, how pip installs packages from PyPI, and how virtual environments (venv, conda) and requirements.txt keep projects from breaking each other. Every term is defined the first time it appears and collected in the Glossary. Every program and terminal session below was run for real (Python 3.13); outputs are unedited.
Two words first: Python and the interpreter
- Python is a programming language: a way to write instructions for a computer in something close to English.
- The interpreter is the program that reads your Python and runs it, line by line, like a live translator at a meeting.
- A script is a
.pyfile that the interpreter runs from top to bottom.
The problem: every tweak reloads everything
An analyst loads a year of sales (slow), then prints a chart title. time.sleep(5) stands in for a slow file read: it just waits 5 seconds.
import time
print("Loading a year of sales...")
time.sleep(5) # pretend: reading a huge file
print("Loaded.")
print("Chart: sunny vs rainy")
Loading a year of sales...
Loaded.
Chart: sunny vs rainy
Her manager asks to capitalise the title. She changes one line, runs again, and waits another 5 seconds for the whole load. With real data the wait is often minutes, on every tweak.
Why: a script always runs from the top, and when it ends, Python forgets every variable. Fix: keep Python running and run one small piece at a time. That's a notebook.
Notebooks: cells and the kernel
| Restaurant | Notebook | Meaning |
|---|---|---|
| Ordering one dish at a time | Cell | One small piece of the notebook you run on its own |
| — | Code cell | A cell that holds Python code; its output appears right below it |
| The waiter who remembers your table | Kernel | A Python process kept running behind the notebook; it runs your cells and remembers every variable |
| The menu's descriptions | Markdown cell | A cell of formatted text (headings, notes), not code |
Markdown is a simple way to format text: # Heading, **bold**, - bullet. Mixing markdown cells and code cells makes a notebook read like a report.
The analyst's work as two cells:
# Cell [1] (run once: 5 seconds)
import time
print("Loading a year of sales...")
time.sleep(5)
sales = 1263
print("Loaded.")
# Cell [2] (re-run as often as you like: instant)
print("Chart: Sunny vs Rainy days, total", sales)
[1] Loading a year of sales...
Loaded.
[2] Chart: Sunny vs Rainy days, total 1263
Cell 2 re-runs instantly because the kernel still remembers sales. Notebook files end in .ipynb (short for IPython notebook); inside, they are a JSON file holding the cells and their outputs.
The trap: hidden state
- You run cell 1 (
total = 1263), then cell 2 (print(total * 2)). Works. - You delete cell 1, or move cell 2 above it. It still works, because the kernel remembers
total. Memory you can't see is called hidden state. - A teammate opens the notebook; their fresh kernel runs top to bottom, and it crashes.
The same code as a script (which, like a fresh kernel, always runs top to bottom):
# Cell [2] was written first in this file...
print(total * 2)
# ...and Cell [1] below it
total = 1263
Traceback (most recent call last):
...
NameError: name 'total' is not defined
Fix: Restart and Run All. Restart wipes the kernel's memory; Run All runs every cell in order, exactly as your teammate will. Do it before you share any notebook: if it survives Restart and Run All, it works for everyone.
Where notebooks run
| Tool | What it is | Install? | Notes |
|---|---|---|---|
| Jupyter Notebook | The classic notebook app; runs on your computer, opens in your browser | Yes | Name from Julia, Python and R. Project Jupyter was spun off from IPython in 2014 |
| JupyterLab | Jupyter's newer interface: files, notebooks and terminals side by side | Yes | Same .ipynb files |
| Google Colab | A hosted Jupyter notebook service (runs on Google's computers) | No | Free to use; notebooks saved in Google Drive (or loaded from GitHub); resources aren't guaranteed, and idle machines are deleted (variables are lost) |
| VS Code | A free code editor | Yes | Opens .ipynb files through the Jupyter extension; has Restart, Run All, a kernel picker and a Variables panel |
Hosted means the software runs on someone else's computers and you use it through the browser.
import, modules, libraries and packages
Python on its own can't read Excel files or draw charts. You bring in ready-made code with the import statement.
import math
print(math.sqrt(144))
import pandas as pd
print(pd.__version__)
import polars
12.0
3.0.5
Traceback (most recent call last):
...
ModuleNotFoundError: No module named 'polars'
import mathloads Python's built-in maths module;math.sqrt(144)is 12.0.import pandas as pdloads pandas (the main data library of this course) under the short nicknamepd.pd.__version__shows which version is installed (yours may differ).polars(another fast data library) isn't installed on this machine, so Python raisesModuleNotFoundError.
| Kitchen | Python | Meaning |
|---|---|---|
| One recipe card | Module | One .py file of ready-made code (e.g. math) |
| A whole cookbook | Library | A collection of ready-made code for one kind of job (e.g. pandas for tables) |
| The cookbook, packed for sale | Package | A library bundled so it can be installed. In everyday talk, library and package mean the same thing |
| The supermarket | PyPI | The Python Package Index: the online store of free packages |
| Home delivery | pip | The installer: "the PyPA recommended tool for installing Python packages", downloads from PyPI |
The fix: pip install
Run this in a terminal (the text window where you type commands). In a notebook cell, write %pip install polars.
$ pip install polars
... (download progress lines skipped) ...
Installing collected packages: polars-runtime-32, polars
Successfully installed polars-2.1.0 polars-runtime-32-2.1.0
$ python -c "import polars; print('polars works')"
polars works
pip also installed polars-runtime-32, a package polars needs. Packages that a package needs are its dependencies.
The next problem: one Python, many projects
Last year's report was built with pandas 2; this year's dashboard needs pandas 3. Versions really do differ: this machine has pandas 3.0.5, while a fresh install a minute later got 3.1.0. If every project shares one Python, upgrading for one project can break another.
Fix: a virtual environment (like a tiffin with separate boxes): a separate, private set of packages for one project, built on top of your normal Python. The Python docs describe it as "isolated from the packages in the base environment, so that only those explicitly installed in the virtual environment are available".
venv: create, activate, install (a real session)
$ mkdir shop && cd shop
$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ python -c "import pandas"
ModuleNotFoundError: No module named 'pandas'
(.venv) $ pip install -q pandas
(.venv) $ python -c "import pandas as pd; print(pd.__version__)"
3.1.0
| Step | macOS / Linux | Windows (cmd) | Windows (PowerShell) |
|---|---|---|---|
| Create | python3 -m venv .venv |
python -m venv .venv |
python -m venv .venv |
| Activate | source .venv/bin/activate |
.venv\Scripts\activate.bat |
.venv\Scripts\Activate.ps1 |
| Leave | deactivate |
deactivate |
deactivate |
venvis the tool built into Python for virtual environments;.venvis just the folder name we chose.- After activating, the prompt starts with
(.venv), so you know which environment you're in. - The new environment starts empty:
import pandasfailed even though this computer has pandas. That's the isolation working.
requirements.txt: the label on the lid
(.venv) $ pip freeze > requirements.txt
(.venv) $ cat requirements.txt
numpy==2.5.4
pandas==3.1.0
python-dateutil==3.0.0
six==1.17.0
tzdata==2026.5
# a teammate rebuilds the same setup with:
$ pip install -r requirements.txt
pip freeze lists every installed package with its exact version. A requirements file stores that list so anyone can rebuild the same environment with pip install -r requirements.txt. Environments are disposable: never put your own code inside one, don't commit it to Git, and recreate it from the requirements file when needed.
Anaconda and conda
| What it is | |
|---|---|
| Anaconda Distribution | One installer with Python, conda, Anaconda Navigator and 600+ data science packages |
| conda | A package manager and environment manager (conda create, conda install): pip's and venv's jobs together |
| Anaconda Navigator | A desktop app to create environments and launch Jupyter with clicks |
| Trade-off | A big download that uses gigabytes of disk; venv + pip is lighter |
Pick one approach per project and stick with it.
Which setup should you use?
| Situation | Use |
|---|---|
| Just starting, or a slow laptop | Google Colab |
| Your own laptop (the setup used at jobs) | Python + VS Code + Jupyter extension, one venv per project |
| Quick snippet | The SyntaxPilot online compiler |
Common mistakes (with the real error messages)
| Mistake | Real message | Why | Fix |
|---|---|---|---|
| Using a variable from a deleted or later cell | NameError: name 'total' is not defined |
Hidden state | Restart and Run All before sharing |
| Importing a package that isn't installed | ModuleNotFoundError: No module named 'polars' |
Not installed in this Python | pip install polars (%pip install in a notebook) |
pip install pandas worked, but import pandas fails |
ModuleNotFoundError: No module named 'pandas' |
pip installed into a different environment | Activate the right venv / pick the right kernel |
Putting project files inside .venv or committing it |
(no error: a huge, broken repo) | Environments are disposable | Commit requirements.txt instead |
Mini project: a reproducible analysis folder
On your laptop (or in a Colab terminal):
$ mkdir chai_analysis && cd chai_analysis
$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install pandas
(.venv) $ pip freeze > requirements.txt
Then create analysis.ipynb with: a markdown cell # Do rainy days hurt chai sales?, a code cell with episode 1's week list, a code cell that cleans it, and a code cell that prints the averages. Finish with Restart and Run All: every cell should run in order with no errors.
Practice
1. Which is which? (a) runs your cells and remembers variables, (b) holds a heading and notes, (c) the online store of packages, (d) the installer.
Solution
(a) kernel, (b) markdown cell, (c) PyPI, (d) pip.
2. You run pip install seaborn in a terminal, but your notebook still says ModuleNotFoundError: No module named 'seaborn'. Give two likely causes.
Solution
The terminal and the notebook use different environments (the venv wasn't activated, or the notebook's kernel is a different Python); or the kernel started before the install and needs a restart. Fix: %pip install seaborn inside the notebook (installs into the kernel's own environment), then restart the kernel.
3. Make a NameError on purpose in a notebook: cell A x = 10, cell B print(x * 2). Run both, delete cell A, run cell B again (it works). Now Restart and Run All. What happens and why?
Solution
After the restart, cell B raises NameError: name 'x' is not defined: x only existed in the old kernel's memory (hidden state).
4. Your teammate needs your exact package versions. What do you send them, and what command do they run?
Solution
Send requirements.txt (made with pip freeze > requirements.txt). They create and activate their own venv and run pip install -r requirements.txt.
Quick quiz
Your notebook worked all day. You click Restart and Run All, and cell [4] fails: NameError: name 'clean' is not defined. Most likely cause?
A. pandas isn't installed B. A cell that creates clean was deleted or sits below cell [4] C. Colab is down D. The kernel is too slow
Answer
B. clean only existed in the old kernel's memory. Restart and Run All exposed the hidden state before your teammate did.
Interview questions
- Why do analysts use notebooks? Run code in small cells, keep data in memory between steps, and mix code, output, charts and explanations in one document.
- What is a kernel? The running language process behind a notebook that executes cells and holds variables.
- What is hidden state, and how do you avoid it? Variables in kernel memory that the visible cells no longer create; avoid it with Restart and Run All before sharing.
- Library vs module vs package? Module = one file of code; library = a collection for one job; package = how it's bundled for installing (often used interchangeably with library).
- Why use virtual environments? To give each project its own package versions so projects can't break each other, and to make setups reproducible with
requirements.txt. - venv vs conda? venv (with pip) is built into Python and light; conda manages both packages and environments and comes with Anaconda.
Glossary
| Term | Meaning | Example |
|---|---|---|
| Python | A programming language close to English | print("hi") |
| Interpreter | The program that reads and runs Python line by line | python3 main.py |
| Script | A .py file run top to bottom |
analysis.py |
| Notebook | A document of cells you run one at a time | analysis.ipynb |
| Jupyter Notebook | The classic notebook app (Julia, Python, R) | Opens in your browser |
| JupyterLab | Jupyter's newer interface with files and terminals | Side-by-side panels |
| Google Colab | A free hosted Jupyter notebook service | colab.research.google.com |
| Hosted | Runs on someone else's computers, used via the browser | Colab |
| Cell | One piece of a notebook you run on its own | Cell [2] |
| Code cell | A cell holding code, with its output below | print(sales) |
| Markdown cell | A cell of formatted text | # Question |
| Markdown | A simple text formatting syntax | **bold** |
| Kernel | The running Python behind a notebook that remembers variables | Restarting it clears memory |
| Hidden state | Variables in memory that no visible cell creates | A deleted cell's total |
| Restart and Run All | Wipe the kernel, then run every cell in order | Do it before sharing |
| .ipynb | The notebook file format (JSON inside) | analysis.ipynb |
| VS Code | A free code editor; opens notebooks via the Jupyter extension | — |
| import statement | Loads a module or library into your program | import pandas as pd |
| Module | One .py file of ready-made code |
math |
| Library | A collection of ready-made code for one job | pandas |
| Package | A library bundled for installing | polars |
| Dependency | A package another package needs | numpy for pandas |
| PyPI | The Python Package Index, the online store of packages | pypi.org |
| pip | The tool that installs packages from PyPI | pip install polars |
| Terminal | A text window for typing commands | $ pip --version |
| ModuleNotFoundError | Python's error when an imported module isn't installed | No module named 'polars' |
| NameError | Python's error for a name that doesn't exist yet | name 'total' is not defined |
| Virtual environment | A private set of packages for one project | .venv |
| venv | Python's built-in tool for virtual environments | python3 -m venv .venv |
| Activate | Switch your terminal into an environment | source .venv/bin/activate |
| requirements.txt | A list of packages with exact versions | pandas==3.1.0 |
| pip freeze | Prints installed packages with versions | pip freeze > requirements.txt |
| Anaconda Distribution | One installer: Python, conda, Navigator, 600+ packages | — |
| conda | A package and environment manager | conda create -n shop pandas |
| Anaconda Navigator | A desktop app for environments and Jupyter | — |
Summary
- A script reruns everything from the top; a notebook runs cells one at a time while the kernel remembers variables.
- Hidden state is the notebook trap; Restart and Run All is the fix.
- Notebooks run in Jupyter Notebook, JupyterLab, Google Colab and VS Code.
importloads modules and libraries; pip installs packages from PyPI; missing ones raiseModuleNotFoundError.- Virtual environments (venv or conda) give each project its own packages; requirements.txt makes the setup reproducible.
- Next episode: Python for data, part 1: variables, types and operators.