SyntaxPilot
Courses/Data Analytics with Python/Lesson 11

Data Analytics with Python · Lesson 11 of 90 · 14 min

pandas 1: the Series

pandasSeriesindexvaluesdtypecreating a Serieslabel-based accessvectorised Series operationsindex alignmentvalue_countsuniquenunique

These are the notes for Data Analytics with Python, episode 11. For three episodes we used NumPy arrays: fast, but just bare numbers with no names. Here we meet pandas and its simplest building block, the Series: values that carry their own labels. We create Series, pick values by label, do vectorised maths, meet the surprise called index alignment, and count things with value_counts, unique and nunique. Every term is defined the first time it appears and collected in the Glossary. Every program below was run for real (Python 3.13, pandas 3.0.5); outputs are unedited.

The problem: numbers without names

Ramesh's week of chai sales, Monday to Sunday, as a NumPy array:

import numpy as np
cups = np.array([320, 280, 410, 300, 350, 390, 450])

The array does not know which number belongs to which day. To find Thursday you count positions in your head (Mon 0, Tue 1, Wed 2, Thu 3) and write cups[3]. Slow, and easy to get wrong.

The obvious fix is a second array of day names. Here is where it breaks. Ramesh wants his top three days, so we sort the cups, biggest first:

import numpy as np
days = np.array(["Mon", "Tue", "Wed", "Thu",
                 "Fri", "Sat", "Sun"])
cups = np.array([320, 280, 410, 300, 350, 390, 450])
print(cups[3])
top = np.sort(cups)[::-1]
print(top[:3])
print(days[:3])
300
[450 410 390]
['Mon' 'Tue' 'Wed']

The numbers are right but the days are wrong: 450 was Sunday, not Monday. Why it failed: the labels live in a different array, and sorting one array does not move the other. We want each label glued to its value so they always move together. That is a Series.

pandas and the Series

Real-life example: a tea stall menu board. Every price is written next to its item name. Rewrite the board in any order and each name still sits next to its own price. You ask "how much is samosa?", never "what's on line 3?".

  • A library (episode 3) is ready-made code you import. pandas is a Python library for working with tables and labelled data. By convention it is imported as pd, just as NumPy is imported as np:
import pandas as pd
  • A Series is a one-dimensional sequence of values where every value has a label.
  • The labels are called the index (like the item names on the board).
  • The data itself is the values (like the prices). Inside, the values are a NumPy array.
  • Whatever you do to a Series (sort, filter, add), each label stays attached to its value.
  • Next episode, a table made of several Series sharing one index is a DataFrame.

Creating a Series with an index

pd.Series(values, index=labels): the values first, then one label per value.

import pandas as pd
cups = pd.Series([320, 280, 410, 300, 350, 390, 450],
                 index=["Mon", "Tue", "Wed", "Thu",
                        "Fri", "Sat", "Sun"])
print(cups)
Mon    320
Tue    280
Wed    410
Thu    300
Fri    350
Sat    390
Sun    450
dtype: int64

The left column is the index, the right column is the values, and the last line is the dtype (data type, from episode 8). int64 means whole numbers stored in 64 bits. The index is not a column of data: it is the labels on the side.

The rest of these notes reuse this cups Series.

Inside a Series: index, values, dtype, size

print(cups.index)
print(cups.values)
print(cups.dtype, cups.size)
days = pd.Series(["Mon", "Tue"])
print(days.dtype)
Index(['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'], dtype='str')
[320 280 410 300 350 390 450]
int64 7
str
Attribute What it is
.index The labels, as an Index object
.values The data as a NumPy array (.to_numpy() does the same and is the recommended spelling)
.dtype The data type of the values
.size How many values (same as len(s))
.name An optional title for the Series (see value_counts below)

pandas 3 note: a Series of text now has dtype str (a dedicated string type). Older pandas versions, and many tutorials online, show object for text. Same idea: the values are text. (Source: the pandas 3.0.0 release notes, 21 January 2026: "Starting with pandas 3.0, a dedicated string data type is enabled by default".)

Three ways to create a Series

import pandas as pd
s = pd.Series([320, 280, 410])
print(s)
print(s.index)
price = pd.Series({"chai": 12, "coffee": 20,
                   "samosa": 15, "bun": 10})
print(price)
rate = pd.Series(5, index=["a", "b", "c"])
print(rate)
0    320
1    280
2    410
dtype: int64
RangeIndex(start=0, stop=3, step=1)
chai      12
coffee    20
samosa    15
bun       10
dtype: int64
a    5
b    5
c    5
dtype: int64
  1. From a list with no index: pandas numbers the values 0, 1, 2. This default index is a RangeIndex: a range of whole numbers (start 0, stop 3, step 1).
  2. From a dict (episode 5): the keys become the index, the dict values become the values.
  3. From a single value (a scalar): the value is repeated once for every label you give.

Mistake: lengths that don't match.

import pandas as pd
s = pd.Series([1, 2, 3], index=["a", "b"])
ValueError: Length of values (3) does not match length of index (2)

Every value needs exactly one label.

Label-based access, .loc and .iloc

Label-based access means getting a value by its label instead of its position.

print(cups["Wed"])
print(cups["Sat":"Sun"])
print(cups[["Mon", "Sun"]])
print(cups.loc["Fri"])
print(cups.iloc[0])
410
Sat    390
Sun    450
dtype: int64
Mon    320
Sun    450
dtype: int64
350
320
  • cups["Wed"]: one value by label. No counting.
  • cups["Sat":"Sun"]: a label slice. It includes the end label (Sun). A position slice (episode 5) stops before the end.
  • cups[["Mon", "Sun"]]: a list of labels picks just those, like fancy indexing in episode 9. Note the double brackets: the outer pair is indexing, the inner pair is the list.
  • .loc[...]: look up by label.
  • .iloc[...]: look up by integer position (the i is for integer).
.loc .iloc
Looks up by label position (0, 1, 2, ...)
Example cups.loc["Mon":"Wed"] → Mon, Tue, Wed cups.iloc[0:3] → positions 0, 1, 2
Slice end included excluded
Not found KeyError IndexError: single positional indexer is out-of-bounds

Mistake: a label that doesn't exist.

print(cups["Holiday"])
KeyError: 'Holiday'

A KeyError means the label is not in the index (the same error as a missing dictionary key). Check with "Holiday" in cups (in on a Series checks the index), which gives False.

Mistake (pandas 3): using a number with [] on a labelled Series.

print(cups[0])
KeyError: 0

Square brackets on a Series look for a label, and there is no label 0. In pandas 3.0.5 (the version used here) [] does not treat the 0 as a position, so some older tutorials that show s[0] working on a labelled Series will fail. For positions, always use cups.iloc[0].

Episode 14 goes much deeper into loc and iloc on tables.

Vectorised Series operations

The values are a NumPy array, so everything from episodes 8-10 still works, and every result keeps its labels. Vectorised means one operation applied to every value at once, with no loop.

print(cups.mean())
print(cups.max(), cups.idxmax())
print(cups[cups > 400])
sales = cups * 12
print(sales.head(3))
print(sales.sum())
357.14285714285717
450 Sun
Wed    410
Sun    450
dtype: int64
Mon    3840
Tue    3360
Wed    4920
dtype: int64
30000
  • mean() is the average: about 357 cups a day.
  • idxmax() returns the label of the largest value (Sun). It is argmax from episode 9, but it gives a label, not a position. idxmin() gives the label of the smallest.
  • A boolean mask (episode 9) works and keeps the labels: Wed and Sun.
  • cups * 12 multiplies every value by the price, 12 rupees.
  • head(n) shows the first n values (default 5); tail(n) shows the last n.

The top-three bug, fixed. sort_values() sorts a Series by its values; ascending=False means biggest first. (Episode 16 covers sorting in depth.)

top = cups.sort_values(ascending=False)
print(top.head(3))
Sun    450
Wed    410
Sat    390
dtype: int64

Correct this time, because each label travelled with its value.

Index alignment: why adding two Series can give NaN

The problem. Ramesh recorded Monday to Wednesday at stall A. His cousin recorded Tuesday to Thursday at stall B. They want combined sales per day.

import pandas as pd
stall_a = pd.Series({"Mon": 320, "Tue": 280,
                     "Wed": 410})
stall_b = pd.Series({"Tue": 200, "Wed": 250,
                     "Thu": 300})
print(stall_a + stall_b)
Mon      NaN
Thu      NaN
Tue    480.0
Wed    660.0
dtype: float64

Index alignment means: before doing maths between two Series, pandas lines them up by label, not by position. Tue and Wed exist in both, so they are added (480, 660). Mon is only in A and Thu only in B: there is nothing to add them to, so the result is NaN ("not a number", from episode 10), which pandas uses to mean missing. Because NaN is a float, the dtype changed from int64 to float64. When the two indexes differ, the result's labels are the union (all labels from both), sorted.

Real-life example: two exam mark sheets. A teacher has a maths sheet and a science sheet with students in different orders. She matches by roll number, never by line number. A student absent from one exam has no mark there: the total is unknown, not zero.

Treat missing as 0 with fill_value. The method form .add() takes fill_value, used when a label is missing on one side:

print(stall_a.add(stall_b, fill_value=0))
Mon    320.0
Thu    300.0
Tue    480.0
Wed    660.0
dtype: float64

.sub(), .mul() and .div() take fill_value the same way.

Order doesn't matter, labels do:

import pandas as pd
x = pd.Series([1, 2, 3], index=["a", "b", "c"])
y = pd.Series([10, 20, 30], index=["c", "b", "a"])
print(x + y)
a    31
b    22
c    13
dtype: int64

a is 1 + 30, b is 2 + 20, c is 3 + 10. Two NumPy arrays [1, 2, 3] + [10, 20, 30] would add by position and give [11 22 33]. Neither is wrong, but you must know which one you are doing. The danger: alignment never raises an error, it silently gives NaN.

Counting values: value_counts, unique, nunique

The problem. Ten orders on Ramesh's bill book. Which item sold most?

import pandas as pd
orders = pd.Series(["chai", "samosa", "chai", "bun",
                    "chai", "coffee", "samosa", "chai",
                    "bun", "chai"])

The long way (episode 6): a loop and a dict.

counts = {}
for o in orders:
    counts[o] = counts.get(o, 0) + 1
print(counts)
{'chai': 5, 'samosa': 2, 'bun': 2, 'coffee': 1}

Correct, but four lines, in first-seen order, not sorted.

The pandas way:

print(orders.value_counts())
chai      5
samosa    2
bun       2
coffee    1
Name: count, dtype: int64

value_counts() counts how often each distinct (different) value appears, biggest first. The result is itself a Series: items in the index, counts in the values. Name: count is the Series' name, a title pandas gave it. This is a frequency table: a table of how often each value occurs.

print(orders.unique())
print(orders.nunique())
print(orders.value_counts(normalize=True))
<StringArray>
['chai', 'samosa', 'bun', 'coffee']
Length: 4, dtype: str
4
chai      0.5
samosa    0.2
bun       0.2
coffee    0.1
Name: proportion, dtype: float64
  • unique(): each distinct value once, in the order it first appeared. For text, pandas 3 returns a StringArray (its own array type for text); for numbers you get a NumPy array.
  • nunique(): just how many distinct values: 4.
  • value_counts(normalize=True): proportions (fractions of the total, adding to 1) instead of counts. Chai is 0.5, half of all orders. Multiply by 100 for percentages.

By default value_counts() and nunique() skip missing values; value_counts(dropna=False) counts NaN too.

Mini project: the day's earnings per item

import pandas as pd
orders = pd.Series(["chai", "samosa", "chai", "bun",
                    "chai", "coffee", "samosa", "chai",
                    "bun", "chai"])
price = pd.Series({"chai": 12, "coffee": 20,
                   "samosa": 15, "bun": 10})
counts = orders.value_counts()
print("Items sold:", len(orders))
print("Different items:", orders.nunique())
money = counts * price
print(money)
print("Total Rs", money.sum())
print("Top earner:", money.idxmax())
Items sold: 10
Different items: 4
bun       20
chai      60
coffee    20
samosa    30
dtype: int64
Total Rs 130
Top earner: chai

counts is ordered chai, samosa, bun, coffee; price is ordered chai, coffee, samosa, bun. Index alignment still matched every count with its own price. No loops, no counting positions.

Common mistakes

Code Real error Why Fix
pd.Series([1, 2, 3], index=["a", "b"]) ValueError: Length of values (3) does not match length of index (2) Every value needs one label Same number of labels as values
cups["Holiday"] KeyError: 'Holiday' The label is not in the index Check cups.index or "Holiday" in cups
cups[0] when labels are days KeyError: 0 pandas 3: [] looks up labels cups.iloc[0]
cups.iloc[7] (7 values) IndexError: single positional indexer is out-of-bounds Positions are 0 to 6 cups.iloc[6] or cups.iloc[-1]
stall_a + stall_b with different days (no error: NaN for Mon and Thu) Unmatched labels become missing stall_a.add(stall_b, fill_value=0)
NumPy days + cups arrays sorted separately (no error: wrong day names) Labels in a separate array don't move Use a Series and sort_values()
cups.sum (no brackets) (no error: prints <bound method Series.sum of ...>) A method must be called cups.sum()
Expecting dtype: object for text (pandas 3 shows dtype: str) pandas 3 has a dedicated string dtype Read str as "text"

Practice

Use this week of screen time in hours:

import pandas as pd
screen = pd.Series([3.5, 2.0, 4.25, 3.0, 5.5, 6.0, 4.0],
                   index=["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"])

1. Print the average, the day with the most screen time (and its value), and the days above 4 hours.

Solution
print(screen.mean())
print(screen.idxmax(), screen.max())
print(screen[screen > 4])
4.035714285714286
Sat 6.0
Wed    4.25
Fri    5.50
Sat    6.00
dtype: float64

2. A second week was only recorded Monday to Friday. Add the two weeks, explain each NaN, then fix it.

week2 = pd.Series({"Mon": 3.0, "Tue": 2.5, "Wed": 4.0, "Thu": 3.5, "Fri": 5.0})
Solution
print(screen + week2)
print(screen.add(week2, fill_value=0))
Fri    10.50
Mon     6.50
Sat      NaN
Sun      NaN
Thu     6.50
Tue     4.50
Wed     8.25
dtype: float64
Fri    10.50
Mon     6.50
Sat     6.00
Sun     4.00
Thu     6.50
Tue     4.50
Wed     8.25
dtype: float64

Sat and Sun exist only in screen, so they have no partner: NaN. Because the two indexes differ, the result's labels are sorted alphabetically (Fri, Mon, Sat, ...).

3. Fifteen snacks your family ate. Print the counts, the number of different snacks, and each snack's share rounded to 2 decimals.

snacks = pd.Series(["poha", "biscuit", "poha", "fruit", "biscuit", "poha", "namkeen",
                    "fruit", "poha", "biscuit", "poha", "fruit", "namkeen", "poha", "biscuit"])
Solution
print(snacks.value_counts())
print(snacks.nunique())
print(snacks.value_counts(normalize=True).round(2))
poha       6
biscuit    4
fruit      3
namkeen    2
Name: count, dtype: int64
4
poha       0.40
biscuit    0.27
fruit      0.20
namkeen    0.13
Name: proportion, dtype: float64

4. Get Sunday's value by position, the total for Mon to Wed by label, and check whether "Holiday" and "Mon" are labels.

Solution
print(screen.iloc[-1], screen.loc["Mon":"Wed"].sum())
print("Holiday" in screen, "Mon" in screen)
4.0 9.75
False True

The label slice "Mon":"Wed" includes Wed: 3.5 + 2.0 + 4.25 = 9.75.

5. Show the first three days by position, and the two lightest days.

Solution
print(screen.iloc[0:3])
print(screen.sort_values().head(2))
Mon    3.50
Tue    2.00
Wed    4.25
dtype: float64
Tue    2.0
Thu    3.0
dtype: float64

Quick quiz

a = pd.Series([1, 2], index=["x", "y"])
b = pd.Series([10, 20], index=["y", "z"])
print(a + b)

A. x 11, y 22 B. x NaN, y 12.0, z NaN C. y 12 D. ValueError

Answer

B.

x     NaN
y    12.0
z     NaN
dtype: float64

Alignment by label: only y is in both (2 + 10 = 12). x and z have no partner, so they become NaN, and the dtype becomes float. A would be adding by position, which pandas does not do.

Interview questions

  1. What is a pandas Series? A one-dimensional labelled array: values (a NumPy array or pandas array) plus an index of labels, a dtype and an optional name.
  2. Series vs NumPy array? A Series has an index, so you can select by label and operations align by label; an array only has positions. A Series also handles missing values and mixed label types more conveniently.
  3. loc vs iloc? loc selects by label (slice end included); iloc selects by integer position (slice end excluded).
  4. What is index alignment? Operations between two Series first match values by index label; labels present in only one Series give NaN in the result.
  5. How do you add two Series without getting NaN for missing labels? a.add(b, fill_value=0).
  6. Why does s[0] raise KeyError: 0 in pandas 3 when the index is strings? [] on a Series is label-based; pandas 3 removed the old fallback to positions. Use s.iloc[0].
  7. unique() vs nunique() vs value_counts()? The distinct values; how many distinct values; how often each value occurs (sorted, biggest first).
  8. Why did an integer Series become float64 after adding? Alignment produced NaN for unmatched labels, and NaN is a float, so the whole result is upcast to float64.
  9. What dtype does pandas 3 use for text? The dedicated string dtype, shown as str (older versions used object).

Glossary

Term Meaning Example
Library Ready-made code you import and use import pandas as pd
pandas A Python library for tables and labelled data pd.Series(...)
pd The usual short name for pandas when importing import pandas as pd
Series A 1-D sequence of values, each with a label pd.Series([320, 280], index=["Mon", "Tue"])
Label A name attached to a value "Wed"
Index All the labels of a Series, as an Index object cups.index
Values The data of a Series (a NumPy array inside) cups.values → [320 280 ...]
dtype The data type of the values int64, float64, str
str dtype pandas 3's dedicated dtype for text (older: object) pd.Series(["Mon"]).dtype → str
object dtype Older pandas' dtype for text and mixed values shown by pandas 2 for text
size How many values a Series holds cups.size → 7
name An optional title for a Series Name: count
RangeIndex The default index 0, 1, 2, ... when you give none RangeIndex(start=0, stop=3, step=1)
Scalar A single value (not a list) pd.Series(5, index=["a", "b"])
Creating a Series Building one from a list, dict or scalar pd.Series({"chai": 12})
Label-based access Getting values by label, not position cups["Wed"] → 410
Label slice A range of labels; the end label is included cups["Sat":"Sun"]
.loc Select by label cups.loc["Fri"]
.iloc Select by integer position cups.iloc[0]
KeyError Error: the label is not in the index KeyError: 'Holiday'
IndexError Error: the position is past the end single positional indexer is out-of-bounds
Vectorised operation One operation on every value at once, no loop cups * 12
idxmax / idxmin Label of the largest / smallest value cups.idxmax() → 'Sun'
head / tail The first / last n values sales.head(3)
sort_values Sort a Series by its values (labels move too) cups.sort_values(ascending=False)
Index alignment Matching values by label before doing maths stall_a + stall_b
NaN "Not a number": pandas' marker for a missing value Mon NaN
Union All labels from both Series Mon, Thu, Tue, Wed
fill_value Value used for a missing side in .add() etc. a.add(b, fill_value=0)
Distinct value A value counted once however often it repeats chai, samosa, bun, coffee
value_counts How often each distinct value occurs, biggest first orders.value_counts()
Frequency table A table of values and how often each occurs chai 5, samosa 2
Proportion A fraction of the total (all add to 1) chai 0.5
normalize=True Makes value_counts give proportions value_counts(normalize=True)
unique Each distinct value once, in first-seen order orders.unique()
StringArray pandas' own array type for text <StringArray> ['chai', ...]
nunique How many distinct values there are orders.nunique() → 4
DataFrame A table of several Series sharing one index (next episode) cups, price, weather per day

Summary

  • A NumPy array has no labels; keeping labels in a separate array breaks as soon as you sort.
  • A Series = values + index. Labels stay glued to values through sorting, filtering and maths.
  • Create it from a list (default RangeIndex), a dict (keys → index) or a scalar. Lengths must match.
  • .index, .values, .dtype (pandas 3 text is str), .size, .name.
  • By label: s["Wed"], s.loc[...]; label slices include the end. By position: s.iloc[...]. In pandas 3, s[0] on a labelled Series is a KeyError.
  • Maths is vectorised and keeps labels; idxmax gives the label of the max; sort_values sorts with labels attached.
  • Index alignment: operations match labels; unmatched labels give NaN (and float dtype). Use .add(..., fill_value=0) when missing should mean 0.
  • value_counts builds a frequency table (normalize=True for shares); unique lists and nunique counts the distinct values.
  • Next episode: pandas 2: the DataFrame.