These are the notes for Cybersecurity Fundamentals (defensive), episode 47. Episode 46 hardened the settings. Today we look at three places where an app takes something from a user and trusts it too much: a file (uploads), a file name (path traversal and zip slip) and a saved object (deserialisation). Every demo runs live, every fix is shown, and every term is explained the first time it appears; the Glossary at the end defines them all again.
Neha, Ravi, the shop and every value are made up. Every folder and archive is a toy the program creates in a temporary folder, the "secret" files are toy files the program writes itself, and nothing outside the machine is contacted. The harmful-looking demos only ever call print(). Python samples run on 12 Nov 2026 with Python 3.13, the Java sample with OpenJDK 21; every output below is copied from a real run. Random values (file names, temporary paths) differ each run.
The problems you'll solve
| Problem | Idea it teaches |
|---|---|
| A "profile photo" that is really a web page | unrestricted file upload, Content-Type, MIME type, extension |
| Labels that lie about a file | extension allowlist, magic bytes (file signatures), size limits |
| Uploads saved under the user's name in the web folder | random file names, storing outside the webroot, malware scanning |
| "Download your invoice" reads a password file | path traversal (../), absolute paths, realpath + commonpath |
Blocking ../ is bypassed |
canonicalisation, URL encoding, strip-once filters |
| A zip that writes outside its folder | archives, zip slip, safe extraction, tarfile filters |
| A cookie that makes the server run code | serialisation, deserialisation, pickle, __reduce__, find_class |
| Attacks made of your own classes | gadget chains (concept), Java serialization, ObjectInputFilter |
| The real fix | safe data formats: JSON with a schema |
Problem 1: a file upload that trusts the label
Basic words first:
- Uploading a file means sending it from your device to a server, for example a profile photo or an invoice.
- When a browser uploads a file it sends three things the user controls: the file's name (
neha.png), the file's bytes, and a Content-Type. - The Content-Type (also called a MIME type) is a label saying what kind of data this claims to be, such as
image/png,image/jpegorapplication/pdf. It is just text in the request, so the uploader can write anything there. OWASP's File Upload Cheat Sheet: "don't trust the Content-Type header as it can be spoofed" (spoofed = faked). - The extension is the end of the file name after the last dot (
.png). It is also chosen by the uploader. - The webroot is the folder a web server shows to the world: anything saved there can be opened by anyone with its address.
- Unrestricted file upload means accepting a file without checking what it really is, what it is called, or where it goes.
Real-life example: a courier counter. The sender writes the label ("documents"). A careful counter X-rays the parcel (magic bytes), gives it its own tracking number instead of the sender's name (our own file name), and keeps parcels in the back room, not the shop window (outside the webroot).
Demo 1: an upload that trusts the label
# Upload v1: trust the browser's Content-Type and the file's own name
import os, tempfile
WEBROOT = tempfile.mkdtemp() # files here are served to everyone
def upload_v1(filename, content_type, data):
if content_type not in ("image/png", "image/jpeg"):
return "rejected: not an image"
path = os.path.join(WEBROOT, filename)
open(path, "wb").write(data)
return "saved " + filename + " in the web folder"
# Neha uploads a real photo
png = b"\x89PNG\r\n\x1a\n" + b"...pixels..."
print(upload_v1("neha.png", "image/png", png))
# someone else: a web page, with a header that *claims* image/png
page = b"<html><script>steal_cookies()</script></html>"
print(upload_v1("profile.html", "image/png", page))
# and a name that replaces the site's own home page
print(upload_v1("index.html", "image/jpeg", page))
print("web folder now:", sorted(os.listdir(WEBROOT)))
Real output:
saved neha.png in the web folder
saved profile.html in the web folder
saved index.html in the web folder
web folder now: ['index.html', 'neha.png', 'profile.html']
Why it failed: the label and the name both came from the user, so the user decided everything. A page with a script, served from our own domain, can run in every visitor's browser (cross-site scripting, episode 40), and index.html replaced our own home page.
Problem 1b: check what the file IS
- An allowlist is a list of the only things accepted; everything else is refused. For uploads: "List allowed extensions" (OWASP), for example only
.png,.jpg,.pdf. A blocklist (list of bad extensions) always misses one. - Magic bytes (a file signature) are the first few bytes of a file that mark its real format, whatever its name says:
| Format | First bytes (hex) | As text |
|---|---|---|
| PNG | 89 50 4E 47 0D 0A 1A 0A |
\x89PNG\r\n\x1a\n |
| JPEG | FF D8 FF |
|
25 50 44 46 2D |
%PDF- |
|
| GIF | 47 49 46 38 37 61 / ...39 61 |
GIF87a / GIF89a |
OWASP adds that a signature check "should not be used on its own": a file can begin with the right bytes and still carry something harmful later, so it is one layer among several.
- A size limit stops one upload from filling the disk ("Set a file size limit").
- Renaming: "Change the filename to something generated by the application". We pick a random name with
secrets.token_hex, so the user cannot choose a name that replaces a file, contains odd characters or climbs out of the folder. Keep the original name in the database if you need to show it.
Demo 2: a safer upload
# Upload v2: check what the file IS, then store it safely
import os, secrets, tempfile
STORE = tempfile.mkdtemp() # outside the web folder, never served
ALLOWED = {".png": b"\x89PNG\r\n\x1a\n", ".jpg": b"\xff\xd8\xff",
".pdf": b"%PDF-"} # extension -> its magic bytes
MAX_BYTES = 2_000_000
def upload_v2(filename, data):
ext = os.path.splitext(filename)[1].lower()
if ext not in ALLOWED:
return f"rejected {filename}: extension {ext or 'none'} not allowed"
if not data.startswith(ALLOWED[ext]):
return f"rejected {filename}: content is not really {ext}"
if len(data) > MAX_BYTES:
return f"rejected {filename}: too big"
new_name = secrets.token_hex(8) + ext # we choose the name
open(os.path.join(STORE, new_name), "wb").write(data)
return f"saved {filename} as {new_name}"
png = b"\x89PNG\r\n\x1a\n" + b"...pixels..."
page = b"<html><script>steal_cookies()</script></html>"
print(upload_v2("neha.png", png))
print(upload_v2("profile.html", page))
print(upload_v2("photo.png", page))
print(upload_v2("index.html.png", page))
print(upload_v2("big.pdf", b"%PDF-" + b"0" * 3_000_000))
Real output (the random name differs each run):
saved neha.png as adffe90c09e800a7.png
rejected profile.html: extension .html not allowed
rejected photo.png: content is not really .png
rejected index.html.png: content is not really .png
rejected big.pdf: too big
Problem 1c: storing and scanning uploads
- Store outside the webroot: OWASP, "Store the files on a different server. If that's not possible, store them outside of the webroot." A file then cannot be opened directly by its address; your code serves it, after checking who may see it.
- Malware scanning: malware (episode 5) is software built to do harm. OWASP: "Run the file through an antivirus or a sandbox if available". Antivirus compares files with signatures of known malware; a sandbox is a sealed test machine where a file is opened and watched.
- Only authorised users upload: "Only allow authorized users to upload files".
- Defence in depth (episode 3): every single check can be fooled, so stack them.
Problem 2: path traversal
- A path is a file's address on disk, such as
invoices/inv-501.pdf. - In a path,
..means the folder one level up, and.means "this folder". - An absolute path starts at the top of the disk (
/etc/hostname); a relative path starts from some folder (invoices/x). - Path traversal (also directory traversal) means using
.., absolute paths or similar tricks in a file name to reach files outside the folder the app meant to use.
Real-life story: "Download your invoice" uses ?file=inv-501.pdf. Someone sends ../config/db.txt instead.
Demo 3: path traversal on a toy folder
# Download an invoice by name: /download?file=inv-501.pdf
import os, tempfile
root = os.path.realpath(tempfile.mkdtemp())
BASE = os.path.join(root, "invoices")
os.makedirs(BASE); os.makedirs(os.path.join(root, "config"))
open(os.path.join(BASE, "inv-501.pdf"), "w").write("Invoice 501: Rs 1,299")
open(os.path.join(root, "config", "db.txt"), "w").write("DB_PASSWORD=toy-pass-123")
def download_v1(name):
return open(os.path.join(BASE, name)).read()
def download_v2(name):
path = os.path.realpath(os.path.join(BASE, name)) # canonical form
if os.path.commonpath([path, BASE]) != BASE:
return "403 Forbidden: outside the invoices folder"
return open(path).read()
for name in ["inv-501.pdf", "../config/db.txt"]:
print(f"v1 {name!r:20} -> {download_v1(name)}")
for name in ["inv-501.pdf", "../config/db.txt", "/etc/hostname"]:
print(f"v2 {name!r:20} -> {download_v2(name)}")
Real output:
v1 'inv-501.pdf' -> Invoice 501: Rs 1,299
v1 '../config/db.txt' -> DB_PASSWORD=toy-pass-123
v2 'inv-501.pdf' -> Invoice 501: Rs 1,299
v2 '../config/db.txt' -> 403 Forbidden: outside the invoices folder
v2 '/etc/hostname' -> 403 Forbidden: outside the invoices folder
os.path.realpathgives the canonical form of a path: absolute, with every.,..and symbolic link worked out.os.path.commonpath([a, b])gives the longest shared folder of two paths, comparing whole folder names.- Watch out:
os.path.join("/srv/invoices", "/etc/passwd")returns/etc/passwd. If the second part is absolute,jointhrows the first part away. Checking the final resolved path catches this too.
Problem 2b: canonicalisation
Real-life example: one flat, many addresses ("Flat 4B", "2nd floor, left", "opposite the lift"). A guard first writes the one official address, then checks it.
- URL encoding writes characters as
%plus two hex digits:/is%2f,.is%2e. Servers decode it before using the value. - Normalising a path means rewriting it in its simplest form (
a/x/../bbecomesa/b). - Canonicalisation means turning every possible spelling of a value into one standard (canonical) form, and only then checking it.
Demo 4: why "just block ../" fails
# Why "just block ../" fails: check the FINAL form, not the raw text
from urllib.parse import unquote
import posixpath
def filter_v1(raw): # check the raw text, decode later
if "../" in raw:
return "blocked"
return "allowed -> " + unquote(raw)
def filter_v2(raw): # delete "../" once
return "allowed -> " + raw.replace("../", "")
def canonical(raw): # decode, normalise, THEN check
path = posixpath.normpath("/invoices/" + unquote(raw))
if not path.startswith("/invoices/"):
return "blocked -> " + path
return "allowed -> " + path
print("v1 ..%2fconfig/db.txt :", filter_v1("..%2fconfig/db.txt"))
print("v2 ....//config/db.txt :", filter_v2("....//config/db.txt"))
for raw in ["inv-501.pdf", "..%2fconfig/db.txt", "../../config/db.txt"]:
print(f"canonical {raw:20}:", canonical(raw))
Real output:
v1 ..%2fconfig/db.txt : allowed -> ../config/db.txt
v2 ....//config/db.txt : allowed -> ../config/db.txt
canonical inv-501.pdf : allowed -> /invoices/inv-501.pdf
canonical ..%2fconfig/db.txt : blocked -> /config/db.txt
canonical ../../config/db.txt : blocked -> /config/db.txt
v1 checked the raw text before decoding; v2 deleted ../ once, which turns ....// into ../. Never try to "clean" a dangerous path. Decode, normalise, resolve, then check against the allowed folder. Best of all, don't use user-supplied names: look files up by an ID stored in your database (see the mini project).
Problem 3: zip slip
- An archive is one file that holds many files, each with its own stored name:
.zip,.tar,.jar. - Extracting (unpacking) writes each entry to disk under its stored name.
- Zip slip: an archive entry whose name contains
../(or an absolute path) is written outside the destination folder. Snyk named it and disclosed it publicly on 5 June 2018; their research calls it "a form of directory traversal" exploited through archive extraction, affecting "thousands of projects" (including ones from HP, Amazon, Apache and Pivotal) across JavaScript, Ruby, .NET, Go and especially Java.
Demo 5: zip slip on a toy archive
# Zip slip: an uploaded zip whose entry name climbs out of the folder
import zipfile, io, os, tempfile
root = os.path.realpath(tempfile.mkdtemp())
DEST = os.path.join(root, "uploads", "batch1"); os.makedirs(DEST)
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w") as z: # a toy archive we build ourselves
z.writestr("photo1.png", "pixels")
z.writestr("../../app_config.py", "DEBUG = True # overwritten!")
def extract_v1(zf): # trusts every entry name
for name in zf.namelist():
out = os.path.join(DEST, name)
open(out, "wb").write(zf.read(name))
def extract_v2(zf): # canonicalise, then check
for name in zf.namelist():
out = os.path.realpath(os.path.join(DEST, name))
if os.path.commonpath([out, DEST]) != DEST:
print(" skipped unsafe entry:", name); continue
open(out, "wb").write(zf.read(name))
extract_v1(zipfile.ZipFile(buf))
print("v1 wrote outside uploads:", os.path.exists(f"{root}/app_config.py"))
os.remove(f"{root}/app_config.py")
extract_v2(zipfile.ZipFile(buf))
print("v2 wrote outside uploads:", os.path.exists(f"{root}/app_config.py"))
Real output:
v1 wrote outside uploads: True
skipped unsafe entry: ../../app_config.py
v2 wrote outside uploads: False
Library safety nets (both checked in Python 3.13 on 12 Nov 2026):
zipfile.ZipFile.extractall()already removes..from entry names, so the same archive's bad entry lands inside the destination asapp_config.py.tarfiletakesfilter="data", which refuses entries that would land outside the destination:
import tarfile, io, os, tempfile
root = os.path.realpath(tempfile.mkdtemp()); dest = os.path.join(root, "out"); os.makedirs(dest)
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w") as t:
data = b"DEBUG = True"
info = tarfile.TarInfo("../app_config.py"); info.size = len(data)
t.addfile(info, io.BytesIO(data))
buf.seek(0)
try:
tarfile.open(fileobj=buf).extractall(dest, filter="data")
except tarfile.FilterError as e:
print(type(e).__name__ + ":", e)
Real output (the temporary path differs each run):
OutsideDestinationError: '../app_config.py' would be extracted to '/tmp/tmppxkdogct/app_config.py', which is outside the destination
Also limit the number and total size of entries: a small compressed archive can unpack to gigabytes (a "zip bomb"). Practice 2 below adds those limits.
Problem 4: serialisation and insecure deserialisation
- Serialisation turns an object in memory into bytes or text so it can be saved (file, cookie, database) or sent (network, message queue).
- Deserialisation turns those bytes back into a live object.
- pickle is Python's built-in serialisation module. Java serialization uses the
Serializableinterface withObjectOutputStream/ObjectInputStream. - Insecure deserialisation means deserialising bytes an attacker could have written or changed, with a format that can do more than hold data.
- Base64 (episode 10) is an encoding, not encryption: anyone can decode and re-encode it.
- A cookie is a small value the server asks the browser to keep and send back; the user can read and change it.
Real-life example: flat-pack furniture. The box holds parts and instructions, and you follow every step. A pickle is the same: data plus steps to rebuild it, and one step can be "call this function". That step is an object's __reduce__ method, which returns a callable and its arguments.
The Python documentation's own warning: "The pickle module is not secure. Only unpickle data you trust." and "It is possible to construct malicious pickle data which will execute arbitrary code during unpickling."
Demo 6: a pickle in a cookie
This is the same harmless demonstration the Python docs use: the "payload" only calls print.
# The shop saves a shopping cart in a cookie with pickle
import pickle, base64
cart = {"user": "neha", "items": ["charger", "cable"]}
cookie = base64.b64encode(pickle.dumps(cart))
print("cookie:", cookie[:40], "...")
print("loaded:", pickle.loads(base64.b64decode(cookie)))
# But the cookie comes back from the BROWSER. Anyone can edit it.
class NotACart:
def __reduce__(self): # "to rebuild me, call this function"
return (print, ("!!! code ran while loading the cookie !!!",))
evil_cookie = base64.b64encode(pickle.dumps(NotACart()))
print("loading the edited cookie...")
pickle.loads(base64.b64decode(evil_cookie)) # the server just ran print()
class SafeUnpickler(pickle.Unpickler): # allow no functions at all
def find_class(self, module, name):
raise pickle.UnpicklingError(f"global '{module}.{name}' is forbidden")
import io
try:
SafeUnpickler(io.BytesIO(base64.b64decode(evil_cookie))).load()
except pickle.UnpicklingError as e:
print("SafeUnpickler:", e)
Real output (the cookie text is the same each run):
cookie: b'gASVMQAAAAAAAAB9lCiMBHVzZXKUjARuZWhhlIwF' ...
loaded: {'user': 'neha', 'items': ['charger', 'cable']}
loading the edited cookie...
!!! code ran while loading the cookie !!!
SafeUnpickler: global 'builtins.print' is forbidden
find_class is the method the unpickler calls whenever the data asks for a class or function; overriding it (the docs' "Restricting Globals" section) lets you forbid all of them or allow a small safe set. It reduces the risk, but the real rule is: never unpickle anything a user can touch.
Problem 4b: gadget chains (the idea)
- A gadget is an ordinary class that already exists in your app or its libraries, whose code runs automatically while an object is being rebuilt (for example a special read method, a setter or a hash function).
- A gadget chain arranges serialised objects so that rebuilding A calls B, which calls C, which finally does something harmful. The attacker sends no new code, only an arrangement of your own classes, which is why patching one class never ends the problem.
- The defence does not depend on knowing the chains: don't deserialise untrusted data; if you must, allow only the exact classes you expect.
Demo 7: Java serialization with an allowlist filter
import java.io.*;
import java.util.*;
public class Main {
record Cart(String user, int items) implements Serializable {}
static byte[] save(Object o) throws IOException {
var out = new ByteArrayOutputStream();
new ObjectOutputStream(out).writeObject(o);
return out.toByteArray();
}
static Object load(byte[] bytes, String filter) throws Exception {
var in = new ObjectInputStream(new ByteArrayInputStream(bytes));
if (filter != null) // allowlist: only these classes may be created
in.setObjectInputFilter(
ObjectInputFilter.Config.createFilter(filter));
return in.readObject();
}
public static void main(String[] args) throws Exception {
String allow = "Main$Cart;java.lang.*;!*";
System.out.println(load(save(new Cart("neha", 2)), allow));
byte[] other = save(new HashMap<>(Map.of("is_admin", true)));
System.out.println("no filter: " + load(other, null));
try { load(other, allow); }
catch (InvalidClassException e) {
System.out.println("filter: " + e.getMessage()); }
}
}
Real output:
Cart[user=neha, items=2]
no filter: {is_admin=true}
filter: filter status: REJECTED
ObjectInputFiltercame with JEP 290, "Filter Incoming Serialization Data", in Java 9. A pattern likeMain$Cart;java.lang.*;!*is read left to right:;separates rules, a trailing*matches a prefix, and!rejects. So: allowCart, allowjava.langclasses, reject everything else.- Oracle's Java 21 docs: "A serialization filter is not enabled or configured by default." You must switch one on (per stream as here, or JVM-wide with the
jdk.serialFilterproperty). An allowlist is safer than a list of known-bad classes.
Problem 4c: the real fix, plain data with a schema
- JSON (JavaScript Object Notation) can hold only objects (dictionaries), arrays (lists), strings, numbers,
true,falseandnull. It cannot name a class or a function, so loading it never runs code. The pickle docs: "Safer serialization formats such as json may be more appropriate if you are processing untrusted data." - A schema is a written rule for the exact shape of the data: which keys, which types, which limits. Validating against it refuses anything unplanned.
Demo 8: JSON with a schema check
# The fix: plain data (JSON) + a schema check. JSON can't call functions.
import json
SCHEMA = {"user": str, "items": list} # exactly these keys and types
def load_cart(text):
data = json.loads(text) # only dicts, lists, text, numbers
if not isinstance(data, dict) or set(data) != set(SCHEMA):
raise ValueError(f"wrong keys: {sorted(data)}")
for key, typ in SCHEMA.items():
if not isinstance(data[key], typ):
raise ValueError(f"{key} must be a {typ.__name__}")
items = data["items"]
if len(items) > 50 or not all(isinstance(i, str) for i in items):
raise ValueError("items: at most 50 strings")
return data
tests = ['{"user": "neha", "items": ["charger", "cable"]}',
'{"user": "neha", "items": ["charger"], "is_admin": true}',
'{"user": "neha", "items": "charger"}',
'{"__reduce__": "print"}']
for t in tests:
try:
print("OK ", load_cart(t))
except ValueError as e:
print("REFUSED", e)
Real output:
OK {'user': 'neha', 'items': ['charger', 'cable']}
REFUSED wrong keys: ['is_admin', 'items', 'user']
REFUSED items must be a list
REFUSED wrong keys: ['__reduce__']
If the user must not change the data at all, either keep it on the server and put only a random ID in the cookie, or sign it with an HMAC (episode 12) and verify the signature before using it.
The whole chain
- A web page was saved as a "photo" → extension allowlist, magic bytes, size limit.
- The user's name went into the web folder → our own random name, storage outside the webroot, malware scanning.
../read the password file → resolve withrealpath, check withcommonpath.- Simple filters were bypassed → canonicalise first, or use IDs instead of names.
- A zip wrote outside its folder → check every entry, use safe extraction, limit size.
- A cookie ran code → no pickle on user data; JSON with a schema; Java filters as a backstop.
Common mistakes (and what they cause)
| Mistake | Real message / result | Why it's wrong | Fix |
|---|---|---|---|
Trusting Content-Type |
saved profile.html in the web folder |
the label is user-chosen | extension allowlist + magic bytes |
| Saving under the user's file name | web folder now: ['index.html', ...] (home page replaced) |
the user chooses where it goes | random name, outside the webroot |
open(os.path.join(BASE, name)) |
v1 '../config/db.txt' -> DB_PASSWORD=toy-pass-123 |
.. and absolute paths escape |
realpath + commonpath |
| Check, then decode | v1 ..%2fconfig/db.txt : allowed -> ../config/db.txt |
checked a spelling, not the meaning | decode, normalise, then check |
path.startswith(BASE) |
../files_backup/x broken=True |
compares text, not folders | os.path.commonpath([path, BASE]) == BASE |
| Extracting archive names as-is | v1 wrote outside uploads: True |
entry names are user input | per-entry check, filter="data" for tar |
pickle.loads(cookie) |
!!! code ran while loading the cookie !!! |
pickle is instructions | JSON + schema, or server-side storage |
| JSON with single quotes | json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1) |
JSON needs double quotes | send real JSON (json.dumps) |
Unfiltered ObjectInputStream |
no filter: {is_admin=true} |
any class on the class path loads | ObjectInputFilter allowlist |
Mini project: a safe document store
Combines everything: allowlist + magic bytes + size limit, random IDs, storage outside any webroot, downloads by ID only with an owner check (the same 404 for "not yours" and "doesn't exist"), and metadata stored as JSON.
# Mini project: a safe document store for the shop
# - uploads: allowlist + magic bytes + size limit + random id, stored outside the webroot
# - downloads: by id only (no user file names), and only by the owner
# - metadata saved as JSON (plain data), never pickle
import json, os, secrets, tempfile
class DocStore:
TYPES = {".pdf": b"%PDF-", ".png": b"\x89PNG\r\n\x1a\n", ".jpg": b"\xff\xd8\xff"}
MAX = 2_000_000
def __init__(self):
self.dir = os.path.realpath(tempfile.mkdtemp()) # not under any webroot
self.meta_path = os.path.join(self.dir, "meta.json")
self.meta = {}
def upload(self, user, filename, data):
ext = os.path.splitext(filename)[1].lower()
if ext not in self.TYPES:
return None, "extension not allowed"
if not data.startswith(self.TYPES[ext]):
return None, "content does not match extension"
if len(data) > self.MAX:
return None, "file too big"
doc_id = secrets.token_hex(12)
with open(os.path.join(self.dir, doc_id + ext), "wb") as f:
f.write(data)
self.meta[doc_id] = {"owner": user, "ext": ext, "original": filename[:100]}
with open(self.meta_path, "w") as f:
json.dump(self.meta, f) # plain data on disk
return doc_id, "saved"
def download(self, user, doc_id):
entry = self.meta.get(doc_id)
if entry is None or entry["owner"] != user: # same answer for both
return None, "404 not found"
with open(os.path.join(self.dir, doc_id + entry["ext"]), "rb") as f:
return f.read(), "200 ok"
store = DocStore()
doc, msg = store.upload("neha", "invoice.pdf", b"%PDF-1.7 toy invoice")
print("neha upload :", msg, "| id length", len(doc))
print("bad upload :", store.upload("neha", "evil.html", b"<html>")[1])
print("fake pdf :", store.upload("neha", "x.pdf", b"<html>")[1])
print("neha download :", store.download("neha", doc))
print("ravi download :", store.download("ravi", doc)[1])
print("path in id :", store.download("neha", "../meta")[1])
Real output:
neha upload : saved | id length 24
bad upload : extension not allowed
fake pdf : content does not match extension
neha download : (b'%PDF-1.7 toy invoice', '200 ok')
ravi download : 404 not found
path in id : 404 not found
Practice
- Add GIF images (
GIF87aorGIF89a) to the upload allowlist. - Make the zip extractor refuse archives with more than 100 entries or more than 10 MB unpacked, without trusting the sizes the archive declares.
- Break the quiz's
startswithcheck with a real input, then fix it. - List three places in apps you use where data is deserialised (cookies, caches, message queues, saved files) and say which format you would use.
Practice 1 solution
# Practice 1 solution: allow GIF images (two possible magic headers)
import os
ALLOWED = {".png": (b"\x89PNG\r\n\x1a\n",), ".jpg": (b"\xff\xd8\xff",),
".pdf": (b"%PDF-",), ".gif": (b"GIF87a", b"GIF89a")}
def check(filename, data):
ext = os.path.splitext(filename)[1].lower()
if ext not in ALLOWED:
return f"rejected {filename}: extension not allowed"
if not data.startswith(ALLOWED[ext]): # startswith accepts a tuple
return f"rejected {filename}: content is not really {ext}"
return f"accepted {filename}"
print(check("cat.gif", b"GIF89a" + b"\x01\x00..."))
print(check("old.gif", b"GIF87a" + b"\x01\x00..."))
print(check("fake.gif", b"<html>hi</html>"))
accepted cat.gif
accepted old.gif
rejected fake.gif: content is not really .gif
Practice 2 solution
# Practice 2 solution: safe extraction with limits on entries and total size
import zipfile, io, os, tempfile
root = os.path.realpath(tempfile.mkdtemp())
DEST = os.path.join(root, "uploads"); os.makedirs(DEST)
MAX_ENTRIES, MAX_TOTAL = 100, 10_000_000 # 100 files, 10 MB unpacked
def extract_safe(zf):
infos = zf.infolist()
if len(infos) > MAX_ENTRIES:
return f"refused: {len(infos)} entries (max {MAX_ENTRIES})"
total = sum(i.file_size for i in infos) # sizes declared in the zip
if total > MAX_TOTAL:
return f"refused: {total:,} bytes unpacked (max {MAX_TOTAL:,})"
written = 0
for info in infos:
out = os.path.realpath(os.path.join(DEST, info.filename))
if os.path.commonpath([out, DEST]) != DEST or info.is_dir():
continue
data = zf.read(info)
written += len(data)
if written > MAX_TOTAL: # don't trust declared sizes
return "refused: real size over the limit"
open(out, "wb").write(data)
return f"extracted {len(infos)} entries, {written:,} bytes"
def make_zip(entries):
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as z:
for name, data in entries: z.writestr(name, data)
return zipfile.ZipFile(buf)
print(extract_safe(make_zip([("a.png", b"pixels"), ("b.png", b"more")])))
print(extract_safe(make_zip([(f"f{i}.txt", b"x") for i in range(500)])))
print(extract_safe(make_zip([("huge.txt", b"0" * 50_000_000)])))
extracted 2 entries, 10 bytes
refused: 500 entries (max 100)
refused: 50,000,000 bytes unpacked (max 10,000,000)
Practice 3 solution
# Practice 3 solution: the quiz check, broken and fixed
import os
BASE = "/srv/files"
def safe_broken(name):
path = os.path.realpath(os.path.join(BASE, name))
return path.startswith(BASE) # compares TEXT
def safe_fixed(name):
path = os.path.realpath(os.path.join(BASE, name))
return os.path.commonpath([path, BASE]) == BASE # compares folders
for name in ["report.pdf", "../files_backup/x", "../../etc/passwd"]:
print(f"{name:20} broken={safe_broken(name)!s:5} fixed={safe_fixed(name)}")
report.pdf broken=True fixed=True
../files_backup/x broken=True fixed=False
../../etc/passwd broken=False fixed=False
Practice 4 discussion
Typical places: session or cart cookies, server caches (Redis/memcached), background job queues, files saved by desktop apps, machine-learning model files. For anything a user or another system could change, use a plain data format (JSON, or Protocol Buffers with a fixed schema), validate it against a schema, and sign it (HMAC) if it must not be changed. Never use pickle or unfiltered Java serialization across a trust boundary.
Quick quiz
BASE = "/srv/files"
def safe(name):
path = os.path.realpath(os.path.join(BASE, name))
return path.startswith(BASE)
Is this path check safe? A: yes, realpath fixes everything. B: no, "../files_backup/x" passes. C: no, realpath is too slow. D: yes, but only on Windows.
Answer
B. /srv/files_backup/x starts with the text /srv/files, so startswith returns True even though it is a different folder. Compare whole folder names: os.path.commonpath([path, BASE]) == BASE, or path.startswith(BASE + os.sep). Practice 3 shows it running.
Interview questions
- Why isn't checking the Content-Type enough for uploads? It is a label the client writes and can be spoofed. Check the extension against an allowlist and the magic bytes, limit size, rename, store outside the webroot and scan.
- Why rename uploaded files? So the user can't overwrite files, inject odd characters, or traverse folders through the name; the server controls the name.
- What is path traversal and how do you prevent it? Using
..or absolute paths in a file name to escape a folder. Avoid user-supplied names (use IDs); otherwise canonicalise (realpath) and check the result is inside the base folder by comparing path components. - What is canonicalisation, and why does order matter? Converting a value to one standard form before checking it. Checking before decoding or normalising lets other spellings (
%2f,....//) through. - What is zip slip? Path traversal through archive entry names during extraction; fix with per-entry canonical checks or library safe modes, plus size and count limits.
- Why is
pickle.loadson untrusted data dangerous? Pickle data can tell the loader to call any function (__reduce__), so loading it can execute arbitrary code. - What is a gadget chain? A sequence of existing classes whose automatic rebuild code calls each other and ends in a harmful action; the attacker supplies only serialised data.
- How do you deserialise safely in Java? Avoid native serialization for untrusted input; if unavoidable, use an
ObjectInputFilterallowlist (JEP 290), and limit depth and size. - What should replace native serialization? A data-only format such as JSON with schema validation, and a signature if integrity matters.
Glossary
| Term | Meaning, with an example |
|---|---|
| Upload | Sending a file from your device to a server, e.g. a profile photo. |
| Unrestricted file upload | Accepting files without checking type, name, size or destination; e.g. a "photo" that is an HTML page. |
| Content-Type / MIME type | A label in the request saying what the data claims to be, e.g. image/png; chosen by the client, can be spoofed. |
| Spoofing | Faking a value to look legitimate, e.g. writing image/png on a web page. |
| Extension | The end of a file name after the last dot, e.g. .pdf; user-chosen. |
| Allowlist | A list of the only accepted values, e.g. .png .jpg .pdf; the opposite of a blocklist. |
| Magic bytes / file signature | The first bytes that mark a file's real format, e.g. %PDF- for PDF. |
| Webroot | The folder a web server serves to the world, e.g. /var/www/html. |
| Storing outside the webroot | Keeping uploads where no URL reaches them, served only through code that checks access. |
| Renaming uploads | Replacing the user's file name with one the server generates, e.g. 8d99de4877cdf0b2.png. |
| Malware scanning | Checking uploads with antivirus signatures or a sandbox before anyone opens them. |
| Antivirus | Software that recognises known malware by signatures and behaviour. |
| Sandbox | A sealed test environment where a file is opened and watched safely. |
| Path | A file's address on disk, e.g. invoices/inv-501.pdf. |
.. |
"The folder one level up" in a path. |
| Absolute / relative path | A path from the top of the disk (/etc/hostname) vs from some folder (config/db.txt). |
| Path (directory) traversal | Using .. or absolute paths to reach files outside the intended folder. |
| Canonical form | The one standard, fully resolved form of a value, e.g. realpath("/a/x/../b") → /a/b. |
| Canonicalisation | Converting every spelling to the canonical form before checking it. |
| URL encoding | Writing characters as % + hex, e.g. / as %2f. |
| Normalising | Rewriting a path in its simplest form, e.g. a/./b → a/b. |
realpath / commonpath |
Python functions that resolve a path completely / find the shared folder of paths. |
| Archive | One file holding many named files, e.g. .zip, .tar, .jar. |
| Zip slip | Path traversal through archive entry names during extraction (Snyk, 2018). |
| Zip bomb | A small archive that unpacks to a huge size. |
| Serialisation | Turning an object into bytes/text to save or send, e.g. pickle.dumps(cart). |
| Deserialisation | Turning those bytes back into an object, e.g. pickle.loads(data). |
| Insecure deserialisation | Deserialising data an attacker could control with a format that can do more than hold data. |
| pickle | Python's built-in serialisation module; "not secure" for untrusted data. |
__reduce__ |
The method telling pickle how to rebuild an object: a function plus its arguments. |
find_class |
The unpickler method called for every class/function the data asks for; override it to restrict them. |
| Gadget | An existing class whose code runs automatically during deserialisation. |
| Gadget chain | Gadgets arranged so rebuilding one triggers the next, ending in a harmful action. |
| Java serialization | Java's built-in object-to-bytes mechanism (Serializable, ObjectInputStream). |
ObjectInputFilter |
Java's filter (JEP 290, Java 9) that accepts or rejects classes during deserialisation. |
| Base64 | An encoding of bytes as text; not encryption, anyone can reverse it. |
| Cookie | A small value the browser stores and sends back; the user can read and edit it. |
| JSON | A data-only text format: objects, arrays, strings, numbers, true, false, null. |
| Schema | A written rule for the exact keys, types and limits data must have. |
| HMAC | A keyed signature that proves data was not changed (episode 12). |
| Defence in depth | Stacking several independent checks so one failure isn't fatal (episode 3). |
Summary
- A file's name and Content-Type are user input. Check the extension against an allowlist, the magic bytes and the size; give the file your own random name; store it outside the webroot; scan it; let only authorised users upload.
- Path traversal: never glue a user's name onto a folder. Use IDs, or resolve with
realpathand check withcommonpath. - Canonicalise before checking: decode and normalise first; never "clean" a path.
- Zip slip: archive entry names are user input; check each one, use
filter="data"for tar, limit count and size. - Insecure deserialisation: pickle and native Java serialization can run code while loading. Use JSON with a schema; sign data you hand to users; use
ObjectInputFilterallowlists where Java serialization can't be avoided.
Next episode: authentication failures and security logging failures: brute force, throttling vs lockout, CAPTCHA, account enumeration, password resets, remember-me tokens, and logging and alerting.