SyntaxPilot
Courses/Cybersecurity Fundamentals/Lesson 72
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Cybersecurity Fundamentals · Lesson 72 of 90

Security logging fundamentals

log sources (OS, application, network, cloud, identity)what to loglog levelsstructured loggingtime synchronisation (NTP)log integrity and tamper resistancelog retentionPII in logscentralised loggingsyslogWindows Event Logs and key event IDsSysmonaudit trails

These are the notes for Cybersecurity Fundamentals (defensive), episode 72, the first episode of the detection and response part of the course. Defences fail; when they do, the only way to answer "what happened?" is a record written at the time. This episode builds security logging step by step: what a useful log entry must contain, where logs come from, separating noise from danger, writing logs a machine can trust and an attacker can't forge, putting every system on one clock, protecting logs from tampering and from leaking secrets, keeping them long enough and away from the attacker, and the Windows events every defender should know. Every term is defined the first time it appears; the Glossary collects them.

Scope. Defensive and educational only. Every program is a toy simulation on made-up data; IP addresses come from the documentation ranges. Samples were run on 14 Nov 2026 in the SyntaxPilot compiler (Python 3).

The morning after

Real-life example: a housing society's gate register. Every visitor is written down: name, flat, time in, time out. When something goes missing, the secretary opens the register. Now a company finds its customer list posted online on Monday morning. The boss asks: who, when, how, how many? The logs say error, error, done.

  • Log: a record a system writes about what it did, one entry (line) per event.
  • Event: anything that happened: a login, an error, a file read, a setting changed.
  • Security logging: deliberately recording the events needed to detect, investigate and prove what happened, and protecting those records.

Step 1: what to log

Demo 1: a log with no story

# Something went wrong last night. Here is everything the app wrote down.
log = ["error", "error", "login ok", "error", "done"]
for line in log:
    print(line)
print("Who? When? From where? Which account? Nobody knows.")

Real output:

error
error
login ok
error
done
Who? When? From where? Which account? Nobody knows.

Demo 2: when, where, who, what

# OWASP: every security event says WHEN, WHERE, WHO and WHAT.
def event(when, where, who, what, outcome):
    return f"{when:10} {where:11} {who:16} {what:12} {outcome}"

print(event("WHEN", "WHERE", "WHO", "WHAT", "OUTCOME"))
print(event("23:41:07Z", "api/login", "ravi 192.0.2.44", "login", "FAIL bad password"))
print(event("23:41:09Z", "api/login", "ravi 192.0.2.44", "login", "FAIL bad password"))
print(event("23:41:15Z", "api/login", "ravi 192.0.2.44", "login", "SUCCESS"))
print(event("23:42:02Z", "admin/users", "ravi 192.0.2.44", "export users", "SUCCESS 9,812 rows"))

Real output:

WHEN       WHERE       WHO              WHAT         OUTCOME
23:41:07Z  api/login   ravi 192.0.2.44  login        FAIL bad password
23:41:09Z  api/login   ravi 192.0.2.44  login        FAIL bad password
23:41:15Z  api/login   ravi 192.0.2.44  login        SUCCESS
23:42:02Z  admin/users ravi 192.0.2.44  export users SUCCESS 9,812 rows

The same night now tells a story: two failed logins on Ravi's account from one IP, a success six seconds later, then the session exports 9,812 users.

  • The four attributes (OWASP Logging Cheat Sheet): when (timestamp in an international format; event time), where (application, host, endpoint, code location), who (user identity and source address), what (event type, severity, outcome, description).
  • Events to always log (OWASP): authentication successes and failures, authorization failures (access denied), input validation failures, session management failures, application errors and system events, application startups and shutdowns, and high-risk actions: creating users, changing privileges, admin actions, use of shared or break-glass accounts, access to sensitive data, data import and export, file uploads.
  • Input validation failure: a request rejected because its data was invalid. OWASP notes these are suspicious events, not proof of an attack: judge them by repetition and context.

Log sources

Source Examples of what it records
operating system logins, new processes, services, privilege use
application business events: payments, exports, password changes
network firewall allow/deny, DNS queries, web proxy, VPN
cloud API calls and configuration changes (cloud audit logs, episode 64)
identity single sign-on, MFA prompts, admin and group changes
  • Log source: any system that writes logs you collect. One attack crosses many sources, so they must be collected together.

Step 2: log levels and syslog

Demo 3: Python's logging module

# Log levels: how serious is each message? Production keeps INFO and above.
import logging, sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO,
                    format="%(levelname)-8s %(message)s")
log = logging.getLogger("chai-pay")
log.debug("cart contents: 2 x masala chai")         # too detailed for production: dropped
log.info("user ravi logged in")
log.warning("5 failed logins for ravi in 1 minute")
log.error("payment service timed out")
log.critical("database unreachable: all payments down")

SYSLOG = ["Emergency", "Alert", "Critical", "Error", "Warning", "Notice", "Informational", "Debug"]
print("syslog severities:", ", ".join(f"{i}={n}" for i, n in enumerate(SYSLOG[:4])), "...")

Real output:

INFO     user ravi logged in
WARNING  5 failed logins for ravi in 1 minute
ERROR    payment service timed out
CRITICAL database unreachable: all payments down
syslog severities: 0=Emergency, 1=Alert, 2=Critical, 3=Error ...
  • Log level: how serious a message is. Python's levels: DEBUG (detail for developers), INFO (normal events), WARNING (something odd), ERROR (something broke), CRITICAL (the system is failing). The configured level is a threshold: level=logging.INFO drops DEBUG.
  • Syslog: the standard way Unix-like systems and network devices send log messages. RFC 5424 defines eight severities, lower is more urgent:
Code Severity Code Severity
0 Emergency 4 Warning
1 Alert 5 Notice
2 Critical 6 Informational
3 Error 7 Debug
  • Security events are not always errors: a successful login is INFO, yet it is exactly what investigators need. Mark security events as security-relevant and never drop them.

Step 3: structured logging

Demo 4: log injection vs JSON

# Log injection: an attacker types a username containing a NEW LINE.
import json
username = "ravi\nINFO login OK user=admin"     # typed into the login box

print("--- plain text log ---")
print(f"WARN login FAIL user={username}")            # one event becomes two lines!

print("--- structured (JSON) log ---")
line = json.dumps({"event": "login FAIL", "user": username})
print(line)
print("a tool reads the field back:", repr(json.loads(line)["user"]))

Real output:

--- plain text log ---
WARN login FAIL user=ravi
INFO login OK user=admin
--- structured (JSON) log ---
{"event": "login FAIL", "user": "ravi\nINFO login OK user=admin"}
a tool reads the field back: 'ravi\nINFO login OK user=admin'
  • Log injection: putting untrusted input into a log in a way that forges or corrupts entries; here a new line character (\n, a line feed; with \r, carriage return, called CR/LF) splits one event into two and fakes "login OK user=admin".
  • Structured logging: each entry is data with named fields, usually JSON (JavaScript Object Notation: {"key": "value"}), not a free-text sentence. The serializer escapes the new line as \n inside the field, so the entry stays one event and tools read back the exact value.
  • OWASP: prefer structured formats like JSON, sanitize and encode all event data, and limit the size of user-supplied values. Structured logs also make questions easy: failures per user, per IP, per hour.

Step 4: one clock for everyone

Demo 5: clock skew

# Two servers, two clocks. The web server's clock is 4 minutes fast.
from datetime import datetime, timedelta
t = datetime(2026, 11, 13, 23, 40, 0)
events = [("db server ", t + timedelta(seconds=30), "9,812 rows read"),
          ("web server", t + timedelta(seconds=10), "admin login from 192.0.2.44")]
skew = {"db server ": timedelta(0), "web server": timedelta(minutes=4)}

def timeline(fix):
    rows = [(when + (timedelta(0) if fix else skew[who]), who, what) for who, when, what in events]
    for when, who, what in sorted(rows): print(f"   {when:%H:%M:%S}  {who}  {what}")

print("clocks as they were (rows read BEFORE the login?):"); timeline(fix=False)
print("all clocks synced by NTP:");                           timeline(fix=True)

Real output:

clocks as they were (rows read BEFORE the login?):
   23:40:30  db server   9,812 rows read
   23:44:10  web server  admin login from 192.0.2.44
all clocks synced by NTP:
   23:40:10  web server  admin login from 192.0.2.44
   23:40:30  db server   9,812 rows read
  • Clock skew: different machines' clocks disagreeing. With a 4-minute fast web server, the rows appear to be read before the admin logged in: the timeline lies.
  • NTP (Network Time Protocol): every machine regularly sets its clock from trusted time servers.
  • UTC (Coordinated Universal Time): log in UTC and write the zone; the Z ("Zulu") at the end of 23:41:07Z means UTC.
  • India: the CERT-In directions of 28 April 2022 require organisations to synchronise ICT system clocks with NTP servers of NIC (National Informatics Centre) or NPL (National Physical Laboratory).

Step 5: logs nobody can fake, leak or erase

Demo 6: a hash chain

# Tamper-evident logs: each entry stores the hash of the entry before it (a hash chain).
from hashlib import sha256
def h(prev, text): return sha256((prev + text).encode()).hexdigest()[:10]

def build(lines):
    chain, prev = [], "0" * 10
    for text in lines:
        prev = h(prev, text); chain.append((text, prev))
    return chain

def verify(chain):
    prev = "0" * 10
    for i, (text, stored) in enumerate(chain):
        if h(prev, text) != stored: return f"TAMPERED at entry {i}: '{text}'"
        prev = stored
    return "chain OK"

log = build(["23:41 ravi login FAIL", "23:41 ravi login OK", "23:42 ravi export users"])
print("original      :", verify(log))
log[2] = ("23:42 ravi viewed dashboard", log[2][1])      # the intruder edits a line
print("after editing :", verify(log))

Real output:

original      : chain OK
after editing : TAMPERED at entry 2: '23:42 ravi viewed dashboard'
  • Log integrity: confidence that log entries haven't been changed or deleted.
  • Tamper-evident: changes can't be hidden; they are detected. (Tamper-proof would mean they can't happen at all.)
  • Hash chain: each entry stores the hash (episode 13) of its own text plus the previous entry's hash; editing any entry breaks every hash after it.
  • OWASP also recommends: build in tamper detection, copy logs to read-only media quickly, keep them in a separate store, write with an account that can only append, restrict read access, and log all access to the logs.

Demo 7: masking PII and secrets

# PII and secrets in logs: mask them BEFORE the line is written.
import re
RULES = [(r"password=\S+", "password=***"),
         (r"token=\S+",    "token=***"),
         (r"\b(\d{2})\d{6}(\d{2})\b", r"\1******\2")]      # 10-digit phone: keep 2 + 2

def safe(line):
    for pattern, repl in RULES: line = re.sub(pattern, repl, line)
    return line

raw = "user=priya phone=9876543210 password=Chai@2026 token=eyJhbGciOi.x"
print("unsafe:", raw)
print("safe  :", safe(raw))

Real output:

unsafe: user=priya phone=9876543210 password=Chai@2026 token=eyJhbGciOi.x
safe  : user=priya phone=98******10 password=*** token=***
  • PII (personally identifiable information): data that identifies a person: name, phone number, address, ID numbers.
  • Never log (OWASP): passwords, access tokens, encryption keys and other primary secrets, payment card or bank data, database connection strings, source code. Remove or hash session IDs; mask, hash or pseudonymise personal data.
  • Masking: hiding part of a value (98******10). Pseudonymisation: replacing an identifier with a consistent stand-in so events still link up.
  • Log retention: how long logs are kept. Too short loses evidence (attacks are often found months later); too long stores risk. CERT-In's 2022 directions: keep logs of all ICT systems for a rolling 180 days, within Indian jurisdiction. India's data protection law (the DPDP Act) is covered in episode 81.

Demo 8: centralised logging

# Centralised logging: every line is copied to a separate log server as it is written.
host_log, central = [], []          # central = a different machine the intruder can't log in to

def write(line):
    host_log.append(line)
    central.append(("web-01", line))         # shipped at once (e.g. syslog over TLS)

write("02:10:09 admin logon from 192.0.2.44")
write("02:12:05 user 'support2' created")
host_log.clear()                             # the intruder wipes the server's own log
write("02:13:40 Security log cleared")

print("on the host   :", host_log)
print("central store :")
for src, line in central: print("   ", src, line)

Real output:

on the host   : ['02:13:40 Security log cleared']
central store :
    web-01 02:10:09 admin logon from 192.0.2.44
    web-01 02:12:05 user 'support2' created
    web-01 02:13:40 Security log cleared
  • Centralised logging: every source ships its logs, as they're written, to a separate log server or service the attacker can't easily reach. The host log was wiped, but the central copy kept the logon, the new user and the clearing itself.
  • Transport: syslog (ideally syslog over TLS, as OWASP recommends) or log-shipping agents.
  • Log aggregation: collecting logs from many sources into one searchable store; next episode feeds it into a SIEM.

Step 6: Windows evidence

Demo 9: key Windows event IDs

# Windows Security log, exported (toy): which event IDs tell a story?
NAMES = {4624: "logon success", 4625: "logon FAILED", 4688: "new process",
         4720: "user account created", 1102: "audit log CLEARED"}
events = [("02:10:01", 4625, "admin from 192.0.2.44")] * 6 + [
          ("02:10:09", 4624, "admin from 192.0.2.44, logon type 10"),
          ("02:11:30", 4688, "cmd.exe started by admin"),
          ("02:12:05", 4720, "new user 'support2'"),
          ("02:13:40", 1102, "Security log cleared by admin")]

fails = sum(1 for _, eid, _ in events if eid == 4625)
print(f"4625 x {fails}  ({NAMES[4625]})")
for t, eid, detail in events:
    if eid != 4625: print(f"{t}  {eid}  {NAMES[eid]:21} {detail}")
if fails >= 5 and any(e == 4624 for _, e, _ in events):
    print("ALERT: many failures, then success: password guessed?")

Real output:

4625 x 6  (logon FAILED)
02:10:09  4624  logon success         admin from 192.0.2.44, logon type 10
02:11:30  4688  new process           cmd.exe started by admin
02:12:05  4720  user account created  new user 'support2'
02:13:40  1102  audit log CLEARED     Security log cleared by admin
ALERT: many failures, then success: password guessed?
  • Windows Event Logs: Windows' built-in logs (Application, System, Security), viewed in Event Viewer. Each entry has an event ID, a number for the kind of event.
Event ID Meaning
4624 an account was successfully logged on
4625 an account failed to log on
4634 an account was logged off
4672 special privileges assigned to new logon (an admin logon)
4688 a new process has been created
4720 a user account was created
4732 a member was added to a security-enabled local group
4740 a user account was locked out
1102 the audit log was cleared
  • Logon type (inside 4624/4625): 2 Interactive (at the keyboard), 3 Network (e.g. a file share), 10 RemoteInteractive (Remote Desktop), among others.
  • Failure status: 4625 carries a status code; Microsoft lists 0xC000006A as a bad password. Many 4625s followed by a 4624 for the same account suggests a guessed password; a 1102 is always worth an alert.

Demo 10: Sysmon process creation

# Sysmon event 1 (process create) records the full command line and the PARENT process.
procs = [
  {"parent": "explorer.exe", "image": "winword.exe",    "cmd": "winword.exe report.docx"},
  {"parent": "winword.exe",  "image": "powershell.exe", "cmd": "powershell -enc SQBFAFgA..."},
  {"parent": "services.exe", "image": "svchost.exe",    "cmd": "svchost.exe -k netsvcs"},
]
for p in procs:
    odd = p["parent"] == "winword.exe" and p["image"] == "powershell.exe"
    print(f"{p['parent']:13} -> {p['cmd']:30}", "<- SUSPICIOUS" if odd else "")

Real output:

explorer.exe  -> winword.exe report.docx        
winword.exe   -> powershell -enc SQBFAFgA...    <- SUSPICIOUS
services.exe  -> svchost.exe -k netsvcs         
  • Sysmon (System Monitor, Microsoft): detailed, low-level Windows activity logging, written to Applications and Services Logs > Microsoft > Windows > Sysmon > Operational, with UTC timestamps.
  • Event 1, Process Create: full command line, parent process (which program started it), file hashes and a process GUID.
  • Event 3 network connection (off by default because of volume), event 11 file created, event 22 DNS query.
  • Parent-child anomaly: an Office app starting PowerShell is a classic sign of a malicious document (episode 69's EDR rule).

Audit trails

  • Audit trail: a log of who did what to which record, and when, ideally with the values before and after (a manager approved a Rs 900 refund).
  • Append-only: entries are added, never edited or deleted.
  • Separation of duties (episode 3): admins can't delete their own audit entries.
  • Audit trails provide accountability and non-repudiation (episode 2): nobody can credibly deny their actions.

The whole chain

  1. "error, error, done" → when, where, who, what, from every log source.
  2. Drowning in lines → log levels, syslog severities.
  3. A forged line through a new line → structured JSON logs.
  4. Clocks disagree → NTP, UTC.
  5. An edited log → hash chain; secrets in logs → masking; how long → retention.
  6. A cleared host log → centralised logging.
  7. Windows → event IDs, Sysmon; people → audit trails.

Common mistakes (and the fix)

Mistake What you see Why Fix
print("error") style logs error / error / done no when, where, who, what log all four attributes
f"login FAIL user={username}" a forged INFO login OK user=admin line new line in user input structured JSON logs, encode values
logging the full request password=Chai@2026 token=eyJ... in the log secrets and PII copied into logs mask before writing; never log secrets
each server's own clock rows read "before" the login clock skew NTP everywhere, UTC timestamps
logs only on the host 1102 and the evidence is gone the intruder controls the host ship logs to a central store at once
DEBUG in production disk fills, secrets leak, real alerts buried too much detail INFO and above; DEBUG only when needed
deleting logs after 7 days breach found in month 3, no evidence retention too short 180 days or more (CERT-In)
nobody reads the logs the attack is logged but unnoticed no monitoring alerts and a SIEM (next episode)

Mini project: a tiny log pipeline

Write a program that: (1) takes raw events as dictionaries, (2) masks PII and secrets, (3) adds a UTC timestamp and level, (4) writes them as JSON lines with a hash chain, (5) "ships" each line to a central list, and (6) runs two detection rules over the central list: five or more login FAIL for one user followed by login OK, and any log cleared event. Then tamper with the host copy and show that verification and the central copy still tell the truth.

Practice

1. Make the mask toy hide email addresses too.

Solution
RULES.append((r"([\w.])[\w.]*@([\w.-]+)", r"\1***@\2"))
print(safe("user=priya email=priya.k@mail.example"))
# user=priya email=p***@mail.example

2. Add event 4740 (account locked out) to the Windows toy.

Solution
NAMES[4740] = "account LOCKED OUT"
events.append(("02:10:08", 4740, "admin locked after 6 failures"))

3. Will the hash-chain toy detect a deleted entry?

Solution

Yes, unless it is the last one. Deleting entry 1 means entry 2's stored hash was computed with entry 1's hash as prev, so the recomputed hash differs and verification reports tampering at the new position. To catch deletion of the last entries too, also store the latest hash (the "head") somewhere separate, such as the central log server.

log = build(["a", "b", "c"]); del log[1]
print(verify(log))   # TAMPERED at entry 1: 'c'

4. Why is a successful login an INFO event but still security-relevant?

Solution

It's normal behaviour, so it isn't an error, but in an investigation it shows who got in, when and from where, and a success after many failures is a key detection signal.

5. Two servers' logs disagree by 4 minutes. Name two fixes.

Solution

Synchronise all clocks with NTP (in India, to NIC or NPL servers), and write timestamps in UTC with the zone marked.

Quick quiz

A: the audit log was cleared
B: a password was guessed after many failed attempts
C: a new user account was created
D: the clock is wrong

The Security log shows many 4625 events for one account, then a 4624. What does it most likely mean?

Answer

B. 4625 is a failed logon and 4624 a successful one: many failures followed by a success is the classic sign of password guessing. 1102 is log cleared; 4720 is user created.

Interview questions

  1. What should every security log entry contain? When, where, who and what (OWASP): timestamp, source system/location, identity and address, event type, severity and outcome.
  2. Which events must always be logged? Authentication successes and failures, authorization failures, validation failures, errors, startups/shutdowns and high-risk actions such as privilege changes and data exports.
  3. What must never be logged? Passwords, tokens, keys, card data, connection strings; session IDs hashed; personal data masked.
  4. What is log injection and how do you prevent it? Forging entries via CR/LF or other control characters in input; use structured logging and encode values.
  5. Why does time synchronisation matter? Correlating events across systems needs one consistent timeline; use NTP and UTC.
  6. How do you make logs tamper-evident? Hash chains or signed logs, append-only/WORM storage, separate central store, restricted and monitored access.
  7. Why centralise logs? An attacker on a host can delete local logs; a separate store preserves evidence and allows correlation.
  8. Name five Windows Security event IDs. 4624 logon, 4625 failed logon, 4688 process created, 4720 user created, 1102 audit log cleared (also 4672, 4732, 4740).
  9. What does Sysmon add? Detailed telemetry such as process creation with command line and parent, network connections, file creation and DNS queries.
  10. Syslog severity 3 vs 6? 3 is Error, 6 is Informational (RFC 5424).
  11. What are India's CERT-In logging rules? Keep ICT logs for a rolling 180 days within Indian jurisdiction, and sync clocks with NIC or NPL NTP servers (directions of April 2022).

Glossary

Term Meaning
log a record a system writes about what it did
entry one line or record in a log
event anything that happened: a login, an error, a change
security logging recording and protecting the events needed to detect and investigate
when / where / who / what OWASP's four attributes of a useful security event
input validation failure a request rejected because its data was invalid
log source any system whose logs you collect
log level how serious a message is (DEBUG, INFO, WARNING, ERROR, CRITICAL)
threshold the lowest level that is kept
syslog the standard protocol for sending log messages on Unix-like systems and devices
severity syslog's 0-7 seriousness scale (RFC 5424)
log injection forging or corrupting log entries through untrusted input
CR/LF carriage return and line feed, the characters that end a line
structured logging logs as data with named fields, usually JSON
JSON JavaScript Object Notation, a text format of keys and values
clock skew machines' clocks disagreeing
NTP Network Time Protocol, which syncs clocks to time servers
UTC Coordinated Universal Time; Z marks it
NIC / NPL India's National Informatics Centre / National Physical Laboratory (time sources)
log integrity confidence logs haven't been changed or deleted
tamper-evident changes are detected, not hidden
hash chain each entry includes the previous entry's hash
read-only / WORM storage storage that can be written once and not changed
PII personally identifiable information
masking hiding part of a value
pseudonymisation replacing an identifier with a consistent stand-in
log retention how long logs are kept
centralised logging shipping logs to a separate central store as they are written
log aggregation collecting logs from many sources into one store
SIEM security information and event management (next episode)
Windows Event Logs Windows' Application, System and Security logs
Event Viewer the Windows tool for reading event logs
event ID the number identifying a kind of Windows event
logon type how a Windows logon happened (2 interactive, 3 network, 10 Remote Desktop)
Sysmon Microsoft System Monitor: detailed Windows activity logging
parent process the program that started another program
audit trail a record of who did what to which record, and when
append-only entries can be added but never edited or removed
accountability actions can be traced to the person who did them
non-repudiation a person can't credibly deny an action

Summary

  • A log entry must answer when, where, who and what; collect logs from every source.
  • Use levels to cut noise, but never drop security events; syslog has 8 severities (RFC 5424).
  • Write structured JSON logs to stop log injection and make questions easy.
  • Sync every clock with NTP, log in UTC.
  • Make logs tamper-evident, mask secrets and PII, keep them 180 days (CERT-In), and ship them to a central store.
  • On Windows: 4624, 4625, 4688, 4720, 1102 and friends, plus Sysmon; for people: audit trails.
  • Next episode: SIEM and detection engineering.

Sources (fetched 14 Nov 2026)

  • OWASP Logging Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
  • IANA, Syslog parameters (RFC 5424 severities): https://www.iana.org/assignments/syslog-parameters/syslog-parameters.xhtml
  • Microsoft Learn, 4625(F): An account failed to log on: https://learn.microsoft.com/windows/security/threat-protection/auditing/event-4625
  • Graylog, Windows Security event log content pack: https://go2docs.graylog.org/illuminate-current/content_packs/windows_security_event_logs_content_pack.html
  • Microsoft Learn, Understanding Sysmon events: https://learn.microsoft.com/en-us/windows/security/operating-system-security/sysmon/sysmon-events
  • ManageEngine, CERT-In directions for Indian enterprises: https://www.manageengine.com/log-management/cyber-security/cert-in-compliance-for-indian-enterprises.html