SyntaxPilot
Courses/Vibe Coding/Lesson 37
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Vibe Coding · Lesson 37 of 40

Analytics, feedback and iteration

page viewseventsprivacy-friendly analyticscollecting feedbackerror trackingshipping small updateschangelog

Vibe Coding: build real apps with AI · Episode 37. Every agent run below is real: Claude Code 2.1.323 in print mode (claude -p --permission-mode acceptEdits), on 13 November 2026 (India time, early morning), in /home/asha/projects/canteen. The agent could edit files but needed approval to run commands, and nobody approved, so it ran nothing. We ran npm test, the server, every Playwright (Chromium) browser test, curl and every screenshot ourselves, on localhost only. Nothing was deployed: the same code works once the app is on a host.

What was written how: the canteen app itself was generated for this video by Claude Code in one earlier run (commit 7182929). Ravi, Meera, Kabir, Neha and Asha are fictional; 98765 43210 is a made-up number.

Sources for product facts (fetched 13 Nov 2026; prices and plans change, check the pages): Plausible data policy, Plausible home page / pricing, Sentry pricing, MDN: User-Agent reduction.

The problem: the chai stall that listens

A good chai stall owner does four things without any app:

At the chai stall In your app
Tally marks on a slate, one per customer Page views and visits: how many came
Notices what people point at ("one samosa") Events: what they do
A suggestion box ("more ginger, please") Feedback: what they say
UPI fails, the customer just walks away Error tracking: what broke

Aunty asks: is anyone using the pre-order app, and which items do students look at? The obvious ways fail:

  • Count the orders in the database: shows only students who finished. Someone who opened the page, looked and left leaves no trace.
  • Ask the class WhatsApp group: three replies, all from friends.

So we'd be guessing. The app itself has to count.

Analytics in plain words

  • Analytics: counting what people do in your app, to decide what to change.
  • Page view: one load of a page. A reload is another page view.
  • Event: one action a person takes: item_added, order_placed.
  • Visit: one sitting of one person, however many reloads. Visitor: the person.
  • Conversion (rate): the share of visits that end in what you wanted (here, an order). 1 order out of 2 visits = 50%.
  • Rule from the start: count actions, not people's identities. Aunty needs numbers, not names.

Run A: the AI adds analytics (real)

$ claude -p "Canteen aunty asks: is anyone actually using the pre-order app, and which items do students look at and order? Add analytics: count page views on the order page and the kitchen page, and track events like 'item added' and 'order placed'. Save them in our own database (no Google Analytics). Add a /stats page for aunty that shows today's numbers."
Edit   src/db.js          + events table, recordPageView(), getStats()
Edit   server.js          GET / counts a page view · POST /api/events
Edit   public/app.js      track('item_added') when a quantity goes 0 -> 1
Write  public/stats.html  /stats: opens, orders, money, per item
Reply: "no Google Analytics, no names and no IP addresses"
 "Opens count page loads, not people, so a refresh counts twice."
$ npm test        # we ran it: tests 4  pass 4

The server-side page view, as the AI wrote it:

app.get('/', (req, res) => {
  db.recordPageView('order');
  res.sendFile(path.join(publicDir, 'index.html'));
});

Our test: two students, six opens (real)

Our own Playwright script (visit.js): Ravi opens the page and orders 2 samosas; Meera, on slow Wi-Fi, reloads twice, picks poha, sets it back to 0, picks it again and leaves. Then a request with WhatsApp's link-preview user agent (what happens when the link is shared in a group) and one with Googlebot's.

"pageViews": {"order": 6}       # 2 students, 6 opens
"Poha":      {"added": 2}       # 1 student
"orders": 1                     # aunty sees: 1 of 6 = 17% order

Why it failed:

  • Reloads: one student, three page views.
  • Bots: a bot is a program that fetches pages automatically, with nobody looking: search-engine crawlers like Googlebot, and link previews (WhatsApp's server opens a shared link to draw the preview card).
  • Every browser and bot sends a user agent: a line saying what it is, e.g. a Chrome phone or Googlebot/2.1.

Run B: skip bots, hash the visitor (real)

$ claude -p "I tested /stats with two real students. ... /stats now says: order page opens 6, poha added 2, orders 1. ... Fix the counting: don't count bots or link previews (WhatsApp, Googlebot and similar). Count unique visitors per day without cookies and without storing IP addresses: like Plausible, hash a daily random salt + IP + user agent, keep only the hash, and delete the salt after the day. Count 'item added' once per item per visitor per day. ..."
Write  src/bots.js   Googlebot, WhatsApp, Telegram, curl, headless ...
Edit   src/db.js     sha256(daily random salt + IP + user agent)
                     daily_salts table · old salt deleted after the day
                     item_added: once per item per visitor per day
Reply: "Only that hash is stored, never the IP or the user agent, and no cookies are used."
 "two phones on the same IP with the identical browser version count as one visitor.
  On the school Wi-Fi each phone normally has its own IP, so this should be rare."
$ npm test        # we ran it: tests 9  pass 9

The words:

  • IP address: the number a device's requests come from on the internet, like a return address on a letter.
  • Hash: a one-way fingerprint. The same input always gives the same output, but you can't turn the output back into the input.
  • Salt: a random secret mixed in before hashing. With a new salt every day and yesterday's deleted, yesterday's hashes can't be matched to anyone. Same phone, same id all day; a stranger tomorrow.

This is the method Plausible describes on its data policy page: hash(daily_salt + website_domain + ip_address + user_agent), with the salt "rotated and deleted every 24 hours".

The AI mistake: three classmates, one visitor (real)

The AI said shared IPs "should be rare" on school Wi-Fi. We tested it: three classmates, each on their own Android phone with Chrome, on the same Wi-Fi. Every request in our test comes from one address (127.0.0.1 plays the school's one public IP), with Chrome's real reduced Android user agent.

User-Agent: Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36
            (KHTML, like Gecko) Chrome/143.0.0.0 Mobile Safari/537.36
Ravi    ordered
Meera   reloaded 2x, poha 1->0->1, left
Kabir   ordered
WhatsApp preview + Googlebot fetched /       # not counted: good

"visitors": {"order": 1}       # three phones, ONE visitor
"orders": 2, "students": 2     # ...and /stats says 100% of visitors ordered

On aunty's /stats page (real screenshot in the video): 1 visitor to the order page, 2 orders, 2 different students who ordered, 100% visitors who ordered.

Why:

  1. One public IP. On school (or home, or office) Wi-Fi, every device reaches servers on the internet through the router, from one shared public IP address. (This is called NAT, network address translation: many devices inside, one address outside.)
  2. One user agent. Chrome now sends a reduced user agent. MDN: "The platform version and device model are always represented by fixed values": Android 10; K on every Android phone.

Same IP + same user agent = same hash = one visitor. The AI's "should be rare" was a guess about a place it had never seen; our three-phone test was a fact.

Run C: a random id per tab (real)

$ claude -p "New test, and the visitor count is wrong. Three classmates, ... /stats says: Visitors 1, Different students who ordered 2, 100%. (1) through the school Wi-Fi, every phone reaches a server on the internet from the SAME public IP, and (2) Chrome on Android sends the same reduced user agent on every phone ... Fix it without cookies and without storing anything personal: let the order page make a random visit id when it opens, keep it only in sessionStorage ..."
Write  public/visit.js   random 32-character id, this tab only
Edit   public/app.js     sends it with page_view, item_added, the order
Edit   src/db.js         IP + user agent hash and daily_salts removed
Reply: "A reload in the same tab reuses it; closing the tab deletes it."
 "A visit is a tab, not a person. A student who closes the tab and comes back ... counts twice."
$ npm test        # we ran it: tests 10  pass 10

# our test again: same three classmates, same Wi-Fi, same phones
"visits": 3   "pageViews": {"order": 5}   "orderingVisits": 2   # 67%

The AI's public/visit.js (shortened):

function getVisitId(storage) {          // storage = sessionStorage
  const KEY = 'canteenVisitId';
  const fresh = () => Array.from(crypto.getRandomValues(new Uint8Array(16)),
                  (b) => b.toString(16).padStart(2, '0')).join('');
  try {
    const saved = storage.getItem(KEY);
    if (/^[0-9a-f]{32}$/.test(saved ?? '')) return saved;   // reload: same id
    const id = fresh();                                     // new tab: new id
    storage.setItem(KEY, id);
    return id;
  } catch {
    return fresh();      // storage blocked: this load is its own visit
  }
}
  • sessionStorage: a small store the browser keeps for one tab: reloads keep it, closing the tab deletes it.
  • Cookie: a small piece of data a site stores in your browser that is sent back with every request; it can follow you for months. This app uses none.

On /stats after the fix (real screenshot in the video): 3 visits, 2 orders, 67% of visits ended in an order.

Privacy-friendly analytics: the rules

  • No cookies, no names, no IP addresses stored.
  • Random, short-lived ids (one tab, or one day).
  • Totals per day, not a diary per person.
  • Skip bots; label every number honestly ("page loads (reloads included)", "visits that ended in an order").
  • Test the counting with real situations: reloads, two tabs, shared Wi-Fi, a link shared in a group.

Error tracking (real)

Neha taps Place order and the Wi-Fi drops. She sees an error, shrugs and buys a vada pav at the counter. Students don't file bug reports; they just stop using the app.

  • Error tracking: the app reports its own errors to you (message, count, where in the code).
  • Stack trace: the list of code lines that led to an error, so you know where to look.
  • Unhandled promise rejection: an async operation (like fetch) failed and no code handled the failure.
$ claude -p "Two more things for aunty. 1) Error tracking: ... Catch errors on the page (window.onerror, unhandled promise rejections, failed fetches) and on the server, and save them in our own database, not a third-party service. Show today's errors on /stats, grouped by message with a count. 2) Feedback: a small 'Tell aunty' box under the order form: one optional message (max 300 characters) and a 1-5 rating. Don't ask for a name or phone number. ..."
Write  public/errors.js    loads first: errors, failed loads, rejections, failed requests (network errors and 5xx)
Write  public/feedback.js  five emoji ratings + optional message
Write  src/scrub.js        removes phone numbers, emails, links
Edit   server.js           any route that throws -> JSON 500, saved
Reply: "Errors that happen while the phone is offline are sent when it comes back online, as long as the tab is still open."
 "Names can't be detected automatically. If a student types their name into the feedback box anyway, it will be saved."
$ npm test        # we ran it: tests 18  pass 18

How errors.js sends a report (the AI's code, shortened): if sending fails (offline), it keeps the report and retries when the browser fires online.

function send(report) {
  realFetch('/api/client-errors', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(report),
    keepalive: true,
  }).catch(() => { if (waiting.length < MAX_REPORTS) waiting.push(report); });
}

Our test: Wi-Fi off at "Place order", then feedback (real)

Wi-Fi off
Neha sees:  Network error. Please try again.
Wi-Fi back
2  "Poha finished by 11 again, please make more"
5  "Pre-order is so fast! Meera 8B, 98765 43210"

"errors":   [{"message": "Network error POST /api/orders (TypeError: Failed to fetch)", "count": 1}]
"feedback": {"average": 3.5, "messages": [
   "Pre-order is so fast! Meera 8B, [removed]",   # the name stays
   "Poha finished by 11 again, please make more"]}

In the video: Neha's phone shows "Network error. Please try again."; aunty's /stats shows a Problems on the order page table (Request failed · Network error POST /api/orders (TypeError: Failed to fetch) · 1 time) and the Tell aunty ratings (2 ratings, average 3.5) and messages.

The phone number was scrubbed (it's a pattern); the name wasn't (names aren't patterns). Read free-text feedback before you share it.

Feedback that people actually give

  • Ask at the right moment: just after ordering.
  • One tap first (a rating), words optional.
  • Anonymous by design, and say "don't write your name".
  • Free text can still hold names: read before sharing.
  • Answer feedback with a change, not just "thanks".

Hosted tools (checked 13 Nov 2026)

Tool What Facts from its own page
Plausible Analytics No cookies; raw IP and user agent never stored; daily salted hash; 30-day free trial, then from $9/month for 10k monthly pageviews; open source, can be self-hosted
Sentry Error tracking Developer plan free: 1 user, 5k errors, 30-day lookback (the page doesn't say per what period); Team from $26/month billed yearly ($29 monthly)

Building it yourself (as here) means no third party sees a single visit. Adding any third-party script means running someone else's code on your page.

Shipping a small update from feedback (real)

$ claude -p "Feedback from the 'Tell aunty' box today: 'Poha finished by 11 again, please make more' (rating 2). Aunty says she can make 30 plates of poha a day, no more. Make one small update: show how many poha plates are left today on the order page, and refuse poha orders when none are left (with a friendly message). Keep the change small: only poha has a limit for now. Then start a CHANGELOG.md ..."
Edit   src/db.js      DAILY_LIMITS = { Poha: 30 }
Edit   public/app.js  "N plates left today" / "Finished for today"
Write  CHANGELOG.md   1.3.0 back to 1.0.0, one per commit
$ npm test            # we ran it: tests 21  pass 21
$ git show --stat HEAD
 8 files changed, 204 insertions(+), 10 deletions(-)

# our check: three orders of 10 plates, then one more
$ curl -s -X POST localhost:4100/api/orders -H 'content-type: application/json' \
    -d '{"name":"Test","className":"8B","items":[{"id":4,"qty":10}]}'
{"orderNumber":1,"total":250}
{"orderNumber":2,"total":250}
{"orderNumber":3,"total":250}
{"error":"Sorry, poha is finished for today! Aunty makes 30 plates a day. Please pick something else, or come early tomorrow."}

The limit in src/db.js (the AI's code, shortened):

export const DAILY_LIMITS = { Poha: 30 };

function getMenu(date = today()) {
  return menuQuery.all().map((r) => {
    const limit = DAILY_LIMITS[r.name];
    if (limit === undefined) return { ...r };
    return { ...r, left: Math.max(0, limit - qtyOrderedByDate.get(date, r.id).n) };
  });
}

On the order page (real screenshot in the video), Poha now says Finished for today in red, and its quantity box is greyed out.

Shipping small updates: change one thing at a time and release it. If tomorrow's numbers move, you know which change moved them.

The changelog

# Changelog

## [1.3.0] - 2026-11-13
### Added
- Poha daily limit: aunty makes 30 plates a day. The order page shows how many plates are
  left today. Asked for in "Tell aunty" feedback ("Poha finished by 11 again").

## [1.2.0] - 2026-11-12
### Added
- Error tracking ... "Tell aunty" feedback form ...

## [1.1.2] - 2026-11-12
### Changed
- Visits are counted with a random id per browser tab ... On school Wi-Fi, phones share one IP
  and the same user agent, so classmates were counted as one visitor.
  • Changelog: a file listing every version, newest first, with what changed and why. This follows the "Keep a Changelog" format.
  • Semantic versioning (MAJOR.MINOR.PATCH): PATCH (1.1.2) for a fix, MINOR (1.3.0) for a new feature, MAJOR (2.0.0) for a change that breaks old behaviour.
  • (Small note: the commits were made after midnight India time, but the container's clock was UTC, so the AI dated them 2026-11-12. Dates in a changelog follow the clock that made the commits.)

The iteration loop

Measure (visits, events) → Listen (feedback, errors) → Change one small thing → Ship + log (CHANGELOG) → measure again. Each turn is an iteration.

Checklist before you trust a number

  • [ ] Test with reloads, two tabs and a shared Wi-Fi.
  • [ ] Share the link in a chat: is the preview counted?
  • [ ] Every label says exactly what it counts.
  • [ ] No cookies, names or IP addresses stored.
  • [ ] Break it on purpose (Wi-Fi off): is the error reported?
  • [ ] Read feedback before you share it.
  • [ ] One change per release, a line in the changelog.

Common mistakes (real)

Mistake What we saw Why Fix
Counting page loads as people 2 students → "order": 6 Reloads count Count visits
Counting robots WhatsApp/2.23..., Googlebot/2.1 Link previews, crawlers Skip bots by user agent
IP + user agent as identity (the AI's) 3 phones → "visitors": 1 Shared public IP, reduced UA Random id per tab
Errors nobody hears about TypeError: Failed to fetch Students give up silently Error tracking, with offline retry
Trusting the scrubber "Meera 8B, [removed]" Names aren't patterns Read before sharing

Mini project: measure, listen, change

  1. Add page-view, visit and event counting to one of your apps, stored in your own database.
  2. Write a Playwright test like ours: two tabs, a reload, a link-preview user agent, three "phones" with the same user agent.
  3. Add a one-tap feedback box and error tracking with an offline retry.
  4. Ship one change that comes from feedback, and start CHANGELOG.md.

Practice questions

  1. A student opens the order page, reloads 3 times and orders. How many page views, visits and orders?
Answer

4 page views (1 load + 3 reloads), 1 visit, 1 order.

  1. Why can't you use IP address + user agent to count students on a school network?
Answer

Every phone reaches the internet from the school's one public IP, and Chrome on Android sends the same reduced user agent ("Android 10; K") on every phone, so they all produce the same hash.

  1. What's the difference between sessionStorage and a cookie for counting visits?
Answer

sessionStorage stays in one tab and is deleted when the tab closes; the page decides what to send. A cookie is sent with every request automatically and can last for months, so it can follow a person across visits.

  1. Your stats show "100% of visitors ordered". What do you suspect first?
Answer

The counter, not the app: visitors are being merged (shared IP/user agent) or non-ordering visits are not being counted. Test with several devices.

  1. Write a changelog line for: "fixed a bug where the total showed Rs 0 for chai".
Answer

Under a PATCH version, e.g. ## [1.3.1] - <date> → ### Fixed → - The total showed Rs 0 when only masala chai was picked.

Quiz

Three classmates open the app on their own Android phones, all on the school Wi-Fi. Which counts them as THREE? A) Count page loads · B) Hash of daily salt + IP address + user agent · C) A random visit id per tab, in sessionStorage · D) Count orders

Answer

C: each tab makes its own random id. A counts reloads, B sees one IP and one user agent, D misses students who don't order.

Interview questions

  1. Page view vs visit vs unique visitor? A page view is one load; a visit is one session of use; a unique visitor is a person/device counted once in a period.
  2. How do you count unique visitors without cookies? A daily salted hash of request data (with the limits shown here), or a random first-party id that lives only as long as the visit (e.g. sessionStorage); store no raw IPs.
  3. Why filter bots? Crawlers and link previews fetch pages with nobody looking; they inflate page views and drag conversion down.
  4. What would you capture in client-side error tracking? Uncaught errors (window.onerror/error event), unhandled rejections, failed requests and 5xx answers, with a message, a location and a count, and nothing personal; queue reports while offline.
  5. Why ship small changes? Small changes are easy to review, test, roll back, and to connect to a change in the numbers.

Glossary

Term Meaning (with an example from this episode)
Analytics Counting what people do in your app to decide what to change (/stats).
Page view One load of a page; a reload is another (Meera: 3 page views).
Event One action a person takes (item_added, order_placed).
Visit One sitting of one person, however many reloads (here: one browser tab).
Visitor / unique visitor The person (or device) counted once in a period.
Conversion rate Share of visits that end in the goal (2 of 3 visits ordered = 67%).
Metric A number you track (visits, orders, errors per day).
Bot A program that fetches pages automatically, with nobody looking (Googlebot).
Link preview A chat app fetching a shared link to draw its preview card (WhatsApp).
User agent The line a browser or bot sends saying what it is (Googlebot/2.1).
User-agent reduction Chrome sending fixed values ("Android 10; K") to protect privacy.
IP address The number a device's requests come from on the internet.
Public IP / NAT The one address a whole Wi-Fi network shows to the internet; NAT is how a router shares it.
Hash A one-way fingerprint: same input, same output, no way back (SHA-256).
Salt A random secret mixed in before hashing, changed daily.
Cookie Data a site stores in your browser and gets back with every request; can last months.
sessionStorage Browser storage for one tab; deleted when the tab closes.
Privacy-friendly analytics Counting without cookies, names or stored IPs; totals, not diaries.
Feedback What people tell you in their own words (the "Tell aunty" box).
Rating A one-tap score (1-5).
Scrub Removing patterns like phone numbers or emails from text before saving.
Error tracking The app reporting its own errors to you, grouped and counted.
window.onerror The browser hook that runs when a JavaScript error isn't caught.
Unhandled promise rejection An async task (like fetch) failed and no code handled it.
TypeError: Failed to fetch What the browser's fetch throws when the network request fails (Wi-Fi off).
Stack trace The list of code lines that led to an error.
5xx / 500 Server error status codes ("the server broke").
Third-party script Code from another company loaded into your page.
Iteration One turn of measure → listen → change → ship.
Shipping a small update Releasing one small change at a time.
Changelog A file listing every version, newest first, with what changed and why.
Semantic versioning MAJOR.MINOR.PATCH: breaking change . new feature . fix (1.3.0).
Playwright A tool that drives a real browser from a script; we used it for every test.

Summary

  • Counting orders and asking friends only shows part of the picture; analytics counts page views (loads), events (actions) and visits (sittings).
  • Reloads and bots (crawlers, link previews) inflate numbers: 2 students became 6 opens.
  • The AI's daily-salted IP + user-agent hash was privacy-friendly but wrong for a school: one public IP and Chrome's fixed "Android 10; K" user agent made 3 classmates 1 visitor. Its "should be rare" was a guess; our three-phone test was a fact.
  • A random id per tab in sessionStorage fixed it: 3 visits, 67%, no cookies, no IPs.
  • Error tracking (with offline retry) and an anonymous feedback box let silent students be heard; scrubbers remove numbers, not names.
  • One piece of feedback became one small update (30 plates of poha a day), recorded in CHANGELOG.md with semantic versioning. Measure → listen → change → ship → measure again.

Next episode: Capstone part 1: plan and build a real product.