SyntaxPilot
Courses/Go Programming/Lesson 65
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Go Programming · Lesson 65 of 89

Building a REST API

REST principlesresource and URL designJSON request decoding and validationresponse encoding helpersconsistent error responses (RFC 9457 problem details)status code choicepaginationfilteringAPI versioningidempotency keysthird-party routers (chi, gorilla/mux, gin, echo) trade-offsOpenAPI

These are the notes for Go episode 65. Raju's chai shop has a phone app that talks to our Go server all day. The API grew without rules: invented verb URLs (/getMenu, /cancelOrderNow) and a 200 OK for every answer, so the app believed a failed cancel had worked. We fix it step by step: resources named by nouns, HTTP methods as the verbs, honest status codes, strict JSON decoding and validation, a response helper that sets headers in the right order, one error shape for everything (problem details, RFC 9457), pagination and filtering for big lists, versioning so old apps keep working, and idempotency keys so retries don't create double orders. Finally we compare the popular routers and describe the API with OpenAPI.

All outputs are from real runs (Go 1.24.7, linux/amd64). Byte counts are real; ports and keys would change between runs.

Facts and sources

  • RFC 9457, "Problem Details for HTTP APIs" (datatracker.ietf.org/doc/html/rfc9457): published July 2023; "This document obsoletes RFC 7807." Media types application/problem+json and application/problem+xml. Standard members: type (a URI reference identifying the problem type), status (the HTTP status code, advisory), title (a short, human-readable summary of the problem type), detail (an explanation specific to this occurrence), instance (a URI reference identifying this specific occurrence). If type is absent it is assumed to be "about:blank", meaning the problem has no meaning beyond the HTTP status code, and the title should then be the standard status phrase (like "Not Found"). Problem types may add extension members.
  • Idempotency-Key header: IETF Internet-Draft draft-ietf-httpapi-idempotency-key-header (HTTPAPI working group, work in progress, not yet an RFC). It targets POST and PATCH, the methods that are not idempotent; an idempotent operation "can be applied multiple times without changing the result beyond the initial application".
  • chi README (github.com/go-chi/chi): "chi is a lightweight, idiomatic and composable router for building Go HTTP services."
  • gorilla/mux README: "Package gorilla/mux implements a request router and dispatcher"; "It implements the http.Handler interface so it is compatible with the standard http.ServeMux"; requests can be matched on "URL host, path, path prefix, schemes, header and query values, HTTP methods", with variables that take "an optional regular expression".
  • Gin README (github.com/gin-gonic/gin): "Gin is a high-performance HTTP web framework written in Go", built on httprouter.
  • Echo README (github.com/labstack/echo): "High performance, extensible, minimalist Go web framework"; it "is built on Go's standard net/http and interoperates with it through echo.WrapHandler and echo.WrapMiddleware", with "a fast radix-tree router, request binding".
  • OpenAPI Specification README (github.com/OAI/OpenAPI-Specification): it "defines a standard, programming language-agnostic interface description for HTTP APIs"; uses include "interactive documentation; code generation for documentation, clients, and servers; and automation of test cases".
  • go doc net/http.MaxBytesReader (Go 1.24.7): "MaxBytesReader is similar to io.LimitReader but is intended for limiting the size of incoming request bodies ... returns a non-nil error of type *MaxBytesError for a Read beyond the limit"; it "prevents clients from accidentally or maliciously sending a large request and wasting server resources."
  • net/http/status.go (Go 1.24.7): StatusUnprocessableEntity = 422 // RFC 9110, 15.5.21; http.StatusText(422) is "Unprocessable Entity".
  • Everything else comes from the real runs below.

The problem: an API without rules

An API (Application Programming Interface) is the set of requests a program accepts and the answers it gives. A web API speaks HTTP: the app sends a request (method + URL + headers + maybe a body) and the server sends a response (a status code + headers + a body). Real-life example: the menu card and the waiter at a restaurant are the restaurant's "API": you can only order what's on the card, in the way the waiter understands.

Raju's API grew over a year. Three developers each invented their own request names, all verbs (/getMenu, /addNewOrder, /cancelOrderNow), and every handler answered with the default status 200 OK, hiding errors inside the body. The app does what most apps do: "if the status is 200, it worked".

package main

import (
    "fmt"
    "io"
    "net/http"
    "net/http/httptest"
)

func main() {
    mux := http.NewServeMux()
    // every action is its own verb-URL, and every answer is 200
    mux.HandleFunc("/getMenu", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, `{"items":["chai","samosa"]}`)
    })
    mux.HandleFunc("/cancelOrderNow", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, `{"error":"order 42 not found"}`) // still a 200!
    })
    srv := httptest.NewServer(mux)
    defer srv.Close()

    // Raju's app: "if the status is 200, it worked"
    for _, p := range []string{"/getMenu", "/cancelOrderNow?id=42", "/getmenu"} {
        resp, _ := http.Get(srv.URL + p)
        body, _ := io.ReadAll(resp.Body)
        resp.Body.Close()
        fmt.Printf("%-22s %d worked=%-5v %s\n", p, resp.StatusCode, resp.StatusCode == 200, body)
    }
}

Output:

/getMenu               200 worked=true  {"items":["chai","samosa"]}
/cancelOrderNow?id=42  200 worked=true  {"error":"order 42 not found"}
/getmenu               404 worked=false 404 page not found

The cancel "worked" (200, worked=true) although the body says order 42 not found, and /getmenu with a small m gives 404 because every URL is an invented name you must remember exactly. Two problems: unguessable URLs, and a status code that doesn't tell the truth.

REST: resources, methods and status codes

REST (Representational State Transfer) is a style for designing web APIs. Real-life example: a well-run library.

In a library In a REST API
shelves labelled by things (Books, Members) resources named by nouns: /orders, /orders/42
the same few actions for everything (look, add, replace, remove) HTTP methods: GET, POST, PUT, PATCH, DELETE
a stamped slip for every visit ("issued", "not found") status codes: 200, 201, 404, ...
you get a photocopy, not the book a representation of the resource (here JSON), not the resource itself
  • A resource is a thing the API manages: one order, the list of orders, the menu.
  • A URL (Uniform Resource Locator) is its address; the path part (/orders/42) names the resource. A URL + method that the API answers is often called an endpoint.
  • A representation is the resource written in some format, here JSON: {"id":42,"item":"chai","qty":2}.
  • Status codes are three-digit numbers: 2xx success, 4xx the client asked wrongly, 5xx the server failed.
  • Stateless: the server does not remember a "conversation". Every request carries everything it needs (for example its login key), so many copies of the server can run side by side and any of them can answer.

Designing the URLs

Instead of Use
GET /getMenu GET /menu
POST /addNewOrder POST /orders
GET /cancelOrderNow?id=42 DELETE /orders/42 (or POST /orders/42/cancel for a state change with rules)
GET /orderItemsOfOrder?id=42 GET /orders/42/items
GET /readyOrders GET /orders?status=ready

Rules: nouns, plural, lowercase, hyphens between words (/menu-items); one item gets its ID in the path; things that belong to it are nested under it; filters, sorting and page numbers go in the query string (the part after ?).

The same API, the REST way

This uses the Go 1.22 routing patterns from episode 61 ("GET /orders/{id}" and r.PathValue("id")). POST answers 201 Created with a Location header saying where the new order lives; DELETE answers 204 No Content; a missing order is 404 Not Found; a method we never registered gets 405 Method Not Allowed automatically, with an Allow header listing the methods that do work.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/http/httptest"
    "strconv"
    "strings"
)

type Order struct {
    ID   int    `json:"id"`
    Item string `json:"item"`
    Qty  int    `json:"qty"`
}

var orders, nextID = map[int]Order{42: {42, "chai", 2}}, 43
func id(r *http.Request) int { n, _ := strconv.Atoi(r.PathValue("id")); return n }
func main() {
    mux := http.NewServeMux() // nouns in the URL, verbs in the method
    mux.HandleFunc("GET /orders/{id}", func(w http.ResponseWriter, r *http.Request) {
        o, ok := orders[id(r)]
        if !ok {
            http.Error(w, "no such order", http.StatusNotFound) // 404
            return
        }
        json.NewEncoder(w).Encode(o)
    })
    mux.HandleFunc("POST /orders", func(w http.ResponseWriter, r *http.Request) {
        var o Order
        json.NewDecoder(r.Body).Decode(&o)
        o.ID, nextID = nextID, nextID+1
        orders[o.ID] = o
        w.Header().Set("Location", "/orders/"+strconv.Itoa(o.ID)) // where the new thing lives
        w.WriteHeader(http.StatusCreated)                          // 201
        json.NewEncoder(w).Encode(o)
    })
    mux.HandleFunc("DELETE /orders/{id}", func(w http.ResponseWriter, r *http.Request) {
        delete(orders, id(r))
        w.WriteHeader(http.StatusNoContent) // 204: done, nothing to send
    })
    srv := httptest.NewServer(mux)
    defer srv.Close()
    for _, t := range []string{"GET /orders/42", `POST /orders {"item":"samosa","qty":3}`,
        "DELETE /orders/42", "GET /orders/42", "PUT /orders/43"} {
        f := strings.SplitN(t+" ", " ", 3)
        req, _ := http.NewRequest(f[0], srv.URL+f[1], strings.NewReader(f[2]))
        resp, _ := http.DefaultClient.Do(req)
        b, _ := io.ReadAll(resp.Body)
        resp.Body.Close()
        fmt.Printf("%-6s %-10s %-22s %s%s %s\n", f[0], f[1], resp.Status, resp.Header.Get("Location"),
            resp.Header.Get("Allow"), strings.TrimSpace(string(b)))
    }
}

Output:

GET    /orders/42 200 OK                  {"id":42,"item":"chai","qty":2}
POST   /orders    201 Created            /orders/43 {"id":43,"item":"samosa","qty":3}
DELETE /orders/42 204 No Content          
GET    /orders/42 404 Not Found           no such order
PUT    /orders/43 405 Method Not Allowed DELETE, GET, HEAD Method Not Allowed

Now every answer tells the truth: 200, 201 + Location: /orders/43, 204, 404, and 405 with Allow: DELETE, GET, HEAD (the mux adds HEAD for every GET route).

Methods, safe and idempotent

  • Safe: the request changes nothing on the server (reading a menu).
  • Idempotent: doing it twice has the same effect as doing it once (switching a light off twice).
Method Meaning Safe Idempotent Typical success
GET read yes yes 200
POST create (or run an action) no no 201 + Location
PUT replace the whole resource no yes 200 or 204
PATCH change part of it no not by default 200
DELETE remove no yes 204

PUT vs PATCH (extra example, not in the video): PUT sends the whole new order, so fields you leave out become zero values; PATCH changes only the fields that were sent. Pointer fields tell "not sent" (nil) from "sent as zero".

package main

import (
    "encoding/json"
    "fmt"
)

type Order struct {
    Item string `json:"item"`
    Qty  int    `json:"qty"`
    Note string `json:"note"`
}

// PATCH body: pointers tell "not sent" (nil) apart from "sent as zero".
type OrderPatch struct {
    Item *string `json:"item"`
    Qty  *int    `json:"qty"`
    Note *string `json:"note"`
}

func main() {
    stored := Order{Item: "chai", Qty: 2, Note: "less sugar"}

    // PUT replaces the WHOLE order: fields left out become zero values.
    var put Order
    json.Unmarshal([]byte(`{"qty":3}`), &put)
    fmt.Printf("after PUT   {\"qty\":3}: %+v\n", put)

    // PATCH changes only the fields that were sent.
    var p OrderPatch
    json.Unmarshal([]byte(`{"qty":3}`), &p)
    patched := stored
    if p.Item != nil {
        patched.Item = *p.Item
    }
    if p.Qty != nil {
        patched.Qty = *p.Qty
    }
    if p.Note != nil {
        patched.Note = *p.Note
    }
    fmt.Printf("after PATCH {\"qty\":3}: %+v\n", patched)
}

Output:

after PUT   {"qty":3}: {Item: Qty:3 Note:}
after PATCH {"qty":3}: {Item:chai Qty:3 Note:less sugar}

Reading JSON safely

Real-life example: order slips with mistakes. Someone writes the quantity in words ("two"), someone misspells "table" as "tabel", and a bad network tears a slip in half. The obvious code decodes the body and hopes:

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
    "strings"
)

type Order struct {
    Item  string `json:"item"`
    Qty   int    `json:"qty"`
    Table int    `json:"table"`
}

func create(w http.ResponseWriter, r *http.Request) {
    var o Order
    json.NewDecoder(r.Body).Decode(&o) // the obvious way: decode and hope
    w.WriteHeader(http.StatusCreated)
    fmt.Fprintf(w, "saved %+v", o)
}

func main() {
    for _, body := range []string{
        `{"item":"chai","qty":2,"table":4}`,
        `{"item":"chai","qty":"two","table":4}`, // qty as text
        `{"item":"chai","qty":-3,"tabel":4}`,    // negative qty, typo in "table"
        `{"item":"chai",`,                       // cut off
    } {
        rec := httptest.NewRecorder()
        create(rec, httptest.NewRequest("POST", "/orders", strings.NewReader(body)))
        fmt.Println(rec.Code, rec.Body)
    }
}

Output:

201 saved {Item:chai Qty:2 Table:4}
201 saved {Item:chai Qty:0 Table:4}
201 saved {Item:chai Qty:-3 Table:0}
201 saved {Item: Qty:0 Table:0}

Everything was saved with 201: "two" became 0, the misspelled tabel vanished (Table:0), -3 went straight in, and the torn body saved an empty order. Why: the decode error was thrown away, and by default encoding/json silently ignores fields it doesn't know.

Strict decoding: DisallowUnknownFields + MaxBytesReader

Decoding means turning the JSON text into a Go value. A careful decoder:

  1. limits the body size with http.MaxBytesReader (1 MB here), so nobody can send us gigabytes;
  2. calls DisallowUnknownFields(), so an unknown field is an error, not silence;
  3. returns Decode's error, and the handler answers 400 Bad Request ("I couldn't read what you sent").
package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
    "strings"
)

type Order struct {
    Item  string `json:"item"`
    Qty   int    `json:"qty"`
    Table int    `json:"table"`
}

func decodeJSON(w http.ResponseWriter, r *http.Request, dst any) error {
    r.Body = http.MaxBytesReader(w, r.Body, 1<<20) // at most 1 MB
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields() // a typo is an error, not silence
    return dec.Decode(dst)
}

func create(w http.ResponseWriter, r *http.Request) {
    var o Order
    if err := decodeJSON(w, r, &o); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest) // 400: we can't read it
        return
    }
    w.WriteHeader(http.StatusCreated)
    fmt.Fprintf(w, "saved %+v\n", o)
}

func main() {
    for _, body := range []string{
        `{"item":"chai","qty":2,"table":4}`,
        `{"item":"chai","qty":"two","table":4}`,
        `{"item":"chai","qty":-3,"tabel":4}`,
        `{"item":"chai",`,
        `{"item":"chai","qty":-3,"table":4}`, // typo fixed, qty still -3
        `{"item":"` + strings.Repeat("a", 2<<20) + `"}`, // a 2 MB body
    } {
        rec := httptest.NewRecorder()
        create(rec, httptest.NewRequest("POST", "/orders", strings.NewReader(body)))
        fmt.Print(rec.Code, " ", rec.Body)
    }
}

Output:

201 saved {Item:chai Qty:2 Table:4}
400 json: cannot unmarshal string into Go struct field Order.qty of type int
400 json: unknown field "tabel"
400 unexpected EOF
201 saved {Item:chai Qty:-3 Table:4}
400 http: request body too large

The real messages: json: cannot unmarshal string into Go struct field Order.qty of type int, json: unknown field "tabel", unexpected EOF (end of file: the JSON stopped in the middle), and http: request body too large. But qty: -3 with the typo fixed is perfectly valid JSON, so it is saved. The decoder checks the shape, not the meaning.

Extra: one JSON value only

Decode reads only the first JSON value, so {"item":"chai","qty":1}{"x":1} would pass. Check that nothing follows by decoding once more and expecting io.EOF. This version (strict.go) adds that check and validation together:

package main

import (
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "net/http/httptest"
    "strings"
)

type Order struct {
    Item  string `json:"item"`
    Qty   int    `json:"qty"`
    Table int    `json:"table"`
}

func (o Order) validate() map[string]string {
    bad := map[string]string{}
    if o.Item == "" {
        bad["item"] = "is required"
    }
    if o.Qty < 1 || o.Qty > 20 {
        bad["qty"] = "must be 1 to 20"
    }
    return bad
}

func decodeJSON(w http.ResponseWriter, r *http.Request, dst any) error {
    r.Body = http.MaxBytesReader(w, r.Body, 1<<20) // at most 1 MB
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields() // a typo is an error, not silence
    if err := dec.Decode(dst); err != nil {
        return err
    }
    if dec.Decode(&struct{}{}) != io.EOF {
        return errors.New("body must hold one JSON object")
    }
    return nil
}

func create(w http.ResponseWriter, r *http.Request) {
    var o Order
    if err := decodeJSON(w, r, &o); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest) // 400: can't read it
        return
    }
    if bad := o.validate(); len(bad) > 0 {
        http.Error(w, fmt.Sprint(bad), http.StatusUnprocessableEntity) // 422: read it, but it's wrong
        return
    }
    w.WriteHeader(http.StatusCreated)
    fmt.Fprintf(w, "saved %+v\n", o)
}

func main() {
    for _, body := range []string{`{"item":"chai","qty":2,"table":4}`, `{"item":"chai","qty":"two","table":4}`,
        `{"item":"chai","qty":-3,"tabel":4}`, `{"item":"chai",`, `{"qty":-3}`, `{"item":"chai","qty":1}{"x":1}`} {
        rec := httptest.NewRecorder()
        create(rec, httptest.NewRequest("POST", "/orders", strings.NewReader(body)))
        fmt.Print(rec.Code, " ", rec.Body)
    }
}

Output:

201 saved {Item:chai Qty:2 Table:4}
400 json: cannot unmarshal string into Go struct field Order.qty of type int
400 json: unknown field "tabel"
400 unexpected EOF
422 map[item:is required qty:must be 1 to 20]
400 body must hold one JSON object

Validation: 400 vs 422

Validation means checking the meaning of data against your own rules: an item is required, a quantity must be 1 to 20. Real-life example: two kinds of wrong slip. One the clerk can't read at all (smudged, torn): 400 Bad Request. The other is readable but impossible ("-3 cups", "50 cups", no item): many APIs answer 422 Unprocessable Entity ("I understood you, but I can't do it"). A good clerk lists every problem at once.

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
    "strings"
)

type Order struct {
    Item  string `json:"item"`
    Qty   int    `json:"qty"`
    Table int    `json:"table"`
}

func (o Order) validate() map[string]string {
    bad := map[string]string{}
    if o.Item == "" {
        bad["item"] = "is required"
    }
    if o.Qty < 1 || o.Qty > 20 {
        bad["qty"] = "must be 1 to 20"
    }
    return bad
}

func decodeJSON(w http.ResponseWriter, r *http.Request, dst any) error {
    r.Body = http.MaxBytesReader(w, r.Body, 1<<20) // at most 1 MB
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields() // a typo is an error, not silence
    return dec.Decode(dst)
}

func create(w http.ResponseWriter, r *http.Request) {
    var o Order
    if err := decodeJSON(w, r, &o); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest) // 400: we can't read it
        return
    }
    if bad := o.validate(); len(bad) > 0 {
        http.Error(w, fmt.Sprint(bad), http.StatusUnprocessableEntity) // 422: we read it, it's wrong
        return
    }
    w.WriteHeader(http.StatusCreated)
    fmt.Fprintf(w, "saved %+v\n", o)
}

func main() {
    for _, body := range []string{`{"item":"chai","qty":2,"table":4}`, `{"item":"chai","qty":-3,"table":4}`, `{"qty":50}`} {
        rec := httptest.NewRecorder()
        create(rec, httptest.NewRequest("POST", "/orders", strings.NewReader(body)))
        fmt.Print(rec.Code, " ", rec.Body)
    }
}

Output:

201 saved {Item:chai Qty:2 Table:4}
422 map[qty:must be 1 to 20]
422 map[item:is required qty:must be 1 to 20]

Order of work in every write handler: decode strictly → validate → do the work → respond.

Response helpers: headers first

Content-Type tells the client what kind of data the body is (application/json). WriteHeader sends the status line and all headers at that moment, so a header set afterwards is ignored. When a response has no Content-Type, Go sniffs the body (guesses its type from the first bytes).

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
)

func writeJSON(w http.ResponseWriter, status int, v any) {
    w.Header().Set("Content-Type", "application/json") // 1. headers
    w.WriteHeader(status)                              // 2. status line
    json.NewEncoder(w).Encode(v)                       // 3. body
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("POST /bad", func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(http.StatusCreated)
        w.Header().Set("Content-Type", "application/json") // too late!
        fmt.Fprint(w, `{"id":43}`)
    })
    mux.HandleFunc("POST /good", func(w http.ResponseWriter, r *http.Request) {
        writeJSON(w, http.StatusCreated, map[string]int{"id": 43})
    })
    srv := httptest.NewServer(mux)
    defer srv.Close()
    for _, p := range []string{"/bad", "/good"} {
        resp, _ := http.Post(srv.URL+p, "application/json", nil)
        resp.Body.Close()
        fmt.Printf("%-6s %s  Content-Type: %s\n", p, resp.Status, resp.Header.Get("Content-Type"))
    }
}

Output:

/bad   201 Created  Content-Type: text/plain; charset=utf-8
/good  201 Created  Content-Type: application/json

The bad handler's Content-Type arrived as text/plain; charset=utf-8 (sniffed). A response helper like writeJSON gets the order right once for every handler: headers → WriteHeader(status) → body.

One shape for every error: problem details (RFC 9457)

Real-life example: at a railway station every clerk explains problems differently (a note, a shout, a stamp), until the station prints one complaint form with the same boxes everywhere. Our errors so far were plain text, a JSON error field and a map, so the app needed special code for each. HTTP has a standard form: problem details, RFC 9457 (an RFC is a numbered internet standards document), sent with the content type application/problem+json.

Member Meaning Example
type URI naming the kind of problem; about:blank = nothing more than the status code https://rajus.in/probs/invalid-order
title short summary of the problem type Not Found
status the HTTP status code 404
detail what went wrong this time no order 99
instance URI of this exact occurrence /orders/99
extension members your own extra fields errors with field messages
package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
)

// Problem is an error body in the RFC 9457 "problem details" shape.
type Problem struct {
    Type     string            `json:"type"`
    Title    string            `json:"title"`
    Status   int               `json:"status"`
    Detail   string            `json:"detail,omitempty"`
    Instance string            `json:"instance,omitempty"`
    Errors   map[string]string `json:"errors,omitempty"` // our own extra member
}

func problem(w http.ResponseWriter, r *http.Request, p Problem) {
    if p.Type == "" {
        p.Type = "about:blank" // "nothing more than the status code"
    }
    p.Title, p.Instance = http.StatusText(p.Status), r.URL.Path
    w.Header().Set("Content-Type", "application/problem+json")
    w.WriteHeader(p.Status)
    json.NewEncoder(w).Encode(p)
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /orders/{id}", func(w http.ResponseWriter, r *http.Request) {
        problem(w, r, Problem{Status: 404, Detail: "no order " + r.PathValue("id")})
    })
    mux.HandleFunc("POST /orders", func(w http.ResponseWriter, r *http.Request) {
        problem(w, r, Problem{Type: "https://rajus.in/probs/invalid-order", Status: 422,
            Detail: "2 fields are invalid",
            Errors: map[string]string{"qty": "must be 1 to 20", "item": "is required"}})
    })
    for _, t := range [][2]string{{"GET", "/orders/99"}, {"POST", "/orders"}} {
        rec := httptest.NewRecorder()
        mux.ServeHTTP(rec, httptest.NewRequest(t[0], t[1], nil))
        fmt.Println(rec.Code, rec.Header().Get("Content-Type"))
        var pretty map[string]any
        json.Unmarshal(rec.Body.Bytes(), &pretty)
        out, _ := json.MarshalIndent(pretty, "", "  ")
        fmt.Println(string(out))
    }
}

Output:

404 application/problem+json
{
  "detail": "no order 99",
  "instance": "/orders/99",
  "status": 404,
  "title": "Not Found",
  "type": "about:blank"
}
422 application/problem+json
{
  "detail": "2 fields are invalid",
  "errors": {
    "item": "is required",
    "qty": "must be 1 to 20"
  },
  "instance": "/orders",
  "status": 422,
  "title": "Unprocessable Entity",
  "type": "https://rajus.in/probs/invalid-order"
}

The 422 gets its own type URI because it carries an extension member (errors); about:blank is for problems that mean nothing beyond their status code.

Choosing the status code

Code When
200 OK read or updated, here's the result
201 Created POST created something (send Location)
204 No Content done, nothing to send
400 Bad Request the body or parameters can't be read
401 Unauthorized we don't know who you are (no or wrong credentials)
403 Forbidden we know you, you're not allowed
404 Not Found no such resource
405 Method Not Allowed wrong method for this URL (send Allow)
409 Conflict clashes with the current state (cancelling a paid order)
422 Unprocessable Entity readable, but breaks the rules
429 Too Many Requests the client is sending too fast
500 Internal Server Error our bug
503 Service Unavailable temporarily can't answer

Never hide an error behind 200.

Pagination and filtering

Raju's shop has a year of orders. The obvious handler sends all of them:

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
)

type Order struct {
    ID     int    `json:"id"`
    Status string `json:"status"`
}

var all []Order

func list(w http.ResponseWriter, r *http.Request) {
    json.NewEncoder(w).Encode(all) // the obvious way: send everything
}

func main() {
    for i := 1; i <= 10000; i++ { // a year of orders at Raju's
        all = append(all, Order{i, []string{"new", "ready", "paid"}[i%3]})
    }
    rec := httptest.NewRecorder()
    list(rec, httptest.NewRequest("GET", "/orders", nil))
    fmt.Printf("GET /orders -> %d, %d orders, %d bytes\n", rec.Code, len(all), rec.Body.Len())
    fmt.Println("...and the app shows only the first 20 of them")
}

Output:

GET /orders -> 200, 10000 orders, 278897 bytes
...and the app shows only the first 20 of them

278,897 bytes on every screen open, for 20 visible rows. Real-life example: nobody reads a thick order register cover to cover; you open one page.

  • Pagination: returning a big list one page at a time.
  • limit: rows per page (default 20, capped at 100: never trust the client's number).
  • offset: how many rows to skip.
  • Filtering: keeping only matching rows (?status=ready), done on the server before paging.
  • next link: the URL of the next page, so the app never builds URLs itself.
package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
    "strconv"
)

type Order struct {
    ID     int    `json:"id"`
    Status string `json:"status"`
}

var all []Order // 10,000 orders
func list(w http.ResponseWriter, r *http.Request) {
    q := r.URL.Query()
    limit := 20 // a sensible default
    if s := q.Get("limit"); s != "" {
        n, err := strconv.Atoi(s)
        if err != nil || n < 1 || n > 100 { // never trust the client's number
            http.Error(w, "limit must be 1 to 100", http.StatusBadRequest)
            return
        }
        limit = n
    }
    offset, _ := strconv.Atoi(q.Get("offset"))
    var hits []Order
    for _, o := range all {
        if s := q.Get("status"); s == "" || o.Status == s { // filtering
            hits = append(hits, o)
        }
    }
    page := hits[min(offset, len(hits)):min(offset+limit, len(hits))] // pagination
    res := map[string]any{"items": page, "total": len(hits)}
    if offset+limit < len(hits) {
        q.Set("offset", strconv.Itoa(offset+limit)) // same filters, next page
        res["next"] = "/orders?" + q.Encode()
    }
    enc := json.NewEncoder(w)
    enc.SetEscapeHTML(false) // keep & as & in the link
    enc.Encode(res)
}

func main() {
    for i := 1; i <= 10000; i++ {
        all = append(all, Order{i, []string{"new", "ready", "paid"}[i%3]})
    }
    for _, u := range []string{"/orders?limit=10000", "/orders?limit=2", "/orders?limit=2&offset=2",
        "/orders?limit=2&offset=9998", "/orders?limit=2&status=ready", "/orders?limit=2&offset=2&status=ready"} {
        rec := httptest.NewRecorder()
        list(rec, httptest.NewRequest("GET", u, nil))
        fmt.Printf("%-38s %d %s", u, rec.Code, rec.Body)
    }
}

Output:

/orders?limit=10000                    400 limit must be 1 to 100
/orders?limit=2                        200 {"items":[{"id":1,"status":"ready"},{"id":2,"status":"paid"}],"next":"/orders?limit=2&offset=2","total":10000}
/orders?limit=2&offset=2               200 {"items":[{"id":3,"status":"new"},{"id":4,"status":"ready"}],"next":"/orders?limit=2&offset=4","total":10000}
/orders?limit=2&offset=9998            200 {"items":[{"id":9999,"status":"new"},{"id":10000,"status":"ready"}],"total":10000}
/orders?limit=2&status=ready           200 {"items":[{"id":1,"status":"ready"},{"id":4,"status":"ready"}],"next":"/orders?limit=2&offset=2&status=ready","total":3334}
/orders?limit=2&offset=2&status=ready  200 {"items":[{"id":7,"status":"ready"},{"id":10,"status":"ready"}],"next":"/orders?limit=2&offset=4&status=ready","total":3334}

SetEscapeHTML(false) keeps & readable in the link (by default encoding/json writes it as &). The last page has no next, and the next link keeps the filter because it is built from the same query with q.Set("offset", ...) and q.Encode().

Offset vs cursor (keyset) pagination

Offset pagination is simple and can jump to page 5, but when new rows arrive while someone is reading, rows shift and a page repeats (or skips) a row; and a big offset is slow because the database still walks past every skipped row. Cursor pagination (also called keyset pagination) says "after id 8" instead of "skip 3": the query is WHERE id < 8 (newest first) or WHERE id > 8 (oldest first). It stays correct while rows are added and is fast with an index. Real run (newest first, a new order 11 arrives after page 1):

package main

import (
    "fmt"
)

type Order struct{ ID int }

// offsetPage skips `offset` rows; cursorPage returns rows with ID < after (newest first).
func offsetPage(all []Order, offset, limit int) []Order {
    return all[min(offset, len(all)):min(offset+limit, len(all))]
}

func cursorPage(all []Order, after, limit int) []Order {
    var page []Order
    for _, o := range all {
        if after == 0 || o.ID < after {
            page = append(page, o)
            if len(page) == limit {
                break
            }
        }
    }
    return page
}

func main() {
    var all []Order // newest first: 10, 9, ..., 1
    for id := 10; id >= 1; id-- {
        all = append(all, Order{id})
    }
    p1 := offsetPage(all, 0, 3)
    c1 := cursorPage(all, 0, 3)
    fmt.Println("page 1 (offset):", p1, "| page 1 (cursor):", c1)

    all = append([]Order{{11}}, all...) // a new order arrives while the user reads page 1

    fmt.Println("page 2 (offset=3):", offsetPage(all, 3, 3)) // shifted: repeats order 8
    fmt.Println("page 2 (after=8): ", cursorPage(all, c1[len(c1)-1].ID, 3))
}

Output:

page 1 (offset): [{10} {9} {8}] | page 1 (cursor): [{10} {9} {8}]
page 2 (offset=3): [{8} {7} {6}]
page 2 (after=8):  [{7} {6} {5}]

Offset page 2 repeats order 8; the cursor page continues correctly with 7, 6, 5.

Versioning

The app is on 5,000 phones, and people update late. We rename qty to quantity and add unit. A change that makes old clients fail is a breaking change.

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/http/httptest"
    "strings"
)

type OrderV1 struct {
    Item string `json:"item"`
    Qty  int    `json:"qty"`
}

type OrderV2 struct { // the new shape: "qty" renamed, a unit added
    Item     string `json:"item"`
    Quantity int    `json:"quantity"`
    Unit     string `json:"unit"`
}

// oldApp is the app already on 5,000 phones: it only knows "qty".
func oldApp(h http.Handler, path string) {
    rec := httptest.NewRecorder()
    h.ServeHTTP(rec, httptest.NewRequest("GET", path, nil))
    var o OrderV1
    json.Unmarshal(rec.Body.Bytes(), &o)
    fmt.Printf("%-14s old app shows: %d x %s   (body %s)\n", path, o.Qty, o.Item, strings.TrimSpace(rec.Body.String()))
}

func main() {
    v1 := func(w http.ResponseWriter, r *http.Request) { json.NewEncoder(w).Encode(OrderV1{"chai", 2}) }
    v2 := func(w http.ResponseWriter, r *http.Request) { json.NewEncoder(w).Encode(OrderV2{"chai", 2, "cup"}) }

    changed := http.NewServeMux() // obvious way: change the one URL in place
    changed.HandleFunc("GET /orders/{id}", v2)
    oldApp(changed, "/orders/42")

    versioned := http.NewServeMux() // fix: keep v1 alive, add v2 next to it
    versioned.HandleFunc("GET /v1/orders/{id}", v1)
    versioned.HandleFunc("GET /v2/orders/{id}", v2)
    oldApp(versioned, "/v1/orders/42")
    oldApp(versioned, "/v2/orders/42") // only new apps should call this
}

Output:

/orders/42     old app shows: 0 x chai   (body {"item":"chai","quantity":2,"unit":"cup"})
/v1/orders/42  old app shows: 2 x chai   (body {"item":"chai","qty":2})
/v2/orders/42  old app shows: 0 x chai   (body {"item":"chai","quantity":2,"unit":"cup"})

Changing the URL in place made the old app show 0 x chai: no crash, no error, just a wrong order (the missing qty decodes to its zero value). Versioning puts the version into the API: /v1/... keeps the old shape, /v2/... sits next to it.

  • Where: most often the URL path (/v1/orders), visible in every log line; some APIs use a header (Accept: application/vnd.rajus.v2+json).
  • Adding a new optional field is not breaking (old apps ignore unknown fields). Renaming, removing, or changing a type or meaning is breaking.
  • Only breaking changes get a new version; announce when the old one will stop.

Idempotency keys

Real-life example: "did my order go through?" You show the same token number at the counter, and the clerk's token book says "token 47, already placed". In episode 63 we did the client side; here's the server side. A retried POST makes a second order because POST is not idempotent. The app makes a unique Idempotency-Key per action and sends it on every try; the server stores key → reply and replays the reply for a repeated key.

package main

import (
    "fmt"
    "net/http"
    "net/http/httptest"
    "sync"
)

var (
    mu     sync.Mutex
    orders int
    seen   = map[string]string{} // Idempotency-Key -> the reply we sent
)

func create(w http.ResponseWriter, r *http.Request) {
    mu.Lock()
    defer mu.Unlock()
    key := r.Header.Get("Idempotency-Key")
    if reply, ok := seen[key]; ok && key != "" {
        fmt.Fprint(w, reply) // a retry: same answer, no new order
        return
    }
    orders++
    reply := fmt.Sprintf("order %d placed", orders)
    if key != "" {
        seen[key] = reply
    }
    w.WriteHeader(http.StatusCreated)
    fmt.Fprint(w, reply)
}

func main() {
    // The phone sends POST /orders, the reply is lost on a bad network, and the app tries again.
    for _, key := range []string{"", "", "7f3a-91c2", "7f3a-91c2"} {
        req := httptest.NewRequest("POST", "/orders", nil)
        req.Header.Set("Idempotency-Key", key)
        rec := httptest.NewRecorder()
        create(rec, req)
        fmt.Printf("key=%-11q %d %-16s orders in the kitchen: %d\n", key, rec.Code, rec.Body, orders)
    }
}

Output:

key=""          201 order 1 placed   orders in the kitchen: 1
key=""          201 order 2 placed   orders in the kitchen: 2
key="7f3a-91c2" 201 order 3 placed   orders in the kitchen: 3
key="7f3a-91c2" 200 order 3 placed   orders in the kitchen: 3

Without a key: two orders for one tap. With key 7f3a-91c2: the retry gets order 3 placed again and the kitchen count stays 3. The mutex matters because real handlers run concurrently. In production keep the key table in the database (key as a unique column, with an expiry of about a day) so it survives restarts and works across servers.

Routers and frameworks

A router matches a request's method and path to a handler. A framework adds much more (binding, validation, its own context type). A dependency is outside code your program pulls in and must keep updated.

Option What it is Handler type Notes
net/http ServeMux (Go 1.22+) standard library router with methods and {wildcards} http.Handler zero dependencies; enough for most REST APIs
chi "a lightweight, idiomatic and composable router" http.Handler route groups, middleware; everything from this course works
gorilla/mux "a request router and dispatcher" http.Handler matches host, headers, query values, regex variables
gin "a high-performance HTTP web framework" gin.Context binding and more; handlers depend on gin
echo "High performance, extensible, minimalist Go web framework" echo.Context radix-tree router, binding; wraps net/http handlers

Trade-off: with gin or echo your handlers take the framework's context type, so leaving later means rewriting them. Start with net/http.

Documenting with OpenAPI

The OpenAPI Specification is a standard, language-independent way to describe an HTTP API in a file written in YAML (an indentation-based data format) or JSON. From it, tools generate interactive documentation, client code, server stubs and tests. Keep the file in the repository and update it in the same commit as the code.

openapi: 3.1.0
info: {title: Raju's Orders API, version: "1.0"}
paths:
  /v1/orders/{id}:
    get:
      summary: Get one order
      parameters:
        - {name: id, in: path, required: true, schema: {type: integer}}
      responses:
        "200": {description: The order}
        "404": {description: No such order}

Mini project: Raju's Orders API

Build /v1/orders with: GET (cursor pagination with after, limit capped at 100, status filter, a next link), POST (strict decoding, validation, 201 + Location, Idempotency-Key), and GET / PATCH / DELETE on /v1/orders/{id}, every error as problem details, and checks for every status code.

Full solution

main.go (standard library only; main drives the API with httptest and prints every answer):

package main

import (
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "net/http/httptest"
    "net/url"
    "strconv"
    "strings"
    "sync"
)

// ---------- types ----------

type Order struct {
    ID     int    `json:"id"`
    Item   string `json:"item"`
    Qty    int    `json:"qty"`
    Status string `json:"status"` // new, ready, paid
}

type OrderPatch struct {
    Qty    *int    `json:"qty"`
    Status *string `json:"status"`
}

type Problem struct {
    Type     string            `json:"type"`
    Title    string            `json:"title"`
    Status   int               `json:"status"`
    Detail   string            `json:"detail,omitempty"`
    Instance string            `json:"instance,omitempty"`
    Errors   map[string]string `json:"errors,omitempty"`
}

// ---------- store ----------

type Store struct {
    mu     sync.Mutex
    orders map[int]Order
    nextID int
    seen   map[string]Order // Idempotency-Key -> created order
}

func NewStore() *Store {
    return &Store{orders: map[int]Order{}, nextID: 1, seen: map[string]Order{}}
}

// ---------- helpers ----------

func writeJSON(w http.ResponseWriter, status int, v any) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(status)
    enc := json.NewEncoder(w)
    enc.SetEscapeHTML(false)
    enc.Encode(v)
}

func problem(w http.ResponseWriter, r *http.Request, p Problem) {
    if p.Type == "" {
        p.Type = "about:blank"
    }
    p.Title, p.Instance = http.StatusText(p.Status), r.URL.Path
    w.Header().Set("Content-Type", "application/problem+json")
    w.WriteHeader(p.Status)
    json.NewEncoder(w).Encode(p)
}

func decodeJSON(w http.ResponseWriter, r *http.Request, dst any) error {
    r.Body = http.MaxBytesReader(w, r.Body, 1<<20)
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields()
    if err := dec.Decode(dst); err != nil {
        return err
    }
    if dec.Decode(&struct{}{}) != io.EOF {
        return errors.New("body must hold one JSON object")
    }
    return nil
}

var statuses = map[string]bool{"new": true, "ready": true, "paid": true}

func validQty(q int) bool { return q >= 1 && q <= 20 }

func pathID(w http.ResponseWriter, r *http.Request) (int, bool) {
    id, err := strconv.Atoi(r.PathValue("id"))
    if err != nil {
        problem(w, r, Problem{Status: 400, Detail: "id must be a number"})
        return 0, false
    }
    return id, true
}

// ---------- handlers ----------

func (s *Store) create(w http.ResponseWriter, r *http.Request) {
    var in Order
    if err := decodeJSON(w, r, &in); err != nil {
        problem(w, r, Problem{Status: 400, Detail: err.Error()})
        return
    }
    bad := map[string]string{}
    if in.Item == "" {
        bad["item"] = "is required"
    }
    if !validQty(in.Qty) {
        bad["qty"] = "must be 1 to 20"
    }
    if len(bad) > 0 {
        problem(w, r, Problem{Type: "https://rajus.in/probs/invalid-order", Status: 422,
            Detail: fmt.Sprintf("%d field(s) are invalid", len(bad)), Errors: bad})
        return
    }
    s.mu.Lock()
    defer s.mu.Unlock()
    key := r.Header.Get("Idempotency-Key")
    if o, ok := s.seen[key]; ok && key != "" {
        writeJSON(w, http.StatusOK, o) // a retry: same order, nothing new
        return
    }
    o := Order{ID: s.nextID, Item: in.Item, Qty: in.Qty, Status: "new"}
    s.nextID++
    s.orders[o.ID] = o
    if key != "" {
        s.seen[key] = o
    }
    w.Header().Set("Location", "/v1/orders/"+strconv.Itoa(o.ID))
    writeJSON(w, http.StatusCreated, o)
}

func (s *Store) get(w http.ResponseWriter, r *http.Request) {
    id, ok := pathID(w, r)
    if !ok {
        return
    }
    s.mu.Lock()
    o, found := s.orders[id]
    s.mu.Unlock()
    if !found {
        problem(w, r, Problem{Status: 404, Detail: "no order " + strconv.Itoa(id)})
        return
    }
    writeJSON(w, http.StatusOK, o)
}

func (s *Store) patch(w http.ResponseWriter, r *http.Request) {
    id, ok := pathID(w, r)
    if !ok {
        return
    }
    var p OrderPatch
    if err := decodeJSON(w, r, &p); err != nil {
        problem(w, r, Problem{Status: 400, Detail: err.Error()})
        return
    }
    bad := map[string]string{}
    if p.Qty != nil && !validQty(*p.Qty) {
        bad["qty"] = "must be 1 to 20"
    }
    if p.Status != nil && !statuses[*p.Status] {
        bad["status"] = "must be new, ready or paid"
    }
    if len(bad) > 0 {
        problem(w, r, Problem{Type: "https://rajus.in/probs/invalid-order", Status: 422,
            Detail: fmt.Sprintf("%d field(s) are invalid", len(bad)), Errors: bad})
        return
    }
    s.mu.Lock()
    defer s.mu.Unlock()
    o, found := s.orders[id]
    if !found {
        problem(w, r, Problem{Status: 404, Detail: "no order " + strconv.Itoa(id)})
        return
    }
    if p.Qty != nil {
        o.Qty = *p.Qty
    }
    if p.Status != nil {
        o.Status = *p.Status
    }
    s.orders[id] = o
    writeJSON(w, http.StatusOK, o)
}

func (s *Store) remove(w http.ResponseWriter, r *http.Request) {
    id, ok := pathID(w, r)
    if !ok {
        return
    }
    s.mu.Lock()
    delete(s.orders, id) // DELETE is idempotent: deleting twice is fine
    s.mu.Unlock()
    w.WriteHeader(http.StatusNoContent)
}

// list: GET /v1/orders?status=ready&limit=2&after=5 (cursor = the last id seen, ascending ids)
func (s *Store) list(w http.ResponseWriter, r *http.Request) {
    q := r.URL.Query()
    limit := 20
    if v := q.Get("limit"); v != "" {
        n, err := strconv.Atoi(v)
        if err != nil || n < 1 || n > 100 {
            problem(w, r, Problem{Status: 400, Detail: "limit must be 1 to 100"})
            return
        }
        limit = n
    }
    after, _ := strconv.Atoi(q.Get("after"))
    status := q.Get("status")
    s.mu.Lock()
    var page []Order
    more := false
    for id := after + 1; id < s.nextID; id++ { // ids ascend, so this is "WHERE id > after ORDER BY id"
        o, ok := s.orders[id]
        if !ok || (status != "" && o.Status != status) {
            continue
        }
        if len(page) == limit {
            more = true
            break
        }
        page = append(page, o)
    }
    s.mu.Unlock()
    res := map[string]any{"items": page}
    if more {
        next := url.Values{"limit": {strconv.Itoa(limit)}, "after": {strconv.Itoa(page[len(page)-1].ID)}}
        if status != "" {
            next.Set("status", status)
        }
        res["next"] = "/v1/orders?" + next.Encode()
    }
    writeJSON(w, http.StatusOK, res)
}

func (s *Store) Routes() http.Handler {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /v1/orders", s.list)
    mux.HandleFunc("POST /v1/orders", s.create)
    mux.HandleFunc("GET /v1/orders/{id}", s.get)
    mux.HandleFunc("PATCH /v1/orders/{id}", s.patch)
    mux.HandleFunc("DELETE /v1/orders/{id}", s.remove)
    return mux
}

// ---------- checks ----------

func main() {
    h := NewStore().Routes()
    call := func(method, path, body, key string) {
        req := httptest.NewRequest(method, path, strings.NewReader(body))
        if key != "" {
            req.Header.Set("Idempotency-Key", key)
        }
        rec := httptest.NewRecorder()
        h.ServeHTTP(rec, req)
        fmt.Printf("%-6s %-38s %d %s", method, path, rec.Code, rec.Body)
        if rec.Body.Len() == 0 {
            fmt.Println()
        }
    }
    for _, item := range []string{"chai", "samosa", "chai", "vada pav", "chai"} {
        call("POST", "/v1/orders", `{"item":"`+item+`","qty":2}`, "")
    }
    call("POST", "/v1/orders", `{"item":"chai","qty":1}`, "k-77")
    call("POST", "/v1/orders", `{"item":"chai","qty":1}`, "k-77") // retry
    call("POST", "/v1/orders", `{"item":"chai","qty":0,"tabel":4}`, "")
    call("POST", "/v1/orders", `{"qty":0}`, "")
    call("PATCH", "/v1/orders/1", `{"status":"ready"}`, "")
    call("PATCH", "/v1/orders/3", `{"status":"ready"}`, "")
    call("PATCH", "/v1/orders/5", `{"status":"done"}`, "")
    call("GET", "/v1/orders/2", "", "")
    call("DELETE", "/v1/orders/2", "", "")
    call("GET", "/v1/orders/2", "", "")
    call("GET", "/v1/orders?limit=2", "", "")
    call("GET", "/v1/orders?after=3&limit=2", "", "")
    call("GET", "/v1/orders?status=ready&limit=1", "", "")
    call("GET", "/v1/orders?status=ready&limit=1&after=1", "", "")
    call("PUT", "/v1/orders/1", `{}`, "")
}

Output:

POST   /v1/orders                             201 {"id":1,"item":"chai","qty":2,"status":"new"}
POST   /v1/orders                             201 {"id":2,"item":"samosa","qty":2,"status":"new"}
POST   /v1/orders                             201 {"id":3,"item":"chai","qty":2,"status":"new"}
POST   /v1/orders                             201 {"id":4,"item":"vada pav","qty":2,"status":"new"}
POST   /v1/orders                             201 {"id":5,"item":"chai","qty":2,"status":"new"}
POST   /v1/orders                             201 {"id":6,"item":"chai","qty":1,"status":"new"}
POST   /v1/orders                             200 {"id":6,"item":"chai","qty":1,"status":"new"}
POST   /v1/orders                             400 {"type":"about:blank","title":"Bad Request","status":400,"detail":"json: unknown field \"tabel\"","instance":"/v1/orders"}
POST   /v1/orders                             422 {"type":"https://rajus.in/probs/invalid-order","title":"Unprocessable Entity","status":422,"detail":"2 field(s) are invalid","instance":"/v1/orders","errors":{"item":"is required","qty":"must be 1 to 20"}}
PATCH  /v1/orders/1                           200 {"id":1,"item":"chai","qty":2,"status":"ready"}
PATCH  /v1/orders/3                           200 {"id":3,"item":"chai","qty":2,"status":"ready"}
PATCH  /v1/orders/5                           422 {"type":"https://rajus.in/probs/invalid-order","title":"Unprocessable Entity","status":422,"detail":"1 field(s) are invalid","instance":"/v1/orders/5","errors":{"status":"must be new, ready or paid"}}
GET    /v1/orders/2                           200 {"id":2,"item":"samosa","qty":2,"status":"new"}
DELETE /v1/orders/2                           204 
GET    /v1/orders/2                           404 {"type":"about:blank","title":"Not Found","status":404,"detail":"no order 2","instance":"/v1/orders/2"}
GET    /v1/orders?limit=2                     200 {"items":[{"id":1,"item":"chai","qty":2,"status":"ready"},{"id":3,"item":"chai","qty":2,"status":"ready"}],"next":"/v1/orders?after=3&limit=2"}
GET    /v1/orders?after=3&limit=2             200 {"items":[{"id":4,"item":"vada pav","qty":2,"status":"new"},{"id":5,"item":"chai","qty":2,"status":"new"}],"next":"/v1/orders?after=5&limit=2"}
GET    /v1/orders?status=ready&limit=1        200 {"items":[{"id":1,"item":"chai","qty":2,"status":"ready"}],"next":"/v1/orders?after=1&limit=1&status=ready"}
GET    /v1/orders?status=ready&limit=1&after=1 200 {"items":[{"id":3,"item":"chai","qty":2,"status":"ready"}]}
PUT    /v1/orders/1                           405 Method Not Allowed

Notes: the retry with key k-77 returns the same order 6 with 200; the unknown field gives a 400 problem, the impossible values a 422 with errors; DELETE then GET gives 204 then 404; PUT was never registered, so 405. The list walks ids upwards from after, which is exactly what WHERE id > ? ORDER BY id LIMIT ? does in SQL (episode 67).

openapi.yaml (part):

openapi: 3.1.0
info: {title: Raju's Orders API, version: "1.0"}
paths:
  /v1/orders:
    get:
      summary: List orders, oldest first
      parameters:
        - {name: status, in: query, schema: {type: string, enum: [new, ready, paid]}}
        - {name: limit, in: query, schema: {type: integer, minimum: 1, maximum: 100, default: 20}}
        - {name: after, in: query, description: id of the last order already seen, schema: {type: integer}}
      responses:
        "200": {description: A page of orders and an optional next link}
        "400": {description: Bad parameters, content: {application/problem+json: {}}}
    post:
      summary: Place an order
      parameters:
        - {name: Idempotency-Key, in: header, schema: {type: string}}
      requestBody:
        content:
          application/json:
            schema:
              type: object
              additionalProperties: false
              required: [item, qty]
              properties:
                item: {type: string}
                qty: {type: integer, minimum: 1, maximum: 20}
      responses:
        "201": {description: Created; Location holds the new order's URL}
        "200": {description: A retry with a known Idempotency-Key}
        "400": {description: Unreadable body, content: {application/problem+json: {}}}
        "422": {description: Invalid fields, content: {application/problem+json: {}}}
  /v1/orders/{id}:
    parameters:
      - {name: id, in: path, required: true, schema: {type: integer}}
    get:
      responses: {"200": {description: The order}, "404": {description: No such order}}
    patch:
      responses: {"200": {description: Updated}, "404": {description: No such order}, "422": {description: Invalid fields}}
    delete:
      responses: {"204": {description: Deleted (also when it was already gone)}}

Common mistakes (and the real messages)

Mistake What you see Why Fix
errors returned with 200 app: worked=true + {"error":"order 42 not found"} clients trust the status code a real 4xx/5xx
verbs in URLs /getmenu → 404 page not found invented names can't be guessed nouns + methods
ignoring Decode's error "two" saved as Qty:0, a torn body saved as an empty order garbage gets saved check it → 400
unknown fields allowed the tabel typo silently dropped default encoding/json ignores them DisallowUnknownFields() → json: unknown field "tabel"
no body limit memory eaten by huge bodies nothing stops them http.MaxBytesReader → http: request body too large
no validation qty: -3 saved with 201 decoding checks shape only validate → 422
Header().Set after WriteHeader Content-Type: text/plain; charset=utf-8 headers were already sent headers first (writeJSON)
every handler its own error format text, {"error"}, maps the app needs special code per error problem details
sending the whole list 278897 bytes for 20 rows no pagination limit (capped) + offset/cursor + next
renaming a field in a live version old app shows: 0 x chai breaking change add /v2, keep /v1
retrying POST without a key orders in the kitchen: 2 for one tap POST isn't idempotent Idempotency-Key

Practice

  1. Redesign these as REST endpoints: /listAllReadyOrders, /updateOrderQty?id=7&qty=3, /removeItemFromOrder?order=7&item=2.
  2. A customer tries to cancel an order that's already paid. Which status code? Write the handler.
  3. Why is {"qty":"two"} a 400 but {"qty":0} a 422?
  4. Your GET /orders uses offset pagination and users complain about duplicates on page 2 during the lunch rush. Why, and what do you change?
  5. You want to add a "unit": "cup" field to /v1 responses. Do you need /v2?
Solutions

1. GET /orders?status=ready; PATCH /orders/7 with {"qty":3}; DELETE /orders/7/items/2.

2. 409 Conflict: the request clashes with the order's current state. Real run:

package main

import (
    "fmt"
    "net/http"
    "net/http/httptest"
    "strconv"
)

var status = map[int]string{1: "new", 2: "paid"}

func cancel(w http.ResponseWriter, r *http.Request) {
    id, _ := strconv.Atoi(r.PathValue("id"))
    s, ok := status[id]
    switch {
    case !ok:
        http.Error(w, "no such order", http.StatusNotFound)
    case s == "paid":
        http.Error(w, "order is already paid", http.StatusConflict) // 409: clashes with the current state
    default:
        status[id] = "cancelled"
        w.WriteHeader(http.StatusNoContent)
    }
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("POST /orders/{id}/cancel", cancel)
    for _, p := range []string{"/orders/1/cancel", "/orders/2/cancel", "/orders/9/cancel"} {
        rec := httptest.NewRecorder()
        mux.ServeHTTP(rec, httptest.NewRequest("POST", p, nil))
        fmt.Printf("%-18s %d %s\n", p, rec.Code, rec.Body)
    }
}
/orders/1/cancel   204 
/orders/2/cancel   409 order is already paid

/orders/9/cancel   404 no such order

(http.Error ends its message with a newline, which is why there's an empty line after the 409.)

3. "two" can't be decoded into an int at all (json: cannot unmarshal string into Go struct field Order.qty of type int), so the body is unreadable: 400. 0 decodes fine but breaks the rule "1 to 20": 422.

4. New orders arrive at the top while people read, so every row shifts down and the next offset page repeats a row (the cursor example above shows it). Switch to cursor pagination: ?after=<last id>.

5. No. Adding a new optional field is not a breaking change; old apps ignore fields they don't know. Renaming or removing qty would need /v2.

Quick quiz

1. The client sends {"item":"chai","qty":0}. We decode strictly (DisallowUnknownFields) and then validate that qty is 1 to 20. Which status? (From the video.)

A. 400 Bad Request · B. 422 Unprocessable Entity · C. 201 Created · D. 200 OK

Answer

B. 422. The JSON is valid and has only known fields, so decoding passes; qty 0 breaks the rule, so validation answers 422.

2. A handler calls w.WriteHeader(201) and then w.Header().Set("Content-Type", "application/json"). What Content-Type does the client get for the body {"id":43}?

Answer

text/plain; charset=utf-8. The header was set too late, so Go sniffed the body.

3. Which of GET, POST, PUT, DELETE are idempotent?

Answer

GET, PUT and DELETE. POST is not (two POSTs make two orders), which is why it needs an Idempotency-Key for safe retries.

Interview questions

  1. What makes an API RESTful? Resources named by URLs (nouns), standard HTTP methods as the actions, representations (usually JSON), meaningful status codes, and stateless requests.
  2. What's the difference between safe and idempotent? Safe changes nothing (GET, HEAD); idempotent means repeating has the same effect as doing it once (GET, PUT, DELETE).
  3. PUT vs PATCH? PUT replaces the whole resource; PATCH changes only the fields sent (use pointer fields to detect "not sent").
  4. How do you decode JSON safely in Go? http.MaxBytesReader, json.NewDecoder with DisallowUnknownFields, check Decode's error, optionally check for trailing data, then validate.
  5. 400 vs 422? 400: the request can't be parsed. 422: it parses but violates business rules.
  6. What is RFC 9457? Problem details for HTTP APIs: a standard JSON error body (type, title, status, detail, instance) with media type application/problem+json; it obsoletes RFC 7807.
  7. Offset vs cursor pagination? Offset is simple and allows jumping, but shifts when rows are added and gets slow for big offsets; cursor (keyset) uses "after this key", stays stable and is fast with an index.
  8. How do you version an API? Usually /v1 in the path; only for breaking changes; keep old versions until clients have moved.
  9. How does a server implement idempotency keys? Store key → response (in the database, unique key, with expiry); on a repeated key return the stored response without doing the work again.
  10. Standard library router or a framework? Since Go 1.22 ServeMux handles methods and wildcards; chi/gorilla stay compatible with http.Handler; gin/echo add features but tie handlers to their context type.

Glossary

Term Meaning Example
API the requests a program accepts and the answers it gives Raju's orders API
Web API an API spoken over HTTP GET /orders/42
REST Representational State Transfer: resources + methods + status codes DELETE /orders/42 → 204
Resource a thing the API manages one order, the list of orders
Representation a resource written in a format {"id":42,"item":"chai","qty":2}
URL the address of a resource /orders/42
Endpoint a method + URL the API answers POST /orders
Query string the part after ?, for filters and pages ?status=ready&limit=20
HTTP method the action of a request GET, POST, PUT, PATCH, DELETE
GET read GET /orders/42
POST create POST /orders
PUT replace the whole resource PUT /orders/42
PATCH change part of a resource PATCH /orders/42 {"qty":3}
DELETE remove DELETE /orders/42
Safe method changes nothing on the server GET
Idempotent twice has the same effect as once PUT, DELETE
Stateless every request carries all it needs the key in each request
Status code three-digit result of a request 201, 404
200 OK success with a result a read
201 Created something new was made after POST
204 No Content success, empty body after DELETE
400 Bad Request can't read the request unexpected EOF
401 Unauthorized who are you? (no/wrong credentials) missing key
403 Forbidden known, but not allowed a waiter asking for sales
404 Not Found no such resource /orders/99
405 Method Not Allowed wrong method for the URL PUT /orders/43
409 Conflict clashes with the current state cancelling a paid order
422 Unprocessable Entity readable but breaks the rules qty: -3
429 Too Many Requests the client is too fast a flooding script
500 Internal Server Error our bug a panic
503 Service Unavailable temporarily can't answer overload, maintenance
Location header URL of the newly created resource Location: /orders/43
Allow header methods a URL supports Allow: DELETE, GET, HEAD
Content-Type what kind of data the body is application/json
Content sniffing guessing the type from the first bytes text/plain; charset=utf-8
JSON decoding turning JSON text into a Go value json.NewDecoder(r.Body).Decode(&o)
DisallowUnknownFields makes unknown JSON fields an error json: unknown field "tabel"
http.MaxBytesReader limits a request body's size 1<<20 (1 MB)
unexpected EOF the JSON ended in the middle a torn body
Validation checking data against your rules qty must be 1 to 20
Response helper one function that writes replies correctly writeJSON(w, 201, o)
WriteHeader sends the status line and headers w.WriteHeader(201)
Problem details RFC 9457 standard error body {"type":"about:blank","title":"Not Found",...}
RFC a numbered internet standards document RFC 9457
application/problem+json media type of problem details error responses
about:blank problem type meaning "just the status code" a plain 404
Extension member your own extra field in a problem errors
omitempty JSON tag option: leave out empty values json:"detail,omitempty"
Pagination returning a list one page at a time ?limit=20
limit rows per page (capped) limit=20, max 100
offset rows to skip offset=40
Cursor (keyset) pagination "after this key" instead of "skip n" ?after=40
Filtering keeping only matching rows ?status=ready
next link URL of the next page in the response "next": "/orders?limit=2&offset=2"
SetEscapeHTML(false) keep &, <, > as-is in JSON output readable next links
Versioning putting a version into the API /v1/orders, /v2/orders
Breaking change a change that makes old clients fail renaming qty
Zero value Go's default value of a type 0 for a missing qty
Idempotency-Key header carrying a unique key per action Idempotency-Key: 7f3a-91c2
Internet-Draft an IETF work-in-progress document draft-ietf-httpapi-idempotency-key-header
IETF the body that publishes internet standards (RFCs) RFC 9457
Mutex a lock so one goroutine at a time touches shared data mu.Lock()
Dependency outside code your program pulls in chi
Router matches method + path to a handler http.ServeMux, chi
Framework a router plus many extra features and its own handler style gin, echo
chi lightweight router using plain http.Handler github.com/go-chi/chi
gorilla/mux router matching host, headers, regex variables github.com/gorilla/mux
gin high-performance web framework gin.Context
echo minimalist web framework echo.Context
OpenAPI standard file describing an HTTP API openapi.yaml
YAML indentation-based data format paths:
httptest test helpers: fake requests, recorders, test servers httptest.NewRecorder()
r.PathValue reads a {wildcard} from the path (Go 1.22) r.PathValue("id")

Summary

  • An API without rules lies: invented verb URLs and 200 for errors made the app think a failed cancel worked.
  • REST: nouns in URLs (resources), methods as verbs, honest status codes, JSON representations, stateless requests. Know which methods are safe and idempotent.
  • Decode strictly (MaxBytesReader, DisallowUnknownFields, check the error) → 400; then validate → 422 with every field problem.
  • Set headers before WriteHeader; use a writeJSON helper.
  • Send every error as problem details (RFC 9457, application/problem+json).
  • Big lists: pagination with a capped limit, offset or a cursor, filtering first, and a next link.
  • Version only for breaking changes (/v1 + /v2); adding optional fields is safe.
  • Idempotency keys make retried POSTs safe.
  • Start with net/http; chi and gorilla/mux stay http.Handler-compatible; gin and echo are frameworks. Describe the API with OpenAPI.

Next episode: Securing Go services: password hashing with bcrypt and Argon2id, sessions and secure cookies, JWT verification, API keys, OAuth 2.0, crypto/tls, crypto/rand vs math/rand, html/template auto-escaping, SQL injection and placeholders, path traversal and os.Root, input size limits, security headers and govulncheck.