SyntaxPilot
Courses/Go Programming/Lesson 35
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Go Programming · Lesson 35 of 89

Encoding: JSON, CSV and more

encoding/jsonMarshalUnmarshalMarshalIndentstruct tags (json:"name,omitempty")unknown fieldsmap[string]any decodingjson.RawMessagejson.Decoder and Encoder streamingDisallowUnknownFieldscustom MarshalJSON and UnmarshalJSONencoding/csvencoding/xmlencoding/base64

These are the notes for Go episode 35. Raju's menu has to travel to a delivery app written in another language. fmt.Println prints Go's own display format, and JSON built by hand with Sprintf breaks on the first quote. So you invent encoding: an agreed format both sides understand. You use encoding/json: json.Marshal and json.MarshalIndent, struct tags (json:"name", omitempty, "-"), json.Unmarshal and its real errors, the silent typo bug fixed with DisallowUnknownFields, map[string]any where numbers become float64, json.RawMessage, streaming with json.Decoder and json.Encoder, and your own format with MarshalJSON and UnmarshalJSON. Then encoding/csv for spreadsheets, encoding/xml, base64 and hex for raw bytes, and encoding/gob.

Every program here runs as-is in the SyntaxPilot compiler (language: Go) with go run main.go; text typed into the program input box is sent to standard input. All outputs and errors are from real runs (Go 1.24.7, linux/amd64).

Facts and sources

From the standard library documentation shipped with Go 1.24.7 (go doc):

  • json.Unmarshal stores JSON into an interface value as: bool for JSON booleans, float64 for JSON numbers, string for strings, []interface{} for arrays, map[string]interface{} for objects, and nil for null. Unknown object keys are "ignored (see Decoder.DisallowUnknownFields for an alternative)".
  • json.Marshal: "The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any array, slice, map, or string of length zero." A field with tag "-" is always omitted. Only exported fields are encoded.
  • encoding/csv "supports the format described in RFC 4180, except that Writer uses LF instead of CRLF as newline character by default."
  • encoding/base64 "implements base64 encoding as specified by RFC 4648."
  • encoding/gob "manages streams of gobs - binary values exchanged between an Encoder (transmitter) and a Decoder (receiver)."
  • json.RawMessage "is a raw encoded JSON value ... can be used to delay JSON decoding or precompute a JSON encoding."

The problem: a format only Go understands

package main

import (
    "encoding/json"
    "fmt"
)

type Item struct {
    Name  string
    Price int
    Tags  []string
}

func main() {
    chai := Item{"Masala Chai", 10, []string{"hot", "sweet"}}
    fmt.Println(chai)          // only Go understands this

    // build the text by hand for the delivery app?
    s := fmt.Sprintf(`{"name":"%s","price":%d}`, `Raju's "Special" Chai`, 15)
    fmt.Println(s)
    fmt.Println("valid JSON?", json.Valid([]byte(s)))
}

Output:

{Masala Chai 10 [hot sweet]}
{"name":"Raju's "Special" Chai","price":15}
valid JSON? false
  • fmt.Println prints Go's display format, meant for people reading Go output. Another program can't tell where the name ends or what 10 means.
  • Building JSON by hand fails as soon as a value contains a ": the quote ends the JSON string early. json.Valid checks whether bytes are correct JSON, and it says false.

Encoding: packing for a courier

Sending a parcel In Go Meaning
Pack in a standard box encoding (serialization) turn a value into an agreed text or byte format
Label each item struct tags names and options attached to fields
Unpack at the other end decoding (deserialization) turn the bytes back into a value
  • Encoding / serialization: turning a value in memory into bytes (text or binary) that can be saved or sent. Decoding / deserialization is the reverse. In encoding/json, encoding is called Marshal and decoding Unmarshal.
  • JSON (JavaScript Object Notation) is a text format: objects in {} with "key": value pairs, arrays in [], strings in double quotes, numbers, true, false and null. It's understood by practically every programming language.
{"name": "Masala Chai", "price": 10, "veg": true, "tags": ["hot", "sweet"], "offer": null}

json.Marshal: Go value to JSON

package main

import (
    "encoding/json"
    "fmt"
)

type Item struct {
    Name  string
    Price int
    Tags  []string
    stock int // lowercase: unexported
}

func main() {
    chai := Item{`Raju's "Special" Chai`, 15, []string{"hot"}, 40}
    data, err := json.Marshal(chai) // Go value → JSON bytes
    fmt.Println(string(data), err)
    fmt.Println("valid JSON?", json.Valid(data))
}

Output:

{"Name":"Raju's \"Special\" Chai","Price":15,"Tags":["hot"]} <nil>
valid JSON? true
  • json.Marshal(v) returns ([]byte, error). It escapes special characters for you (\" for a quote).
  • A slice becomes a JSON array, a struct an object, a map an object, nil becomes null.
  • It reads the struct's fields at run time using reflection (a program looking at its own types and fields while it runs; episode 39). It can only see exported fields (starting with a capital letter, episode 19). stock vanished without any error.

Struct tags and omitempty

A struct tag is a string literal in backquotes after a field's type. Packages read it with reflection. encoding/json reads the json:"..." key.

package main

import (
    "encoding/json"
    "fmt"
)

type Item struct {
    Name   string   `json:"name"`
    Price  int      `json:"price"`
    Tags   []string `json:"tags,omitempty"` // left out if empty
    Secret string   `json:"-"`              // never sent
}

func main() {
    menu := []Item{
        {Name: "Masala Chai", Price: 10, Tags: []string{"hot"}, Secret: "ginger"},
        {Name: "Samosa", Price: 15},
    }
    data, _ := json.Marshal(menu)
    fmt.Println(string(data))

    pretty, _ := json.MarshalIndent(menu[1], "", "  ")
    fmt.Println(string(pretty))
}

Output:

[{"name":"Masala Chai","price":10,"tags":["hot"]},{"name":"Samosa","price":15}]
{
  "name": "Samosa",
  "price": 15
}
Tag Meaning
json:"name" use name as the JSON key
json:"tags,omitempty" leave the field out when it is empty
json:"-" never encode or decode this field
json:",omitempty" keep the Go name, but omit when empty
json:"price,string" encode the number as a JSON string ("15")
  • Empty, for omitempty, means false, 0, a nil pointer or interface, and any array, slice, map or string of length zero. A zero struct is not "empty" (use a pointer, or the newer omitzero option in Go 1.24).
  • json.MarshalIndent(v, prefix, indent) adds line breaks and indentation for humans.

json.Unmarshal: JSON to Go value

package main

import (
    "encoding/json"
    "fmt"
)

type Item struct {
    Name  string `json:"name"`
    Price int    `json:"price"`
}

func main() {
    var it Item
    err := json.Unmarshal([]byte(`{"name":"Poha","price":20}`), &it)
    fmt.Printf("%+v %v\n", it, err)

    err = json.Unmarshal([]byte(`{"name":"Poha","price":"twenty"}`), &it)
    fmt.Println("err:", err)

    err = json.Unmarshal([]byte(`{"name":"Poha"}`), it) // forgot &
    fmt.Println("err:", err)
}

Output:

{Name:Poha Price:20} <nil>
err: json: cannot unmarshal string into Go struct field Item.price of type int
err: json: Unmarshal(non-pointer main.Item)
  • Pass a pointer (&it, episode 11) so Unmarshal can write into your variable. Without one: json: Unmarshal(non-pointer main.Item).
  • Types must match: a JSON string can't go into an int field: json: cannot unmarshal string into Go struct field Item.price of type int.
  • Keys are matched to field names or tags case-insensitively ("NAME" also fills Name).
  • Fields missing from the JSON keep their current value (the zero value for a fresh variable).

Unknown fields: the silent typo

package main

import (
    "encoding/json"
    "fmt"
    "strings"
)

type Item struct {
    Name  string `json:"name"`
    Price int    `json:"price"`
}

func main() {
    in := `{"name":"Vada Pav","prise":25}` // typo: prise
    var a Item
    err := json.Unmarshal([]byte(in), &a)
    fmt.Printf("%+v %v\n", a, err) // price silently 0

    var b Item
    dec := json.NewDecoder(strings.NewReader(in))
    dec.DisallowUnknownFields()
    fmt.Println("err:", dec.Decode(&b))
}

Output:

{Name:Vada Pav Price:0} <nil>
err: json: unknown field "prise"
  • Unmarshal ignores keys it doesn't know, so a typo like prise leaves Price at 0 with no error.
  • A json.Decoder with DisallowUnknownFields() reports json: unknown field "prise". Turn it on for input you control (config files, your own API). Leave it off when a newer sender may add fields you don't need yet.

map[string]any: when you don't know the shape

package main

import (
    "encoding/json"
    "fmt"
)

func main() {
    in := `{"name":"Thali","price":120,"veg":true,"items":["dal","rice"]}`
    var m map[string]any // any JSON object
    json.Unmarshal([]byte(in), &m)
    for _, k := range []string{"name", "price", "veg", "items"} {
        fmt.Printf("%-5s %-14T %v\n", k, m[k], m[k])
    }
    price := m["price"].(float64) // numbers become float64
    fmt.Println("price + 10 =", price+10)
    n := m["price"].(int)
    fmt.Println(n)
}

Output:

name  string         Thali
price float64        120
veg   bool           true
items []interface {} [dal rice]
price + 10 = 130
panic: interface conversion: interface {} is float64, not int

goroutine 1 [running]:
main.main()
    main.go:17 +0x348
exit status 2
  • any is the empty interface (episode 21): it can hold any value. Decoding into map[string]any accepts any JSON object.
  • JSON numbers become float64, arrays []interface {}, objects map[string]interface {}.
  • The type assertion .(int) panics (episode 22): interface conversion: interface {} is float64, not int. Use .(float64), the comma-ok form, or better, a struct.
  • dec.UseNumber() makes numbers json.Number (text) instead of float64, which keeps large integers exact.

json.RawMessage: decode in two steps

package main

import (
    "encoding/json"
    "fmt"
)

type Event struct {
    Type string          `json:"type"`
    Data json.RawMessage `json:"data"` // kept as raw bytes
}
type Order struct {
    Item string `json:"item"`
    Qty  int    `json:"qty"`
}
type Review struct {
    Stars int `json:"stars"`
}

func main() {
    in := `[{"type":"order","data":{"item":"chai","qty":2}},
           {"type":"review","data":{"stars":5}}]`
    var events []Event
    json.Unmarshal([]byte(in), &events)
    for _, e := range events {
        switch e.Type { // decide, THEN decode the rest
        case "order":
            var o Order
            json.Unmarshal(e.Data, &o)
            fmt.Printf("order %+v\n", o)
        case "review":
            var r Review
            json.Unmarshal(e.Data, &r)
            fmt.Printf("review %+v (raw %s)\n", r, e.Data)
        }
    }
}

Output:

order {Item:chai Qty:2}
review {Stars:5} (raw {"stars":5})
  • json.RawMessage is a []byte that holds a piece of JSON undecoded.
  • Decode the outer part first, look at the type, then decode Data into the right struct with a switch (episode 7, 22).

Streaming: json.Decoder and json.Encoder

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "os"
)

type Order struct {
    Item string `json:"item"`
    Qty  int    `json:"qty"`
}

func main() {
    dec := json.NewDecoder(os.Stdin) // reads JSON values one by one
    enc := json.NewEncoder(os.Stdout)
    total := 0
    for {
        var o Order
        err := dec.Decode(&o)
        if err == io.EOF {
            break
        }
        if err != nil {
            fmt.Println("bad order:", err)
            return
        }
        total += o.Qty
        enc.Encode(map[string]int{o.Item: o.Qty}) // one line each
    }
    fmt.Println("cups:", total)
}

Program input (stdin):

{"item":"chai","qty":2}
{"item":"coffee","qty":1}
{"item":"chai","qty":3}

Output:

{"chai":2}
{"coffee":1}
{"chai":3}
cups: 6
  • Streaming means handling data piece by piece as it arrives, instead of loading it all first.
  • json.NewDecoder(r io.Reader): each Decode(&v) reads the next JSON value; at the end it returns io.EOF (episode 33). Memory stays small however many values arrive.
  • json.NewEncoder(w io.Writer): each Encode(v) writes one value followed by a newline. enc.SetIndent("", " ") pretty-prints.
  • One JSON value per line is a common log and data format, often called NDJSON or JSON Lines.
  • Use Marshal/Unmarshal for one value already in memory; Encoder/Decoder for files, standard input and network connections (episode 33's io.Reader and io.Writer).

Your own format: MarshalJSON and UnmarshalJSON

Money must be exact (float64 can't store 0.1 exactly), so Raju stores prices as whole paise: a defined type (episode 19).

package main

import (
    "encoding/json"
    "fmt"
)

// Paise holds money exactly, as whole paise (100 = Rs 1).
type Paise int

// MarshalJSON makes Paise a json.Marshaler.
func (p Paise) MarshalJSON() ([]byte, error) {
    return json.Marshal(fmt.Sprintf("Rs %d.%02d", p/100, p%100))
}

type Item struct {
    Name  string `json:"name"`
    Price Paise  `json:"price"`
}

func main() {
    data, err := json.Marshal(Item{"Cutting Chai", 1250})
    fmt.Println(string(data), err)
}

Output:

{"name":"Cutting Chai","price":"Rs 12.50"} <nil>

And the way back:

package main

import (
    "encoding/json"
    "fmt"
)

type Paise int

// UnmarshalJSON reads "Rs 40.50" back into 4050 paise.
func (p *Paise) UnmarshalJSON(b []byte) error {
    var rs, ps int
    if _, err := fmt.Sscanf(string(b), `"Rs %d.%d"`, &rs, &ps); err != nil {
        return fmt.Errorf("bad price %s: %w", b, err)
    }
    *p = Paise(rs*100 + ps)
    return nil
}

type Item struct {
    Name  string `json:"name"`
    Price Paise  `json:"price"`
}

func main() {
    var it Item
    err := json.Unmarshal([]byte(`{"name":"Lassi","price":"Rs 40.50"}`), &it)
    fmt.Println(it.Price, err)
    err = json.Unmarshal([]byte(`{"name":"Lassi","price":"forty"}`), &it)
    fmt.Println("err:", err)
}

Output:

4050 <nil>
err: bad price "forty": input does not match format
  • If a type has MarshalJSON() ([]byte, error), it satisfies the json.Marshaler interface and json.Marshal calls it.
  • UnmarshalJSON([]byte) error (the json.Unmarshaler interface) has a pointer receiver (episode 20) because it changes the value.
  • fmt.Sscanf scans a string using a format, like Printf in reverse; its error input does not match format is wrapped with %w (episode 25).
  • The same idea exists for plain text: encoding.TextMarshaler (MarshalText), used by JSON for map keys and by many other packages.

CSV with encoding/csv

CSV (comma-separated values) is a text table: one record per line, fields separated by commas. Spreadsheets open and save it. The rules: a field that contains a comma, a quote or a newline is put in double quotes, and a quote inside it is doubled ("").

package main

import (
    "fmt"
    "strings"
)

func main() {
    line := `"Chai, Masala",10`  // a name with a comma inside
    fields := strings.Split(line, ",")
    fmt.Printf("%d fields: %q\n", len(fields), fields)
}

Output:

3 fields: ["\"Chai" " Masala\"" "10"]

strings.Split knows none of those rules. encoding/csv does:

package main

import (
    "encoding/csv"
    "fmt"
    "os"
    "strings"
)

func main() {
    w := csv.NewWriter(os.Stdout)
    w.Write([]string{"name", "price"})
    w.Write([]string{"Chai, Masala", "10"}) // comma: quoted for us
    w.Write([]string{`Raju's "Special"`, "15"})
    w.Flush()

    in := "name,price\n\"Chai, Masala\",10\nSamosa,15\n"
    rows, err := csv.NewReader(strings.NewReader(in)).ReadAll()
    for _, r := range rows {
        fmt.Printf("%d fields: %q\n", len(r), r)
    }
    fmt.Println("err:", err)

    _, err = csv.NewReader(strings.NewReader("a,b\nc\n")).ReadAll()
    fmt.Println("err:", err)
}

Output:

name,price
"Chai, Masala",10
"Raju's ""Special""",15
2 fields: ["name" "price"]
2 fields: ["Chai, Masala" "10"]
2 fields: ["Samosa" "15"]
err: <nil>
err: record on line 2: wrong number of fields
  • csv.NewWriter(w): Write(record) per row; it's buffered (episode 33), so call Flush and check w.Error().
  • csv.NewReader(r): Read() returns one record at a time (until io.EOF); ReadAll() returns all.
  • Every record must have the same number of fields as the first one: record on line 2: wrong number of fields. Set r.FieldsPerRecord = -1 to allow any number.
  • r.Comma = ';' reads semicolon-separated files; r.Comment = '#' skips comment lines.

XML with encoding/xml

XML (Extensible Markup Language) is an older text format of nested elements in angle brackets, with attributes inside the opening tag.

package main

import (
    "encoding/xml"
    "fmt"
)

type Item struct {
    XMLName xml.Name `xml:"item"`
    Name    string   `xml:"name"`
    Price   int      `xml:"price,attr"`
}

func main() {
    data, _ := xml.MarshalIndent(Item{Name: "Chai", Price: 10}, "", "  ")
    fmt.Println(string(data))

    var it Item
    err := xml.Unmarshal([]byte(`<item price="15"><name>Samosa</name></item>`), &it)
    fmt.Println(it.Name, it.Price, err)
}

Output:

<item price="10">
  <name>Chai</name>
</item>
Samosa 15 <nil>
  • Same ideas as JSON: struct tags, xml.Marshal, xml.MarshalIndent, xml.Unmarshal, xml.NewDecoder.
  • XMLName xml.Name with tag xml:"item" names the element; ,attr makes a field an attribute; ,chardata the text inside.

Raw bytes as text: hex and base64

JSON, CSV and XML carry text. A photo, a key or a hash is raw bytes, which may not be valid text. Two encodings turn bytes into safe characters:

package main

import (
    "encoding/base64"
    "encoding/hex"
    "fmt"
)

func main() {
    raw := []byte{0xFF, 0x00, 'h', 'i', 0xFE} // bytes, not text
    fmt.Printf("%q\n", raw)

    fmt.Println("hex:   ", hex.EncodeToString(raw))
    fmt.Println("base64:", base64.StdEncoding.EncodeToString(raw))
    fmt.Println("url:   ", base64.URLEncoding.EncodeToString(raw))

    back, err := base64.StdEncoding.DecodeString("/wBoaf4=")
    fmt.Println(back, err)
    _, err = hex.DecodeString("ff0g")
    fmt.Println("err:", err)
}

Output:

"\xff\x00hi\xfe"
hex:    ff006869fe
base64: /wBoaf4=
url:    _wBoaf4=
[255 0 104 105 254] <nil>
err: encoding/hex: invalid byte: U+0067 'g'
Encoding How Size Used for
hex (hexadecimal) each byte → two digits 0-9a-f (episode 18) 2× hashes, colours, debugging
base64 every 3 bytes → 4 characters A-Z a-z 0-9 + /, padded with = about 1.33× email attachments, images in JSON, tokens
base64 URL encoding - and _ instead of + and / about 1.33× data inside web addresses

base64.RawURLEncoding leaves out the = padding. A []byte field in a struct is encoded as base64 by json.Marshal automatically.

encoding/gob: Go to Go

package main

import (
    "bytes"
    "encoding/gob"
    "encoding/json"
    "fmt"
)

type Item struct {
    Name  string
    Price int
}

func main() {
    menu := []Item{{"Chai", 10}, {"Samosa", 15}, {"Poha", 20}}

    var buf bytes.Buffer
    err := gob.NewEncoder(&buf).Encode(menu) // Go-only binary format
    fmt.Println("gob bytes:", buf.Len(), err)
    j, _ := json.Marshal(menu)
    fmt.Println("json bytes:", len(j))

    var back []Item
    err = gob.NewDecoder(&buf).Decode(&back)
    fmt.Println(back, err)
}

Output:

gob bytes: 85 <nil>
json bytes: 84
[{Chai 10} {Samosa 15} {Poha 20}] <nil>
  • gob is Go's own binary format: no tags, and it handles Go types directly. The stream starts with a description of the type, then the values.
  • For three items it's no smaller than JSON; it wins on long streams of many values of the same type.
  • Only Go can read it. For anything another language or a human must read, use JSON. For fast cross-language binary formats, people use Protocol Buffers (a separate library).

Which format when?

Need Use
web APIs, config, talking to other languages JSON (encoding/json)
spreadsheets, data exports CSV (encoding/csv)
older systems and document formats XML (encoding/xml)
bytes inside text (JSON, URLs, email) base64; hex for hashes
Go program to Go program gob

Mini project: menu converter (CSV → JSON → back)

  1. Read menu.csv with csv.NewReader, skipping the header row.
  2. Make a []Item with tags, prices as Paise.
  3. Write it with MarshalIndent.
  4. Read it back with DisallowUnknownFields and check it matches.
Solution
package main

import (
    "bytes"
    "encoding/csv"
    "encoding/json"
    "fmt"
    "reflect"
    "strconv"
    "strings"
)

type Paise int

func (p Paise) MarshalJSON() ([]byte, error) {
    return json.Marshal(fmt.Sprintf("Rs %d.%02d", p/100, p%100))
}

func (p *Paise) UnmarshalJSON(b []byte) error {
    var s string
    if err := json.Unmarshal(b, &s); err != nil {
        return err
    }
    var rs, ps int
    if _, err := fmt.Sscanf(s, "Rs %d.%d", &rs, &ps); err != nil {
        return fmt.Errorf("bad price %s: %w", s, err)
    }
    *p = Paise(rs*100 + ps)
    return nil
}

type Item struct {
    Name  string `json:"name"`
    Price Paise  `json:"price"`
    Veg   bool   `json:"veg"`
}

func main() {
    in := "name,paise,veg\n\"Chai, Masala\",1000,true\nSamosa,1550,true\nChicken Roll,8000,false\n"
    rows, err := csv.NewReader(strings.NewReader(in)).ReadAll()
    if err != nil {
        fmt.Println("csv:", err)
        return
    }
    var menu []Item
    for _, r := range rows[1:] { // skip the header
        p, err1 := strconv.Atoi(r[1])
        veg, err2 := strconv.ParseBool(r[2])
        if err1 != nil || err2 != nil {
            fmt.Println("bad row:", r)
            continue
        }
        menu = append(menu, Item{r[0], Paise(p), veg})
    }

    out, _ := json.MarshalIndent(menu, "", "  ")
    fmt.Println(string(out))

    var back []Item
    dec := json.NewDecoder(bytes.NewReader(out))
    dec.DisallowUnknownFields()
    err = dec.Decode(&back)
    fmt.Println("decode:", err, "same:", reflect.DeepEqual(menu, back))
}

Output:

[
  {
    "name": "Chai, Masala",
    "price": "Rs 10.00",
    "veg": true
  },
  {
    "name": "Samosa",
    "price": "Rs 15.50",
    "veg": true
  },
  {
    "name": "Chicken Roll",
    "price": "Rs 80.00",
    "veg": false
  }
]
decode: <nil> same: true

Common mistakes (and the real messages)

Mistake Real message / result Why Fix
building JSON with Sprintf invalid JSON when a value has a " (json.Valid = false) no escaping json.Marshal
lowercase struct field field silently missing from the JSON unexported fields are invisible capitalise it, add a json tag
json.Unmarshal(data, it) json: Unmarshal(non-pointer main.Item) it must write into your variable json.Unmarshal(data, &it)
"price":"twenty" into an int json: cannot unmarshal string into Go struct field Item.price of type int types must match fix the input, or custom UnmarshalJSON
a typo in a JSON key the field stays 0, no error unknown keys are ignored Decoder.DisallowUnknownFields() → json: unknown field "prise"
m["price"].(int) on map[string]any panic: interface conversion: interface {} is float64, not int JSON numbers → float64 a struct, .(float64), or UseNumber
strings.Split on a CSV line 3 fields instead of 2 quoted commas encoding/csv
a short CSV record record on line 2: wrong number of fields fields per record must match fix the data or FieldsPerRecord = -1
forgetting csv.Writer.Flush no output the writer is buffered w.Flush(), then w.Error()
a bad hex digit encoding/hex: invalid byte: U+0067 'g' only 0-9a-f validate the input
float64 for money rounding errors binary fractions whole paise in an integer type

Practice

  1. (Homework) Build the CSV → JSON converter (solution: the mini project).
  2. (Homework) Send yourself a typo field and catch it with DisallowUnknownFields (shown above).
  3. (Homework) Add the custom Paise type to your project (shown above).
  4. (Homework) Base64-encode a small file: os.ReadFile it and print base64.StdEncoding.EncodeToString(data).
  5. Decode [{"item":"chai","qty":2},{"item":"vada pav","qty":1}] into a slice and print the total quantity.
Solution 5
package main

import (
    "encoding/json"
    "fmt"
)

type Order struct {
    Item string `json:"item"`
    Qty  int    `json:"qty"`
}

func main() {
    in := `[{"item":"chai","qty":2},{"item":"vada pav","qty":1}]`
    var orders []Order
    if err := json.Unmarshal([]byte(in), &orders); err != nil {
        fmt.Println("error:", err)
        return
    }
    total := 0
    for _, o := range orders {
        total += o.Qty
    }
    fmt.Println(len(orders), "orders, total qty", total)
}

Output:

2 orders, total qty 3
  1. Make a struct with a *int field Discount and tag json:"discount,omitempty". What's the difference between a nil pointer and a pointer to 0 when you marshal?
Solution 6
package main

import (
    "encoding/json"
    "fmt"
)

type Item struct {
    Name     string `json:"name"`
    Discount *int   `json:"discount,omitempty"`
}

func main() {
    zero := 0
    a, _ := json.Marshal(Item{Name: "Chai"})
    b, _ := json.Marshal(Item{Name: "Chai", Discount: &zero})
    fmt.Println(string(a))
    fmt.Println(string(b))
}

Output:

{"name":"Chai"}
{"name":"Chai","discount":0}

A nil pointer is empty, so it's omitted; a pointer to 0 is not nil, so "discount":0 is sent. Pointers let you tell "not given" from "zero".

Quick quiz

1. What does this print? (From the video.)

type Item struct {
    Name  string `json:"name"`
    Price int    `json:"price,omitempty"`
    qty   int
}
b, _ := json.Marshal(Item{"Chai", 0, 2})
fmt.Println(string(b))

A. {"name":"Chai","price":0,"qty":2} · B. {"name":"Chai"} · C. {"name":"Chai","price":0} · D. {"Name":"Chai","qty":2}

Answer

B. Price is 0, an empty value, so omitempty drops it; qty is unexported, so it's never encoded. Real output: {"name":"Chai"}.

2. You decode {"price": 120} into map[string]any. What is the dynamic type of m["price"]?

Answer

float64.

3. Which reads a never-ending stream of JSON objects from a network connection: json.Unmarshal or json.NewDecoder(conn).Decode?

Answer

json.NewDecoder(conn).Decode, called in a loop: it reads one value at a time from an io.Reader. Unmarshal needs all the bytes in memory first.

4. How many base64 characters encode 6 bytes?

Answer

8: every 3 bytes become 4 characters, and 6 is a multiple of 3, so no padding.

Interview questions

  1. Why are unexported fields not encoded by encoding/json? The package uses reflection from outside your package, and Go's visibility rules don't let it read unexported fields.
  2. What does omitempty consider empty? false, 0, nil pointers and interfaces, and zero-length arrays, slices, maps and strings. Not zero structs.
  3. How do you reject unknown fields? Use a json.Decoder and call DisallowUnknownFields() before Decode.
  4. What type do JSON numbers get in interface{}? float64, unless you use Decoder.UseNumber(), which gives json.Number.
  5. When would you use json.RawMessage? To delay decoding part of a message until you know its type (polymorphic messages), or to pass JSON through unchanged.
  6. Marshal vs Encoder? Marshal returns bytes for one value in memory; Encoder writes values to an io.Writer as a stream. Same for Unmarshal vs Decoder.
  7. How do you customise how a type is encoded? Implement json.Marshaler (MarshalJSON) and json.Unmarshaler (UnmarshalJSON, pointer receiver), or encoding.TextMarshaler for plain text.
  8. Why not split CSV with strings.Split? CSV has quoting rules: commas, quotes and newlines inside quoted fields. encoding/csv follows RFC 4180.
  9. base64 vs hex? Both turn bytes into text. Hex doubles the size and is easy to read; base64 is about 33% bigger and more compact.
  10. When is gob a good choice? Between Go programs, for streams of Go values (a typical use, per its docs, is remote procedure calls with net/rpc). Not for other languages.

Glossary

Term Meaning Example
Encoding / serialization turning a value into bytes in an agreed format Go struct → JSON
Decoding / deserialization turning those bytes back into a value JSON → Go struct
Format the agreed rules for how data is written JSON, CSV, XML
Display format how fmt prints values for people {Masala Chai 10 [hot sweet]}
JSON JavaScript Object Notation: a text format of objects, arrays, strings, numbers, booleans, null {"name":"Chai"}
JSON object {} with "key": value pairs {"price": 10}
JSON array an ordered list in [] ["hot","sweet"]
null JSON's "no value" (Go nil) "offer": null
encoding/json Go's JSON package import "encoding/json"
Marshal encode a value (Go's name) json.Marshal(v)
Unmarshal decode into a value (Go's name) json.Unmarshal(b, &v)
json.Valid reports whether bytes are valid JSON json.Valid(b)
Escaping writing special characters safely, like \" for a quote "Raju's \"Special\""
Reflection a program inspecting its own types and fields at run time how json finds fields
Exported field a field starting with a capital letter, visible outside the package Name
Struct tag a backquoted string after a field's type, read by packages `json:"name"`
omitempty tag option: leave the field out when it's empty json:"tags,omitempty"
json:"-" never encode or decode the field Secret string
MarshalIndent Marshal with line breaks and indentation json.MarshalIndent(v, "", " ")
Pointer argument &v, so the function can fill your variable json.Unmarshal(b, &it)
Unknown field a JSON key with no matching struct field; ignored by default "prise"
DisallowUnknownFields make a Decoder reject unknown fields dec.DisallowUnknownFields()
map[string]any a map that can hold any JSON object var m map[string]any
any the empty interface: holds any value var x any = 5
Type assertion x.(T): get the concrete value from an interface (episode 22) m["price"].(float64)
json.RawMessage undecoded JSON bytes, for decoding later Data json.RawMessage
Streaming handling data piece by piece as it arrives reading orders all day
json.Decoder reads JSON values one by one from an io.Reader json.NewDecoder(os.Stdin)
json.Encoder writes JSON values to an io.Writer json.NewEncoder(os.Stdout)
NDJSON / JSON Lines one JSON value per line {"item":"chai","qty":2}
json.Marshaler interface with MarshalJSON() ([]byte, error) func (p Paise) MarshalJSON()
json.Unmarshaler interface with UnmarshalJSON([]byte) error func (p *Paise) UnmarshalJSON(b []byte)
Paise 1/100 of a rupee; stored as an integer for exact money 1250 = Rs 12.50
fmt.Sscanf scan a string using a format fmt.Sscanf(s, "Rs %d.%d", &rs, &ps)
CSV comma-separated values: a text table name,price
Record / field a CSV row / one value in it Samosa,15
Quoting (CSV) wrapping a field with commas or quotes in "...", doubling inner quotes "Chai, Masala"
encoding/csv Go's CSV reader and writer (RFC 4180) csv.NewReader(r)
RFC "Request for Comments": a published internet standard RFC 4180 (CSV)
XML Extensible Markup Language: nested elements in angle brackets <item price="10">
Element / attribute an XML tag pair / a name="value" inside a tag <name>, price="10"
Hex each byte as two hexadecimal digits ff00
Base64 3 bytes → 4 safe characters /wBoaf4=
Padding (=) filler at the end of base64 when the length isn't a multiple of 3 ...=
URL encoding (base64) base64 with - and _ instead of + and / _wBoaf4=
gob Go's own binary encoding for Go-to-Go data gob.NewEncoder(&buf)
Binary format bytes not meant to be read as text gob, images

Summary

  • Programs share data through an agreed format; Go's display format and hand-built JSON break.
  • json.Marshal encodes exported fields and escapes for you; struct tags rename (json:"name"), omit empty values (omitempty) or hide fields ("-"); MarshalIndent is for humans.
  • json.Unmarshal needs a pointer and matching types, with real error messages.
  • Unknown keys are silently ignored: use a Decoder with DisallowUnknownFields.
  • In map[string]any, numbers are float64. json.RawMessage decodes in two steps.
  • json.Decoder and json.Encoder stream values over any io.Reader / io.Writer.
  • MarshalJSON / UnmarshalJSON give a type its own JSON form.
  • CSV needs encoding/csv (quoting rules, same field count); XML works like JSON; base64 and hex carry bytes as text; gob is Go-to-Go only.

Next episode: Working with time: time.Time, time.Duration, Add, Sub, Since, formatting with the reference layout 2006-01-02 15:04:05, time.Parse, time zones, the monotonic clock and Unix timestamps.