These are the notes for Go episode 35. Raju's menu has to travel to a delivery app written in another language. fmt.Println prints Go's own display format, and JSON built by hand with Sprintf breaks on the first quote. So you invent encoding: an agreed format both sides understand. You use encoding/json: json.Marshal and json.MarshalIndent, struct tags (json:"name", omitempty, "-"), json.Unmarshal and its real errors, the silent typo bug fixed with DisallowUnknownFields, map[string]any where numbers become float64, json.RawMessage, streaming with json.Decoder and json.Encoder, and your own format with MarshalJSON and UnmarshalJSON. Then encoding/csv for spreadsheets, encoding/xml, base64 and hex for raw bytes, and encoding/gob.
Every program here runs as-is in the SyntaxPilot compiler (language: Go) with go run main.go; text typed into the program input box is sent to standard input. All outputs and errors are from real runs (Go 1.24.7, linux/amd64).
Facts and sources
From the standard library documentation shipped with Go 1.24.7 (go doc):
json.Unmarshalstores JSON into an interface value as:boolfor JSON booleans,float64for JSON numbers,stringfor strings,[]interface{}for arrays,map[string]interface{}for objects, andnilfornull. Unknown object keys are "ignored (see Decoder.DisallowUnknownFields for an alternative)".json.Marshal: "The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any array, slice, map, or string of length zero." A field with tag"-"is always omitted. Only exported fields are encoded.encoding/csv"supports the format described in RFC 4180, except that Writer uses LF instead of CRLF as newline character by default."encoding/base64"implements base64 encoding as specified by RFC 4648."encoding/gob"manages streams of gobs - binary values exchanged between an Encoder (transmitter) and a Decoder (receiver)."json.RawMessage"is a raw encoded JSON value ... can be used to delay JSON decoding or precompute a JSON encoding."
The problem: a format only Go understands
package main
import (
"encoding/json"
"fmt"
)
type Item struct {
Name string
Price int
Tags []string
}
func main() {
chai := Item{"Masala Chai", 10, []string{"hot", "sweet"}}
fmt.Println(chai) // only Go understands this
// build the text by hand for the delivery app?
s := fmt.Sprintf(`{"name":"%s","price":%d}`, `Raju's "Special" Chai`, 15)
fmt.Println(s)
fmt.Println("valid JSON?", json.Valid([]byte(s)))
}
Output:
{Masala Chai 10 [hot sweet]}
{"name":"Raju's "Special" Chai","price":15}
valid JSON? false
fmt.Printlnprints Go's display format, meant for people reading Go output. Another program can't tell where the name ends or what10means.- Building JSON by hand fails as soon as a value contains a
": the quote ends the JSON string early.json.Validchecks whether bytes are correct JSON, and it saysfalse.
Encoding: packing for a courier
| Sending a parcel | In Go | Meaning |
|---|---|---|
| Pack in a standard box | encoding (serialization) | turn a value into an agreed text or byte format |
| Label each item | struct tags | names and options attached to fields |
| Unpack at the other end | decoding (deserialization) | turn the bytes back into a value |
- Encoding / serialization: turning a value in memory into bytes (text or binary) that can be saved or sent. Decoding / deserialization is the reverse. In
encoding/json, encoding is called Marshal and decoding Unmarshal. - JSON (JavaScript Object Notation) is a text format: objects in
{}with"key": valuepairs, arrays in[], strings in double quotes, numbers,true,falseandnull. It's understood by practically every programming language.
{"name": "Masala Chai", "price": 10, "veg": true, "tags": ["hot", "sweet"], "offer": null}
json.Marshal: Go value to JSON
package main
import (
"encoding/json"
"fmt"
)
type Item struct {
Name string
Price int
Tags []string
stock int // lowercase: unexported
}
func main() {
chai := Item{`Raju's "Special" Chai`, 15, []string{"hot"}, 40}
data, err := json.Marshal(chai) // Go value → JSON bytes
fmt.Println(string(data), err)
fmt.Println("valid JSON?", json.Valid(data))
}
Output:
{"Name":"Raju's \"Special\" Chai","Price":15,"Tags":["hot"]} <nil>
valid JSON? true
json.Marshal(v)returns([]byte, error). It escapes special characters for you (\"for a quote).- A slice becomes a JSON array, a struct an object, a map an object,
nilbecomesnull. - It reads the struct's fields at run time using reflection (a program looking at its own types and fields while it runs; episode 39). It can only see exported fields (starting with a capital letter, episode 19).
stockvanished without any error.
Struct tags and omitempty
A struct tag is a string literal in backquotes after a field's type. Packages read it with reflection. encoding/json reads the json:"..." key.
package main
import (
"encoding/json"
"fmt"
)
type Item struct {
Name string `json:"name"`
Price int `json:"price"`
Tags []string `json:"tags,omitempty"` // left out if empty
Secret string `json:"-"` // never sent
}
func main() {
menu := []Item{
{Name: "Masala Chai", Price: 10, Tags: []string{"hot"}, Secret: "ginger"},
{Name: "Samosa", Price: 15},
}
data, _ := json.Marshal(menu)
fmt.Println(string(data))
pretty, _ := json.MarshalIndent(menu[1], "", " ")
fmt.Println(string(pretty))
}
Output:
[{"name":"Masala Chai","price":10,"tags":["hot"]},{"name":"Samosa","price":15}]
{
"name": "Samosa",
"price": 15
}
| Tag | Meaning |
|---|---|
json:"name" |
use name as the JSON key |
json:"tags,omitempty" |
leave the field out when it is empty |
json:"-" |
never encode or decode this field |
json:",omitempty" |
keep the Go name, but omit when empty |
json:"price,string" |
encode the number as a JSON string ("15") |
- Empty, for
omitempty, meansfalse,0, anilpointer or interface, and any array, slice, map or string of length zero. A zero struct is not "empty" (use a pointer, or the neweromitzerooption in Go 1.24). json.MarshalIndent(v, prefix, indent)adds line breaks and indentation for humans.
json.Unmarshal: JSON to Go value
package main
import (
"encoding/json"
"fmt"
)
type Item struct {
Name string `json:"name"`
Price int `json:"price"`
}
func main() {
var it Item
err := json.Unmarshal([]byte(`{"name":"Poha","price":20}`), &it)
fmt.Printf("%+v %v\n", it, err)
err = json.Unmarshal([]byte(`{"name":"Poha","price":"twenty"}`), &it)
fmt.Println("err:", err)
err = json.Unmarshal([]byte(`{"name":"Poha"}`), it) // forgot &
fmt.Println("err:", err)
}
Output:
{Name:Poha Price:20} <nil>
err: json: cannot unmarshal string into Go struct field Item.price of type int
err: json: Unmarshal(non-pointer main.Item)
- Pass a pointer (
&it, episode 11) soUnmarshalcan write into your variable. Without one:json: Unmarshal(non-pointer main.Item). - Types must match: a JSON string can't go into an
intfield:json: cannot unmarshal string into Go struct field Item.price of type int. - Keys are matched to field names or tags case-insensitively (
"NAME"also fillsName). - Fields missing from the JSON keep their current value (the zero value for a fresh variable).
Unknown fields: the silent typo
package main
import (
"encoding/json"
"fmt"
"strings"
)
type Item struct {
Name string `json:"name"`
Price int `json:"price"`
}
func main() {
in := `{"name":"Vada Pav","prise":25}` // typo: prise
var a Item
err := json.Unmarshal([]byte(in), &a)
fmt.Printf("%+v %v\n", a, err) // price silently 0
var b Item
dec := json.NewDecoder(strings.NewReader(in))
dec.DisallowUnknownFields()
fmt.Println("err:", dec.Decode(&b))
}
Output:
{Name:Vada Pav Price:0} <nil>
err: json: unknown field "prise"
Unmarshalignores keys it doesn't know, so a typo likepriseleavesPriceat 0 with no error.- A
json.DecoderwithDisallowUnknownFields()reportsjson: unknown field "prise". Turn it on for input you control (config files, your own API). Leave it off when a newer sender may add fields you don't need yet.
map[string]any: when you don't know the shape
package main
import (
"encoding/json"
"fmt"
)
func main() {
in := `{"name":"Thali","price":120,"veg":true,"items":["dal","rice"]}`
var m map[string]any // any JSON object
json.Unmarshal([]byte(in), &m)
for _, k := range []string{"name", "price", "veg", "items"} {
fmt.Printf("%-5s %-14T %v\n", k, m[k], m[k])
}
price := m["price"].(float64) // numbers become float64
fmt.Println("price + 10 =", price+10)
n := m["price"].(int)
fmt.Println(n)
}
Output:
name string Thali
price float64 120
veg bool true
items []interface {} [dal rice]
price + 10 = 130
panic: interface conversion: interface {} is float64, not int
goroutine 1 [running]:
main.main()
main.go:17 +0x348
exit status 2
anyis the empty interface (episode 21): it can hold any value. Decoding intomap[string]anyaccepts any JSON object.- JSON numbers become
float64, arrays[]interface {}, objectsmap[string]interface {}. - The type assertion
.(int)panics (episode 22):interface conversion: interface {} is float64, not int. Use.(float64), the comma-ok form, or better, a struct. dec.UseNumber()makes numbersjson.Number(text) instead offloat64, which keeps large integers exact.
json.RawMessage: decode in two steps
package main
import (
"encoding/json"
"fmt"
)
type Event struct {
Type string `json:"type"`
Data json.RawMessage `json:"data"` // kept as raw bytes
}
type Order struct {
Item string `json:"item"`
Qty int `json:"qty"`
}
type Review struct {
Stars int `json:"stars"`
}
func main() {
in := `[{"type":"order","data":{"item":"chai","qty":2}},
{"type":"review","data":{"stars":5}}]`
var events []Event
json.Unmarshal([]byte(in), &events)
for _, e := range events {
switch e.Type { // decide, THEN decode the rest
case "order":
var o Order
json.Unmarshal(e.Data, &o)
fmt.Printf("order %+v\n", o)
case "review":
var r Review
json.Unmarshal(e.Data, &r)
fmt.Printf("review %+v (raw %s)\n", r, e.Data)
}
}
}
Output:
order {Item:chai Qty:2}
review {Stars:5} (raw {"stars":5})
json.RawMessageis a[]bytethat holds a piece of JSON undecoded.- Decode the outer part first, look at the
type, then decodeDatainto the right struct with aswitch(episode 7, 22).
Streaming: json.Decoder and json.Encoder
package main
import (
"encoding/json"
"fmt"
"io"
"os"
)
type Order struct {
Item string `json:"item"`
Qty int `json:"qty"`
}
func main() {
dec := json.NewDecoder(os.Stdin) // reads JSON values one by one
enc := json.NewEncoder(os.Stdout)
total := 0
for {
var o Order
err := dec.Decode(&o)
if err == io.EOF {
break
}
if err != nil {
fmt.Println("bad order:", err)
return
}
total += o.Qty
enc.Encode(map[string]int{o.Item: o.Qty}) // one line each
}
fmt.Println("cups:", total)
}
Program input (stdin):
{"item":"chai","qty":2}
{"item":"coffee","qty":1}
{"item":"chai","qty":3}
Output:
{"chai":2}
{"coffee":1}
{"chai":3}
cups: 6
- Streaming means handling data piece by piece as it arrives, instead of loading it all first.
json.NewDecoder(r io.Reader): eachDecode(&v)reads the next JSON value; at the end it returnsio.EOF(episode 33). Memory stays small however many values arrive.json.NewEncoder(w io.Writer): eachEncode(v)writes one value followed by a newline.enc.SetIndent("", " ")pretty-prints.- One JSON value per line is a common log and data format, often called NDJSON or JSON Lines.
- Use
Marshal/Unmarshalfor one value already in memory;Encoder/Decoderfor files, standard input and network connections (episode 33'sio.Readerandio.Writer).
Your own format: MarshalJSON and UnmarshalJSON
Money must be exact (float64 can't store 0.1 exactly), so Raju stores prices as whole paise: a defined type (episode 19).
package main
import (
"encoding/json"
"fmt"
)
// Paise holds money exactly, as whole paise (100 = Rs 1).
type Paise int
// MarshalJSON makes Paise a json.Marshaler.
func (p Paise) MarshalJSON() ([]byte, error) {
return json.Marshal(fmt.Sprintf("Rs %d.%02d", p/100, p%100))
}
type Item struct {
Name string `json:"name"`
Price Paise `json:"price"`
}
func main() {
data, err := json.Marshal(Item{"Cutting Chai", 1250})
fmt.Println(string(data), err)
}
Output:
{"name":"Cutting Chai","price":"Rs 12.50"} <nil>
And the way back:
package main
import (
"encoding/json"
"fmt"
)
type Paise int
// UnmarshalJSON reads "Rs 40.50" back into 4050 paise.
func (p *Paise) UnmarshalJSON(b []byte) error {
var rs, ps int
if _, err := fmt.Sscanf(string(b), `"Rs %d.%d"`, &rs, &ps); err != nil {
return fmt.Errorf("bad price %s: %w", b, err)
}
*p = Paise(rs*100 + ps)
return nil
}
type Item struct {
Name string `json:"name"`
Price Paise `json:"price"`
}
func main() {
var it Item
err := json.Unmarshal([]byte(`{"name":"Lassi","price":"Rs 40.50"}`), &it)
fmt.Println(it.Price, err)
err = json.Unmarshal([]byte(`{"name":"Lassi","price":"forty"}`), &it)
fmt.Println("err:", err)
}
Output:
4050 <nil>
err: bad price "forty": input does not match format
- If a type has
MarshalJSON() ([]byte, error), it satisfies thejson.Marshalerinterface andjson.Marshalcalls it. UnmarshalJSON([]byte) error(thejson.Unmarshalerinterface) has a pointer receiver (episode 20) because it changes the value.fmt.Sscanfscans a string using a format, likePrintfin reverse; its errorinput does not match formatis wrapped with%w(episode 25).- The same idea exists for plain text:
encoding.TextMarshaler(MarshalText), used by JSON for map keys and by many other packages.
CSV with encoding/csv
CSV (comma-separated values) is a text table: one record per line, fields separated by commas. Spreadsheets open and save it. The rules: a field that contains a comma, a quote or a newline is put in double quotes, and a quote inside it is doubled ("").
package main
import (
"fmt"
"strings"
)
func main() {
line := `"Chai, Masala",10` // a name with a comma inside
fields := strings.Split(line, ",")
fmt.Printf("%d fields: %q\n", len(fields), fields)
}
Output:
3 fields: ["\"Chai" " Masala\"" "10"]
strings.Split knows none of those rules. encoding/csv does:
package main
import (
"encoding/csv"
"fmt"
"os"
"strings"
)
func main() {
w := csv.NewWriter(os.Stdout)
w.Write([]string{"name", "price"})
w.Write([]string{"Chai, Masala", "10"}) // comma: quoted for us
w.Write([]string{`Raju's "Special"`, "15"})
w.Flush()
in := "name,price\n\"Chai, Masala\",10\nSamosa,15\n"
rows, err := csv.NewReader(strings.NewReader(in)).ReadAll()
for _, r := range rows {
fmt.Printf("%d fields: %q\n", len(r), r)
}
fmt.Println("err:", err)
_, err = csv.NewReader(strings.NewReader("a,b\nc\n")).ReadAll()
fmt.Println("err:", err)
}
Output:
name,price
"Chai, Masala",10
"Raju's ""Special""",15
2 fields: ["name" "price"]
2 fields: ["Chai, Masala" "10"]
2 fields: ["Samosa" "15"]
err: <nil>
err: record on line 2: wrong number of fields
csv.NewWriter(w):Write(record)per row; it's buffered (episode 33), so callFlushand checkw.Error().csv.NewReader(r):Read()returns one record at a time (untilio.EOF);ReadAll()returns all.- Every record must have the same number of fields as the first one:
record on line 2: wrong number of fields. Setr.FieldsPerRecord = -1to allow any number. r.Comma = ';'reads semicolon-separated files;r.Comment = '#'skips comment lines.
XML with encoding/xml
XML (Extensible Markup Language) is an older text format of nested elements in angle brackets, with attributes inside the opening tag.
package main
import (
"encoding/xml"
"fmt"
)
type Item struct {
XMLName xml.Name `xml:"item"`
Name string `xml:"name"`
Price int `xml:"price,attr"`
}
func main() {
data, _ := xml.MarshalIndent(Item{Name: "Chai", Price: 10}, "", " ")
fmt.Println(string(data))
var it Item
err := xml.Unmarshal([]byte(`<item price="15"><name>Samosa</name></item>`), &it)
fmt.Println(it.Name, it.Price, err)
}
Output:
<item price="10">
<name>Chai</name>
</item>
Samosa 15 <nil>
- Same ideas as JSON: struct tags,
xml.Marshal,xml.MarshalIndent,xml.Unmarshal,xml.NewDecoder. XMLName xml.Namewith tagxml:"item"names the element;,attrmakes a field an attribute;,chardatathe text inside.
Raw bytes as text: hex and base64
JSON, CSV and XML carry text. A photo, a key or a hash is raw bytes, which may not be valid text. Two encodings turn bytes into safe characters:
package main
import (
"encoding/base64"
"encoding/hex"
"fmt"
)
func main() {
raw := []byte{0xFF, 0x00, 'h', 'i', 0xFE} // bytes, not text
fmt.Printf("%q\n", raw)
fmt.Println("hex: ", hex.EncodeToString(raw))
fmt.Println("base64:", base64.StdEncoding.EncodeToString(raw))
fmt.Println("url: ", base64.URLEncoding.EncodeToString(raw))
back, err := base64.StdEncoding.DecodeString("/wBoaf4=")
fmt.Println(back, err)
_, err = hex.DecodeString("ff0g")
fmt.Println("err:", err)
}
Output:
"\xff\x00hi\xfe"
hex: ff006869fe
base64: /wBoaf4=
url: _wBoaf4=
[255 0 104 105 254] <nil>
err: encoding/hex: invalid byte: U+0067 'g'
| Encoding | How | Size | Used for |
|---|---|---|---|
| hex (hexadecimal) | each byte → two digits 0-9a-f (episode 18) |
2× | hashes, colours, debugging |
| base64 | every 3 bytes → 4 characters A-Z a-z 0-9 + /, padded with = |
about 1.33× | email attachments, images in JSON, tokens |
| base64 URL encoding | - and _ instead of + and / |
about 1.33× | data inside web addresses |
base64.RawURLEncoding leaves out the = padding. A []byte field in a struct is encoded as base64 by json.Marshal automatically.
encoding/gob: Go to Go
package main
import (
"bytes"
"encoding/gob"
"encoding/json"
"fmt"
)
type Item struct {
Name string
Price int
}
func main() {
menu := []Item{{"Chai", 10}, {"Samosa", 15}, {"Poha", 20}}
var buf bytes.Buffer
err := gob.NewEncoder(&buf).Encode(menu) // Go-only binary format
fmt.Println("gob bytes:", buf.Len(), err)
j, _ := json.Marshal(menu)
fmt.Println("json bytes:", len(j))
var back []Item
err = gob.NewDecoder(&buf).Decode(&back)
fmt.Println(back, err)
}
Output:
gob bytes: 85 <nil>
json bytes: 84
[{Chai 10} {Samosa 15} {Poha 20}] <nil>
- gob is Go's own binary format: no tags, and it handles Go types directly. The stream starts with a description of the type, then the values.
- For three items it's no smaller than JSON; it wins on long streams of many values of the same type.
- Only Go can read it. For anything another language or a human must read, use JSON. For fast cross-language binary formats, people use Protocol Buffers (a separate library).
Which format when?
| Need | Use |
|---|---|
| web APIs, config, talking to other languages | JSON (encoding/json) |
| spreadsheets, data exports | CSV (encoding/csv) |
| older systems and document formats | XML (encoding/xml) |
| bytes inside text (JSON, URLs, email) | base64; hex for hashes |
| Go program to Go program | gob |
Mini project: menu converter (CSV → JSON → back)
- Read
menu.csvwithcsv.NewReader, skipping the header row. - Make a
[]Itemwith tags, prices asPaise. - Write it with
MarshalIndent. - Read it back with
DisallowUnknownFieldsand check it matches.
Solution
package main
import (
"bytes"
"encoding/csv"
"encoding/json"
"fmt"
"reflect"
"strconv"
"strings"
)
type Paise int
func (p Paise) MarshalJSON() ([]byte, error) {
return json.Marshal(fmt.Sprintf("Rs %d.%02d", p/100, p%100))
}
func (p *Paise) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err != nil {
return err
}
var rs, ps int
if _, err := fmt.Sscanf(s, "Rs %d.%d", &rs, &ps); err != nil {
return fmt.Errorf("bad price %s: %w", s, err)
}
*p = Paise(rs*100 + ps)
return nil
}
type Item struct {
Name string `json:"name"`
Price Paise `json:"price"`
Veg bool `json:"veg"`
}
func main() {
in := "name,paise,veg\n\"Chai, Masala\",1000,true\nSamosa,1550,true\nChicken Roll,8000,false\n"
rows, err := csv.NewReader(strings.NewReader(in)).ReadAll()
if err != nil {
fmt.Println("csv:", err)
return
}
var menu []Item
for _, r := range rows[1:] { // skip the header
p, err1 := strconv.Atoi(r[1])
veg, err2 := strconv.ParseBool(r[2])
if err1 != nil || err2 != nil {
fmt.Println("bad row:", r)
continue
}
menu = append(menu, Item{r[0], Paise(p), veg})
}
out, _ := json.MarshalIndent(menu, "", " ")
fmt.Println(string(out))
var back []Item
dec := json.NewDecoder(bytes.NewReader(out))
dec.DisallowUnknownFields()
err = dec.Decode(&back)
fmt.Println("decode:", err, "same:", reflect.DeepEqual(menu, back))
}
Output:
[
{
"name": "Chai, Masala",
"price": "Rs 10.00",
"veg": true
},
{
"name": "Samosa",
"price": "Rs 15.50",
"veg": true
},
{
"name": "Chicken Roll",
"price": "Rs 80.00",
"veg": false
}
]
decode: <nil> same: true
Common mistakes (and the real messages)
| Mistake | Real message / result | Why | Fix |
|---|---|---|---|
building JSON with Sprintf |
invalid JSON when a value has a " (json.Valid = false) |
no escaping | json.Marshal |
| lowercase struct field | field silently missing from the JSON | unexported fields are invisible | capitalise it, add a json tag |
json.Unmarshal(data, it) |
json: Unmarshal(non-pointer main.Item) |
it must write into your variable | json.Unmarshal(data, &it) |
"price":"twenty" into an int |
json: cannot unmarshal string into Go struct field Item.price of type int |
types must match | fix the input, or custom UnmarshalJSON |
| a typo in a JSON key | the field stays 0, no error | unknown keys are ignored | Decoder.DisallowUnknownFields() → json: unknown field "prise" |
m["price"].(int) on map[string]any |
panic: interface conversion: interface {} is float64, not int |
JSON numbers → float64 |
a struct, .(float64), or UseNumber |
strings.Split on a CSV line |
3 fields instead of 2 |
quoted commas | encoding/csv |
| a short CSV record | record on line 2: wrong number of fields |
fields per record must match | fix the data or FieldsPerRecord = -1 |
forgetting csv.Writer.Flush |
no output | the writer is buffered | w.Flush(), then w.Error() |
| a bad hex digit | encoding/hex: invalid byte: U+0067 'g' |
only 0-9a-f |
validate the input |
float64 for money |
rounding errors | binary fractions | whole paise in an integer type |
Practice
- (Homework) Build the CSV → JSON converter (solution: the mini project).
- (Homework) Send yourself a typo field and catch it with
DisallowUnknownFields(shown above). - (Homework) Add the custom
Paisetype to your project (shown above). - (Homework) Base64-encode a small file:
os.ReadFileit and printbase64.StdEncoding.EncodeToString(data). - Decode
[{"item":"chai","qty":2},{"item":"vada pav","qty":1}]into a slice and print the total quantity.
Solution 5
package main
import (
"encoding/json"
"fmt"
)
type Order struct {
Item string `json:"item"`
Qty int `json:"qty"`
}
func main() {
in := `[{"item":"chai","qty":2},{"item":"vada pav","qty":1}]`
var orders []Order
if err := json.Unmarshal([]byte(in), &orders); err != nil {
fmt.Println("error:", err)
return
}
total := 0
for _, o := range orders {
total += o.Qty
}
fmt.Println(len(orders), "orders, total qty", total)
}
Output:
2 orders, total qty 3
- Make a struct with a
*intfieldDiscountand tagjson:"discount,omitempty". What's the difference between anilpointer and a pointer to 0 when you marshal?
Solution 6
package main
import (
"encoding/json"
"fmt"
)
type Item struct {
Name string `json:"name"`
Discount *int `json:"discount,omitempty"`
}
func main() {
zero := 0
a, _ := json.Marshal(Item{Name: "Chai"})
b, _ := json.Marshal(Item{Name: "Chai", Discount: &zero})
fmt.Println(string(a))
fmt.Println(string(b))
}
Output:
{"name":"Chai"}
{"name":"Chai","discount":0}
A nil pointer is empty, so it's omitted; a pointer to 0 is not nil, so "discount":0 is sent. Pointers let you tell "not given" from "zero".
Quick quiz
1. What does this print? (From the video.)
type Item struct {
Name string `json:"name"`
Price int `json:"price,omitempty"`
qty int
}
b, _ := json.Marshal(Item{"Chai", 0, 2})
fmt.Println(string(b))
A. {"name":"Chai","price":0,"qty":2} · B. {"name":"Chai"} · C. {"name":"Chai","price":0} · D. {"Name":"Chai","qty":2}
Answer
B. Price is 0, an empty value, so omitempty drops it; qty is unexported, so it's never encoded. Real output: {"name":"Chai"}.
2. You decode {"price": 120} into map[string]any. What is the dynamic type of m["price"]?
Answer
float64.
3. Which reads a never-ending stream of JSON objects from a network connection: json.Unmarshal or json.NewDecoder(conn).Decode?
Answer
json.NewDecoder(conn).Decode, called in a loop: it reads one value at a time from an io.Reader. Unmarshal needs all the bytes in memory first.
4. How many base64 characters encode 6 bytes?
Answer
8: every 3 bytes become 4 characters, and 6 is a multiple of 3, so no padding.
Interview questions
- Why are unexported fields not encoded by
encoding/json? The package uses reflection from outside your package, and Go's visibility rules don't let it read unexported fields. - What does
omitemptyconsider empty?false,0,nilpointers and interfaces, and zero-length arrays, slices, maps and strings. Not zero structs. - How do you reject unknown fields? Use a
json.Decoderand callDisallowUnknownFields()beforeDecode. - What type do JSON numbers get in
interface{}?float64, unless you useDecoder.UseNumber(), which givesjson.Number. - When would you use
json.RawMessage? To delay decoding part of a message until you know its type (polymorphic messages), or to pass JSON through unchanged. MarshalvsEncoder?Marshalreturns bytes for one value in memory;Encoderwrites values to anio.Writeras a stream. Same forUnmarshalvsDecoder.- How do you customise how a type is encoded? Implement
json.Marshaler(MarshalJSON) andjson.Unmarshaler(UnmarshalJSON, pointer receiver), orencoding.TextMarshalerfor plain text. - Why not split CSV with
strings.Split? CSV has quoting rules: commas, quotes and newlines inside quoted fields.encoding/csvfollows RFC 4180. - base64 vs hex? Both turn bytes into text. Hex doubles the size and is easy to read; base64 is about 33% bigger and more compact.
- When is gob a good choice? Between Go programs, for streams of Go values (a typical use, per its docs, is remote procedure calls with
net/rpc). Not for other languages.
Glossary
| Term | Meaning | Example |
|---|---|---|
| Encoding / serialization | turning a value into bytes in an agreed format | Go struct → JSON |
| Decoding / deserialization | turning those bytes back into a value | JSON → Go struct |
| Format | the agreed rules for how data is written | JSON, CSV, XML |
| Display format | how fmt prints values for people |
{Masala Chai 10 [hot sweet]} |
| JSON | JavaScript Object Notation: a text format of objects, arrays, strings, numbers, booleans, null | {"name":"Chai"} |
| JSON object | {} with "key": value pairs |
{"price": 10} |
| JSON array | an ordered list in [] |
["hot","sweet"] |
null |
JSON's "no value" (Go nil) |
"offer": null |
encoding/json |
Go's JSON package | import "encoding/json" |
| Marshal | encode a value (Go's name) | json.Marshal(v) |
| Unmarshal | decode into a value (Go's name) | json.Unmarshal(b, &v) |
json.Valid |
reports whether bytes are valid JSON | json.Valid(b) |
| Escaping | writing special characters safely, like \" for a quote |
"Raju's \"Special\"" |
| Reflection | a program inspecting its own types and fields at run time | how json finds fields |
| Exported field | a field starting with a capital letter, visible outside the package | Name |
| Struct tag | a backquoted string after a field's type, read by packages | `json:"name"` |
omitempty |
tag option: leave the field out when it's empty | json:"tags,omitempty" |
json:"-" |
never encode or decode the field | Secret string |
MarshalIndent |
Marshal with line breaks and indentation | json.MarshalIndent(v, "", " ") |
| Pointer argument | &v, so the function can fill your variable |
json.Unmarshal(b, &it) |
| Unknown field | a JSON key with no matching struct field; ignored by default | "prise" |
DisallowUnknownFields |
make a Decoder reject unknown fields |
dec.DisallowUnknownFields() |
map[string]any |
a map that can hold any JSON object | var m map[string]any |
any |
the empty interface: holds any value | var x any = 5 |
| Type assertion | x.(T): get the concrete value from an interface (episode 22) |
m["price"].(float64) |
json.RawMessage |
undecoded JSON bytes, for decoding later | Data json.RawMessage |
| Streaming | handling data piece by piece as it arrives | reading orders all day |
json.Decoder |
reads JSON values one by one from an io.Reader |
json.NewDecoder(os.Stdin) |
json.Encoder |
writes JSON values to an io.Writer |
json.NewEncoder(os.Stdout) |
| NDJSON / JSON Lines | one JSON value per line | {"item":"chai","qty":2} |
json.Marshaler |
interface with MarshalJSON() ([]byte, error) |
func (p Paise) MarshalJSON() |
json.Unmarshaler |
interface with UnmarshalJSON([]byte) error |
func (p *Paise) UnmarshalJSON(b []byte) |
| Paise | 1/100 of a rupee; stored as an integer for exact money | 1250 = Rs 12.50 |
fmt.Sscanf |
scan a string using a format | fmt.Sscanf(s, "Rs %d.%d", &rs, &ps) |
| CSV | comma-separated values: a text table | name,price |
| Record / field | a CSV row / one value in it | Samosa,15 |
| Quoting (CSV) | wrapping a field with commas or quotes in "...", doubling inner quotes |
"Chai, Masala" |
encoding/csv |
Go's CSV reader and writer (RFC 4180) | csv.NewReader(r) |
| RFC | "Request for Comments": a published internet standard | RFC 4180 (CSV) |
| XML | Extensible Markup Language: nested elements in angle brackets | <item price="10"> |
| Element / attribute | an XML tag pair / a name="value" inside a tag |
<name>, price="10" |
| Hex | each byte as two hexadecimal digits | ff00 |
| Base64 | 3 bytes → 4 safe characters | /wBoaf4= |
Padding (=) |
filler at the end of base64 when the length isn't a multiple of 3 | ...= |
| URL encoding (base64) | base64 with - and _ instead of + and / |
_wBoaf4= |
| gob | Go's own binary encoding for Go-to-Go data | gob.NewEncoder(&buf) |
| Binary format | bytes not meant to be read as text | gob, images |
Summary
- Programs share data through an agreed format; Go's display format and hand-built JSON break.
json.Marshalencodes exported fields and escapes for you; struct tags rename (json:"name"), omit empty values (omitempty) or hide fields ("-");MarshalIndentis for humans.json.Unmarshalneeds a pointer and matching types, with real error messages.- Unknown keys are silently ignored: use a
DecoderwithDisallowUnknownFields. - In
map[string]any, numbers arefloat64.json.RawMessagedecodes in two steps. json.Decoderandjson.Encoderstream values over anyio.Reader/io.Writer.MarshalJSON/UnmarshalJSONgive a type its own JSON form.- CSV needs
encoding/csv(quoting rules, same field count); XML works like JSON; base64 and hex carry bytes as text; gob is Go-to-Go only.
Next episode: Working with time: time.Time, time.Duration, Add, Sub, Since, formatting with the reference layout 2006-01-02 15:04:05, time.Parse, time zones, the monotonic clock and Unix timestamps.