These are the notes for Go episode 33. Raju's order log comes from a string today, from the keyboard tomorrow, and from a file or the internet next week. A function that takes a string can't read the keyboard, so you invent the idea of a source you can read from bit by bit: io.Reader. You read by hand with a small byte "bucket", see the classic bug of ignoring n, and learn that io.EOF is the normal end of input. Then io.Writer (os.Stdout, bytes.Buffer, strings.Builder), io.Copy and io.ReadAll, the wrappers io.TeeReader, io.MultiWriter, io.LimitReader and io.Pipe, the combined interfaces io.Closer and io.ReadWriter, and the bufio package: a buffered Writer (and the forgotten Flush that makes output vanish), bufio.Reader.ReadString, and bufio.Scanner with its split functions and its 64 KB token too long limit. Every program here runs as-is in the SyntaxPilot compiler (language: Go) with go run main.go; text typed into the program input box is sent to standard input. All outputs and errors are from real runs (Go 1.24.7, linux/amd64).
Facts and sources
From the standard library documentation shipped with Go 1.24.7 (go doc):
io.Reader"is the interface that wraps the basic Read method. Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered." Callers "should always process the n > 0 bytes returned before considering the error err."io.EOF"is the error returned by Read when no more input is available", and "Functions should return EOF only to signal a graceful end of input." Callers test for it with==.bufio.MaxScanTokenSize = 64 * 1024: "the maximum size used to buffer a token unless the user provides an explicit buffer with Scanner.Buffer".- The default buffer size of
bufio.NewWriterandbufio.NewReaderis 4096 bytes (defaultBufSize = 4096inbufio/bufio.go).
The problem: data from everywhere
package main
import (
"fmt"
"os"
"strings"
)
// countLines counts the orders in a log.
func countLines(s string) int {
return strings.Count(s, "\n")
}
func main() {
fmt.Println(countLines("chai 10\nsamosa 15\npoha 20\n"))
fmt.Println(countLines(os.Stdin)) // the keyboard?
}
Output:
# command-line-arguments
./main.go:16:28: cannot use os.Stdin (variable of type *os.File) as string value in argument to countLines
Why it fails: countLines accepts only a string, data that is already complete in memory. The keyboard (os.Stdin, standard input, episode 18), a file and a network connection all deliver data bit by bit, and they are different types (os.Stdin is a *os.File). We'd need one countLines per source.
io.Reader: the tap and the bucket
Real-life example: a water tap.
| At the water tank | In Go |
|---|---|
| A tap: you don't care which tank is behind it | io.Reader: anything with a Read method |
| You bring a bucket; it fills up to n litres | Read(p []byte) (n int, err error) fills p and returns how many bytes n |
| "Tank empty" | io.EOF: the normal end of input |
type Reader interface {
Read(p []byte) (n int, err error)
}
pis a byte slice (episode 17), raw bytes in a row; it's your bucket.nsays how many bytes were put in this time. Onlyp[:n]is new data.errisnil,io.EOFat the end, or a real error. EOF stands for "end of file", but it means the end of any input. It is not a failure.
strings.NewReader(s) turns a string into a Reader (a *strings.Reader). Reading by hand with an 8-byte bucket, and printing the whole bucket every time:
package main
import (
"fmt"
"io"
"strings"
)
func main() {
r := strings.NewReader("chai 10\nsamosa 15\npoha 20\n")
buf := make([]byte, 8) // a bucket of 8 bytes
for {
n, err := r.Read(buf)
fmt.Printf("n=%d %q\n", n, buf) // BUG: prints the whole bucket
if err == io.EOF {
break
}
}
}
Output:
n=8 "chai 10\n"
n=8 "samosa 1"
n=8 "5\npoha 2"
n=2 "0\npoha 2"
n=0 "0\npoha 2"
The log is 26 bytes: 8 + 8 + 8 + 2. On the fourth read n is 2, but the bucket still holds 6 old bytes from the third read (poha 2), and we printed them. Then n is 0 with io.EOF, and the same stale bytes again.
The fix: use only buf[:n], and process the bytes before checking the error:
package main
import (
"fmt"
"io"
"strings"
)
func main() {
r := strings.NewReader("chai 10\nsamosa 15\npoha 20\n")
buf := make([]byte, 8)
for {
n, err := r.Read(buf)
if n > 0 {
fmt.Printf("n=%d %q\n", n, buf[:n]) // only the n new bytes
}
if err == io.EOF {
fmt.Println("end of input:", err)
break
}
}
}
Output:
n=8 "chai 10\n"
n=8 "samosa 1"
n=8 "5\npoha 2"
n=2 "0\n"
end of input: EOF
Rules: always use n; treat io.EOF as a normal ending; and handle the n > 0 bytes first, because a Reader may return data and io.EOF in the same call.
One function, any source: io.ReadAll
io.ReadAll(r) reads until io.EOF and returns all the bytes (and nil error at a normal end). Now countLines takes an io.Reader:
package main
import (
"bytes"
"fmt"
"io"
"os"
"strings"
)
// countLines works for ANY source that has a Read method.
func countLines(r io.Reader) (int, error) {
data, err := io.ReadAll(r)
if err != nil {
return 0, err
}
return bytes.Count(data, []byte("\n")), nil
}
func main() {
fmt.Println(countLines(strings.NewReader("chai 10\nsamosa 15\npoha 20\n")))
fmt.Println(countLines(os.Stdin)) // the program input box
}
Program input (typed into the input box):
chai 10
samosa 15
poha 20
thali 120
Output:
3 <nil>
4 <nil>
One function, two very different sources. os.Stdin has a Read method, so it is a Reader; so are files (*os.File, next episode), network connections, HTTP request bodies and compressed streams. This is "accept interfaces" from episode 22: ask for the smallest interface you need.
You can write your own Reader too: any type with the right Read method.
package main
import (
"fmt"
"io"
)
// Tokens is our own Reader: it produces "T1 T2 T3 " and then io.EOF.
type Tokens struct{ next, last int }
func (t *Tokens) Read(p []byte) (int, error) {
if t.next > t.last {
return 0, io.EOF
}
s := fmt.Sprintf("T%d ", t.next)
if len(p) < len(s) {
return 0, io.ErrShortBuffer
}
t.next++
return copy(p, s), nil
}
func main() {
data, err := io.ReadAll(&Tokens{next: 1, last: 3})
fmt.Printf("%q %v\n", data, err)
}
"T1 T2 T3 " <nil>
copy(p, s) copies the string's bytes into the bucket and returns how many it copied: exactly the n to return. io.ReadAll returns a nil error because io.EOF is the expected end.
io.Writer, bytes.Buffer, strings.Builder, io.Copy
type Writer interface {
Write(p []byte) (n int, err error)
}
Write writes the bytes of p and returns how many were written and an error. fmt.Fprintf(w, ...) (F for "file") formats text and writes it to any Writer.
package main
import (
"bytes"
"fmt"
"io"
"os"
"strings"
)
// printReceipt writes to ANY io.Writer.
func printReceipt(w io.Writer, item string, price int) {
fmt.Fprintf(w, "%-8s Rs %3d\n", item, price)
}
func main() {
printReceipt(os.Stdout, "chai", 10)
var buf bytes.Buffer // a Writer that keeps bytes in memory
printReceipt(&buf, "samosa", 15)
printReceipt(&buf, "thali", 120)
fmt.Println("buffer holds", buf.Len(), "bytes")
var sb strings.Builder // a Writer that builds one string
printReceipt(&sb, "poha", 20)
fmt.Printf("%q\n", sb.String())
io.Copy(os.Stdout, &buf) // Reader → Writer
}
Output:
chai Rs 10
buffer holds 32 bytes
"poha Rs 20\n"
samosa Rs 15
thali Rs 120
| Type | What it is |
|---|---|
os.Stdout |
standard output, the screen (a *os.File, a Writer) |
os.Stderr |
standard error: a second output for messages and errors |
bytes.Buffer |
a growable chunk of bytes in memory; both a Writer and a Reader |
strings.Builder |
a Writer that efficiently builds one string; read it with .String() |
strings.Reader |
a Reader over a string (strings.NewReader) |
io.Copy(dst, src) |
reads from a Reader and writes to a Writer until io.EOF; returns bytes copied and an error |
Each receipt line is 16 bytes (%-8s pads the item to 8 characters, then Rs, a 3-wide number and a newline), so two lines are 32 bytes. &buf is needed because Write has a pointer receiver on *bytes.Buffer (episode 20).
Wrappers: io.TeeReader, io.LimitReader, io.MultiWriter
Because the interfaces are tiny, the io package can wrap them to add features without changing the code that reads or writes:
io.TeeReader(r, w): a Reader; everything read fromris also written tow, like a T-shaped pipe joint.io.LimitReader(r, n): a Reader that returnsio.EOFafternbytes.io.MultiWriter(w1, w2, ...): a Writer; each write goes to all of them.
package main
import (
"bytes"
"fmt"
"io"
"os"
"strings"
)
func main() {
var logCopy bytes.Buffer
// TeeReader: whatever is read from r is also written to logCopy
r := io.TeeReader(strings.NewReader("chai 10\nsamosa 15\npoha 20\n"), &logCopy)
// LimitReader: read at most 12 bytes
first, _ := io.ReadAll(io.LimitReader(r, 12))
fmt.Printf("read %q\n", first)
fmt.Printf("copy %q\n", logCopy.String())
// MultiWriter: one write goes to the screen AND the buffer
var saved bytes.Buffer
both := io.MultiWriter(os.Stdout, &saved)
fmt.Fprintln(both, "order 4: thali 120")
fmt.Println("saved", saved.Len(), "bytes")
}
Output:
read "chai 10\nsamo"
copy "chai 10\nsamo"
order 4: thali 120
saved 19 bytes
We read exactly 12 bytes (chai 10\n is 8, plus samo), and the tee kept an identical copy. order 4: thali 120\n is 19 bytes, printed and saved. Real programs use these to log what they read, cap upload sizes and save copies.
Saving a copy of standard input with io.TeeReader (homework), with input chai 10:
var saved bytes.Buffer
r := io.TeeReader(os.Stdin, &saved)
data, _ := io.ReadAll(r)
fmt.Println("read", len(data), "bytes; saved copy:", saved.String())
read 7 bytes; saved copy: chai 10
io.Closer, io.ReadWriter and friends
Some sources must be closed when you're done (files, network connections), to give the resource back to the operating system. io.Closer has one method, Close() error. The io package combines the small interfaces by embedding (episode 23):
| Interface | Methods |
|---|---|
io.Reader |
Read |
io.Writer |
Write |
io.Closer |
Close |
io.ReadWriter |
Read + Write |
io.ReadCloser |
Read + Close |
io.WriteCloser |
Write + Close |
io.ReadWriteCloser |
Read + Write + Close |
A compile-time check (episode 21's var _ I = value trick):
package main
import (
"bytes"
"fmt"
"io"
"os"
"strings"
)
func main() {
var _ io.ReadWriteCloser = os.Stdin // *os.File: Read, Write, Close
var _ io.ReadWriter = &bytes.Buffer{} // Read and Write
var _ io.Closer = strings.NewReader("chai")
fmt.Println("ok")
}
Output:
# command-line-arguments
./main.go:14:23: cannot use strings.NewReader("chai") (value of type *strings.Reader) as io.Closer value in variable declaration: *strings.Reader does not implement io.Closer (missing method Close)
A *os.File can read, write and close; a *bytes.Buffer can read and write; but a *strings.Reader has no Close: a string in memory has nothing to give back. Lesson: ask only for the methods you need.
io.Pipe
io.Pipe() returns a connected pair: a *io.PipeReader and a *io.PipeWriter. Whatever is written to the writer can be read from the reader. A goroutine (the go keyword before a function call) runs that function at the same time as main; goroutines get full episodes from episode 43.
package main
import (
"fmt"
"io"
"os"
)
func main() {
pr, pw := io.Pipe() // what is written to pw can be read from pr
go func() { // a goroutine: runs at the same time as main
for i := 1; i <= 3; i++ {
fmt.Fprintf(pw, "kitchen: order %d ready\n", i)
}
pw.Close() // the reader now gets io.EOF
}()
n, err := io.Copy(os.Stdout, pr)
fmt.Println(n, "bytes, err:", err)
}
Output:
kitchen: order 1 ready
kitchen: order 2 ready
kitchen: order 3 ready
69 bytes, err: <nil>
pw.Close()makes the reader seeio.EOF, soio.Copyknows it's done. Forget it, and the reader waits forever.- A pipe has no storage: each
Writewaits until a reader has taken the data. Three lines of 23 bytes = 69 bytes.
bufio.Writer and Flush
Real-life example: the delivery boy. One samosa per trip wastes the day on the road; fill a tray and make one trip; at closing time, send the half-full tray anyway.
- Each small
Writeto the screen or a file is a request to the operating system, a system call: relatively slow. bufio.NewWriter(w)wraps a Writer with a buffer, a waiting area in memory (4096 bytes by default). Small writes collect there; it sends them in one go when it's full.Flush()sends whatever is waiting now.Buffered()tells you how many bytes are waiting.
Forgetting Flush:
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
w := bufio.NewWriter(os.Stdout) // collects writes in a 4096-byte buffer
for i := 1; i <= 3; i++ {
fmt.Fprintf(w, "token %d\n", i)
}
fmt.Fprintln(os.Stderr, "bytes waiting in the buffer:", w.Buffered())
// forgot w.Flush()
}
Output:
bytes waiting in the buffer: 24
No tokens at all. The message went to standard error directly; the 24 bytes of tokens never filled the 4096-byte buffer, and main ended with them still waiting. No error message: the output just vanishes.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
w := bufio.NewWriter(os.Stdout)
defer w.Flush() // send what's left before main returns
for i := 1; i <= 3; i++ {
fmt.Fprintf(w, "token %d\n", i)
}
}
Output:
token 1
token 2
token 3
defer w.Flush() right after creating the Writer is the habit. In real programs, check its error (if err := w.Flush(); err != nil): that's where a full disk or a closed connection shows up.
bufio.Reader and ReadString
bufio.NewReader(r) wraps a Reader with a buffer and adds handy methods. ReadString('\n') reads up to and including the delimiter (the character that marks the end of a piece). At the end it returns what it has plus io.EOF. fmt.Sscan (S for string) scans values out of a string.
package main
import (
"bufio"
"fmt"
"io"
"os"
)
func main() {
r := bufio.NewReader(os.Stdin)
total := 0
for {
line, err := r.ReadString('\n') // up to and including '\n'
var item string
var price int
if _, e := fmt.Sscan(line, &item, &price); e == nil {
fmt.Printf("%-7s %4d\n", item, price)
total += price
}
if err == io.EOF {
break
}
}
fmt.Println("total", total)
}
With the same input as before:
Output:
chai 10
samosa 15
poha 20
thali 120
total 165
Use the line first, then check io.EOF: if the input doesn't end with a newline, the last line arrives together with io.EOF.
bufio.Scanner and split functions
bufio.NewScanner(r) is the simplest way to read input piece by piece. Each piece is a token. Scan() moves to the next token and returns false at the end (or on an error); Text() returns the token as a string (without the newline). A split function decides how input is cut into tokens:
| Split function | Token |
|---|---|
bufio.ScanLines (default) |
one line, without \n (and without a trailing \r) |
bufio.ScanWords |
one word, split on spaces and newlines |
bufio.ScanRunes |
one Unicode character (rune, episode 17) |
bufio.ScanBytes |
one byte |
package main
import (
"bufio"
"fmt"
"os"
"strings"
)
func main() {
sc := bufio.NewScanner(os.Stdin) // default split: ScanLines
lines := 0
for sc.Scan() {
lines++
fmt.Printf("%d: %s\n", lines, sc.Text())
}
words := bufio.NewScanner(strings.NewReader("one masala chai\ntwo vada pav"))
words.Split(bufio.ScanWords) // a split function: one word per token
n := 0
for words.Scan() {
n++
}
fmt.Println("lines", lines, "words", n)
}
Output:
1: chai 10
2: samosa 15
3: poha 20
4: thali 120
lines 4 words 6
ScanRunes counts characters, not bytes:
sc := bufio.NewScanner(strings.NewReader("चाय tea"))
sc.Split(bufio.ScanRunes)
n := 0
for sc.Scan() {
n++
}
fmt.Println("runes:", n, "bytes:", len("चाय tea"))
runes: 7 bytes: 13
The 64 KB limit: token too long
package main
import (
"bufio"
"fmt"
"strings"
)
func main() {
long := strings.Repeat("x", 70000) + "\n" // one 70,000-byte line
sc := bufio.NewScanner(strings.NewReader(long))
for sc.Scan() {
fmt.Println("got a line of", len(sc.Text()))
}
if err := sc.Err(); err != nil {
fmt.Println("error:", err)
}
}
Output:
error: bufio.Scanner: token too long
A token can be at most bufio.MaxScanTokenSize = 64 KB by default. Scan returned false straight away; without the sc.Err() check you'd never know why. Always check sc.Err() after the loop.
package main
import (
"bufio"
"fmt"
"strings"
)
func main() {
long := strings.Repeat("x", 70000) + "\n"
sc := bufio.NewScanner(strings.NewReader(long))
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024) // allow lines up to 1 MB
for sc.Scan() {
fmt.Println("got a line of", len(sc.Text()))
}
if err := sc.Err(); err != nil {
fmt.Println("error:", err)
}
}
Output:
got a line of 70000
sc.Buffer(buf, max) gives the Scanner a starting buffer and a maximum token size (here 1 MB). For truly huge or unlimited lines, use bufio.Reader instead.
Which reading tool?
| Need | Use |
|---|---|
| everything at once (small input) | io.ReadAll |
| line by line, words, runes | bufio.Scanner (check Err) |
| up to a delimiter, any length | bufio.Reader.ReadString / ReadBytes |
| move data from a Reader to a Writer | io.Copy |
| full control over each chunk | Read with your own []byte (use p[:n]) |
Mini project: the order log tool
Summarise(r io.Reader, w io.Writer) error: scan item price lines, write one neat line per order and a total through a buffered Writer, skip bad lines, return sc.Err() and Flush's error. Test it with a strings.Reader and a bytes.Buffer.
Solution
package main
import (
"bufio"
"bytes"
"fmt"
"io"
"os"
"strings"
)
// Summarise reads "item price" lines from r and writes a neat bill to w.
func Summarise(r io.Reader, w io.Writer) error {
sc := bufio.NewScanner(r)
bw := bufio.NewWriter(w)
total := 0
for sc.Scan() {
var item string
var price int
if _, err := fmt.Sscan(sc.Text(), &item, &price); err != nil {
fmt.Fprintf(bw, "skipped %q\n", sc.Text())
continue
}
fmt.Fprintf(bw, "%-8s Rs %4d\n", item, price)
total += price
}
if err := sc.Err(); err != nil {
return err
}
fmt.Fprintf(bw, "%-8s Rs %4d\n", "TOTAL", total)
return bw.Flush()
}
func main() {
// a test: a string in, a buffer out, no keyboard needed
var out bytes.Buffer
err := Summarise(strings.NewReader("chai 10\nsamosa 15\noops\nthali 120\n"), &out)
fmt.Print(out.String())
fmt.Println("err:", err, "| bytes:", out.Len())
// the real thing: standard input to standard output
if err := Summarise(os.Stdin, os.Stdout); err != nil {
fmt.Println("error:", err)
}
}
Program input:
poha 20
lassi 30
Output:
chai Rs 10
samosa Rs 15
skipped "oops"
thali Rs 120
TOTAL Rs 145
err: <nil> | bytes: 83
poha Rs 20
lassi Rs 30
TOTAL Rs 50
Because Summarise takes interfaces, the same code serves a test (string in, buffer out) and the real program (keyboard in, screen out).
Common mistakes (and the real messages)
| Mistake | Real message / result | Why | Fix |
|---|---|---|---|
func countLines(s string) called with os.Stdin |
cannot use os.Stdin (variable of type *os.File) as string value in argument to countLines |
tied to one source | func countLines(r io.Reader) |
printing buf after Read |
stale bytes: n=2 "0\npoha 2" |
only n bytes are new |
buf[:n] |
treating io.EOF as a failure |
(program reports an error on every normal end) | EOF is the normal end | if err == io.EOF { break } |
var _ io.Closer = strings.NewReader("chai") |
*strings.Reader does not implement io.Closer (missing method Close) |
no Close method |
ask only for io.Reader |
forgetting pw.Close() on an io.Pipe |
(the reader waits forever) | the reader never sees EOF | close the writer when done |
bufio.NewWriter(os.Stdout) without Flush |
(no output at all) | bytes wait in the 4096-byte buffer | defer w.Flush() |
a 70,000-byte line in a Scanner |
bufio.Scanner: token too long (only if you check sc.Err()) |
64 KB default limit | sc.Buffer(buf, max), or bufio.Reader |
Practice
- (Homework) Build
Summarise(solution: the mini project). - (Homework) Remove
Flushand watch the output vanish (shown above). - (Homework) Count runes with
bufio.ScanRunes(shown above). - (Homework) Save a copy of stdin with
io.TeeReader(shown above). - Write
upper(r io.Reader, w io.Writer) errorthat copies text and makes every letter a capital.
Solution 5
func upper(r io.Reader, w io.Writer) error {
data, err := io.ReadAll(r)
if err != nil {
return err
}
_, err = w.Write(bytes.ToUpper(data))
return err
}
upper(strings.NewReader("chai"), os.Stdout) prints CHAI.
- Predict the output of the quiz program below, then explain why
fmt.Printand abufio.Writerover the sameos.Stdoutcan come out in a different order.
Solution 6
samosa chai. The bufio.Writer holds "chai " in its own buffer until Flush; fmt.Print writes to os.Stdout directly. Both end up on the same screen, but in the order the bytes actually reach os.Stdout. Don't mix buffered and unbuffered writes to the same destination.
- Why does
io.Copy(os.Stdout, &buf)print nothing if you call it a second time on the samebytes.Buffer?
Solution 7
Reading from a bytes.Buffer consumes the bytes. After the first io.Copy, the buffer is empty, so the second one copies 0 bytes.
Quick quiz
1. What does this print? (From the video.)
w := bufio.NewWriter(os.Stdout)
fmt.Fprint(w, "chai ")
fmt.Print("samosa ")
w.Flush()
fmt.Println()
A. chai samosa · B. samosa chai · C. samosa · D. chai
Answer
B. "chai " waits in the buffer; fmt.Print goes straight to os.Stdout, so samosa appears first; Flush then sends chai. Real output: samosa chai.
2. A Read call returns n = 5 and err = io.EOF. What should you do with the 5 bytes?
Answer
Use them. Process p[:n] first, then stop because of io.EOF.
3. Which type is both a Reader and a Writer: strings.Reader, strings.Builder or bytes.Buffer?
Answer
bytes.Buffer.
Interview questions
- What is
io.Readerand why is it so widely used? A one-method interface,Read(p []byte) (n int, err error). Because it's tiny, files, network connections, HTTP bodies, buffers, strings and compressors all implement it, so one function taking anio.Readerworks with all of them, and wrappers (limit, tee, decompress) compose freely. - How should a caller handle
nanderrfromRead? Processp[:n]first (n may be > 0 even with an error), then checkerr;io.EOFmeans a normal end. - Is
io.EOFan error? It's anerrorvalue, but it signals a graceful end of input, not a failure. Compare with==(orerrors.Is);io.ReadAllandio.Copyturn it into anilerror. - What does
io.Copydo? Reads from a Reader and writes to a Writer until EOF or an error, returning the byte count. - Why use
bufio.Writer, and what's the catch? It batches small writes into fewer system calls; the catch is you must callFlush(and check its error), or buffered data is lost. bufio.Scannervsbufio.Reader? Scanner is the simplest for lines/words/runes but has a token size limit (64 KB by default; change withBuffer) and you must checkErr(). Reader'sReadString/ReadByteshandle lines of any length.- What is
io.Pipefor? Connecting code that writes to code that reads, synchronously, usually across goroutines, without a buffer in between; closing the writer gives the reader EOF. - What's the difference between
bytes.Bufferandstrings.Builder?bytes.Bufferis a read-write byte buffer (Reader and Writer);strings.Builderis write-only and optimised for building a single string. - How do you test a function that reads input? Make it take an
io.Reader(andio.Writerfor output), then passstrings.NewReader("...")and abytes.Bufferin the test.
Glossary
| Term | Meaning | Example |
|---|---|---|
io package |
standard package of basic interfaces and helpers for input and output | import "io" |
| Source | where data comes from: a string, the keyboard, a file, the network | os.Stdin |
| Byte slice | []byte: raw bytes in a row (episode 17) |
make([]byte, 8) |
io.Reader |
interface with Read(p []byte) (n int, err error) |
strings.NewReader("chai") |
Read |
fills p with up to len(p) bytes and returns how many (n) |
n, err := r.Read(buf) |
n |
how many bytes Read/Write actually handled; only p[:n] is new |
buf[:n] |
io.EOF |
the error Read returns when no more input is available; a normal end |
if err == io.EOF |
| EOF | "end of file", used for the end of any input | the tank is empty |
io.ReadAll |
reads everything until EOF and returns the bytes | data, err := io.ReadAll(r) |
io.Writer |
interface with Write(p []byte) (n int, err error) |
os.Stdout |
fmt.Fprintf |
formatted printing to any io.Writer |
fmt.Fprintf(w, "%d", 10) |
fmt.Sscan |
reads space-separated values out of a string | fmt.Sscan(line, &item, &price) |
os.Stdin / os.Stdout / os.Stderr |
standard input (keyboard / input box), standard output (screen), standard error (messages) | io.Copy(os.Stdout, r) |
*os.File |
Go's type for an open file; also used for stdin/stdout | os.Stdin |
strings.Reader |
a Reader over a string | strings.NewReader(log) |
strings.Builder |
a Writer that builds one string efficiently | sb.String() |
bytes.Buffer |
an in-memory, growable byte buffer: a Reader and a Writer | var buf bytes.Buffer |
io.Copy |
copies from a Reader to a Writer until EOF | io.Copy(os.Stdout, &buf) |
| Wrapper | a Reader or Writer built around another one to add a feature | io.LimitReader(r, 12) |
io.TeeReader |
Reader that also writes everything it reads to a Writer | keep a copy of input |
io.LimitReader |
Reader that stops (EOF) after n bytes | read at most 12 bytes |
io.MultiWriter |
Writer that writes to several Writers at once | screen + buffer |
io.Pipe |
a connected in-memory reader/writer pair with no storage | kitchen → counter |
| Goroutine | a function running at the same time as the rest of the program, started with go (episode 43) |
go func() { ... }() |
io.Closer |
interface with Close() error |
files, connections |
io.ReadWriter |
Reader + Writer, combined by embedding | *bytes.Buffer |
io.ReadWriteCloser |
Reader + Writer + Closer | *os.File |
| Embedding (interfaces) | listing one interface inside another to combine their methods (episode 23) | type ReadWriter interface { Reader; Writer } |
| Buffer | a waiting area in memory where data collects before moving on | the tray |
| System call | a request from a program to the operating system, e.g. to write to the screen | each unbuffered Write |
bufio package |
buffered I/O: Writer, Reader, Scanner |
import "bufio" |
bufio.Writer |
a Writer that collects small writes in a buffer (4096 bytes by default) | bufio.NewWriter(os.Stdout) |
Flush |
sends all bytes waiting in a bufio.Writer now |
defer w.Flush() |
Buffered |
how many bytes are waiting in a bufio.Writer |
w.Buffered() |
bufio.Reader |
a buffered Reader with extra methods | bufio.NewReader(os.Stdin) |
ReadString |
reads up to and including a delimiter | r.ReadString('\n') |
| Delimiter | a character that marks where one piece ends | '\n' |
bufio.Scanner |
reads input token by token: Scan, Text, Err |
for sc.Scan() { sc.Text() } |
| Token | one piece of input produced by a Scanner | one line, one word |
| Split function | decides how a Scanner cuts input into tokens | bufio.ScanWords |
ScanLines / ScanWords / ScanRunes / ScanBytes |
built-in split functions: lines (default), words, characters, bytes | sc.Split(bufio.ScanRunes) |
bufio.MaxScanTokenSize |
default maximum token size, 64 KB | token too long beyond it |
sc.Buffer |
sets a Scanner's starting buffer and maximum token size | sc.Buffer(buf, 1024*1024) |
sc.Err |
the error that stopped a Scanner (nil at a normal end) | if err := sc.Err(); err != nil |
| Rune | a Unicode character (episode 17) | च |
Summary
- A
stringparameter only reads strings; anio.Readerparameter reads from anywhere. Read(p)fills your bucket and returns n: use onlyp[:n], process data before the error, and treatio.EOFas the normal end.io.Writeris the other side:os.Stdout,bytes.Buffer,strings.Builder;fmt.Fprintfwrites to any Writer;io.Copyandio.ReadAllmove the data.- Wrappers add features:
TeeReader,LimitReader,MultiWriter, andio.Pipeconnects a writer to a reader. - Small interfaces combine:
io.Closer,io.ReadWriter,io.ReadWriteCloser. Ask for the smallest one you need. bufio.Writersaves system calls, but alwaysFlush(and check its error).bufio.Reader.ReadStringreads up to a delimiter;bufio.Scannergives tokens with a split function, has a 64 KB default limit, and needs anErr()check.
Next episode: Files and the os package: os.Open, os.Create, permissions, defer f.Close() and close errors, os.ReadFile, filepath.WalkDir, io/fs, embed, and environment variables.