These are the notes for Go episode 32. Raju's app stores a day of one million orders, and he wants just the first three orders over 50 rupees. Returning a filtered slice works but checks every order and builds a huge slice. Passing a callback avoids the slice but can't break. So you invent a callback that returns a bool, the yield function, and then use Go 1.23's range over function types so a normal for ... range loop with a normal break works. You'll meet iter.Seq and iter.Seq2, see the run-time panic when an iterator ignores yield's false, use the iterators in slices and maps (All, Values, Collect, Keys, Sorted), write custom iterators (even infinite ones), and learn push vs pull iterators with iter.Pull. Every program here runs as-is in the SyntaxPilot compiler (language: Go) with go run main.go. All outputs and errors are from real runs (Go 1.24.7, linux/amd64).
Facts and sources
From the Go blog post "Range Over Function Types" by Ian Lance Taylor (20 August 2024):
- "Range over function types is a new language feature in the Go 1.23 release."
- The new
iterpackage definestype Seq[V any] func(yield func(V) bool)andtype Seq2[K, V any] func(yield func(K, V) bool). - "A push iterator pushes each value in a sequence to a yield function." A pull iterator works the other way: each call returns the next value.
iter.Pulltakes a push iterator and "returns a pair of functions": the next function and a stop function. slices.Allandslices.Valuesreturn iterators over a slice;slices.Collectgathers an iterator's values into a slice;maps.All,maps.Keysandmaps.Valuesreturn iterators over a map.
The problem: a million orders
package main
import "fmt"
var checked int
// bigOrders returns every order over 50 rupees.
func bigOrders(orders []int) []int {
var out []int
for _, o := range orders {
checked++
if o > 50 {
out = append(out, o)
}
}
return out
}
func main() {
orders := make([]int, 1_000_000)
for i := range orders {
orders[i] = i * 37 % 100 // a fake day of order amounts
}
big := bigOrders(orders)
fmt.Println("first three:", big[:3])
fmt.Println("checked", checked, "orders, built a slice of", len(big))
}
Output:
first three: [74 85 59]
checked 1000000 orders, built a slice of 490000
i * 37 % 100 just makes up a day of order amounts between 0 and 99. The answer is right, but we checked all 1,000,000 orders and built a slice of 490,000 numbers to show three. That's wasted time and memory. Worse, if the data were endless (a live stream of orders), this function would never return.
Callbacks can't break
A callback is a function you pass to another function, to be called back for each item. No slice is built:
package main
import "fmt"
// forEachBig calls f for every order over 50.
func forEachBig(orders []int, f func(int)) {
for _, o := range orders {
if o > 50 {
f(o)
}
}
}
func main() {
orders := make([]int, 1_000_000)
for i := range orders {
orders[i] = i * 37 % 100 // a fake day of order amounts
}
n := 0
forEachBig(orders, func(o int) {
fmt.Println(o)
n++
if n == 3 {
break
}
})
}
Output:
# command-line-arguments
./main.go:24:13: break is not in a loop, switch, or select
Why: break sits inside the function literal, not inside a loop. The loop lives in forEachBig, a different function, and break can't cross a function boundary. A return would only end this one call of the callback; forEachBig would just call it again for the next order.
Inventing yield
Real-life example: the chai seller on the train.
| On the train | In Go |
|---|---|
| The seller hands you one cup | yield(v): the iterator hands you one value |
| "One more!" | yield returns true: keep going |
| "Bas, enough!" | yield returns false: stop (and clean up) |
The fix is one small change: the callback returns a bool. true means "send more", false means "stop". By convention this callback is called yield:
package main
import "fmt"
var checked int
// yield returns false when the caller wants to stop.
func forEachBig(orders []int, yield func(int) bool) {
for _, o := range orders {
checked++
if o > 50 {
if !yield(o) {
return
}
}
}
}
func main() {
orders := make([]int, 1_000_000)
for i := range orders {
orders[i] = i * 37 % 100 // a fake day of order amounts
}
n := 0
forEachBig(orders, func(o int) bool {
fmt.Println(o)
n++
return n < 3 // false after the third: stop
})
fmt.Println("checked", checked, "orders")
}
Output:
74
85
59
checked 8 orders
Only 8 orders were checked (orders 0 to 7; the big ones are at positions 2, 5 and 7). No big slice, and the work stopped the moment the caller said enough. This is the iterator pattern: a function that produces values one at a time, on demand.
Range over functions (Go 1.23)
Writing return n < 3 is clumsy. Since Go 1.23 you can range over a function that takes a yield:
package main
import (
"fmt"
"iter"
)
var checked int
// bigOrders returns an iterator: a func that takes yield.
func bigOrders(orders []int) iter.Seq[int] {
return func(yield func(int) bool) {
for _, o := range orders {
checked++
if o > 50 && !yield(o) {
return
}
}
}
}
func main() {
orders := make([]int, 1_000_000)
for i := range orders {
orders[i] = i * 37 % 100 // a fake day of order amounts
}
n := 0
for o := range bigOrders(orders) {
fmt.Println(o)
n++
if n == 3 {
break
}
}
fmt.Println("checked", checked, "orders")
}
Output:
74
85
59
checked 8 orders
bigOrdersnow returns an iterator:iter.Seq[int], which is justfunc(yield func(int) bool).for o := range bigOrders(orders)calls that function. Go writes theyieldcallback for you from the loop body.- When the body finishes normally (or hits
continue),yieldreturnstrue. When the body doesbreak(orreturn, orgotoout of the loop),yieldreturnsfalse. - An iterator is any function that hands out a sequence of values like this. A sequence is just a series of values, one after another.
Which functions can you range over? Exactly three shapes:
| Function type | Loop |
|---|---|
func(yield func() bool) |
for range f { ... } |
func(yield func(V) bool) (iter.Seq[V]) |
for v := range f { ... } |
func(yield func(K, V) bool) (iter.Seq2[K, V]) |
for k, v := range f { ... } |
Anything else is rejected:
func notIter(f func(int)) {}
for v := range notIter { ... }
cannot range over notIter (value of type func(f func(int))): func must be func(yield func(...) bool): yield func does not return bool
What break really does: always check yield
package main
import (
"fmt"
"iter"
)
// tokens hands out token numbers 1, 2, 3, ... forever.
func tokens() iter.Seq[int] {
return func(yield func(int) bool) {
for t := 1; ; t++ {
yield(t) // BUG: ignores what yield returns
}
}
}
func main() {
for t := range tokens() {
fmt.Println("token", t)
if t == 2 {
break
}
}
}
Output:
token 1
token 2
panic: runtime error: range function continued iteration after function for loop body returned false
goroutine 1 [running]:
...
After break, yield returned false, but the iterator called it again. Go detects this at run time and panics (crashes), so the bug can't hide. The rule: if !yield(v) { return }, every time.
package main
import (
"fmt"
"iter"
)
// tokens hands out token numbers 1, 2, 3, ... forever.
func tokens() iter.Seq[int] {
return func(yield func(int) bool) {
for t := 1; ; t++ {
if !yield(t) {
fmt.Println("(counter closed)")
return
}
}
}
}
func main() {
for t := range tokens() {
fmt.Println("token", t)
if t == 2 {
break
}
}
}
Output:
token 1
token 2
(counter closed)
- The spot where the iterator returns after
falseis where it cleans up: closing a file, a network connection, and so on. tokensis an infinite sequence: it never ends by itself. That's fine, because values are made only when the loop asks for them. This "only when needed" style is called lazy.
iter.Seq2: two values per step
package main
import (
"fmt"
"iter"
)
// numbered yields two values each time: a number and an item.
func numbered(menu []string) iter.Seq2[int, string] {
return func(yield func(int, string) bool) {
for i, item := range menu {
if !yield(i+1, item) {
return
}
}
}
}
func main() {
menu := []string{"chai", "samosa", "poha"}
for n, item := range numbered(menu) {
fmt.Println(n, item)
}
}
Output:
1 chai
2 samosa
3 poha
Seq gives one value per step; Seq2 gives pairs, like index and value or key and value. Both are generic types (episode 30) defined in the iter package.
Iterators in slices and maps
package main
import (
"fmt"
"iter"
"maps"
"slices"
)
func cheap(prices map[string]int) iter.Seq[string] {
return func(yield func(string) bool) {
for item, p := range maps.All(prices) {
if p <= 15 && !yield(item) {
return
}
}
}
}
func main() {
menu := []string{"chai", "samosa", "poha"}
for i, v := range slices.All(menu) {
fmt.Print(i, "=", v, " ")
}
fmt.Println()
again := slices.Collect(slices.Values(menu))
fmt.Println(again, len(again))
prices := map[string]int{"chai": 10, "samosa": 15, "thali": 120}
fmt.Println(slices.Sorted(maps.Keys(prices)))
fmt.Println(slices.Sorted(cheap(prices)))
}
Output:
0=chai 1=samosa 2=poha
[chai samosa poha] 3
[chai samosa thali]
[chai samosa]
| Function | Gives | Kind |
|---|---|---|
slices.All(s) |
index and value | iter.Seq2[int, E] |
slices.Values(s) |
values only | iter.Seq[E] |
slices.Collect(seq) |
runs an iterator, returns a new slice | []E |
slices.Sorted(seq) |
collects and sorts (needs cmp.Ordered values) |
[]E |
maps.All(m) |
key and value | iter.Seq2[K, V] |
maps.Keys(m) |
keys | iter.Seq[K] |
maps.Values(m) |
values | iter.Seq[V] |
Map iteration order is random (episode 15), so slices.Sorted(maps.Keys(m)) is the standard way to get the keys in order. Our own cheap iterator, built on maps.All, plugs into slices.Sorted just like a library one: iterators compose like pipes.
Iterators plug together: a lazy pipeline
Filter takes any iterator and returns a new iterator. It builds nothing up front. orders is an infinite source that counts how many amounts it made:
package main
import (
"fmt"
"iter"
)
func Filter[V any](seq iter.Seq[V], keep func(V) bool) iter.Seq[V] {
return func(yield func(V) bool) {
for v := range seq {
if keep(v) && !yield(v) {
return
}
}
}
}
var made int // how many orders were made
func orders() iter.Seq[int] {
return func(yield func(int) bool) {
for i := 0; ; i++ {
made++
if !yield(i * 37 % 100) {
return
}
}
}
}
func main() {
big := Filter(orders(), func(o int) bool { return o > 50 })
for o := range big {
fmt.Println(o)
if o == 59 { // the third big order
break
}
}
fmt.Println("orders made:", made)
}
Output:
74
85
59
orders made: 8
Only 8 orders were ever made. When the loop breaks, Filter's yield returns false, so Filter returns; that makes the range seq loop inside Filter stop, so orders's yield returns false too. The stop signal travels back down the whole chain, like kitchen stations passing plates one at a time and all stopping when the waiter says enough. Filter is a generic function (episode 30), so it works for an iterator of any type.
Push vs pull
Real-life example: two bank counters.
- Push: the chai seller comes to you and decides the pace. That's every iterator so far: the iterator pushes each value into your
yield.for ... rangeuses push iterators. - The problem: merge two counters' sorted token queues into one list, smallest first. You must walk both sequences side by side, and one
rangeloop walks only one sequence. - Pull: you go to the counter and say "next!". You pull each value when you want it.
iter.Pull(seq) turns a push iterator into a pull iterator. It returns two functions:
next()returns the next value andok(falsewhen the sequence is finished);stop()ends the iteration early and lets the iterator clean up. Always call it, usually withdefer stop()(episode 10).
package main
import (
"fmt"
"iter"
"slices"
)
// merge walks two sorted queues side by side, smallest first.
func merge(a, b iter.Seq[int]) []int {
nextA, stopA := iter.Pull(a)
defer stopA()
nextB, stopB := iter.Pull(b)
defer stopB()
var out []int
x, okA := nextA()
y, okB := nextB()
for okA || okB {
if okA && (!okB || x <= y) {
out = append(out, x)
x, okA = nextA()
} else {
out = append(out, y)
y, okB = nextB()
}
}
return out
}
func main() {
counter1 := slices.Values([]int{1, 4, 9})
counter2 := slices.Values([]int{2, 3, 10})
fmt.Println(merge(counter1, counter2))
}
Output:
[1 2 3 4 9 10]
A tiny pull example, step by step:
package main
import (
"fmt"
"iter"
)
func countdown(n int) iter.Seq[int] {
return func(yield func(int) bool) {
for i := n; i >= 1; i-- {
if !yield(i) {
return
}
}
}
}
func main() {
next, stop := iter.Pull(countdown(2))
defer stop()
v, ok := next()
fmt.Println(v, ok)
v, ok = next()
fmt.Println(v, ok)
v, ok = next()
fmt.Println(v, ok)
}
2 true
1 true
0 false
After the sequence ends, next returns the zero value and false. Use push (a range loop) by default: it's simpler and faster. Use pull only when you must control the pace yourself, like walking two sequences at once.
Mini project: an order pipeline
Tasks: Filter (keep orders over 50, still lazy), Take (first n values), an infinite Fib, and Zip (pair two sequences with iter.Pull into an iter.Seq2).
Solution
package main
import (
"fmt"
"iter"
"slices"
)
// Filter yields only the values for which keep returns true. Still lazy.
func Filter[V any](seq iter.Seq[V], keep func(V) bool) iter.Seq[V] {
return func(yield func(V) bool) {
for v := range seq {
if keep(v) && !yield(v) {
return
}
}
}
}
// Take yields at most n values, then stops the source too.
func Take[V any](seq iter.Seq[V], n int) iter.Seq[V] {
return func(yield func(V) bool) {
if n <= 0 {
return
}
i := 0
for v := range seq {
if !yield(v) {
return
}
i++
if i == n {
return
}
}
}
}
// Fib yields 0, 1, 1, 2, 3, 5, ... forever.
func Fib() iter.Seq[int] {
return func(yield func(int) bool) {
a, b := 0, 1
for {
if !yield(a) {
return
}
a, b = b, a+b
}
}
}
// Zip pairs up two sequences until either one runs out.
func Zip[A, B any](as iter.Seq[A], bs iter.Seq[B]) iter.Seq2[A, B] {
return func(yield func(A, B) bool) {
nextA, stopA := iter.Pull(as)
defer stopA()
nextB, stopB := iter.Pull(bs)
defer stopB()
for {
a, okA := nextA()
b, okB := nextB()
if !okA || !okB || !yield(a, b) {
return
}
}
}
}
func main() {
orders := slices.Values([]int{74, 12, 85, 40, 59, 90})
big := Filter(orders, func(o int) bool { return o > 50 })
fmt.Println(slices.Collect(big))
fmt.Println(slices.Collect(Take(big, 2)))
fmt.Println(slices.Collect(Take(Fib(), 10)))
tokens := slices.Values([]int{101, 102, 103})
names := slices.Values([]string{"Asha", "Priya"})
for t, name := range Zip(tokens, names) {
fmt.Println(t, name)
}
}
Output:
[74 85 59 90]
[74 85]
[0 1 1 2 3 5 8 13 21 34]
101 Asha
102 Priya
Notice that big can be used twice: an iter.Seq built from a slice starts over each time it is ranged. Take returns as soon as it has n values, which makes the inner range seq loop break, so Fib sees false and stops: infinite sequences are safe.
Common mistakes (and the real messages)
| Mistake | Real message / result | Why | Fix |
|---|---|---|---|
| returning a filtered slice to show 3 values | checked 1000000 orders, built a slice of 490000 |
all the work and memory up front | return an iter.Seq[int] |
break inside a callback |
break is not in a loop, switch, or select |
the loop is in another function | range over an iterator |
calling yield without checking |
panic: runtime error: range function continued iteration after function for loop body returned false |
the iterator kept going after stop | if !yield(v) { return } |
| ranging over a func whose callback returns nothing | cannot range over notIter (...): func must be func(yield func(...) bool): yield func does not return bool |
only func(yield func(...) bool) shapes are allowed |
make the callback return bool |
next, _ := iter.Pull(seq) |
(no message: the iterator never cleans up) | stop was never called |
next, stop := iter.Pull(seq); defer stop() |
expecting maps.Keys in a fixed order |
(different order on each run) | map order is random | slices.Sorted(maps.Keys(m)) |
Practice
- (Homework) Build
Filter,TakeandFib(solution: the mini project). - (Homework) Remove the
if !yieldcheck from an iterator and read the panic (shown above). - (Homework) Print a map sorted by key with
slices.Sorted(maps.Keys(m)).
Solution 3
prices := map[string]int{"chai": 10, "samosa": 15}
for _, k := range slices.Sorted(maps.Keys(prices)) {
fmt.Println(k, prices[k])
}
chai 10
samosa 15
- (Homework) Zip two sequences with
iter.Pull(solution: the mini project'sZip). - Write
countdown(n int) iter.Seq[int]that yields n, n-1, ..., 1, and print3 2 1 go!.
Solution 5
func countdown(n int) iter.Seq[int] {
return func(yield func(int) bool) {
for i := n; i >= 1; i-- {
if !yield(i) {
return
}
}
}
}
for i := range countdown(3) {
fmt.Print(i, " ")
}
fmt.Println("go!")
3 2 1 go!
- Write
Enumerate[V any](seq iter.Seq[V]) iter.Seq2[int, V]that numbers the values of any sequence from 0.
Solution 6
func Enumerate[V any](seq iter.Seq[V]) iter.Seq2[int, V] {
return func(yield func(int, V) bool) {
i := 0
for v := range seq {
if !yield(i, v) {
return
}
i++
}
}
}
- Why is
slices.Collect(Fib())a bad idea?
Solution 7
Fib never stops and Collect never says false, so it runs forever, using more and more memory. Cut it first: slices.Collect(Take(Fib(), 10)).
Quick quiz
1. What does this print? (From the video.)
func evens(yield func(int) bool) {
for i := 0; ; i += 2 {
if !yield(i) {
return
}
}
}
for v := range evens {
if v > 4 { break }
fmt.Print(v, " ")
}
A. 0 2 4 · B. 0 2 4 6 · C. it never stops · D. compile error
Answer
A. At v = 6 the loop breaks, yield returns false, and evens returns. Real output: 0 2 4.
2. What does yield return when the loop body runs continue?
Answer
true: continue just moves on to the next value.
3. Which function turns a push iterator into a pull one, and what two functions does it return?
Answer
iter.Pull; it returns next (value, ok) and stop.
Interview questions
- What is an iterator in Go? Since Go 1.23, a function of the form
func(yield func(V) bool)(or with zero or two values) that produces values one at a time by callingyield.iter.Seq[V]anditer.Seq2[K, V]name these types. - How does
breakwork with range over func? The compiler turns the loop body into theyieldfunction; onbreak(orreturn),yieldreturnsfalse, and the iterator must stop and return. - What happens if an iterator ignores
false? The runtime panics: "range function continued iteration after function for loop body returned false". - Why use an iterator instead of returning a slice? No up-front work or memory, early stopping, and it works for very large or infinite sequences. A slice is still better when you need random access or the data many times.
- Push vs pull? Push: the iterator calls your code for each value (
for range). Pull: you callnext()for each value (iter.Pull), useful for walking several sequences at once. Pull iterators must be stopped withstop(). - How do you get a map's keys in sorted order?
slices.Sorted(maps.Keys(m)). - Where does an iterator do cleanup? Right after
yieldreturnsfalse(or when its loop ends), before it returns, often withdefer. - Can an iterator be infinite? Yes. Values are produced lazily, so it's safe as long as the consumer stops (with
break,Take, orstop()).
Glossary
| Term | Meaning | Example |
|---|---|---|
| Iterator | a function that produces a sequence of values one at a time | bigOrders(orders) |
| Sequence | a series of values, one after another | 74, 85, 59, ... |
| Callback | a function passed to another function, to be called for each item | forEachBig(orders, func(o int) {...}) |
yield |
the callback an iterator calls with each value; returns true (more) or false (stop) |
if !yield(o) { return } |
| Range over func | (Go 1.23) for ... range over a function of the form func(yield func(...) bool) |
for o := range bigOrders(orders) |
iter package |
(Go 1.23) standard package defining Seq, Seq2 and Pull |
import "iter" |
iter.Seq[V] |
func(yield func(V) bool): a sequence of single values |
iter.Seq[int] |
iter.Seq2[K, V] |
func(yield func(K, V) bool): a sequence of pairs |
index and item |
| Early break | leaving a range-over-func loop before the end; makes yield return false |
if n == 3 { break } |
| Infinite sequence | a sequence that never ends on its own | tokens(), Fib() |
| Lazy | values are produced only when asked for | Filter builds nothing up front |
| Pipeline | iterators chained so each passes values to the next, one at a time | Filter(orders(), keep) |
| Push iterator | the iterator calls your code with each value | for v := range seq |
| Pull iterator | your code asks for each value with next() |
next, stop := iter.Pull(seq) |
iter.Pull |
turns a push iterator into a pull iterator; returns next and stop |
merge two queues |
next |
the pull function: returns the next value and ok |
v, ok := next() |
stop |
ends a pull iteration and lets the iterator clean up | defer stop() |
| Cleanup | releasing resources (files, connections) when work is done | after yield returns false |
slices.All |
iterator of index and value over a slice | for i, v := range slices.All(menu) |
slices.Values |
iterator of the values of a slice | slices.Values(menu) |
slices.Collect |
runs an iterator and returns its values as a new slice | slices.Collect(seq) |
slices.Sorted |
collects an iterator's values and sorts them | slices.Sorted(maps.Keys(m)) |
maps.All |
iterator of key and value over a map | for k, v := range maps.All(m) |
maps.Keys |
iterator of a map's keys | maps.Keys(prices) |
maps.Values |
iterator of a map's values | maps.Values(prices) |
| Merge | combining two sorted lists into one sorted list | [1 2 3 4 9 10] |
| Function literal | a function written inline without a name (episode 10) | func(o int) bool { ... } |
| Panic | a run-time crash that stops the program (episode 26) | range function continued iteration ... |
| Generic type | a type with type parameters (episode 30) | iter.Seq[V any] |
Summary
- A filtered slice does all the work even to show three values; a plain callback can't break.
- Make the callback return a
bool: that'syield.true= more,false= stop. - Since Go 1.23 you can range over functions;
breakmakesyieldreturnfalse. iter.Seq[V]anditer.Seq2[K, V]name the iterator function types.- Always
if !yield(v) { return }, or Go panics; return is where you clean up. - Iterators are lazy and can be infinite.
slices.All,Values,Collect,Sortedandmaps.All,Keys,Valuesare ready-made iterators and helpers.- Push (range) is the default; pull (
iter.Pull→next,stop) is for when you set the pace, like merging two sequences. Always callstop.
Next episode: Standard library tour: io and bufio. One function that reads from a string, the keyboard, a file or the network, and the one forgotten line that makes your output vanish.