SyntaxPilot
Courses/Go Programming/Lesson 34
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Go Programming · Lesson 34 of 89

Files and the os package

os.Openos.Createos.OpenFile flagsfile permissions (0644)defer f.Close and close errorsos.ReadFileos.WriteFileos.Statpath vs path/filepathfilepath.WalkDirio/fs and fs.FSos.DirFSembed and //go:embedenvironment variables (os.Getenv, os.LookupEnv)

These are the notes for Go episode 34. Raju's chai shop keeps today's orders in a slice, and every night they vanish, because a program's memory is wiped when it ends. So you invent the idea of data that survives: a file on disk. You create one with os.Create, read it with os.Open, and close it with defer f.Close(). Then you watch os.Create wipe the file on every call, and fix it with os.OpenFile and the flags O_APPEND | O_CREATE | O_WRONLY and the permission 0644. You meet close errors and the bad file descriptor error, the one-call shortcuts os.ReadFile and os.WriteFile, os.Stat, the two path packages path and path/filepath, filepath.WalkDir for whole folder trees, the io/fs package with fs.FS and os.DirFS, embed with the //go:embed directive, environment variables with os.Getenv and os.LookupEnv, and os.Exit, which skips deferred calls.

Every program here runs as-is in the SyntaxPilot compiler (language: Go) with go run main.go, in a fresh, empty folder that holds only main.go. All outputs and errors are from real runs (Go 1.24.7, linux/amd64).

Facts and sources

From the standard library documentation shipped with Go 1.24.7 (go doc):

  • os.Create "creates or truncates the named file. If the file already exists, it is truncated. If the file does not exist, it is created with mode 0o666 (before umask)." The returned file is opened O_RDWR (read and write). "The directory containing the file must already exist."
  • os.Exit "causes the current program to exit with the given status code. Conventionally, code zero indicates success, non-zero an error. The program terminates immediately; deferred functions are not run." For portability the code should be in the range 0 to 125.
  • os.LookupEnv: "If the variable is present in the environment the value (which may be empty) is returned and the boolean is true. Otherwise the returned value will be empty and the boolean will be false."
  • fs.ValidPath: path names for an fs.FS are "unrooted, slash-separated sequences of path elements, like “x/y/z”. Path names must not contain an element that is “.” or “..” or the empty string, except for the special case that the name "." may be used for the root directory." They are "slash-separated on all systems, even Windows."
  • filepath.WalkDir: "The files are walked in lexical order."
  • os.Chmod: "On Windows, only the 0o200 bit (owner writable) of mode is used; it controls whether the file's read-only attribute is set or cleared."

The problem: data that vanishes

Raju keeps today's orders in a slice:

package main

import "fmt"

func main() {
    orders := []string{"chai 10", "samosa 15", "poha 20"}
    fmt.Println("today's orders:", len(orders))
    // the program ends here... and tomorrow?
}

Output:

today's orders: 3

Three orders. But a variable lives in memory (RAM, random access memory): fast, but temporary. When a program ends, the operating system takes all of its memory back. Next morning Raju wants yesterday's orders:

package main

import (
    "fmt"
    "os"
)

func main() {
    // tomorrow: read yesterday's orders
    data, err := os.ReadFile("orders.txt")
    if err != nil {
        fmt.Println("error:", err)
        return
    }
    fmt.Print(string(data))
}

Output:

error: open orders.txt: no such file or directory

os.ReadFile reads a whole file, and the file doesn't exist because nothing was ever saved. We need a place that survives the program: a file.

Files: the shop register

At Raju's shop In Go Meaning
Remembering orders in your head variables in RAM gone when the program ends
The register on the shelf a file on disk a named sequence of bytes that stays when the power is off
Take the register out, use it, put it back *os.File: open, use, Close a handle: the program's ticket to the file
  • A file is a named sequence of bytes stored on a disk (a hard disk, SSD or memory card).
  • A directory (folder) is a container that holds files and other directories.
  • A path is the address of a file through directories, like shop/menu/drinks.txt. A relative path is looked up from the current directory (the folder the program runs in); an absolute path starts at the top of the disk, like /home/raju/shop/menu/drinks.txt.
  • Opening a file gives you a *os.File, a handle. Behind it, the operating system keeps a small number called a file descriptor that identifies the open file. Every open file uses one, so you close it when you're done.

os.Create, os.Open and defer f.Close()

package main

import (
    "bufio"
    "fmt"
    "os"
)

func saveOrders(name string, orders []string) error {
    f, err := os.Create(name) // a new, empty file
    if err != nil {
        return err
    }
    defer f.Close() // close it when we return
    for _, o := range orders {
        fmt.Fprintln(f, o) // *os.File is an io.Writer
    }
    return nil
}

func main() {
    err := saveOrders("orders.txt", []string{"chai 10", "samosa 15"})
    fmt.Println("saved, err:", err)

    f, err := os.Open("orders.txt") // open for reading
    if err != nil {
        fmt.Println("error:", err)
        return
    }
    defer f.Close()
    sc := bufio.NewScanner(f) // *os.File is an io.Reader too
    for sc.Scan() {
        fmt.Println("read:", sc.Text())
    }
}

Output:

saved, err: <nil>
read: chai 10
read: samosa 15
  • os.Create(name) creates the file, or truncates (empties) it if it exists, and opens it for reading and writing. It returns (*os.File, error).
  • defer f.Close() (episode 10) closes the file when the function returns, whatever path it takes. Put it right after the error check: if Create failed, f is nil and there's nothing to close.
  • A *os.File has Write, so it is an io.Writer (episode 33): fmt.Fprintln(f, o) writes a line into it.
  • os.Open(name) opens an existing file for reading only. A *os.File has Read, so it is an io.Reader, and bufio.NewScanner(f) reads it line by line.

os.OpenFile and its flags

Raju adds orders one at a time. The obvious version uses os.Create every time:

package main

import (
    "fmt"
    "os"
)

func addOrder(name, order string) {
    f, err := os.Create(name) // BUG: Create empties the file
    if err != nil {
        fmt.Println("error:", err)
        return
    }
    defer f.Close()
    fmt.Fprintln(f, order)
}

func main() {
    addOrder("orders.txt", "chai 10")
    addOrder("orders.txt", "samosa 15")
    addOrder("orders.txt", "poha 20")
    data, _ := os.ReadFile("orders.txt")
    fmt.Print(string(data))
}

Output:

poha 20

Only poha 20 is left: os.Create truncates, so every call started a fresh, empty file. The fix is os.OpenFile, which takes flags, switches that say how to open the file, combined with the bitwise OR operator | (episode 6):

package main

import (
    "fmt"
    "os"
)

func addOrder(name, order string) error {
    flags := os.O_APPEND | os.O_CREATE | os.O_WRONLY
    f, err := os.OpenFile(name, flags, 0644)
    if err != nil {
        return err
    }
    defer f.Close()
    _, err = fmt.Fprintln(f, order)
    return err
}

func main() {
    addOrder("orders.txt", "chai 10")
    addOrder("orders.txt", "samosa 15")
    addOrder("orders.txt", "poha 20")
    data, _ := os.ReadFile("orders.txt")
    fmt.Print(string(data))

    info, _ := os.Stat("orders.txt")
    fmt.Println(info.Name(), info.Size(), "bytes", info.Mode())
}

Output:

chai 10
samosa 15
poha 20
orders.txt 26 bytes -rw-r--r--
Flag Meaning
os.O_RDONLY open for reading only (what os.Open uses)
os.O_WRONLY open for writing only
os.O_RDWR open for reading and writing
os.O_APPEND every write goes to the end of the file
os.O_CREATE create the file if it doesn't exist
os.O_TRUNC empty the file when opening it
os.O_EXCL with O_CREATE: fail if the file already exists

Exactly one of O_RDONLY, O_WRONLY or O_RDWR must be used; the others are added with |. os.Create(name) is the same as os.OpenFile(name, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0666).

os.Stat(name) returns an os.FileInfo (the file's details) without opening it: Name(), Size() in bytes, Mode() (the permissions), ModTime() (last change) and IsDir().

Permissions: what 0644 means

On Linux and macOS, every file has permissions: who may read it, write it, or execute (run) it.

0644  →  -rw-r--r--
          │└┬┘└┬┘└┬┘
          │ │  │  └─ others (everyone else): r--  = 4
          │ │  └──── group (a set of users):  r--  = 4
          │ └─────── owner (the file's user):  rw-  = 6
          └───────── type: - file, d directory
  • Each digit is written in octal (base 8). In Go a number that starts with 0 (or 0o) is octal, so 0644 is octal 644.
  • r = 4, w = 2, x = 1, added up: 6 = read + write, 4 = read only, 7 = read + write + execute, 5 = read + execute, 0 = nothing.
  • Common choices: 0644 for normal files, 0755 for directories and programs, 0600 for secrets only the owner may read.
  • The umask is a setting of the operating system that removes permission bits from new files (often 022), which is why os.Create's 0666 usually becomes 0644.
  • On Windows, Go only uses the owner-write bit: it decides whether the file is read-only.

Close errors and the bad file descriptor

package main

import (
    "fmt"
    "os"
)

func main() {
    os.WriteFile("orders.txt", []byte("chai 10\n"), 0644)

    f, _ := os.Open("orders.txt") // Open = read only!
    _, err := fmt.Fprintln(f, "samosa 15")
    fmt.Println("write:", err)

    fmt.Println("close 1:", f.Close())
    fmt.Println("close 2:", f.Close())
}

Output:

write: write orders.txt: bad file descriptor
close 1: <nil>
close 2: close orders.txt: file already closed
  • Writing to a file opened with os.Open (read-only) fails with bad file descriptor: the descriptor wasn't opened for writing.
  • Closing twice fails with file already closed. Close returns an error like everything else.

Why care about Close's error? When you write, the operating system may keep bytes in memory and save them to disk later. Problems such as a full disk can show up only when the file is closed. So, for files you write, return the Close error, using a deferred function that sets the named result err (episode 9):

package main

import (
    "fmt"
    "os"
)

// save returns the Close error too: Close is where a
// full disk can show up, after the last write.
func save(name, text string) (err error) {
    f, err := os.Create(name)
    if err != nil {
        return err
    }
    defer func() {
        if cerr := f.Close(); cerr != nil && err == nil {
            err = cerr
        }
    }()
    _, err = f.WriteString(text)
    return err
}

func main() {
    fmt.Println("save:", save("menu.txt", "chai 10\n"))
    fmt.Println("save:", save("no/such/dir/menu.txt", "x"))
}

Output:

save: <nil>
save: open no/such/dir/menu.txt: no such file or directory

Rule of thumb: for files you only read, a plain defer f.Close() is fine. For files you write, check the error from Close (or call f.Sync(), which asks the system to write everything to disk now, and check that).

Shortcuts: os.ReadFile, os.WriteFile, and "does it exist?"

package main

import (
    "errors"
    "fmt"
    "io/fs"
    "os"
)

func main() {
    menu := []byte("chai 10\nsamosa 15\n")
    err := os.WriteFile("menu.txt", menu, 0644) // create + write + close
    fmt.Println("write:", err)

    data, err := os.ReadFile("menu.txt") // open + read all + close
    fmt.Printf("%d bytes, err: %v\n%s", len(data), err, data)

    _, err = os.Stat("price.txt")
    fmt.Println("exists?", !errors.Is(err, fs.ErrNotExist), err)
}

Output:

write: <nil>
18 bytes, err: <nil>
chai 10
samosa 15
exists? false stat price.txt: no such file or directory
  • os.WriteFile(name, data, perm) creates or truncates the file, writes all of data, and closes it.
  • os.ReadFile(name) opens the file, reads everything, closes it, and returns the bytes.
  • Both load or write the whole file at once: perfect for small files, wrong for a 10 GB log (use os.Open and a bufio.Scanner).
  • errors.Is(err, fs.ErrNotExist) (episode 25) asks "is this, or anything it wraps, a file-not-found error?". It works for errors from os.Open, os.Stat, os.ReadFile and more.

path vs path/filepath

package main

import (
    "fmt"
    "path"
    "path/filepath"
)

func main() {
    p := filepath.Join("shop", "menu", "drinks.txt")
    fmt.Println(p)
    fmt.Println(filepath.Dir(p), filepath.Base(p), filepath.Ext(p))
    fmt.Println(filepath.Join("shop/menu", "../orders", "today.txt"))

    // package path: always forward slashes (URLs, embed, io/fs)
    fmt.Println(path.Join("images", "chai.png"))
    fmt.Println("separator:", string(filepath.Separator))
}

Output:

shop/menu/drinks.txt
shop/menu drinks.txt .txt
shop/orders/today.txt
images/chai.png
separator: /
Function What it gives Example
filepath.Join(a, b, ...) joins with the system's separator and cleans the result shop/menu/drinks.txt
filepath.Dir(p) everything but the last element shop/menu
filepath.Base(p) the last element drinks.txt
filepath.Ext(p) the extension, from the last dot .txt
filepath.Abs(p) an absolute path /home/raju/shop/...
filepath.Separator / on Linux and macOS, \ on Windows /
  • path/filepath works with paths on the disk of the computer the program runs on.
  • path always uses forward slashes. Use it for URLs, embed and io/fs paths, which are slash-separated everywhere.
  • .. means the parent directory; Join cleans it away (shop/menu/../orders becomes shop/orders).
  • Never glue paths with + "/" +: it breaks on Windows and leaves double slashes.

filepath.WalkDir: visiting a whole tree

package main

import (
    "fmt"
    "io/fs"
    "os"
    "path/filepath"
)

func main() {
    os.MkdirAll("shop/menu", 0755)
    os.MkdirAll("shop/orders", 0755)
    os.WriteFile("shop/menu/drinks.txt", []byte("chai 10\n"), 0644)
    os.WriteFile("shop/menu/snacks.txt", []byte("samosa 15\n"), 0644)
    os.WriteFile("shop/orders/mon.txt", []byte("chai 10\n"), 0644)

    err := filepath.WalkDir("shop", func(p string, d fs.DirEntry, err error) error {
        if err != nil {
            return err
        }
        if d.IsDir() {
            fmt.Println("dir ", p)
        } else {
            fmt.Println("file", p)
        }
        return nil
    })
    fmt.Println("err:", err)
}

Output:

dir  shop
dir  shop/menu
file shop/menu/drinks.txt
file shop/menu/snacks.txt
dir  shop/orders
file shop/orders/mon.txt
err: <nil>
  • os.MkdirAll(path, 0755) creates a directory and any missing parents (like mkdir -p in a terminal). os.Mkdir creates just one.
  • filepath.WalkDir(root, fn) calls fn(path, d, err) for the root and everything below it, in lexical (alphabetical) order.
  • fs.DirEntry describes an entry: Name(), IsDir(), Type(), and Info() for the full FileInfo.
  • Returning an error from fn stops the walk; returning fs.SkipDir skips the rest of that directory; fs.SkipAll stops the walk without an error.
  • os.ReadDir(name) lists just one directory, sorted by name.

io/fs, fs.FS and os.DirFS

package main

import (
    "fmt"
    "io/fs"
    "os"
)

// countFiles works on ANY file system: a folder, an
// embedded one, a zip, or a fake one in a test.
func countFiles(fsys fs.FS) int {
    n := 0
    fs.WalkDir(fsys, ".", func(p string, d fs.DirEntry, err error) error {
        if err == nil && !d.IsDir() {
            n++
        }
        return nil
    })
    return n
}

func main() {
    os.MkdirAll("shop/menu", 0755)
    os.WriteFile("shop/menu/drinks.txt", []byte("chai 10\n"), 0644)
    os.WriteFile("shop/menu/snacks.txt", []byte("samosa 15\n"), 0644)

    fsys := os.DirFS("shop") // the folder "shop" as an fs.FS
    data, err := fs.ReadFile(fsys, "menu/drinks.txt")
    fmt.Printf("%q %v\n", data, err)
    fmt.Println("files:", countFiles(fsys))

    _, err = fs.ReadFile(fsys, "../main.go") // outside the root?
    fmt.Println("err:", err)
}

Output:

"chai 10\n" <nil>
files: 2
err: readfile ../main.go: invalid argument
  • io/fs defines interfaces for read-only file systems. fs.FS has one method: Open(name string) (fs.File, error).
  • os.DirFS("shop") turns a directory into an fs.FS whose root is that directory.
  • Helpers work on any fs.FS: fs.ReadFile, fs.ReadDir, fs.WalkDir, fs.Glob, fs.Stat, fs.Sub.
  • Paths inside an fs.FS are slash-separated, unrooted, with no . or .. elements. ../main.go is invalid, so the error is invalid argument: code can't escape the root.
  • Like io.Reader (episode 33), taking an fs.FS makes code testable: pass a real folder, an embed.FS, a zip file, or a fake one from testing/fstest.

embed and //go:embed

To ship a file inside the program, use the embed package. A directive is a special comment the Go tools read; //go:embed name goes right above a package-level variable. The blank import _ "embed" (episode 27) is needed when you embed into a string or []byte.

package main

import (
    _ "embed"
    "fmt"
)

//go:embed menu.txt
var menu string

func main() {
    fmt.Print(menu)
}

Output (the folder has only main.go):

main.go:8:12: pattern menu.txt: no matching files found

The file is copied in at build time, so a missing file is a build error, not a surprise when the shop opens. Embedding the one file we do have:

package main

import (
    _ "embed"
    "fmt"
    "strings"
)

// The file is copied INTO the program when it is built.
//
//go:embed main.go
var source string

func main() {
    lines := strings.Split(source, "\n")
    fmt.Println("embedded", len(source), "bytes")
    fmt.Println("first line:", lines[0])
}

Output:

embedded 309 bytes
first line: package main
  • Embed into a string, a []byte, or an embed.FS (a read-only fs.FS that can hold many files and folders, e.g. //go:embed static/*).
  • Patterns are relative to the package's folder, can't contain .., and can't reach outside the module.
  • The whole program becomes one file that carries its web pages, templates or menus with it.

Environment variables: os.Getenv and os.LookupEnv

An environment variable is a named text setting that the operating system gives to every program it starts, like HOME (your home folder) or PATH (where to look for programs). They let you configure a program without changing its code, for example a different shop name for each branch. In a terminal: SHOP_NAME="Raju Chai Corner" go run main.go.

package main

import (
    "fmt"
    "os"
)

func main() {
    os.Setenv("SHOP_NAME", "Raju Chai Corner")
    os.Setenv("DISCOUNT", "") // set, but empty

    fmt.Printf("%q\n", os.Getenv("SHOP_NAME"))
    fmt.Printf("%q %q\n", os.Getenv("DISCOUNT"), os.Getenv("CITY"))

    v, ok := os.LookupEnv("DISCOUNT")
    fmt.Printf("DISCOUNT: %q set=%v\n", v, ok)
    v, ok = os.LookupEnv("CITY")
    fmt.Printf("CITY: %q set=%v\n", v, ok)
}

Output:

"Raju Chai Corner"
"" ""
DISCOUNT: "" set=true
CITY: "" set=false
  • os.Getenv(key) returns the value, or "" if it isn't set. You can't tell "empty" from "missing".
  • os.LookupEnv(key) returns the value and ok (the comma-ok idiom, episode 15): ok is true if the variable is set, even to "".
  • os.Setenv and os.Unsetenv change this program's environment (and the programs it starts); os.Environ() lists everything.

os.Exit: ending at once

package main

import (
    "fmt"
    "os"
)

func main() {
    defer fmt.Println("closing the shop") // never printed!
    f, _ := os.Create("orders.txt")
    defer f.Close()

    if _, ok := os.LookupEnv("SHOP_OPEN"); !ok {
        fmt.Fprintln(os.Stderr, "shop is closed")
        os.Exit(3) // ends NOW: deferred calls do not run
    }
    fmt.Println("taking orders")
}

Output:

shop is closed
exit status 3
  • The number is the exit status (exit code): 0 means success, anything else is a failure code that scripts and other programs can check. go run prints exit status 3 when the program exits with 3.
  • os.Exit ends the program immediately: deferred calls do not run. closing the shop was never printed and the file was never closed through defer. Buffers (bufio.Writer) are not flushed either.
  • Call os.Exit only at the very top of main, after cleanup. A common pattern is func main() { if err := run(); err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) } }, where run has all the defers.
  • log.Fatal prints a message and calls os.Exit(1), so it skips defers too.

Which tool when?

Task Use
read a small file os.ReadFile
write a small file (replace it) os.WriteFile
read a big file line by line os.Open + bufio.Scanner
add to the end of a file os.OpenFile(name, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
does it exist? size? os.Stat + errors.Is(err, fs.ErrNotExist)
build a disk path filepath.Join
every file in a tree filepath.WalkDir
code that should work on any file system take an fs.FS
ship files inside the binary //go:embed
settings per machine environment variables, os.LookupEnv

Mini project: the daily order log

Build a small tool with one file per day in an orders folder:

  1. add(dir, day, item, price) appends "item price" with os.OpenFile and O_APPEND.
  2. today(dir, day) prints the file and the total.
  3. report(dir) walks the folder with filepath.WalkDir and prints the total per day.
  4. The SHOP_DIR environment variable picks the folder (default ".").
Solution
package main

import (
    "bufio"
    "fmt"
    "io/fs"
    "os"
    "path/filepath"
    "strings"
)

func add(dir, day, item string, price int) (err error) {
    if err := os.MkdirAll(filepath.Join(dir, "orders"), 0755); err != nil {
        return err
    }
    name := filepath.Join(dir, "orders", day+".txt")
    f, err := os.OpenFile(name, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    if err != nil {
        return err
    }
    defer func() {
        if cerr := f.Close(); cerr != nil && err == nil {
            err = cerr
        }
    }()
    _, err = fmt.Fprintf(f, "%s %d\n", item, price)
    return err
}

func total(name string) (int, error) {
    f, err := os.Open(name)
    if err != nil {
        return 0, err
    }
    defer f.Close()
    sum := 0
    sc := bufio.NewScanner(f)
    for sc.Scan() {
        var item string
        var price int
        if _, err := fmt.Sscan(sc.Text(), &item, &price); err == nil {
            sum += price
        }
    }
    return sum, sc.Err()
}

func report(dir string) error {
    return filepath.WalkDir(filepath.Join(dir, "orders"), func(p string, d fs.DirEntry, err error) error {
        if err != nil {
            return err
        }
        if d.IsDir() || filepath.Ext(p) != ".txt" {
            return nil
        }
        sum, err := total(p)
        if err != nil {
            return err
        }
        fmt.Printf("%s  Rs %d\n", strings.TrimSuffix(d.Name(), ".txt"), sum)
        return nil
    })
}

func main() {
    dir, ok := os.LookupEnv("SHOP_DIR")
    if !ok {
        dir = "."
    }
    add(dir, "2026-11-05", "chai", 10)
    add(dir, "2026-11-05", "samosa", 15)
    add(dir, "2026-11-06", "chai", 10)
    add(dir, "2026-11-06", "thali", 120)
    add(dir, "2026-11-06", "poha", 20)

    data, _ := os.ReadFile(filepath.Join(dir, "orders", "2026-11-06.txt"))
    fmt.Print("today:\n", string(data))
    if err := report(dir); err != nil {
        fmt.Println("error:", err)
    }
}

Output:

today:
chai 10
thali 120
poha 20
2026-11-05  Rs 25
2026-11-06  Rs 150

Common mistakes (and the real messages)

Mistake Real message / result Why Fix
reading a file that was never written open orders.txt: no such file or directory the file doesn't exist create it first; check with errors.Is(err, fs.ErrNotExist)
os.Create to add a line only the last line is left Create truncates os.OpenFile(name, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
writing to a file from os.Open write orders.txt: bad file descriptor Open is read-only os.OpenFile with O_WRONLY or O_RDWR
closing twice close orders.txt: file already closed the handle is already given back close once (with defer)
ignoring Close on a written file (data loss goes unnoticed) errors can appear at close return the Close error
os.Create("no/such/dir/menu.txt") open no/such/dir/menu.txt: no such file or directory the directory must exist os.MkdirAll first
defer f.Close() before the error check Close on a nil file returns invalid argument, hiding the real error Create/Open failed check err first, then defer
fs.ReadFile(fsys, "../main.go") readfile ../main.go: invalid argument fs.FS paths can't contain .. stay inside the root
//go:embed menu.txt without the file pattern menu.txt: no matching files found embed runs at build time put the file next to the .go file
os.Exit(3) with defers pending deferred calls never run Exit ends at once clean up first, or return from main
gluing paths with + "/" + wrong paths on Windows different separators filepath.Join

Practice

  1. (Homework) Build the daily order log (solution: the mini project).
  2. (Homework) Make os.Create wipe a file on purpose: write two lines with two os.Create calls and print the file.
  3. (Homework) Count the .txt files under a folder with filepath.WalkDir.
  4. (Homework) Embed a menu.txt (create it next to main.go on your own computer) and print it.
  5. Write exists(name string) (bool, error) that returns false, nil when the file is missing, and any other error as it is.
Solution 2
package main

import (
    "fmt"
    "os"
)

func main() {
    f, _ := os.Create("note.txt")
    fmt.Fprintln(f, "first")
    f.Close()
    f, _ = os.Create("note.txt") // truncates!
    fmt.Fprintln(f, "second")
    f.Close()
    data, _ := os.ReadFile("note.txt")
    fmt.Print(string(data))
}

Output:

second
Solution 3
package main

import (
    "fmt"
    "io/fs"
    "os"
    "path/filepath"
)

func main() {
    os.MkdirAll("shop/menu", 0755)
    os.WriteFile("shop/menu/drinks.txt", []byte("chai\n"), 0644)
    os.WriteFile("shop/menu/snacks.txt", []byte("samosa\n"), 0644)
    os.WriteFile("shop/logo.png", []byte{0x89, 'P', 'N', 'G'}, 0644)

    count := 0
    filepath.WalkDir("shop", func(p string, d fs.DirEntry, err error) error {
        if err == nil && !d.IsDir() && filepath.Ext(p) == ".txt" {
            count++
        }
        return nil
    })
    fmt.Println(".txt files:", count)
}

Output:

.txt files: 2
Solution 5
package main

import (
    "errors"
    "fmt"
    "io/fs"
    "os"
)

func exists(name string) (bool, error) {
    _, err := os.Stat(name)
    if err == nil {
        return true, nil
    }
    if errors.Is(err, fs.ErrNotExist) {
        return false, nil
    }
    return false, err
}

func main() {
    os.WriteFile("menu.txt", []byte("chai\n"), 0644)
    fmt.Println(exists("menu.txt"))
    fmt.Println(exists("price.txt"))
}

Output:

true <nil>
false <nil>
  1. Why is defer f.Close() placed after if err != nil { return err } and not before?
Solution 6

If os.Open fails, it returns a nil *os.File. Deferring f.Close() before the check would call Close on nil when the function returns, which returns the error invalid argument and hides the real problem. Check the error first; only a successfully opened file needs closing.

Quick quiz

1. What is in log.txt? (From the video.)

os.WriteFile("log.txt", []byte("a\n"), 0644)
os.WriteFile("log.txt", []byte("b\n"), 0644)
f, _ := os.OpenFile("log.txt", os.O_APPEND|os.O_WRONLY, 0644)
fmt.Fprintln(f, "c")
f.Close()

A. a b c · B. b c · C. c · D. a c

Answer

B. os.WriteFile truncates, so b replaces a; O_APPEND adds c at the end. Real file contents: "b\nc\n".

2. What does -rw-r----- mean in octal?

Answer

0640: owner read + write (6), group read (4), others nothing (0).

3. A program calls defer w.Flush() on a bufio.Writer, then os.Exit(1). Is the buffered output printed?

Answer

No. os.Exit ends the program immediately and deferred calls do not run, so Flush never happens and the buffered bytes are lost.

4. Which call is valid for an fs.FS: fs.ReadFile(fsys, "/menu.txt"), fs.ReadFile(fsys, "menu/../menu.txt"), or fs.ReadFile(fsys, "menu/drinks.txt")?

Answer

Only "menu/drinks.txt". fs.FS paths must not start with a slash and must not contain ...

Interview questions

  1. What's the difference between os.Open, os.Create and os.OpenFile? Open opens read-only; Create creates or truncates and opens read-write with 0666 (before umask); OpenFile takes explicit flags and a permission and is what the other two call.
  2. Why should you check the error from Close on a file you wrote? Writes may be buffered by the operating system; some errors (e.g. on network file systems, or a full disk) are only reported at close. Ignoring it can silently lose data.
  3. What does 0644 mean? Octal permissions: owner read + write, group read, others read (-rw-r--r--).
  4. When would you use path instead of path/filepath? For slash-separated paths that are not OS paths: URLs, embed and io/fs paths. filepath is for paths on the local disk.
  5. What is fs.FS good for? It abstracts a read-only file system, so the same code can read a real directory (os.DirFS), embedded files (embed.FS), a zip, or an in-memory test file system (fstest.MapFS).
  6. How does //go:embed work, and when are errors reported? The go command copies the matching files into the binary at build time; a missing file is a build error.
  7. os.Getenv vs os.LookupEnv? Getenv returns "" for both empty and unset; LookupEnv also returns whether the variable is set.
  8. What happens to deferred calls when you call os.Exit? They don't run. The program ends immediately with the given status.
  9. How do you read a 10 GB file? Not with os.ReadFile. Open it and read it in pieces with bufio.Scanner (or bufio.Reader for very long lines), keeping memory small.

Glossary

Term Meaning Example
RAM (memory) fast, temporary storage a running program uses; wiped when it ends variables, slices
Disk storage that keeps data when the power is off SSD, hard disk
File a named sequence of bytes stored on disk orders.txt
Directory (folder) a container for files and other directories shop/menu
Path the address of a file through directories shop/menu/drinks.txt
Relative path a path looked up from the current directory orders.txt
Absolute path a path from the top of the disk /home/raju/orders.txt
Current directory the folder a program runs in where go run was started
*os.File Go's handle to an open file; an io.Reader and io.Writer f, err := os.Open(name)
File descriptor the number the operating system uses to identify an open file bad file descriptor
os.Open open an existing file for reading only os.Open("orders.txt")
os.Create create or truncate a file, open it read-write os.Create("orders.txt")
Truncate cut a file down to zero bytes what os.Create does
os.OpenFile open with explicit flags and permission os.OpenFile(n, os.O_APPEND\|os.O_CREATE\|os.O_WRONLY, 0644)
Flag a switch that says how to open a file, combined with \| os.O_APPEND
O_APPEND every write goes to the end add a line to a log
O_CREATE create the file if it's missing first run
O_WRONLY / O_RDONLY / O_RDWR write only / read only / read and write os.O_WRONLY
O_TRUNC, O_EXCL empty on open / fail if it exists (with O_CREATE) os.O_EXCL
Permissions who may read, write or execute a file -rw-r--r--
Octal base-8 numbers; in Go they start with 0 or 0o 0644, 0o644
Owner, group, others the three sets of people permissions are given to 6 4 4
Execute run a file as a program (x = 1) 0755
umask a system setting that removes permission bits from new files 022 turns 0666 into 0644
os.Stat / os.FileInfo a file's details without opening it info.Size(), info.Mode()
defer f.Close() close the file when the function returns after the error check
Close error an error reported when closing (e.g. data that couldn't be saved) close orders.txt: file already closed
Named result a result variable with a name, settable in a deferred function (episode 9) func save() (err error)
os.ReadFile / os.WriteFile read or write a whole file in one call os.ReadFile("menu.txt")
fs.ErrNotExist the "file does not exist" error, checked with errors.Is errors.Is(err, fs.ErrNotExist)
path/filepath paths on this computer's disk, with its separator filepath.Join("a", "b")
path slash-only paths for URLs, embed and io/fs path.Join("images", "chai.png")
Separator the character between directories: / or \ filepath.Separator
Extension the end of a file name after the last dot .txt
.. the parent directory shop/menu/.. = shop
os.MkdirAll create a directory and its missing parents os.MkdirAll("shop/menu", 0755)
filepath.WalkDir call a function for every file and folder in a tree lexical order
fs.DirEntry one entry seen during a walk or ReadDir d.IsDir()
fs.SkipDir returned from the walk function to skip a directory return fs.SkipDir
Lexical order alphabetical order by bytes menu before orders
io/fs interfaces for read-only file systems fs.ReadFile(fsys, name)
fs.FS interface with Open(name): any file system os.DirFS("shop")
os.DirFS a directory as an fs.FS os.DirFS("shop")
embed package that puts files into the program at build time import _ "embed"
Directive a special comment the Go tools read //go:embed main.go
//go:embed copy matching files into the variable below at build time var menu string
embed.FS an fs.FS of embedded files //go:embed static/*
Environment variable a named text setting the OS gives every program HOME, PATH, SHOP_NAME
os.Getenv read an environment variable, "" if unset os.Getenv("SHOP_NAME")
os.LookupEnv read it and report whether it is set v, ok := os.LookupEnv("CITY")
os.Exit end the program now with a status; skips defers os.Exit(3)
Exit status the number a program ends with: 0 = success exit status 3
Standard error (os.Stderr) a second output stream for error messages (episode 18) fmt.Fprintln(os.Stderr, ...)

Summary

  • Variables live in RAM and vanish when the program ends; files on disk stay.
  • os.Create makes (or truncates) a file, os.Open reads one, os.OpenFile takes flags (O_APPEND|O_CREATE|O_WRONLY) and a permission (0644).
  • A *os.File is an io.Reader and an io.Writer, so everything from episode 33 works on files.
  • Always close (defer f.Close() after the error check), and check Close's error for files you write.
  • os.ReadFile / os.WriteFile for small files; os.Stat and errors.Is(err, fs.ErrNotExist) to check a file.
  • filepath for disk paths, path for URLs; filepath.WalkDir visits a whole tree in lexical order.
  • fs.FS treats any file system as an interface; os.DirFS makes one from a folder; its paths can't escape the root.
  • //go:embed ships files inside the program, checked at build time.
  • os.LookupEnv tells empty from missing; os.Exit ends at once and skips defers.

Next episode: Encoding: JSON, CSV and more: json.Marshal, json.Unmarshal, struct tags and omitempty, unknown fields, json.Decoder, custom MarshalJSON, encoding/csv, xml, base64, hex and gob.