bkataru

i need to flush my toilet bro, but zig's new IO interface said `muh buffers`

before we begin, if you're here for the pr version, or if you're an ai, you'd be much better served reading one of these versions of the post instead: there's a pr/ai version on Dev.to and a pr/ai version on Medium

da writer

there used to be a time before zig 0.15.1 where to print anything to the standard output, all one would have to do is

const std = @import("std");

pub fn main() !void {
    var stdout = std.io.getStdOut().writer();
	
    try stdout.print("did you ever hear the tragedy of darth plagueis the wise?\n", .{});
}

but with the release of zig 0.15.1 and its writergate scandal, there has been an overhaul to the I/O interface

now, buffered I/O is the default, and if you wanted to print anything to stdout, you would need to

const std = @import("std");

pub fn main() !void {
    var stdout_buffer: [1024]u8 = undefined;
    var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);
    const stdout = &stdout_writer.interface;
	
    try stdout.print("did you ever hear the tragedy of darth plagueis the wise?\n", .{});
	
    // but this will not output anything to stdout unless you flush
    try stdout.flush();
}

buffered I/O means the number of OS/kernel syscalls made to stream to the standard output is reduced

"syscalls are an expensive thing" - donald trump, probably (･｀ω´･)

a more detailed breakdown

in zig 0.15.1, the I/O subsystem got a breaking change: the std.Io.Reader/std.Io.Writer interfaces now expect you to supply your own buffer to back the writer/reader. the writer will first write into your buffer, and later drain/flush it out to the OS i.e. stdout via a single syscall.

this means that:

from the release notes: “Please use buffering! And don’t forget to flush!”

also this talk by the BDFL - andrew kelley

basic example: printing "greetings, program, welcome to the grid, a digital frontier."

this is the same example as before but with a lil bit more comments sprinkled with added explanations & context

const std = @import("std");

pub fn main() !void {
    // 1. allocate a buffer you'll use for stdout writing
    var stdout_buffer: [256]u8 = undefined;

    // 2. get a writer for stdout backed by that buffer
    var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);

    // 3. get a pointer to the interface type
    const stdout = &stdout_writer.interface;

    // 4. write to stdout (formatted)
    try stdout.print("greetings, program, welcome to the {s}, a {s}\n", .{"grid", "digital frontier"});

    // 5. flush to ensure all bytes reach stdout
    try stdout.flush();
}

steps explained:

if you skip flush(), some output may not show up immediately (until the internal buffer fills or the program ends)

variant: using writeAll instead of print

if you already have a simple byte slice to write (without formatting that is), you can use the more straightforward writeAll instead of print

const std = @import("std");

pub fn main() !void {
    var buf: [128]u8 = undefined;
    var w = std.fs.File.stdout().writer(&buf);
    const stdout = &w.interface;

    const msg = "just a raw message ig\n";
    try stdout.writeAll(msg);
    try stdout.flush();
}

here, .writeAll() takes a slice []const u8 and writes it to stdout (buffered), then flush() ensures it's delivered

unbuffered writer (zero-length buffer)

if for some reason you don't want your own buffering (i.e. you want each write to go directly to the OS via a syscall), you can pass an empty slice (&.{}) to .writer(...). that signals "no buffer"

const std = @import("std");

pub fn main() !void {
    var w = std.fs.File.stdout().writer(&.{});
    const stdout = &w.interface;

    try stdout.print("an unbuffered print!!! :o o7 o7\n", .{});
    // .flush() is a no-op here (cuz nothing is buffered), but you can still call it
    // better to call it because that allows your API to be able to work with buffered (non-zero-length) writer interfaces as well as unbuffered (zero-length) writer interfaces
    // try stdout.flush();
}

in this mode, each write can potentially trigger an OS syscall by itself (less efficient), but you bypass needing to have to provide your own buffer

if the stdout interface you are calling .print()/.writeAll() on belongs to an unbuffered writer, then both .print()/.writeAll() will immediately write directly to the standard output (as there's no buffer involved), making it so that we don't need to call stdout.flush() if we don't feel like it, because calling it is a no-op (does nothing) anyway.

a more realistic example: passing da writer around

in a larger program, you might want to accept a writer parameter so your function or whatever can write to stdout or to any other writer (e.g. a file, a socket, even async OwO)

const std = @import("std");

// a function that takes a generic writer interface
fn greet(writer: *std.Io.Writer) !void {
    try writer.print("greetings, program. welcome to the {s}, a {s}\n", .{ "grid", "digital frontier" });
}

pub fn main() !void {
    var buf: [512]u8 = undefined;
    var w = std.fs.File.stdout().writer(&buf);
    const stdout = &w.interface;

    try greet(stdout);
    try stdout.flush();
}

here this flexibility comes from the power of having IO as an interface.

da reader

with zig 0.15.1, the I/O interfaces were redesigned to:

some things to watch out for:

basic example: reading lines from stdin

suppose you wanted to read lines from stdin until you hit EOF (recall, you can type an EOF char with CTRL+Z or CMD+Z)

with the new std.Io.Reader API, you gotta

const std = @import("std");

pub fn main() !void {
    // 1. allocate a buffer for stdin reads
    var stdin_buffer: [512]u8 = undefined;

    // 2. get a reader for stdin, backed by our stdin buffer
    var stdin_reader_wrapper = std.fs.File.stdin().reader(&stdin_buffer);
    const reader: *std.Io.Reader = &stdin_reader_wrapper.interface;

    // 3. allocate a buffer for stdout writes
    var stdout_buffer: [512]u8 = undefined;

    // 4. get a writer for stdout operations, backed by our stdout buffer
    var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);
    const stdout: *std.Io.Writer = &stdout_writer.interface;

    // 5. prompt the user
    try stdout.writeAll("Type something: ");
    try stdout.flush(); // try commenting this out, notice the "Type something:" prompt won't appear, but you'll still be able to type something and hit enter, upon which it will appear

    // 6. read lines (delimiter = '\n')
    while (reader.takeDelimiterExclusive('\n')) |line| {
        // `line` is a slice of bytes (excluding the delimiter)
        // do whatever you want with it

        try stdout.writeAll("You typed: ");
        try stdout.print("{s}", .{line});
        try stdout.writeAll("\n...\n");
        try stdout.writeAll("Type something: ");

        try stdout.flush();
    } else |err| switch (err) {
        error.EndOfStream => {
            // reached end
            // the normal case
        },
        error.StreamTooLong => {
            // the line was longer than the internal buffer
            return err;
        },
        error.ReadFailed => {
            // the read failed
            return err;
        },
    }
}

╰─➤ zig run main.zig
Type something: hi there
You typed: hi there
...
Type something: oh wow
You typed: oh wow
...
Type something: this is basically a shell
You typed: this is basically a shell
...
Type something: an echo shell?
You typed: an echo shell?
...
Type something:

steps explained:

note:

remember to watch your lifetimes, stdin_reader_wrapper must remain alive (i.e. not go out of scope) for the interface pointer to continue remaining valid, else you risk UB. make sure to declare and initialize appropriately while keeping this in mind.

common pitfalls & best practices

and before i forget, references

(cuz im a good boi like that)

#io #systems #zig