The last post shared a technique for making a Linux machine automatically sleep and then wake up on demand. At the time, I felt I'd found something useful for my own purposes, but wasn't sure if it would arouse any interest beyond my own. After I posted it to Hacker News, the post ended up garnering attention and discussion, seeming to show that this is a common problem looking for a simple solution.

Reading through those discussions, there were a few common themes in feedback:

• Lack of IPv6 support: the post suggested entirely disabling IPv6 on the server, which seemed excessive.
• Complicated installation: the post suggested installing a Ruby service called ARP Stand-in to respond to ARP requests on behalf of the sleeping server. Doing so wasn't straightforward due to differences in Ruby environments across machines (Ruby version, gem installation path, etc.) and required manually setting up a systemd service.
• Resource requirements: Being written in Ruby, ARP Stand-in was not ideally suited to resource-constrained devices like network routers.

New network service

I decided to address the above weaknesses with a rewrite of ARP Stand-in in Zig, a low-level, compiled language. The new program is called Friendly Neighbor and achieves the following improvements:

• Full support for IPv6 using Neighbor Discovery Protocol (NDP) in addition to ARP.
• Simple and fast installation using the provided Snap package, which takes care of running the service as a daemon.
• Lightweight and performant, with a 10x reduction in memory (RAM) usage compared to ARP Stand-in. Below is the output of pmap -x [pid] for the old service and the new one:

Copycopy code to clipboard
Address           Kbytes     RSS   Dirty
========================================
ARP Stand-in:
total kB           87828   20468   13024
========================================
Friendly Neighbor:
total kB            8532    2600     324

New documentation and forum

I also copied the instructions from the previous blog post into documentation stored in the new repository and hosted with Github Pages. This way, the instructions can continue to evolve independently with version-controlled history, and are also open for pull requests from anyone interested to contribute. The instructions have been updated to incorporate Friendly Neighbor and are now simpler than before, thanks to IPv6 support and quicker setup of the network service. Check out the latest documentation.

The new repository serves as a dedicated space to discuss issues with the network service itself (using Github Issues), but also with the larger topics of sleep-on-idle and wake-on-demand for Linux (using Github Discussions).

Design decisions

Hobby projects are different from professional projects. Design decisions in professional projects are driven by business constraints, organizational considerations (e.g. technologies/languages already used by the team), available infrastructure, and the like. For me, hobby projects are different: they're driven by wherever my interest happens to wander, whatever happens to be firing my curiosity at the moment. I try to channel that curious, excited energy into something that will have some practical value at least for myself, and ideally for others too.

Using a compiled language

The original service (ARP Stand-in) was written in Ruby, an interpreted language. I chose Ruby at the time because I was familiar with it and because it has ready-to-use libraries for network packet capture and manipulation. That choice proved to be great for quickly prototyping, but led to two of the main weaknesses mentioned above: high resource consumption and complicated installation.

I had recently read Just For Fun, the story of how Linus Tovalds wrote the early versions of Linux. I enjoyed the book overall, and a few themes sparked my interest in particular. One thing that stood out was Tovalds' fluency in assembly language and ease with programming the hardware directly, without any abstractions in between. This idea really struck me, having myself worked much of my career in web development, abstracted many levels away from the hardware. I found myself craving that same kind of bare-metal programming, the same experience of complete control over the computer (and the attendant risk of completely messing it up). In my spare time, I found myself poking around reference manuals for the 80386, asking ChatGPT about syscalls, and reading about programming interfaces for MS-DOS.

So when I decided to rewrite ARP Stand-in, I already knew I wanted to try using something really low-level, but not quite as hardcore as assembly language. That ruled out bytecode-compiled languages like Java and left me considering machine code-compiled languages like C.

Choosing Zig

I knew that C was the classic choice for writing Unix network services, used in such projects as Apache HTTPd, BIND DNS server, Postfix, OpenSSH, and many others. I learned C and used it during university studies over a decade ago, but I also remembered the painful parts of it, like null pointers. If I'd need to spend time getting up to speed on C again, why not invest that energy into something more modern instead?

I first thought of Rust. I'd seen its rapid growth over the last few years along with the crop of new, high-quality programs created with it. I was keen on it, but a bit hesitant about the learning curve. I had also caught wind of Zig via Hacker News and was intrigued by its relative simplicity, its interoperability with existing C code, and its degree of control over memory.

As I dug deeper, I realized that a distinguishing feature of Zig was its memory model: rather than solving heap memory management with borrow checking (like Rust) or garbage collection (like Go), Zig retains the manual memory management paradigm of C. What it does offer are the defer and errdefer keywords to make manual memory management simpler and less error-prone. Zig's memory model seemed aligned with what I was after: full control, even if it might mean getting myself into trouble. I also liked the explicitness of it, that I could see exactly where memory was being allocated and freed.

I came to realize that while Zig and Rust are both compiled languages without runtimes, Rust is actually a higher-level language than Zig, mainly due to Rust's ownership and borrowing system. Programming in Rust means thinking in Rust concepts (e.g. references, borrowing), while programming in Zig means thinking in concepts closer to the hardware itself (e.g. pointers, memory allocation). The fact that a programming language is higher-level doesn't inherently make it better or worse; some people and projects are better served by higher-level abstractions. In my case, since I was already super keen on going low-level, Zig was the obvious choice.

How it went

In a word, I'd describe programming with Zig as fun. It followed through on its promise of low-level programming, minus many of the typical annoyances of C. Some features I really enjoyed were Zig's error handling, built-in unit testing, and slices. Below, I'll expand on some specific parts of the experience.

This project turned out to be a great example of prototyping at a higher level before spending time optimizing at a lower level, or the idea that "premature optimization is the root of all evil." Prototyping in a high-level language (Ruby) was the right way to prove out assumptions and answer questions like:

• Can a network device answer ARP requests on behalf of a sleeping machine?
• Will other network devices accept ARP replies originating from a different machine than the one requested?
• Can a user-space program listen for and process ARP packets, considering that they are sent at a layer below TCP and UDP?
• Will wake-on-unicast work if another device answers ARP requests on behalf of the sleeping machine?
• What's better for sending/receiving ARP packets: libpcap, or a raw socket?

With those questions answered and the major design decisions in place (e.g. using libpcap), it then made sense to think about optimizing. The optimizations ended up being significant, with the new Zig program (Friendly Neighbor) using significantly less memory and being smaller/simpler to install than the original Ruby program (ARP Stand-in).

Packed structs for network packets

Network packets follow standard bit layouts, where groups of bits represent fields in the packet. For example, below is the structure of a Neighbor Discovery Protocol (NDP) neighbor solicitation packet, as per RFC 4861:

Copycopy code to clipboard
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Type      |     Code      |          Checksum             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           Reserved                            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
+                                                               +
|                                                               |
+                       Target Address                          +
|                                                               |
+                                                               +
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   Options ...
+-+-+-+-+-+-+-+-+-+-+-+-

Thanks to Zig's packed structs, it's easy to craft outgoing network packets and access fields in an incoming packet – doing so is as simple as working with structs (barring a few gotchas, like fields less than a single byte long). All I needed to do was to define a packed struct matching the layout from the RFC standard. For example, here's how an ARP packet struct is defined (source):

Copycopy code to clipboard
const EthernetArpFrame = packed struct {
    eth_dst_addr: u48,
    eth_src_addr: u48,
    payload_type: u16,
    hardware_type: u16,
    protocol_type: u16,
    hardware_addr_len: u8,
    protocol_addr_len: u8,
    operation: u16,
    sender_hardware_addr: u48,
    sender_protocol_addr: u32,
    target_hardware_addr: u48,
    target_protocol_addr: u32,
};

With the packed struct layout defined, crafting an outgoing packet was a matter of creating an instance of the struct and populating its fields. For incoming packets, I was able to access the fields after casting the received bytes to the struct type. Check out the source code to see more of how it works.

Takeaway: Packed structs are a subtle yet powerfully versatile language feature in Zig.

Argument parsing

Friendly Neighbor is a network service started from the command line, and I wanted it to take configuration values from the command line interface (CLI). Implementing CLI argument parsing turned out to be more difficult than I had hoped.

The Zig standard library doesn't provide CLI argument parsing beyond std.process.argsWithAllocator, which just takes the CLI invocation string for the current process and breaks it up into arguments by whitespace. This means that anything more advanced needs to be handled by a library or code written from scratch.

I wanted to spend my time solving the problem at hand, not reinventing the wheel by implementing complete argument parsing. So I opted to use zig-clap, which is a high-quality library, but still relatively immature (v0.6) and very much lacking in features compared with the clap crate for Rust. Adding the dependency on third-party code also ended up introducing a build-related challenge detailed below.

In the end, argument parsing works nicely, but getting it working demonstrated Zig's overall immaturity as a language and ecosystem. Its standard library is barebones and mostly undocumented, its package manager has the bare minimum feature set, and its library ecosystem is much smaller than established languages. Zig's C interoperability does make it straightforward to leverage existing C libraries, but not every C library will have optimal developer ergonomics when used in Zig code.

This isn't a critique of Zig, just a remark about where it currently stands. I expect these shortcomings to shrink or disappear in the coming years as Zig matures.

Takeaway: Leveraging libraries is currently much harder than in mainstream languages, especially high-level ones like Javascript or Python. Expect to either use C libraries or spend time reinventing the wheel.

Log levels

While exploring how to configure the verbosity level for the program's output, I also learned some interesting things. Zig provides nice logging utilities in the standard library (std.log). I wanted to set the log level to be INFO by default, and to include DEBUG messages if the --verbose flag was passed during invocation. What I didn't expect was that, by default, the log level can only be changed at comptime, meaning that it can't be changed based on a CLI option.

I learned that this choice was driven by optimization: by setting the log level at comptime, debug messages can be left throughout the code without incurring the runtime performance penalty of checking the current log level each time. If the log level at comptime doesn't include debug messages, those messages simply aren't included in the compiled program. This design decision was a tradeoff in favour of runtime performance and at the expense of runtime flexibility. It made sense once I understood it, but it was an unfamiliar kind of choice coming from high-level languages like Ruby and Javascript. It reminded me that Zig is a low-level language and makes tradeoffs as such.

Takeaway: As a low-level language, Zig makes tradeoffs that might be surprising to those coming from high-level languages.

Package Manager

It turned out that having an external library dependency (zig-clap) made the build quite a bit more complicated. Being brand new, Zig's package manager is basic and lacking documentation. The challenge I ran into was building the project using Snapcraft's Build from Github feature. The cloud-based build environment blocks direct requests out to the internet, but allows them through a dedicated proxy. The problem was that Zig's build utility doesn't support using a proxy (see issue), so I needed to add a workaround that manually downloads the package and stores it in the Zig cache before building. This workaround gets the job done, but feels hacky and adds maintenance overhead, since the package details (checksum, path) are now defined in two places. You can see the full workaround here.

Takeaway: Zig's package manager is still in its early days and has a barebones feature set.

Dynamic linking due to libpcap dependency

One of Zig's really nice features is its ease with compiling static binaries (binaries that don't have any dependencies on system libraries). Having static binaries greatly simplifies distribution of a given program, because all that's needed is a static binary for each architecture + operating system combination. System-specific packages are unnecessary because there are no dependent libraries to install.

I had decided to base Friendly Neighbor on libpcap, mainly for performance reasons: libpcap can filter packets in kernel space, which is more lightweight than filtering packets in user space. This choice, however, meant a dependency on the libpcap system library (defined here).

I only realized later that this choice made it infeasible to statically compile Friendly Neighbor, since Zig doesn't seem to support static compilation when linking against system libraries (see issue). Even if this was possible, it probably wouldn't make sense to do so given the size of libpcap and its respective dependencies, as the resulting static binary would be giant.

This didn't matter in the end because I opted to distribute the binaries primarily via Snap package, which conveniently bundles the required system libraries and also greatly simplifies running the program as a service (daemon). Regardless, it's useful to know for future projects that linking to system libraries makes static builds impossible (or at least not straightforward).

Takeaway: Linking system libraries is a hindrance to building a static binary.

Conclusion

This project was a lot of fun. I'm thankful to the Zig community for giving us a great new language and set of tooling. I'm very pleased to have Zig as a new tool in my toolbelt. Based on what I've learned, I'll keep the following in mind when considering Zig for future projects.

I would choose Zig for a project that…

• … needs to run in a resource-constrained environment, like an embedded device
• … benefits from very fine-grained control over memory allocation
• … is for low-level tinkering/hacking, where I want to build something quickly without getting bogged down by minutiae of language rules and compiler errors (like in Rust, for example)
• … deals directly with hardware, e.g. a toy operating system
• … aims to leverage or extend existing C code
• … builds to WebAssembly, because Zig makes it easy and produces lean binaries

I would hesitate to choose Zig for a project that…

• … deals primarily with high-level business logic or data manipulation, like a web application backend. A low-level language like Zig is a drag in this case because it requires lots of repetitive code for trivial things like manipulating strings.
  • Consider Javascript, Ruby, or Python instead
• … benefits from high-level code abstractions, because Zig doesn't offer them.
  • Consider Rust instead
• … depends on libraries beyond the Zig standard library and libc, due to challenges finding high-quality libraries and limitations with building static binaries.
  • Consider Rust or Go instead
  • I expect this to change as Zig and its library ecosystem mature
• … will be developed/maintained by a large group of programmers with varying levels of experience, due to pitfalls for beginners dealing with manual memory management.
  • Consider Rust or Go instead