The chain starts as follows: you have a program that’s written in some programming language. If it’s a compiled language (like C++ or Rust), you’ll need to compile it in order to run it.

$ cat > program.c <<EOF
#include <stdio.h>

int main() {
  puts("Hello, world!");
  return 0;
}
EOF
$ cc program.c -o program
$ ./program
Hello, world!

If it’s an interpreted language (like Python or JavaScript), there’s no need to compile it. Just pass the program to the interpreter.

$ python3 -c "print('Hello, world!')"
Hello, world!

The interpreter is usually a compiled program, however. So the actual chain of events, amortized across multiple parties, can be simplified to:

$ # At your distribution...
$ cc python3.c -o python3
$ upload_to_server ./python3

$ # At your computer...
$ apt install python3
$ python3 -c "print('Hello, world!')"
Hello, world!

No matter what we do, at some point a compiler has to compile some code.

Of course, this is glossing over a lot that can happen between these two extremes of “interpreted” and “compiled” languages. Some interpreters are optimized by turning them into really fast compilers.

However, the compiler is itself a program that’s written in some programming language. Therefore, at the start, you need a compiler to compile that compiler. But then that compiler itself needs a compiler. Then that compiler. So on and so forth.

This is the bootstrap chain; the collection of software that is required to build your software. Usually the bootstrap chain ends at a certain trusted root, like the Debian software repositories. For most organizational contexts, this is as far as your need to go.

But, any sufficiently good programmer will eventually get a little more curious. They want to pull on the bootstrap chain a little further than that. The majority of compilers are bootstrapped; they’re written in their own language and can be used to compile themselves.

The short version of the story is that the previous version of a compiler is used to compile the next version of a compiler. GCC 14 is used to compile GCC 15. GCC 13 is used to compile GCC 14. Rinse and repeat, all the way back to GCC 1 being used to compile GCC 2.

I can’t really find any history on how the first versions of GCC were compiled¹, but I imagine they were compiled using the simpler C compilers of the time. If you were to follow those compilers back, you’d eventually get ones that were implemented in assembly language. Meaning, you’d need an assembler to assemble them.

Then, who assembles the assembler? Back in the 70’s, when this code was first being written, it was common for programmers to just not use assemblers. They would write out the intended assembly code, “assemble” it into the binary code by hand, then load that binary code into the computer. David Wheeler gets credit for inventing the first assembler, which would convert textual assembly code into binary.

For most people, that’s where the story ends. Developers toggled bytes into early computers to get assemblers, early assemblers built early compilers, then early compilers built the modern compilers and interpreters we enjoy today.

For us, it’s just the beginning.

Ken Thompson and Trusting Trust

You may notice that this entire extended “bootstrap chain” takes place over almost 80 years of computing progress. Specifically, large parts of the chain involve programs that have since been lost to time, or compilers that were always closed source. It makes you wonder: what would happen if someone in that chain was actively malicious? What if someone in the chain wanted to control every program at every step after?

Ken Thompson is a living legend. He created Unix, and along the way also helped create the C programming language alongside Dennis Ritchie, as well as a million other little things like ed and UTF-8.

For his massive contribution to computer science, he won the Turing Award in 1983. Upon receiving the award, he demonstrated a possible attack on compiler infrastructure that could be used to compromise all software, everywhere.

He first demonstrated a fairly typical attack: a compiler that intentionally mis-compiled a program with a known source input. Effectively, the compiler changes the behavior of the program it’s compiling. For example, imagine we had a program that validates some input as part of a login program.

#include <stdio.h>
#include <string.h>

const char *password = "foobar";

int main() {
  char buffer[80];

  printf("Input password: ");
  fgets(buffer, 80, stdin);
  if (strcmp(buffer, password) == 0) {
    puts("Password matches!");
    return 0;
  } else {
    puts("Incorrect password!");
    return 1;
  }
}

Now, you go to compile this program using the system’s C compiler, cc. But, let’s say the compiler is written like this:

const char *program_to_look_for = "int main() { ... }";
const char *program_to_replace_it_with = "int main() { ... }";

char *program_source = read_c_code_from_file();
if (strcmp(program_source, program_to_look_for) == 0) {
  // Compiler our custom program instead.
  program_source = program_to_replace_it_with;
}
do_the_actual_compilation(program_source);

For most programs, the compiler will behave normally as intended. But, it’s looking for your specific login program. If it sees it, it will replace the source code it should compile with something else. So even if you pass the compiler the correct source code, it can do whatever it wants with it.

Such a compiler could, for example, insert a backdoor into your login program.

$ good-cc login.c -o login
$ ./login
Input password: secret_backdoor
Incorrect password!

$ # Now, using our backdoored compiler:
$ cc login.c -o login
$ ./login
Input password: secret_backdoor
Password matches!

You can replace the “source code matching” code above with any other kind of heuristic. For example, analyzing the RTL for some kind of security subroutine, then subtly replacing it with a backdoored version.

This is somewhat disturbing, but can be easily avoided by just using a trusted compiler. Except, remember, compilers are themselves programs that need to be compiled by compilers. It’s possible for a compiler to tell when it’s compiling another compiler, then compile it with the backdoor code in it.

Imagine a backdoor programmed like this:

let program_source = read_program_code();
if (strcmp(program_source, c_compiler_source_code) == 0) {
  program_source = backdoored_c_compiler_source_code;
} else if (strcmp(program_source, login_source_code) == 0) {
  program_source = backdoored_login_source_code;
}
do_the_actual_compilation(program_source);

Effectively, this is a self-reproducing backdoor; a “virus” of sorts that infects every subsequent compiler. All a malicious actor has to do is insert this program early in the bootstrap chain. Then, every subsequent compiler in the bootstrap chain will have the backdoor as well. These compilers will compile the backdoor into other compilers, which will then compiler the backdoor in your program.

Impact

The terrifying thing about this virus is that there is no reliable way to detect it. There’s no real way to inspect binaries to ensure the bootloader isn’t there.

You could, in theory, hexdump the programs and see if there are any errant instructions in there. But the hexdumper could be backdoored as well, to silently not show you the backdoor. Ken Thompson goes to great lengths to indicate that the backdoor can effect any program that handles code; whether it’s the assembler, the dynamic loader, or even the hardware’s microcode.

You could debug the program and check to see exactly what it’s executing. But, the debugger could be backdoored. It could be silently skipping the instructions that insert the backdoor.

You might be saying “no worries, we have reproducible builds. We can just compare the program across builds to ensure that the hash is the same.” Except, this doesn’t work. How are you checking that the program is the same? sha256sum? GnuPG? These can be backdoored too.

You would have to manually inspect the hard drive with an electron microscope to actually verify the exact contents. But even then, this isn’t a bulletproof defense. Modern CPU’s run microcode to translate the instruction code to the underlying CPU microarchitecture. That microcode can be compromised and made to mis-translate certain code.

This isn’t a theoretical attack, either. Viruses like this have already been discovered in the wild. Granted, in this case, this virus only infected Delphi environments and was mostly harmless.

But now, imagine what a nation state could do with this attack. PRISM has existed for some time now. In fact, there are individual software companies (and not that big of ones, either) that could put backdoors into popular open source software that could be replicated throughout the ecosystem.

This affects all software, closed and open source.

The Solution

The solution to the source of compiled software being in question is to have a way to create software without relying on other software. Back in the 70’s, the boot program for a computer was toggled into the front panel by a series of levers. This program would be the bare minimum needed to load the rest of the operating system from the disk.

This is still theoretically possible today. For example, it’s possible to take a floppy disk and a pencil, then make holes in the floppy disk with the pencil. The floppy disk reader can interpret the holes as binary, which effectively lets you program with a pencil.

You could use this strategy to write a compiler for a very, very basic programming language. Since you can trust the compiler you just wrote, you can trust anything that compiler has output. Then, you use this language to write a compiler for a slightly more advanced programming language.

You keep going until you create an assembler, then keep going after that until you have a C compiler. With a C compiler you can start compiling old GNU utilities until you arrive at GCC. Once you have GCC, you can compile anything, including Linux.

This is what the stage0 project has been doing for some time. There is a hex interpreter that starts with around 380 bytes of code. This is feasible enough to toggle into a computer, then verified manually, by hand. Then after a couple of more advanced hex interpreters, it brings up a macro assembler, which then assembles a basic C compiler. This then compiles TinyCC, which can compile a basic operating system and userland. This is enough to get to GCC up, which then brings up Linux.

This is glossing over a lot. The full bootstrap chain is here. I also talk about bootstrappable builds a lot in this post.

Of course, this isn’t bulletproof. stage0 requires a BIOS or UEFI implementation to run. BIOS and UEFI is written in code; code that can be very easily compromised. If we’re going this far, who’s the say the microcode or boot ROM on the CPU isn’t compromised? In fact, how do we know the RTL on the CPU isn’t backdoored? It’s not like anyone has ever bothered to check.

Conclusion

I wanted to share a little of my computing paranoia in this post. Computers can never be 100 percent secure. Until someone manages to build a CPU in their garage out of nothing but electrical wire and duct tape, then bring up Linux on that CPU, it’s possible for any compiler to be compromised.

I don’t know if there’s an actual name for this idiom; I’m calling it the “block pattern” for lack of a better word. I find myself reaching for it frequently in code, and I think other Rust code could become cleaner if it followed this pattern. If there’s an existing name for this, please let me know!

The pattern comes from blocks in Rust being valid expressions. For example, this code:

let foo = { 1 + 2 };

…is equal to this code:

let foo = 1 + 2;

…which is, in turn, equal to this code:

let foo = {
    let x = 1;
    let y = 2;
    x + y
};

So, why does this matter?

Let’s say you have a function that loads a configuration file, then sends a few HTTP requests based on that config file. In order to load that config file, first you need to load the raw bytes of that file from the disk. Then you need to parse whatever the format of the configuration file is. For the sake of having a complex enough program to demonstrate the value of this pattern, let’s say it’s JSON with comments. You would need to remove the comments first using the regex crate, then parse the resulting JSON with something like serde-json.

Such a function would look like this:

use regex::{Regex, RegexBuilder};
use std::{fs, sync::LazyLock};

/// Format of the configuration file.
#[derive(serde::Deserialize)]
struct Config { /* ... */ }

// Always make sure to cache your regexes!
static STRIP_COMMENTS: LazyLock<Regex> = LazyLock::new(|| {
    RegexBuilder::new(r"//.*").multi_line(true).build().expect("regex build failed")
});

/// Function to load the config and send some HTTP requests.
fn foo(cfg_file: &str) -> anyhow::Result<()> {
    // Load the raw bytes of the file.
    let config_data = fs::read(cfg_file)?;

    // Convert to a string to the regex can work on it.
    let config_string = String::from_utf8(&config_data)?;

    // Strip out all comments.
    let stripped_data = STRIP_COMMENTS.replace(&config_string, "");

    // Parse as JSON.
    let config = serde_json::from_str(&stripped_data)?;

    // Do some work based on this data.
    send_http_request(&config.url1)?;
    send_http_request(&config.url2)?;
    send_http_request(&config.url3)?;

    Ok(())
}

This is fairly simple, and just leverages a few Rust crates and language features to parse JSON and then do something with it.

However, there are a few weaknesses here. In the foo function, we declare four new variables (config_data, config_string, stripped_data, config) only for only one of those variables to be used after the configuration parsing (config). In addition, let’s say you didn’t know what this code was for going in, and you didn’t have these comments (or you had bad comments). One might ask why you’re declaring the regular expression STRIP_COMMENTS, or why you’re loading data from a file.

When I write code, I try to make it immediately obvious what the purpose of the code is, and why it’s written that way. This is why I generally avoid C’s “bottom-up” strategy for organizing code. It’s like being given a few screws and being expected to implicitly understand that it should be built into a chair. In Rust, I like that you are able to define your top-level functions first, and then go down and define all the bits and pieces after.

Although, we can do a little bit better. What if we organized the foo function like this:

/// Function to load the config and send some HTTP requests.
fn foo(cfg_file: &str) -> anyhow::Result<()> {
    // Load the configuration from the file.
    let config = {
        // Cached regular expression for stripping comments.
        static STRIP_COMMENTS: LazyLock<Regex> = LazyLock::new(|| {
            RegexBuilder::new(r"//.*").multi_line(true).build().expect("regex build failed")
        });

        // Load the raw bytes of the file.
        let raw_data = fs::read(cfg_file)?;

        // Convert to a string to the regex can work on it.
        let data_string = String::from_utf8(&raw_data)?;

        // Strip out all comments.
        let stripped_data = STRIP_COMMENTS.replace(&config_string, "");

        // Parse as JSON.
        serde_json::from_str(&stripped_data)?
    };

    // Do some work based on this data.
    send_http_request(&config.url1)?;
    send_http_request(&config.url2)?;
    send_http_request(&config.url3)?;

    Ok(())
}

In this function, we’ve moved all of the configuration-related code (parsing, loading, even the static regex) into the block. This works because Rust lets you have items, statements and expressions inside of a block, hence why we were able to move everything inside. This pattern has three immediate advantages:

• The block starts with the intent of the code (let config = ...). We can see that we’re working to resolve some kind of configuration object right off the bat. Only then do we move into the implementation details of the code.
• It reduces pollution of the namespace of both the foo function and the top-level module. Now in foo, the variable names config_data, config_string et al are no longer used. In addition to allowing these variable names to be re-used, it makes this code a lot more “idiot-proof”. If someone else were to edit the foo function, they would only be able to use config. They wouldn’t be able to use the raw_data or STRIP_COMMENTS items, which are only meant to be used by the config parser.
• The variables raw_data and data_string go out of scope at the end of the block, which means they are dropped, freeing up resources.

As an aside, all three of these advantages also come if you were to refactor the block out into its own function. However, this pattern has two key advantages over that:

• The code flow is still inline with the rest of the function. For shorter blocks, this improves reading comprehension, since it means you don’t have to go to a different part of the code to fully understand the function.
• If there are a lot of variables that the block would use, it prevents needing to explicitly name those variables as parameters.

There is one more benefit that’s not exposed in the above example: erasure of mutability. Let’s say you construct some object for use in a later part of the function:

let mut data = vec![];
data.push(1);
data.extend_from_slice(&[4, 5, 6, 7]);

data.iter().for_each(|x| println!("{x}"));
return data[2];

The issue is that data is declared as mutable, which means the rest of the function can mutate it. Since a lot of bugs come from data being mutated when it isn’t supposed to be mutated, we’d like to restrict the mutability of the data to a certain area of the function. This is also possible with the block pattern:

let data = {
    let mut data = vec![];
    data.push(1);
    data.extend_from_slice(&[4, 5, 6, 7]);
    data
};

data.iter().for_each(|x| println!("{x}"));
return data[2];

This effectively “closes” the mutability to a certain section of the function.

Closing Thoughts

I don’t know if this pattern is already well known to the Rust community. Even if it is, I figure it’s still a good idea to bring it to people who may be inexperienced in Rust.

Disclaimer: I will take a look at some publicly available Rust crates today. Please do not harass their authors, maintainers or users.

I’m not kidding. Let’s check out ripgrep, one of the most popular Rust programs of all time. We can check the number of dependencies fairly easily by cloning the project, then running cargo tree through a nightmarish sed invocation.

$ git clone https://github.com/BurntSushi/ripgrep
$ cargo tree -e no-dev --prefix none | \
    sed -e 's/(\*)//g' -e 's/^[ \t]*//;s/[ \t]*$//' | \
    sort -u | wc -l
33

To break down each step of the shell command:

• cargo tree prints the dependencies of the current Cargo package in a “tree” format, that lets you see which dependencies are brought in by which other dependencies. -e no-dev skips dev dependencies, which are only used for testing. By default cargo uses a tree formatting that we can skip using the option --prefix none.
• sed is a simple core utility that lets you run regular expressions on each line of a command’s output. Each expression is denoted by -e.
  • The first expression, s/(\*)//g, is a simple replacement that says “replace (s) every instance of (*) with nothing, globally”. cargo tree adds this to repeated dependencies and I’d like to remove that.
  • The second expression s/^[ \t]*//;s/[ \t]*$// removes whitespace at the start and end of each line. This just cleans up the output in such a way that it doesn’t mess with our uniqueness test later.
• sort -u sorts every line and deletes duplicated lines.
• wc -l prints the total number of lines.

Thirty-tree dependencies isn’t that bad for a highly advanced text matching program. Most of the dependencies are things like the Rust regex engine, or highly optimized text matching libraries. Frankly, I can find a fairly good reason for these dependencies. Then again, it’s a small command line utility, so it’s not like I’d expect it to have a million dependencies. Great job!

So let’s try something a little more complicated. Specifically, let’s try a networking application, since those are known for having unneeded dependencies. I’ll pick one off the awesome list… ah, miniserve.

It should be a relatively small application, right? So let’s check the dependency count.

$ git clone https://github.com/svenstaro/miniserve
$ cargo tree --no-default-features -e no-dev --prefix none | \
    sed -e 's/(\*)//g' -e 's/^[ \t]*//;s/[ \t]*$//' | \
    sort -u | wc -l
281

I specifically added --no-default-features to remove unneeded dependencies, and we still have a grand total of two hundred and eighty one dependencies. That’s quite a few! Looking into the dependency list, we can see:

• A QR code generator.
• The rand crate, both version v0.8.5 and version v0.9.1, as well as fastrand.
• actix-web, which brings in all of tokio and various crates for handling internet edge cases.
• Further duplicates of base64, hashbrown, syn and zerocopy.

I can justify a lot of these dependencies. The internet is a complicated place, and anything that needs to face the public ‘net and respond to 99% of web clients needs to deal with that. Still, thats 281 pieces of code that the maintainers have to audit. Imagine if one of those dependencies is compromised, or just becomes unmaintained. Not to mention the fact that there are two copies of these pieces of code compiled into every one of these crates.

I’m not here to gawk at dependency graphs. Anyone can do that. I’d like to identify the scope of the problem, and see if we can figure out some solutions.

Dependency Dragback

To be clear, this problem isn’t just something randoms complain about. I’ve been on both sides of this issue. I’ve been in security audits where “the amount of code that gets pulled in” is an active demerit against integrating Rust into a project. I’ve also been the author of libraries where lowering the dependency count becomes a big issue.

I’d also like to be clear that this isn’t a problem unique to Rust. People have been complaining about extraneous packages on JavaScript and Python ever since they got package managers. JavaScript is famous for its “leftpad” incident, as well as simple pieces of code requiring two hundred dependencies. Even C++ runs into dependency problems once you add Boost to the equation.

In all of these languages, there’s two types of dependencies, in my opinion.

• Dependencies that do something you don’t want to do yourself. You don’t want to implement an HTTP server, so you pull in axum.
• Dependencies that act as the canonical interface to some system facility or hardware device. Think the C library for making system calls¹, or the OpenGL library for putting things on the screen with the GPU.

To be clear, there are very good reasons to pull in the first kind of dependency. For things like cryptography or networking, writing low-level operations yourself is a one-way trip to security-breach-ville. Even for less critical operations, it’s usually better to use a tried-and-true, tested library than going and writing it yourself.

There’s also something to be said here about code reuse; there’s no reason why there should be more than one implementation of some algorithm in the Rust community, necessitating the splitting of work between experts. Why have one group of people review one piece of code and have another group of people review another piece of nearly identical code, when it would be better to have both groups review only one crate. This idea is the core reason why the x11rb-protocol crate was created.

That being said, there are some thing that are just so trivial that using separate crates for them is far-and-away overkill. I’ve had to deal with crates that depend on heavy hitters like regex and nom for byte-munching operations that were able to be implemented in basic slicing.

The worst offender for this problem is scopeguard. There are very few use cases where this crate is economical over a few extra lines of simple Rust code. Here is a quick polyfill:

// Before:
scopeguard::defer! {
  do_the_thing();
}

// After:
struct CallOnDrop<F: FnMut()>(F);
impl<F: FnMut()> Drop for CallOnDrop<F> {
  fn drop() {
    (self.0)();
  }
}

let _bomb = CallOnDrop(|| do_the_thing());

Safety Spinoff

Going back to my earlier point. These two types of dependencies exist in pretty much all languages. I argue that Rust has a third type:

• “Safety quarantine” crates that wrap some unsafe features in a safe wrapper.

There are simpler ones like bytemuck which just wrap around simple data transmutations in a way that the standard library hasn’t gotten around to exposing yet. Then there are the C library wrappers like zstd which take a C library and make it safe. I’m largely talking about the bytemuck-type wrappers here.

When used effectively, these kind of “safety quarantine” crates let you isolate unsafe code to specific parts of your dependency graph so you can be sure the rest is safe. If the other crates in your dependency tree use #![forbid(unsafe_code)], you can be sure that any behavioral problems will only come from those “quarantined” unsafe code crates.

In the dependency tree of smol, we use this strategy to great effect. It’s impossible to create a performant executor without some level of unsafe code. So we isolate all of the unsafe code to async-task, so async-executor can have almost no unsafe code. Similar, efficient lock-free channels require unsafe code to implement on some level. So we isolate all of the unsafe code to concurrent-queue so that the async-channel crate can be #![forbid(unsafe_code)]. While smol may have quite a few dependencies, many of these dependencies exist in such a way that it completely eliminates unsafe code.

I think it would be nicer if more crates did this. eframe, one of the more popular GUI crates on the market, has around 120 dependencies. I wonder how much more palatable it would be if more of those dependencies were written with entirely safe code.

What do?

Still, many crates end up just having a smidgeon of unsafe code in them, however justified. Not to mention, safe code is code you still have to audit. So while there’s nothing wrong with handfuls of micro-crates that each do something well, we should still seek a way to reduce the amount of dependencies in our dependency tree.

The first measure we can take is somewhat obvious, for both application and library developers. It’s possible to minimize features, which often minimizes dependencies. You can run cargo add with the --no-default-features flag, like so:

$ cargo add nom --no-default-features

In my own applications, I usually start by adding dependencies without default features, and then adding features piecemeal whenever I need them. Because of this, I can usually keep my dependency tree to a minimum.

When this fails, don’t be afraid to switch to another, parallel crate. For many dependency-heavy crates, there’s usually an alternative crate with much fewer dependencies, often while retaining the core functionality you depend on. For example, instead of futures, consider using futures-lite. Instead of actix-web, maybe see if your use case can be fulfilled by axum.

I’ve found that, with these two strategies, you can minimize the dependencies of your crate or application. Usually, to a much more manageable level.

This post is a long overdue follow-up to my earlier post about how blocking code is a leaky abstraction. I’ve written a lot of words on this blog defending one of Rust’s most controversial features: async/await syntax. Many view it as an overcomplication in an otherwise elegant language; I see it as the missing piece thats lets you model asynchronous functionality and treat I/O operations as data.

However, in these blogposts I realize that I have made a significant error. I have presented async/await as the only way to write asynchronous programs in Rust. There are many other ways to write programs with non-standard control flow in Rust; in fact, many of these strategies predate async/await as a whole.

Most of these strategies rely on a callback-based event loop; where you pass a callback to an event source, and then that event source triggers the callback when an event happens. This is a tried and true model; libuv is an example of an implementation of callback loops that’s used in millions of applications by way of node.js. As for Rust, winit and the appropriately named calloop are examples of popular callback event loops.

For this post, I’d like to take a closer look at calloop. I feel as though it’s a pretty good example of callback event loops, and in fact the Linux backend for winit is built on calloop. I’ll compare calloop to async/await (or specifically smol) and see what applications in which they’re a good fit.

Disclaimer: I maintain an async runtime, smol. However, I also maintain calloop and winit (although I am on hiatus). So I am biased in both directions.

Breakdown Boogie

What you must first understand is that the underlying design philosophies for async/await and calloop have very different goals.

async/await was designed specifically for networking applications; where you have to scale to handling millions of requests at once across dozens of CPU cores. Some writers have asserted that this means async I/O for smaller use cases is a “should be a weird thing that you resort to for niche use cases”. I argue that it means you can scale with relative ease. If you know code can handle 5,000,000 concurrent tasks, that means it can handle 5 with no issues.

calloop, meanwhile, is designed for single threaded applications. As in, an application where all of your logic runs on one single CPU core. This does not necessarily mean the application is slower or meant to be used less; Redis notably runs on a single-threaded architecture. Although, calloop is explicitly not meant for performance; this is made quite explicit in its documentation:

The main target use of this event loop is thus for apps that expect to spend most of their time waiting for events and wishes to do so in a cheap and convenient way. It is not meant for large scale high performance IO.

async/await is the definite winner when it comes to high performance applications that expect to scale vertically. However, the calloop model is used more frequently in the GUI ecosystem. This makes sense once you realize calloop was created by the same person who created the Rust Wayland implementation and calloop is effectively a Rust implementation of the Wayland event loop. As I mentioned earlier, winit is built on calloop.

In the past, I’ve proposed that we could work async/await into the GUI ecosystem and extolled the possible benefits. Needless to say, I think async/await could have its place in the GUI ecosystem; a place that calloop has traditionally occupied.

Advantage Async

To cut this off at the pass; you don’t have to exclusively use async/await or calloop in your program. They are compatible! Best friends! Roommates! smooching

calloop goes out of its way to add async/await compatibility, making it so you can easily run Futures inside of the event loop. Meanwhile, you can use the async-io crate to poll an EventLoop on certain platforms:

use async_io::Async;
use calloop::EventLoop;

// Wrap the event loop into the `smol` runtime.
let mut event_loop = Async::new(EventLoop::try_new()?)?;

// Dispatch events when needed.
loop {
    event_loop.read_with(|event_loop| {
        event_loop.dispatch(None, &mut ())
    }).await?;
}

Since GUI programs are mostly single threaded, async/await’s multithreaded advantages don’t really apply. But that doesn’t mean async/await is powerless here; there are single threaded runtimes. The main disadvantage of these runtimes is that Waker, the backbone of async/await, is multithreaded and assumes it’s going to be sent to other threads. This condition requires that you make all of your primitives dependent on synchronous primitives like Mutex, which adds a performance drag to the program. LocalWaker would fix this issue if it’s ever merged to mainline Rust.

There are two principle advantages to async/await in this model. The first is that async/await brings composability, a property I believe is valuable in GUI applications. calloop is admittedly not composable, preferring multiple event sources over being able to chain event sources together.

I see libraries like accesskit, ui_events_winit and wgpu that are begging for composability. wgpu even has an official middleware pattern. Not to mention how composable GUI widgets need to be able to layer on top of eachother in order to actually be useful.

The second advantage is easy integration with the rest of the Rust async ecosystem. Many useful GUI apps end up having to make network calls at some point; imagine being able to seamlessly make network requests from your GUI application without freezing up like every other Win32 application seems to do. Not to mention, the async ecosystem has mature event delivery mechanisms that could solve the “event delivery” problem Rust GUI frequently has.

I’ve already written about this at length here, if you’re interested in me going into more detail.

Shared State Scenario

However, calloop has one crucial advantage that async/await will never have. One critical strength that means there are very good reasons why programmers will and should choose calloop for many use cases. The silver bullet, the kryptonite: calloop allows for shared state in a way that async/await does not.

calloop is designed around having a shared structure that’s passed around to every event source. You pass it some shared state in dispatch…

struct MyState {
    name: String,
    counter: i32,
    foobar: File
}

let mut state = MyState { /* ... */ };
event_loop.dispatch(None, &mut state);

…and suddenly, every event source has direct &mut access to state. This sharing works because calloop is just calling event sources in order on a single thread in a loop; it can trivially pass &mut MyState to an event source while it’s polling it for completion.

async/await has no good answer for this. Usually, network applications are designed in accordance with the actor model, where each task has its own specific state and only shares it with other tasks via channels. Many GUI applications are designed like this as well. But many more are designed around only having one single blob of mutable state that every widget needs to have access to.

Granted, async/await has no problem sharing immutable state (via immutable references, with &state). Using primitives like RefCell in single-threaded setting, it’s possible to turn this into mutable state. But this solution introduces ugly, brittle interior mutability into the Rust program. It’s a pale shadow of what calloop is able to achieve so effortlessly.

There is some form of shared state in async/await, via the Context parameter that is passed to all Futures. But as of the time of writing, this value only holds a Waker, and is often re-created wholesale during things like async task polling.

There is a proposed ext() method that would allow holding arbitrary extension data inside of the Context. However, even this is a shallow imitation. It only holds an &mut dyn Any, a type erased value that will need to be cast into whatever value you need. Even if you’re okay with that, it will take some work for ext() to be handled by the Rust async ecosystem.

So yes, async/await has no refutal for this problem.

Conclusion

While async/await holds many advantages for programs that are using the actor model, it will take some work and compromise for it to be integrated into programs that use a significant amount of shared state. I cope by saying shared state is an antipattern in Rust anyways.

Alt: notgull slumped on the floor holding a bottle of Absolut Vodka.

In terms of open source code, 2023 was fairly decent for me. I was the maintainer of two popular projects that were used in a number of real world usecases. I was also working on some new, exciting projects that took these existing usecases and did something interesting with them. I even had consistent posts on this blog, which brought me some attention from people I really respected. By all records, I was doing great.

In 2024, I suddenly stopped contributing so much. Projects that I was previously spending a significant amount of time on suddenly stopped receiving updates. PRs went unreviewed. Issues went unanswered. I had people asking me if something was wrong. I went five months on this blog without a single post, with the one post just being a completion of a mostly-finished article I had before.

By 2025, I had announced that I was going into hiatus and stepped back from all of my projects. Even then, this announcement was largely a formalization of the existing state of affairs. Any project that I had been maintaining was effectively abandoned.

I’d like to analyze my burnout in greater depth. Specifically I’d like to go into the experience of being burnt out, and the reasons why I felt this way. Mostly, just as a way of getting all of those emotions into a rational form. Partially, so other people can avoid falling into the same pitfalls I did.

In case you can’t tell, this one is going to be very personal, in a way that articles on this blog haven’t been so far. Feel free to skip if that’s not your thing.

Prelude

At the beginning of 2024, I still felt pretty good.

The past couple of years had been pretty transformative for me. I’d been writing open source software for while, but not the kind of software that ends up being used. It all started when the fantastic Taiki Endo had offered me a maintainer spot in smol-rs. I was still writing code, but all of a sudden, people were actually taking that code and doing something with it. Like if you walked into your model train set, and all of a sudden there were actual passengers using it to get to their day jobs. I felt ecstatic.

In the time since then I’d become somewhat well known in the Rust community for maintaining smol. I’d also started writing code for rust-windowing, further expanding my repertoire. This led to a couple of cool one-off projects, like theo and async-winit. Even if those projects wouldn’t see wide use, you don’t know how fun it was to tinker with a codebase like theo and watch it grow into something actually usable.

Not to mention, I also started writing this blog! It’s quite rewarding to write something, and then see people actually share your opinions. It’s like seeing yourself on TV.

I think it’s a very common experience for an open-source programmer to start at a programming job, and then just completely halt their open-source activity. I started at my current position at my current company in the summer of 2023, after I’d graduated college. Even with my full-time job, I was still writing open source code and keeping a pretty good output overall. I’d felt like I’d beaten the odds; I could have my cake and eat it too.

Even going into 2024, I still felt like I was on top of the world. At some point I promised myself that I would write one blogpost every two weeks. Of course, that didn’t pan out, but who cares? I was still writing tons of code, both for my job and for the open-source community. I could keep going forever if I wanted to.

So why did everything change?

The Experience of being Burnt Out

All of a sudden, I was having trouble getting into my open source projects. The passion that had previously pushed me forwards was waning. I still liked reading through issues and solving them. But it started to take a grating cadence in my mind; it was something I’m doing because I was forcing myself to, not because I wanted to.

I still kept at it, though. I’m nothing if not stubborn. But that approach slowly wore down. The act of reviewing pull requests became a chore that I never looked forward to.

By May, I could barely bring myself to look at the code. Even opening GitHub induced a visceral reaction. Reading through the code that I’d previously enjoyed skimming for fun was like trudging through molasses. A struggle.

In a few months time, I couldn’t even bring myself to look at the smol codebase.

Lack of progress brought shame, and shame induced further lack of progress. I knew there were issues piling up on my projects, but what could I do about them? Like sewage water filling up my neighbor’s apartment, there were urgent problems but I couldn’t do anything about them. Those tasks just seemed to loom large on my to-do list. Like statues that I had been tasked to knock down myself, with nothing but a single sledge hammer and a bottle of hydrochloric acid.

Maybe I could’ve kept going. But at some point I just realized, I couldn’t even feign it anymore.

Maybe this was why I publicly announced dozer, which was a mistake in my opinion. dozer wasn’t in anywhere near the state where it would be productive to have multiple people working on it. I ended up having to completely re-do the backend if I wanted to handle even a fraction of Rust’s core language. A re-do that is, to this day, incomplete.

I’m sure you know the experience of starting a new project. All of the potential in front of you, ready to be seized and turned into something wonderful. It’s a high like pure crack cocaine. But soon you’re hit with the cold reality that you can’t just build castles in the clouds. Eventually you’re going to have to implement the bits of the project you never expected to implement, or completely change your plans to fit your API.

In the end, you eventually have to slog through those bits. Especially when you’re already burnt out from other projects, each of those bits feels like you’re carving them with a dark obsidian knife out of your very soul.

In March of 2025, I did something long overdue. I announced my hiatus, and said I was taking a break. This, thankfully, relieved the pressure that I felt on me, real or imagined. It’s given me time to think, time to digest my experience.

I feel better now, by the way. I’m not sure I want to return from my hiatus just yet; I imagine it will be a gradual return once I am ready. But, I’m writing code again. For fun! I’m excited to see where it goes.

The Cause of Burnout

In the above paragraphs, I realize I’ve made an error; I’ve made it seem like burnout was an inevitable fact, a direct cause-an-effect of long-term open source development. I don’t think it is. I’d like to go into certain external factors within my life and see what I can pick out.

Obviously, there’s work. Around the start of 2024, my responsibilities were suddenly expanded. To make a very long story short, I’d been assigned to a project that was an order of magnitude more important and more pressurizing than any I’d been assigned before. I found myself writing and testing a significant amount of code, and it began to eat into my energy budget.

I genuinely enjoy writing code. As I’ve alluded above it’s like building a model train set that people can actually ride. But, even I have my limits as to how much code I can write in day.

I’ve been told this is likely an example of the overjustification effect. If you give something with intrinsic motivation to perform a task incentives to complete that task, they will lose that intrinsic motivation.

As an example, let’s say you see a kid who plays on a swing in the park every day. You go up to the kid and say “I will give you twenty dollars every time I see you playing on this swing”. Of course the kid will keep playing on the swing when you come by every day. It’s a win-win!

But then let’s say you stop coming by with the money. Will the kid keep playing on the swing? No. Because at some point it stopped being something the kid did for fun and started being a chore they had to do for money.

Right now I feel like I may be the kid on the swing, and someone’s suddenly offered me a lot of money to keep swinging.

But maybe it’s unfair to blame it all on work. While the projects I’m assigned to may be intense, I still find them quite fun. To be clear, I think the overjustification effect is part of it, but not quite the whole problem.

Circa April 2024, I was going through something that I would quite describe as a downward spiral, but it certainly felt like one at the time. I can describe it as “finding out a lot of things the hard way”.

In the end, I had the feeling of falling even though I was standing on solid ground. This kind of panic definitely contributed to my burnout, along with a handful of longstanding personal issues I was going through at the time.

It’s never just one thing, it seems, that overwhelms the defenses. It’s always at least two or three.

What now?

As I said before, I’m feeling better. A lot of things in my life have stabilized for the time being, and I’m finally seeing a psychologist to help me through my varied emotions. Finally, I’m feeling that passion that originally drew me to open source come back.

I don’t want to immediately burn through that passion, though. I want to nurture it and let it grow again, like it did the first time. I’m slowly but surely taking up some of my projects again. Maybe someday we’ll see a half-decent dozer, or a smol-rs as lively as it used to be.

Maybe 2025 will be the year of the notgull.

Disclaimer: I am one of the maintainers for smol, a small and fast async runtime for Rust.

I’ve been involved in the Rust community for four years at this point. At this point, I’ve seen a lot of criticism of async. I’ve found it to be an elegant model for programming that easily outclasses alternatives. I use it frequently in my own programs when it fits. There are a lot of programs that would be improved with the presence of async, that don’t use it because people are scared of it. In fact, many organizations have a “hard ban” on async code.

Some of this criticism is valid. async code is a little hard to wrap your head around, but that’s true with many other concepts in Rust, like the borrow checker and the weekly blood sacrifices. Many popular async libraries are explicitly tied to heavyweight crates like tokio and futures, which aren’t good picks for many types of programs. There are also a lot of language features that need to be released for async to be used without an annoying amount of Boxing dynamic objects.

There’s one point, though, that I’ve heard quite frequently at this point. I think it’s misleading. Let’s talk about it.

What’s in a leak?

I’ve seen a lot of people say that async is a “leaky abstraction”. What this means is that the presence of async in a program forces you to bend the program’s control flow to accommodate it. If you have 100 files in your program, and one of those files uses async, you have to either write the entire program in async or resort to bohemian, mind-bending hacks to contain it. Just like an inlaw moving into your spare bedroom.

I do not mean memory leaks, which is what happens if you fail to free memory that you allocate. Neither async nor blocking code has a problem with memory leaking intrinsically.

Dependency Dog: If you want to see a good example of a leaky abstraction, consider AppKit. Not only is AppKit thread-unsafe to the point where many functions can only safely be called on the main() thread, it forces you into Apple’s terrifying Objective-C model. Basically any program that wants to have working GUI on macOS now needs to interface in Apple’s way, with basically no alternatives.

I’ve seen the “What Color is Your Function?” blogpost by Bob Nystrom referenced a lot in these discussions. This blogpost was originally written with JavaScript’s callbacks in mind. Fair enough. The callback model is hard to deal with, and its enduring popularity in the Rust ecosystem is something I have to write a blogpost about. He also mentions async/await as a potential solution to this problem, although one that he is unsatisfied with, as it still divides the ecosystem into asynchronous and synchronous halves.

While this blogpost may be correct when it comes to JavaScript and other, higher-level languages, I believe that Rust stands out in such a way that it’s not true for this language. In fact, I believe the opposite is true. Non-async code (or “blocking” code) is the real leaky abstraction.

Object Class: Safe

I’d like to discuss how you call blocking code from async code, and vice versa. That way we can compare.

Let’s make a table to describe how it goes calling functions from one “color” to another. You can call blocking code from blocking code without any issues. You can also call asynchronous code from asynchronous code trivially. There is also a strategy for calling asynchronous code from blocking code that I will go into shortly. So our table looks like this:

Note that not all code fits cleanly into the async/blocking categories. A notorious example is GUI code, which uses blocking semantics but overall acts a lot like async code in that it’s not allowed to block. But that’s a topic for another post.

When you write an async function, it returns a Future, which represents a value that will eventually be resolved. There are a lot of things you can do with a Future. You can race it against another Future, spawn it on an executor, and any number of other operations. It’s a point I delve deeper into in this post.

However, one of the simpler operations is to just wait for a Future to complete. Often, the waiting is done by blocking the current thread. So by “blocking on” the Future, we can effectively turn an async function into a synchronous call.

async fn my_async_code() { /* ... */ }

fn my_main_blocking_code() {
    use futures_lite::future::block_on;
    block_on(my_async_code());
}

block_on takes any Future, whether it’s !Send or not 'static or if it’s about to explode. So literally any async function can be called from synchronous code.

It’s relatively simple, too. block_on is implemented like this:

pub fn block_on<T>(future: impl Future<Output = T>) -> T{
    // A `Context` with a `Waker` is needed to poll a `Future`.
    let waker = waker_that_blocks_current_thread();
    let mut context = Context::from_waker(&waker);

    std::pin::pin!(future); // This used to require `unsafe` code, but doesn't anymore!

    // Poll the future in a loop, blocking the thread while we wait.
    loop {
        match future.as_mut().poll(&mut context) {
            Poll::Ready(value) => return value,
            Poll::Pending => block_thread_until_waker_wakes_us(),
        }
    }
}

Dependency Dog: The actual block_on is a little more complicated. It has some logic to reuse the waker between function calls, to reduce the overhead to one thread-local key access and nothing else.

Okay, but what if you don’t want futures_lite in your dependency tree? futures_lite isn’t the heaviest dependency on the block (that’s futures), but it’s still a non-negligible amount of code. No need to worry! There’s also pollster, which has zero (required) dependencies and consists of less than 100 lines of code.

fn my_main_blocking_code() {
    use pollster::block_on;
    block_on(my_async_code());
}

So, calling async code from blocking code is easy. Just call block_on. It’s that simple!

It’s not that simple

Of course it’s not that simple. I’m sure people familiar with actually calling async code from blocking code are screaming at the screen right now. So let’s address that.

There are a substantial number of async crates out there that run on top of tokio. They use tokio’s primitives, tokio’s executor, and tokio’s I/O semantics. Because of this, they rely on tokio’s runtime to be running in the background. If you try the above strategy for a crate that relies on tokio, it will fail at runtime with a panic.

No need to fear. We can start a tokio runtime and let it peacefully run in the background, forever. The libraries are able to pick up on this runtime and use it.

In main(), during your program initialization, put this:

use std::{future, thread};

fn main() {
    // Create a runtime.
    let rt = tokio::runtime::Builder::new_current_thread()
        .enable_all()
        .build()
        .unwrap();

    // Clone a handle to the runtime and send it to another thread.
    thread::spawn({
        let handle = rt.handle().clone();

        // Run the handle on this thread, forever.
        move || handle.block_on(future::pending::<()>())
    });

    // "Enter" the runtime and let it sit there.
    let _guard = rt.enter();

    // Block on any futures.
    pollster::block_on(my_async_function());
}

For any block_on calls in your application, the runtime will already be available. Note that you will need to call enter() on any new threads that use tokio primitives. Thankfully you can get a Handle to the runtime, which can be sent to any thread and is also cheaply clonable.

But that’s really it. Once you have the runtime humming away in the background, tokio futures should Just Work!

As an aside, another hitch is that block_on and functions like it are only available on std-enabled platforms. But the no_std async story is a blogpost for another day.

A Quick Segue into tokio::main

I’ve seen some people recommend using the tokio::main attribute to turn an async function into a blocking function, then calling that from your real code. For example:

#[tokio::main(flavor = "current_thread")]
async fn my_async_code() { /* ... */ }

fn main() {
    // `tokio::main` transparently converts `my_async_code` into a blocking function.
    my_async_code();
}

It’s a little impressive, if not a little hacky. The async function is turned into a blocking function using the proc macro.

But… just don’t do this. It means that, every time my_async_code is called, it spins up a tokio runtime, runs the code, then immediately throws that runtime away. For functions that are called a lot, it really adds up. In addition it makes the function signature misleading. It’s a blocking function, not an async function!

Meanwhile, for blocking code…

First off, I find async code to be more predictable than blocking code, in a weird way. Look at this function signature:

async fn my_async_function() { /* ... */ }

What does this tell you? I know that the Future returned by this function won’t block. I can place it in my executor of choice, or race it against any other Futures, without worrying that it will hog the execution loop. By convention poll() will probably run in a time period close to “instant”, before yielding and then letting something else take over.

Yes, there are buggy Futures out there. But well-formed Futures complete quickly.

Now look at this function signature:

fn my_function() { /* ... */ }

By looking at this function signature, can you tell how long it will take to run? Maybe it will complete instantly. Maybe it reads from a file and can potentially take between a few microseconds to a few whole seconds, depending on the file system. Maybe it blocks on a network socket. Maybe it processes a bunch of data in a loop, meaning that for large datasets it could run for a long time.

Yes, you can check the docs. But the docs usually fail to mention any of the above behavior, even for functions in the standard library. All of this ignores behavior dependent on generics/traits, too. It doesn’t matter how well-formed it is, you can’t tell how this function will act.

Often, when writing async programs, I have to be extra sure when I use blocking functions that I’m not accidentally blocking, which would lock up the entire event loop. In most cases this requires me to read the entire code of the function to understand what can go wrong.

If I can’t be sure it won’t block, I’ll need to wrap it in a Future that runs it on its own isolated thread. smol provides the blocking threadpool to run code on other threads, while tokio has a spawn_blocking function.

use blocking::unblock;

fn my_blocking_function() { /* ... */ }

async fn my_async_main() {
    unblock(|| my_blocking_function()).await;
}

This method comes at a cost. At the very least it’s an allocation for the blocking task’s state, as well as a few atomic operations to push it and then pop it from some thread pool’s task queue. At worst it spawns an entire new thread. Compare this to the cost of block_on which is usually one thread-local access.

But wait! unblock will send the function to another thread to be run. So, the function needs to be Send and 'static. This strategy doesn’t even work if the function relies on some kind of thread-unsafe state, like a RefCell. If the function takes a reference to some data you may need to wrap it in an Arc<Mutex>.

use blocking::unblock;
use std::sync::{Arc, Mutex};

fn my_blocking_function(data: &mut Foo) { /* ... */ }

async fn my_async_main() {
    let data = Arc::new(Mutex::new(/* ... */));
    unblock({
        let data = data.clone();
        move || my_blocking_function(&mut data.lock().unwrap())
    }).await;
}

I know this is a common complain with tokio’s style of async/await, but it’s just as bad the other way as well.

For the record, you can call block_on with any kind of borrowed data, with no hassles.

async fn my_async_code(foo: &mut Foo) { /* ... */ }

fn my_main_blocking_code() {
    use futures_lite::future::block_on;

    let mut data = /* ... */;
    block_on(my_async_code(&mut data)); // This works!
}

In order to avoid these issues I often have to segment out code that might block into their own sections. This lets me avoid the overhead of unblock for each function as a bonus.

fn some_blocking_segment(mut data: Foo) {
    do_something(&mut data);
    data.postprocess();
    print_the_data(&data);
}

async fn my_async_main() {
    // This doesn't work if `Foo` is `!Send`.
    let data = /* ... */;
    unblock(
        move || my_blocking_function(&mut data.lock().unwrap())
    }).await;
}

However this requires me to re-architect parts of my code into these segments. It gets difficult to interweave further async code into this sub-section as well. Yes, I can call async functions from block_on, but I’d really prefer to .await on it.

Say, doesn’t this seem very… leaky, to you?

Let’s Fix This

I don’t like to bring up a problem without also mentioning a possible solution. I mentioned documentation above; it would be nice if there was some kind of indicator that a function blocked.

/// Does a thing.
/// 
/// # Blocking
/// 
/// This function will block the first time it is called, as it is reading from
/// `/dev/random` to seed the random number generator.
fn my_blocking_function(data: &mut Foo) { /* ... */ }

It would be a Herculean effort, and I don’t think it’s a sustainable approach. If you’re writing a higher level library, it would be a lot to ask to check if your dependency’s dependency’s dependency maybe reads from a socket.

From a language standpoint, it would be nice if there was some kind of #[blocking] attribute to indicate that a function blocked, like so:

#[blocking]
fn my_blocking_function(data: &mut Foo) { /* ... */ }

Maybe there could even be some kind of tree-traversal to see if you were calling a #[blocking] function from async code, and then raise a warning. Unfortunately I’m unsure if this would work either. There are function that might block once and never again, or functions that only block under specific circumstances that the Rust compiler can’t predict. Not to mention, it would be difficult to solve the problem of data being processed in a tight loop.

So, I don’t know. There are some clever people on the language design team, so maybe they have better ideas.

Parting Shots

Frankly, I don’t think async code is leaky at all, and the ways that it does leak are largely due to library problems. Meanwhile blocking code leaks by its fundamental design. I hope you found this helpful and that it might remove some reservations about using async code in the future.

Perceptive Rustaceans may have noticed my activity has gone down as of late. There are a handful of different reasons for this. I’ve been the subject of a truly apocalyptic series of life events, including the death of a relative that rattled me to my core. I’ve had more responsibilities at work, leaving me with less time and energy to contribute. Maybe I’ve also lost a little bit of the college-kid enthusiasm that brought me to open source in the first place.

There’s another reason, too. I’ve been cooking up a project that’s been taking up most of my time. It’s certainly the largest project I’ve created in the open source world, and if I complete it, it will certainly be my crowning achievement.

I am writing a Rust compiler in pure C. No C++. No flex or yacc. Not even a Makefile. Nothing but pure C.

It’s called Dozer.

Wait, Why?

To understand why I’ve followed this path of madness, you first need to understand bootstrapping and why it is important.

Let’s say that you’ve written some code in Rust. In order to run this code, you need to compile it. A compiler is a program that parses your code, validates its correctness, and then transforms it into machine code that the CPU can understand.

Dependency Dog: Yes, it’s significantly more complicated than that. Except when it’s less complicated than that. Compilers are tricky to even describe.

For Rust, your main compiler is rustc. If you don’t know, this is the underlying program that cargo calls when you run cargo build. It’s fantastic software, and frankly a gem of the open source community. Its code quality is up there with the Linux kernel and the Quake III source code.

However, rustc itself is a program. So it needs a compiler to compile it from its source code to machine code. Say, what language is rustc written in?

Ah, rustc is a Rust program. Written in Rust, for the purpose of compiling Rust code. But, think about this for a second. If rustc is written in Rust, and rustc is needed to compile Rust code, that means you need to use rustc to compile rustc. Which is fine for us users, since we can just download rustc from the internet and use it.

But, who compiled the first rustc? There had to be a chicken before the egg, right? Where does it start?

…

Actually, that’s fairly simple. Every new version of rustc was compiled with the previous version of rustc. So rustc version 1.80.0 was compiled with rustc version 1.79.0. Which was, in turn, compiled with rustc version 1.78.0. And so on and so forth, all the way back to version 0.7 if the compiler. At that point, the compiler was written in OCaml. So all you needed was an OCaml compiler to get a fully functioning rustc program.

There, problem solved! We’ve figured out how to create rustc from first principles! All is well, let’s go back to business.

Just one more thing. We still need a version of the OCaml compiler for all of this to work. So what language is the OCaml compiler written in?

faceplant

Okay, okay, no worries! There is a project that can successfully compile the OCaml compiler using Guile, which is one of the many variants of Scheme, which is one of many variants of Lisp. Not to mention, Guile’s interpreter is written in C.

So this brings us, as all eventually things do, to the C programming language. We just compile it using GCC, and everything works out. So we just need to compile GCC, which is written using… C++?!

Okay, that’s a little unfair. GCC was written in C until version 5, and it’s not like there’s a shortage of C compilers written in C out there. For instance, consider TinyCC, which is written in C and handles not only compiling, but assembly and linking too.

…but that still doesn’t answer our question. What was the first C compiler written in? Assembly? Then what was the first assembler written in?

The Descent Principle

This is where we introduce the Bootstrappable Builds project. To me, this is one of the most fascinating projects in the open source community. It’s basically code alchemy.

Their Linux bootstrap process starts with a 512-byte binary seed. This seed contains what’s possibly the simplest compiler you can imagine: it takes hexadecimal digits and outputs the corresponding raw bytes. As an example, here part of the “source code” that’s compiled with this compiler.

31 C0           # xor ax, ax
8E D8           # mov ds, ax
8E C0           # mov es, ax
8E D0           # mov ss, ax
BC 00 77        # mov sp, 0x7700
FC              # cld ; clear direction flag
88 16 15 7C     # mov [boot_drive], dl

Note that everything after the pound sign is a comment, and all whitespace is stripped. Frankly, I’m not even sure this can be called a programming language. Still, it is technically analyzable, dissectable source code.

From here, this compiler compiles a very simple operating system, a barebones shell, and a slightly more advanced compiler. That compiler compiles a slightly more advanced compiler. A few steps later, you have something that roughly looks like assembly code.

DEFINE cmp_ebx,edx 39D3
DEFINE je 0F84
DEFINE sub_ebx, 81EB

:loop_options
    cmp_ebx,edx                         # Check if we are done
    je %loop_options_done               # We are done
    sub_ebx, %2                         # --options

Man, it’s weird to think of assembly code as being higher-level than anything else, right?

This is enough to get them to a very basic subset of C. Then they compile a slightly more advanced C compiler written in this subset. A few steps later they can compile TinyCC. From there they can bootstrap yacc, basic coreutils, Bash, autotools, and eventually GCC and Linux.

I’m not doing this justice, it’s a fascinating process. Every step is listed here.

Anyhow, you’ve essentially gone from “a binary blob small enough to be manually analyzed” to Linux, GCC, and basically everything else. But let’s start again from TinyCC.

Right now, Rust shows up very late into this process. They use mrustc, an alternative Rust implementation written in C++ that can compile rustc version 1.56. From here, they then compile up to modern Rust code.

The main issue here is that, by the time C++ is introduced into the bootstrap chain, the bootstrap is basically over. So if you wanted to use Rust at any point before C++ is introduced, you’re out of luck.

So, for me, it would be really nice if there was a Rust compiler that could be bootstrapped from C. Specifically, a Rust compiler that can be bootstrapped from TinyCC, while assuming that there are no tools on the system yet that could be potentially useful.

That’s Dozer.

The Plan

I’ve been working on Dozer for the past two months, putting my anemic free time to work on writing in a language that I kind of hate.

Dependency Dog: That’s a little unfair. C has some elegant qualities to it. Reality truly is what you make of it. It’s just that I would not let this code anywhere near production.

It’s written with no extensions, and so far both TinyCC and cproc are able to compile it with no issues. I’m using QBE as a backend. Other than that, I assume no tools exist on the system. Just a C compiler, some very basic shell implementation, and nothing else.

I won’t get into the raw experience of writing a compiler in this blogpost. But so far, I have the lexer done, as well as a sizable part of the parser. Macro/module expansion is something I’m putting off as long as possible, typechecking only supports i32, and codegen is a little bit rough. But it’s a start.

I can successfully compile this code:

fn rust_main() -> i32 {
    (2 - 1) * 6 + 3
}

So, where to from here? Here’s my plan.

• Slowly advance Dozer until it can compile some basic libc-using samples, then libcore, then rustc.
  • For the record, I’m planning on compiling rustc’s Cranelift backend, which is written entirely in Rust. Since we’re assuming we don’t have C++ yet, we can’t compile LLVM.
• Create a cargo equivalent that can use Dozer to compile Rust packages.
• Find out which sources in rustc are automaticaly generated and then strip them out. By the Bootstrappable project’s rules, automatically generated code is not allowed.
• Create a process that can be used to compile rustc and then cargo, then use our compiled versions of rustc/cargo to re-compile canonical versions of rustc/cargo.

This will definitely be the hardest project I’ve ever undertaken. Part of me doubts that I will be able to finish it. But you know what? It’s better to have tried and lost than to never have tried at all.

Stay tuned for more Dozer updates, as well as an explanation of the architecture I have planned.

If you want to hear me talk about smol, winit, open source in general, and how I think people should learn async Rust, consider giving it a watch!

I pride myself on smol packages being very easy to parse for anyone with a beginner’s level of experience in Rust. By that I mean, if you want to know how smol works, it should be very easy to pick up the source code, read through it, and understand how each individual part works.

Dependency Dog: Wait, do people normally read source code for fun?

notgull: No, I think that’s just a “me” thing.

There’s a few crates that are a little harder to take as bathroom reading. There’s polling, which does a lot of low-level system interaction to make asynchronous I/O work. I’ve done my best to make it interesting, but there’s not a whole lot to say about a crate that’s basically following the OS’s instruction manual.

Then there’s async-task. async-task’s philosophy runs counter to the rest of smol. When it comes to optimization, smol generally tries to go for safety and reasonability over crazy optimizations with diminishing returns. For async-task however, we take the gloves off. We go all out to make sure tasks are as small and use as few resources as possible.

notgull: This is actually because async-task predates smol! It was originally used as the task implementation for async-std.

I’d like to provide this series of blogposts as a reference for how async-task works, how you might arrive to an implementation like async-task organically, and how it was optimized into its current state.

notgull: As a heads up: most posts for this blog assume an intermediate knowledge of Rust. However, this post is intended for readers who may not already be familiar with concepts like executors or dynamic dispatch.

Of course, it may be a good idea to review the basics, even if you’re an expert.

Background Basics

Let’s say you have two Futures; blocks of asynchronous code that can be ran concurrently. You want to run both of them at once.

// Future #1
let foo = async {
    let x = my_function().await;
    do_something(x).await;
};

// Future #2
let bar = async {
    for _ in 0..50 {
        respond_to_user().await;
    }
};

Dependency Dog: Wait… Future? async? await? What’s that?

notgull: They’re Rust’s user-space concurrency building blocks! If you need a refresher on what these mean, it may be worth it to read the async book.

Running two futures at a time can be done very easily. First, we bring in the futures-lite crate:

$ cargo add futures-lite
    Updating crates.io index
      Adding futures-lite v2.3.0 to dependencies
             Features:
             + alloc
             + fastrand
             + futures-io
             + parking
             + race
             + std
             - memchr
    Updating crates.io index

Then, we can use the zip combinator to run both Futures at the same time. Finally, we can use block_on to poll the resulting Future until it completes. It looks like this:

use futures_lite::future;

// Run the two futures in parallel.
let combined = future::zip(foo, bar);

// Block on the combined future until it completes.
future::block_on(combined);

How the zip combinator works is as follows:

• It tries to poll the first Future. If it is ready, it takes the result and saves it the memory. It remembers not to poll the first Future again.
• It does the same thing for the second Future. It polls it if it hasn’t finished. If it has, it saves the result.
• Once both Futures are finished, it returns a tuple of the result.

Using this strategy, we can poll two Futures at the same time. The following diagram shows what this looks like in practice:

Note that, even though only one thread of execution is used, it appears as though the futures are run at the same time.

Scalability Solutions

The zip combinator works for very simple cases of concurrency, but falls apart for higher-level scenarios. Let’s say you want to poll four futures at once. (The horror!)

let baz = async { /* ... */ };
let cap = async { /* ... */ };

Then, you would need to call zip three times!

let combined = future::zip(
    future::zip(foo, bar),
    future::zip(baz, cap)
);

You run into the some problems too, like:

• You can only run a fixed number of futures at once. If you might run a variable number of futures, you’re out of luck.
• What if you want to cancel one of the futures halfway through?
• Each future is polled every time zip is woken up. This means polling zip is an O(n) operation. This is sometimes known as the “thundering herd” problem.

Let’s try to solve these problems. Without any prior art, I mean. We can solve the “fixed number of futures” problem pretty easily. Consider the [slab] crate, which lets us set up an indexed list of objects. It’s similar to an arena. We can fit our futures in there.

$ cargo add slab
    Updating crates.io index
      Adding slab v0.4.9 to dependencies
             Features:
             + std
             - serde
    Updating crates.io index

Let’s also box the Futures, so we can use multiple different implementors of Future in our same zip.

use slab::Slab;
use std::future::Future;
use std::pin::Pin;

struct GiantZip {
    // Completed futures are represented by `None`.
    futures: Slab<Option<Pin<Box<dyn Future<Output = ()>>>>>
}

impl GiantZip {
    fn new() -> Self {
        Self {
            futures: Slab::new()
        }
    }
}

Let’s have an insert method that can be used to add a new Future to this new zip-equivalent. It will return a key that can be used to look up the future in our list.

impl GiantZip {
    fn insert<F: Future<Output = ()> + 'static>(&mut self, future: F) -> usize {
        self.futures.insert(Some(Box::pin(future)))
    }
}

Dependency Dog: The Future needs to be 'static because it’s being boxed and pinned on the heap without a lifetime. It’s possible to work around this by adding a lifetime to GiantZip here, but let’s keep it simple for now.

Finally, let’s make it so polling the GiantZip tries to resolve every one of the futures contained within.

impl Future for GiantZip {
    type Output = ();

    fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
        let mut unfinished = false;

        for (_, future_slot) in self.futures.iter_mut() {
            if let Some(future) = future_slot.as_mut() {
                // Try to poll this future.
                match future.as_mut().poll(cx) {
                    Poll::Ready(()) => {
                        // Set the future to `None`.
                        *future_slot = None;
                    },

                    Poll::Pending => {
                        // We are unfinished; return Pending.
                        unfinished = true; 
                    }
                }
            }
        }

        if unfinished {
            Poll::Pending
        } else {
            Poll::Ready(())
        }
    }
}

Finally, we can test this out on futures that actually do something.

$ cargo add async-channel
    Updating crates.io index
      Adding async-channel v2.2.0 to dependencies
             Features:
             + std
    Updating crates.io index

// Create a channel with a capacity of 1.
let (sender, recv) = async_channel::bounded(1);

// This is basically an `async fn` that sends a number over the channel.
let our_future = |i: i32| {
    let sender = sender.clone();
    async move { sender.send(i).await.ok(); }
};

// Create a future that reads from the channel.
let reader = async move {
    for _ in 0..3 {
        println!("{}", recv.recv().await.unwrap());
    }
};

// Use the GiantZip to poll all of these at once.
let mut zipper = GiantZip::new();
zipper.insert(our_future(1));
zipper.insert(our_future(2));
zipper.insert(our_future(3));
zipper.insert(reader);

// Wait for them to finish.
future::block_on(zipper);

When we run it, we see this:

$ time cargo run -q
1
2
3
cargo run -q  0.03s user 0.03s system 93% cpu 0.064 total

That’s pretty fast, but that’s only because we have a low number of futures. If we have 10,000,000 futures (not an unrealistic number for a web server!), it will run much slower.

$ time cargo run -q
0
1
2
cargo run -q  9.16s user 0.70s system 100% cpu 9.803 total

notgull: It’s hard to express in the textual format, but each line had a few seconds’ delay between each of them. So it’s taking a while to get to the future that actually prints the line.

In addition to being inefficient, we’ve also stumbled upon another issue: GiantZip is unfair. The reader Future is at the very end of the futures list, which means it’s processed last when polling. Since a lot of the futures end up being blocked on reader, it means polling the GiantZip takes a lot longer than it normally should.

Thankfully, we can solve the O(n) problem and also (kind of) solve the fairness problem in one fell swoop. Instead of polling every future every time we poll, we should only poll the ones whose Wakers have been woken up. Since we know those ones are ready, we should only poll those.

Let’s add a queue structure to the GiantZip that contains the indexes of the futures that are ready to be woken. I’m wrapping it in an Arc and a Mutex for reasons that will become obvious later.

use std::collections::VecDeque;
use std::sync::{Arc, Mutex};

struct GiantZip {
    // Completed futures are represented by `None`.
    futures: Slab<Option<Pin<Box<dyn Future<Output = ()>>>>>,

    // NEW: Queue of futures that are waiting to be woken up.
    queue: Arc<Mutex<VecDeque<usize>>>,
}

impl GiantZip {
    fn new() -> Self {
        Self {
            futures: Slab::new(),
            queue: Arc::new(Mutex::new(VecDeque::new()))
        }
    }
}

Now, when we first insert the Future into the GiantZip, we have to mark it as ready. This is done by just pushing the index of the future into the queue.

impl GiantZip {
    fn insert<F: Future<Output = ()> + 'static>(&mut self, future: F) -> usize {
        // NEW: Save the index and push it to the back of the queue before returning.
        let index = self.futures.insert(Some(Box::pin(future)));
        self.queue.lock().unwrap().push_back(index);
        index
    }
}

We also need to have a way for the Future to mark itself as ready. The Future calls the Waker when it is ready to be woken up, so we can just create a Waker that wraps around the top-level Waker, but also marks the current future as ready.

We’ll bring in waker-fn to make this easier. A Waker is just a glorified callback, so we can easily represent it as one.

$ cargo add waker-fn
    Updating crates.io index
      Adding waker-fn v1.1.1 to dependencies
    Updating crates.io index

Let’s make creating the Waker a helper function on GiantZip, to keep things clean.

use waker_fn::waker_fn;
use std::task::Waker;

impl GiantZip {
    /// Create a waker that wakes the future in the provided slot.
    fn waker_for_slot(&self, index: usize, toplevel: &Waker) -> Waker {
        // Clone shared resources.
        // *This* is why we made `queue` wrapped in an `Arc`, by the way.
        let queue = self.queue.clone();
        let toplevel = toplevel.clone();

        // Create a waker.
        waker_fn(move || {
            // Mark the future as ready.
            queue.lock().unwrap().push_back(index);

            // Wake the toplevel `block_on` waker, so the GiantZip poll()
            // implementation is ran again.
            toplevel.wake_by_ref();
        })
    }
}

Finally, we can adjust the poll() implementation for GiantZip such that it pops from the queue instead of polling each and every future in the list.

impl GiantZip {
    /// Get the next index in the list.
    fn next_index(&self) -> Option<usize> {
        self.queue.lock().unwrap().pop_front()
    }
}

impl Future for GiantZip {
    type Output = ();

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
        // Get around Rust's pinning rules.
        let this = self.get_mut();

        // NEW: We drain the "queue" instead of iterating over every future.
        // Make sure not to hold the lock while polling; if a future is woken by another future,
        // it would deadlock otherwise.
        while let Some(index) = this.next_index() {
            // NEW: Create a waker to poll this future with.
            let waker = this.waker_for_slot(index, cx.waker());
            let mut slot_context = Context::from_waker(&waker);

            let future_slot = match this.futures.get_mut(index) {
                Some(slot) => slot,
                None => continue
            };
            if let Some(future) = future_slot.as_mut() {
                // Try to poll this future.
                match future.as_mut().poll(&mut slot_context) {
                    Poll::Ready(()) => {
                        // Set the future to `None`.
                        *future_slot = None;
                    },

                    Poll::Pending => {}
                }
            }
        }

        if this.futures.iter().any(|(_, fut)| fut.is_some()) {
            Poll::Pending
        } else {
            Poll::Ready(())
        }
    }
}

When we run the program, we see the following output:

$ time cargo run -q
0
1
2
cargo run -q  5.03s user 0.62s system 101% cpu 5.583 total

There is still quite a bit of contention caused by the initial burst of futures as well as the last burst of futures. But, there is no more delay between the printing of the numbers. This indicates that the runtime is being used more efficiently.

This also solves our cancellation problem; we can just add a remove method to the GiantZip to remove a keyed future from the list.

impl GiantZip {
    fn remove(&mut self, index: usize) {
        self.futures.remove(index);
    }
}

let key = zipper.insert(async { panic!() });
// Actually, might not be the best idea to run that task.
zipper.remove(key);

Now we’ve solved all of our problems… and introduced a million new ones.

Persistent Problems

I’ve deliberately made some mistakes in the above example, in order to illustrate how fixing those mistakes can lead to a very important data pattern. So let’s discuss those mistakes.

Dependency Dog: “Deliberately”, you say?

notgull: Hey, I’ll have you know, I’m just following the natural evolution of the async/await pattern.

Dependency Dog: Did the “natural evolution” force you to re-allocate a new Waker every time you polled a future?

The first issue is that all of this is very inefficient. Ignoring our suboptimal queueing structure, we have three main allocations here:

• We have a Vec to hold our futures inside of.
• Each individual Future requires its own Box.
• Every time the GiantZip is polled we have to create a Waker to poll it with. The [waker_fn] crate allocates this inside of an Arc.

Specifically we should be concerned about the Waker allocation, since it occurs on the hot path. We should try our best to make sure that we can create a Waker without allocating.

There are some other persistent problems, like:

• It’s very easy to misuse the API. usize isn’t a great type to index by, for a collection of hotly-polled futures.
• You can’t get the result of a Future after it completes… or use a Future that returns anything other than (), for that matter.
• It is very difficult (albeit not impossible) to remove a Future from the list while it is running.

We’ll begin to address these problems in the next blog post, when we start to build a real task abstraction.

This is a common question that I’ve seen a lot in the Rust community. Frankly, I completely understand where it’s coming from.

Rust is a low-level language that doesn’t hide the complexity of coroutines from you. This is in opposition to languages like Go, where async happens by default, without the programmer needing to even consider it.

Smart programmers try to avoid complexity. So, they see the extra complexity in async/await and question why it is needed. This question is especially pertinent when considering that a reasonable alternative exists in OS threads.

Let’s take a mind-journey through async and see how it stacks up.

Background Blitz

Rust is a low-level language. Normally, code is linear; one thing runs after another. It looks like this:

fn main() {
    foo();
    bar();
    baz();
}

Nice and simple, right?

However, sometimes you will want to run many things at once. The canonical example for this is a web server. Consider the following written in linear code:

fn main() -> io::Result<()> {
    let socket = TcpListener::bind("0.0.0.0:80")?;

    loop {
        let (client, _) = socket.accept()?;
        handle_client(client)?;
    }
}

Imagine if handle_client takes a few milliseconds, and two clients try to connect to your webserver at the same time. You’ll run into a serious problem!

• Client #1 connects to the webserver, and is accepted by the accept() function. It starts running handle_client().
• Client #2 connects to the webserver. However, since accept() is not currently running, we have to wait for handle_client() for Client #1 to finish running.
• After waiting a few milliseconds, we get back to accept(). Client #2 can connect.

Now imagine that instead of two clients, there are two million simultaneous clients. At the end of the queue, you’ll have to wait several minutes before the web server can help you. It becomes un-scalable very quickly.

Obviously, the embryonic web tried to solve this problem. The original solution was to introduce threading. By saving the value of some registers and the program’s stack into memory, the operating system can stop a program, run another program in its place, then resume running that program later. Essentially, it allows for multiple routines (or “threads”, or “processes”) to run on the same CPU.

Using threads, we can rewrite the above code as follows:

fn main() -> io::Result<()> {
    let socket = TcpListener::bind("0.0.0.0:80")?;

    loop {
        let (client, _) = socket.accept()?;
        thread::spawn(move || handle_client(client));
    }
}

Now, the client is being handled by a separate thread than the one handling waiting for new connections. Great! This avoids the problem by allowing concurrent thread access.

• Client #1 is accepted by the server. The server spawns a thread that calls handle_client.
• Client #2 tries to connect to the server.
• Eventually, handle_client blocks on something. The OS saves the thread handling Client #1 and brings back the main thread.
• The main thread accepts Client #2. It spawns a separate thread to handle Client #2. With only a few microseconds of delay, Client #1 and Client #2 are run in parallel.

Threads work especially well when you consider that production-grade web servers have dozens of CPU cores. It’s not just that the OS can give the illusion that all of these threads run at the same time; it’s that the OS can actually make them all run at once.

Eventually, for reasons I’ll elaborate later, programmers wanted to bring this concurrency out of the OS space and into the user space. There are many different models for userspace concurrency. There is event-driven programming, actors, and coroutines. The one Rust settled on is async/await.

To oversimplify, you compile the program as a grab-bag of state machines that can all be run independently of another. Rust itself provides a mechanism for creating state machines; the mechanism of async and await. The above program in terms of async/await would look like this, written using smol:

#[apply(smol_macros::main!)]
async fn main(ex: &smol::Executor) -> io::Result<()> {
    let socket = TcpListener::bind("0.0.0.0:80").await?;

    loop {
        let (client, _) = socket.accept().await?;
        ex.spawn(async move {
            handle_client(client).await;
        }).detach();
    }
}

• The main function is preceded with the async keyword. This means that it is not a traditional function, but one that returns a state machine. Roughly, the function’s contents correspond to that state machine.
• await includes another state machine as a part of the currently running state machine. For accept(), it means that the state machine will include it as a step.
• Eventually, one of the inner functions will yield, or give up control. For example, when accept() waits for a new connection. At this point the entire state machine will yield its execution to the higher-level executor. For us, that is smol::Executor.
• Once execution is yielded, the Executor will replace the current state machine with another one that is running concurrently, spawned through the spawn function.
• We pass an async block to the spawn function. This block represents an entire new state machine, independent of the one created by the main function. All this state machine does is run the handle_client function.
• Once main yields, one of the clients is selected to run in its place. Once that client yields, the cycle repeats.
• You can now handle millions of simultaneous clients.

Of course, user-space concurrency like this introduces an uptick in complexity. When you’re using threads, you don’t have to deal with executors and tasks and state machines and all.

If you’re a reasonable person, you might be asking “why do we need to do all of this? Threads work well; for 99% of programs, we don’t need to involve any kind of user-space concurrency. Introducing new complexity is technical debt, and technical debt costs us time and money.

“So why wouldn’t we use threads?”

Timeout Trouble

Perhaps one of Rust’s biggest strengths is composability. It provides a set of abstractions that can be nested, built upon, put together, and expanded upon.

I recall that the thing that made me stick with Rust is the Iterator trait. It blew my mind that you could make something an Iterator, apply a handful of different combinators, then pass the resulting Iterator into any function that took an Iterator.

It continues to impress me how powerful it is. Let’s say you want to receive a list of integers from another thread, only take the ones that are immediately available, discard any integers that aren’t even, add one to all of them, then push them onto a new list.

That would be fifty lines and a helper function in some other languages. In Rust it can be done in five:

let (send, recv) = mpsc::channel();
my_list.extend(
    recv.try_iter()
        .filter(|x| x & 1 == 0)
        .map(|x| x + 1)
);

The best thing about async/await is that it lets you apply this composability to I/O-bound functions. Let’s say you have a new client requirement; you want to add a timeout to your above function. Assume that our handle_client above function looks like this:

async fn handle_client(client: TcpStream) -> io::Result<()> {
    let mut data = vec![];
    client.read_to_end(&mut data).await?;
    
    let response = do_something_with_data(data).await?
    client.write_all(&response).await?;

    Ok(())
}

If we want to add, say, a three-second timeout, we can combine two combinators to do that:

• The race function takes two futures and runs them at the same time.
• The Timer future waits for some time before returning.

Here is what the final code looks like:

async fn handle_client(client: TcpStream) -> io::Result<()> {
    // Future that handles the actual connection.
    let driver = async move {
        let mut data = vec![];
        client.read_to_end(&mut data).await?;
        
        let response = do_something_with_data(data).await?
        client.write_all(&response).await?;

        Ok(())
    };

    // Future that handles waiting for a timeout.
    let timeout = async {
        Timer::after(Duration::from_secs(3)).await;

        // We just hit a timeout! Return an error.
        Err(io::ErrorKind::TimedOut.into())
    };

    // Run both in parallel.
    driver.race(timeout).await
}

I find this to be a very easy process. All you have to do is wrap your existing code in an async block and race it against another future.

An added bonus of this approach is that it works with any kind of stream. Here, we use a TcpStream. However we can easily replace it with anything that implements impl AsyncRead + AsyncWrite. It could be a GZIP stream on top of the normal stream, or a Unix socket, or a file. async just slides into whatever pattern you need from it.

Thematic Threads

What if we wanted to implement this in our threaded example above?

fn handle_client(client: TcpStream) -> io::Result<()> {
    let mut data = vec![];
    client.read_to_end(&mut data)?;
    
    let response = do_something_with_data(data)?
    client.write_all(&response)?;

    Ok(())
}

Well, it’s not easy. Generally, you can’t interrupt the read or write system calls in blocking code, without doing something catastrophic like closing the file descriptor (which can’t be done in Rust).

Thankfully, TcpStream has two functions set_read_timeout and set_write_timeout that can be used to set the timeouts for reading and writing, respectively. However, we can’t just use it naively. Imagine a client that sends one byte every 2.9 seconds, just to reset the timeout.

So we have to program a little defensively here. Due to the power of Rust combinators, we can write our own type wrapping around the TcpStream to program the timeout.

// Deadline-aware wrapper around `TcpStream.
struct DeadlineStream {
    tcp: TcpStream,
    deadline: Instant
}

impl DeadlineStream {
    /// Create a new `DeadlineStream` that expires after some time.
    fn new(tcp: TcpStream, timeout: Duration) -> Self {
        Self {
            tcp,
            deadline: Instant::now() + timeout,
        }
    }
}

impl io::Read for DeadlineStream {
    fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
        // Set the deadline.
        let time_left = self.deadline.saturating_duration_since(Instant::now());
        self.tcp.set_read_timeout(Some(time_left))?;

        // Read from the stream.
        self.tcp.read(buf)
    }
}

impl io::Write for DeadlineStream {
    fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
        // Set the deadline.
        let time_left = self.deadline.saturating_duration_since(Instant::now());
        self.tcp.set_write_timeout(Some(time_left))?;

        // Read from the stream.
        self.tcp.write(buf)
    }
}

// Create the wrapper.
let client = DeadlineStream::new(client, Duration::from_secs(3));

let mut data = vec![];
client.read_to_end(&mut data)?;

let response = do_something_with_data(data)?
client.write_all(&response)?;

Ok(())

On one hand, it could be argued that this is elegant. We used Rust’s capabilities to solve the problem with a relatively simple combinator. I’m sure it would work well enough.

On the other hand, it’s definitely hacky.

• We’ve locked ourselves into using TcpStream. There’s no trait in Rust to abstract over using the set_read_timeout and set_write_timeout types. So it would take a lot of additional work to make it use any kind of writer.
• It involves an extra system call for setting the timeout.
• I imagine this type is much more unwieldy to use for the kinds of actual logic that web servers demand.

If I saw this code in production, I would ask the author why they avoided using async/await to solve this problem. This is the phenomenon I was describing in my post “Why you might actually want async in your project”. Quite frequently I encounter a pattern where synchronous code can’t be used without contortion, so I have to rewrite it in async.

Async Success Stories

There’s a reason why the HTTP ecosystem has adopted async/await as its primary runtime mechanism, even for clients. You can take any function that makes an HTTP call, and make it fit whatever hole or use case you want it to.

tower is probably the best example of this phenomenon I can think of, and it’s really the thing that made me realize how powerful async/await can be. If you implement your service as an async function, you get timeouts, rate limiting, load balancing, hedging and back-pressure handling. All of that for free.

It doesn’t matter what runtime you used, or what you’re actually doing in your service. You can throw tower at it to make it more robust.

macroquad is a miniature Rust game engine that aims to make game development as easy as possible. Its main function uses async/await in order to run its engine. This is because async/await is really the best way in Rust to express a linear function that needs to be stopped in order to wait for something else.

In practice, this can be extremely powerful. Imagine simultaneously polling a network connection to your game server and your GUI framework, on the same thread. The possibilities are endless.

Improving Async’s Image

I don’t think the issue is that some people think threads are better than async. I think the issue is that the benefits of async aren’t widely broadcast. This leads some people to be misinformed about the benefits of async.

If this is an educational problem, I think it’s worth taking a look at the educational material. Here’s what the Rust Async Book says when comparing async/await to operating system threads.

OS threads don’t require any changes to the programming model, which makes it very easy to express concurrency. However, synchronizing between threads can be difficult, and the performance overhead is large. Thread pools can mitigate some of these costs, but not enough to support massive IO-bound workloads.

- Rust Async Book, various authors

I think this is a consistent problem throughout the async community. When someone asks the question of “why do we want to use this over OS threads”, people have a tendency to kind of wave their hand and say “async has less overhead. Other than that, everything’s the same.”

This is the reason why web server authors switched to async/await. It’s how they solved the C10k problem. But, it’s not going to be the reason why everyone else switches to async/await.

Performance gains are fickle and can disappear in the wrong circumstances. There are plenty of cases where a threaded workflow can be faster than an equivalent async workflow (mostly, in the case of CPU bound tasks). I think that we, as a community, have over-emphasized the ephemeral performance benefits of async Rust while downplaying its semantic benefits.

In the worst case, it leads to people shrugging off async/await as “a weird thing that you resort to for niche use cases”. It should be seen as a powerful programming model that lets you succinctly express patterns that can’t be expressed in synchronous Rust without dozens of threads and channels.

I also think there’s a tendency to try to make async Rust “just like sync Rust” in a way that encourages negative comparison. By “tendency”, I mean that it’s the stated roadmap for the Rust project, saying that “that writing async Rust code should be as easy as writing sync code, apart from the occasional async and await keyword.”.

I reject this framing because it’s fundamentally impossible. It’s like trying to host a pizza party on a ski slope. Sure, you can probably get 99% of the way there, especially if you’re really talented. But there are differences that the average bear will notice, no matter how good you are.

We shouldn’t be trying to force our model into unfriendly idioms to appease programmers who refuse to adopt another type of pattern. We should be trying to highlight the strengths of Rust’s async/await ecosystem; its composability and its power. We should be trying to make it so async/await is the default choice whenever a programmer reaches for concurrency. Rather than trying to make sync Rust and async Rust the same, we should embrace the differences.

In short, we shouldn’t be using technical reasons to argue for a semantic model. We should be using semantic reasons.