The async ecosystem and its arguments

This lesson is mostly Part 6 arriving early: the social machine around async Rust. Your goal here is not to learn an API but to be able to hold a position in a conversation and know what the arguments actually are.

Why the runtime choice is not a detail

std defines Future and stops. It provides no executor, no async I/O, no timers, no async channels.

So a runtime supplies all of that — and each supplies its own incompatible versions. tokio::time::sleep does not work on a smol executor. tokio::net::TcpStream is a different type from the async-std equivalent. A library that uses tokio’s I/O types forces tokio on everything above it.

The consequence: the runtime choice propagates through your dependency tree, and it is very hard to change later. This is unlike Node, where there is one event loop, or Go, where the runtime is the language.

Whether that was the right call is genuinely contested. The argument for: an embedded target and a web server have irreconcilable requirements, and freezing one executor into std would have been worse. The argument against: nearly everyone picks tokio anyway, so the ecosystem paid for flexibility almost nobody uses.

Both positions are respectable. Having a view, and knowing it is contested, is the point.

The landscape

Work-stealing versus thread-per-core

Tokio’s default scheduler is work-stealing: tasks live in per-worker queues, and an idle worker steals from a busy one. Load balances itself, no tuning.

The cost is that a task may resume on a different thread from the one it started on. That is precisely why tokio::spawn requires Send (3.6), and it costs cache locality — the task’s data is warm in one core’s cache and it wakes up on another.

Thread-per-core pins each task to one core for its whole life. glommio and monoio do this, usually with io_uring. Benefits: better cache behaviour, no cross-thread synchronisation, and no Send requirement, so Rc and RefCell become usable in async code again.

The cost is that a slow task cannot be rebalanced — one core can be saturated while others idle — so it wants uniform, short tasks.

You can get most of the way there with tokio alone by using Builder::new_current_thread per core, which some teams do.

For a design discussion, the honest summary: work-stealing is the right default and thread-per-core is a specialisation for high-throughput, uniform, Linux-only workloads. If someone proposes thread-per-core, the question is whether they have measured.

Function colouring

The best-known complaint, from a 2015 essay usually referenced as “What Color Is Your Function?”.

An async fn can only be .awaited from an async context. So async-ness propagates upward through every caller. Make one leaf function async and you may have to change fifty call sites.

Practical consequences you will actually meet:

• Libraries often ship both versions (reqwest and reqwest::blocking), duplicating the API.
• Trait implementations you do not control cannot be made async, which is why spawn_blocking and block_in_place exist as escape hatches.
• Drop cannot be async — there is no async fn drop. Async cleanup is an open problem, and it is why you see explicit .close().await methods that you must remember to call. This is a real, current, unfixed gap.

The counter-argument, which is fair: colouring is not arbitrary. An async function genuinely is different — it can suspend, it can be cancelled, it needs an executor. Hiding that (as Go does with goroutines) buys ergonomics and gives up control over scheduling and cancellation.

Async traits

Historically the biggest pain point, and substantially better now.

Since Rust 1.75, async fn in traits works:

trait Fetcher {
    async fn fetch(&self, url: &str) -> Result<String>;
}

Two limitations remain, and they are the reason you still see the old workaround:

You cannot use it with dyn. An async fn in a trait desugars to a method returning impl Future, and return-position impl Trait is not dyn-compatible (3.3). So Box<dyn Fetcher> does not work.

You cannot require the returned future to be Send. Which means a trait method’s future may not be spawnable, and there is no way to demand it in the trait definition. The trait_variant crate generates a Send variant of a trait to work around this.

So the #[async_trait] macro — which boxes every returned future — is still common. When you see it, read: this trait needs dyn, or needs Send futures, and predates or works around the current limitations. It costs an allocation per call, which is usually irrelevant and occasionally is not.

Did you need async at all?

The question worth asking loudest, because it is asked least.

Async exists for many concurrent, I/O-bound operations. Tens of thousands of open connections mostly waiting. That is what it is good at, and it is genuinely excellent at it.

If your service handles a few hundred concurrent requests, threads are probably simpler and fast enough:

• No colouring — any function can call any function.
• Ordinary stack traces and ordinary debuggers.
• No runtime dependency, no Send + 'static puzzles, no cancellation-safety concerns.
• OS threads scale further than most people assume — thousands are fine on Linux.

The failure mode is a team adopting async because it sounds like the serious choice, then spending weeks on Send bounds and Arc<Mutex<...>> for a service that would have been three hundred lines of std::thread and a channel.

The tech-lead version: async Rust is a performance tool with a real complexity cost. Reach for it when connection counts are high and work is I/O-bound. Do not reach for it because the blog posts do.

Part 4 checkpoint

1. T: Sync is equivalent to what statement about &T? (4.1)
2. What exactly does Rust prevent — and what does it not? (4.1)
3. What does thread::scope buy over spawn? (4.2)
4. What does calling an async fn do? (4.3)
5. Why are two sequential .awaits not concurrent? (4.3)
6. Why does holding a MutexGuard across .await break spawn? (4.3)
7. Why is choosing a runtime a bigger decision in Rust than in Node? (4.4)

Next: 5.1 — unsafe does not turn off the borrow checker.