Part 3 · The compile-time machine II — types & traits
Where does this method come from?
Auto-ref, auto-deref, and coercion happen silently on every method call. This is the single most useful reading skill in Rust, and almost nobody teaches it.
What this lesson is trying to break (3)
- A method call is a simple lookup on the receiver's type.
- If a method is not in the type's documentation, it does not exist.
- `&String` and `&str` are converted by some special-case rule for strings.
You have been reading Rust for three parts now, and every method call you have seen involved machinery nobody has described. This lesson describes it.
It is the highest-leverage reading skill on the site. The question “where does this method come from?” is the one that stalls people most often in unfamiliar code, and it has a precise, learnable answer.
The problem
let s = String::from("hello world");
let n = s.len();
let first = s.split(' ').next();
let up = s.to_uppercase();
Now open the docs for String. len is not there as a String method in the
way you would expect. split is not on String at all. to_uppercase is not on
String either.
All three are methods on str. And s is a String, which is a different type.
Nothing about this is a special case for strings. It is the general algorithm.
The algorithm
For receiver.method(args), the compiler:
Step 1 — build a candidate list of receiver types. Start with the receiver’s
type T, then repeatedly apply Deref to get *T, **T, and so on until it
cannot deref further. Then append one unsized coercion (e.g. [T; N] to [T]).
For s: String that gives: String, then str (because
String: Deref<Target = str>).
Step 2 — for each candidate type U, in order, look for a method whose
receiver is:
U— by value&U— by shared reference (auto-ref)&mut U— by exclusive reference (auto-ref)
Step 3 — the first match wins. Inherent methods (in a plain impl Type
block) are checked before trait methods. Trait methods only count if the trait is
in scope.
So s.len():
candidate `String` → is there String::len taking String, &String, &mut String?
(no inherent `len` on String matching)
candidate `str` → is there str::len taking str? no.
→ taking &str? YES ✓
so this is really str::len(&*s)
The compiler inserted a deref and a reference for you. That is all
s.len() ever was.
#[derive(Clone)]
struct Wrapper<T>(T);
struct NotClone;
fn main() {
let w = Wrapper(NotClone);
let r = &w;
let x = r.clone();
}Does this compile, and if so what is x?
It compiles, and x is &Wrapper<NotClone> — not a Wrapper.
This is one of the nastiest silent gotchas in Rust and it catches experienced people.
#[derive(Clone)] on Wrapper<T> generates impl<T: Clone> Clone for Wrapper<T>
— note the added bound. NotClone does not implement Clone, so
Wrapper<NotClone> is not Clone.
Now run the algorithm on r.clone() where r: &Wrapper<NotClone>:
- Candidate
&Wrapper<NotClone>by value: is there aclonetaking&Wrapper<NotClone>? The implClone for &Texists — references are alwaysClone— and its receiver is&&Wrapper<...>. Try by-value first: no. Try&U: yes,<&Wrapper<NotClone> as Clone>::clonematches.
So you cloned the reference. It compiles, does nothing useful, and there is no warning.
The symptom appears later, as a confusing type error somewhere downstream, or as
a mysterious performance win from a “clone” that never copied anything. When
someone says “my .clone() isn’t doing anything”, this is almost always why.
clippy has a lint for it (clone_double_ref / clone_on_copy depending on the
shape), which is a decent argument for running clippy in CI.
Deref coercion
The same Deref relation also applies at coercion sites — function
arguments, let with a type annotation, and returns:
fn takes(s: &str) {}
let owned = String::from("hi");
takes(&owned); // &String coerced to &str
The rule: &T coerces to &U when T: Deref<Target = U>, and it applies
transitively.
Varies: only the smart pointer wrapping the same underlying data
| You have | Coerces to | Because |
|---|---|---|
&String | &str | String: Deref<Target = str> |
&Vec<T> | &[T] | Vec<T>: Deref<Target = [T]> |
&Box<T> | &T | Box<T>: Deref<Target = T> |
&Rc<String> | &str | transitively: Rc → String → str |
&mut Vec<T> | &mut [T] | DerefMut, the same rule for exclusive refs |
This is why idiomatic signatures take &str and &[T] rather than &String and &Vec
That caption is a real API-design guideline and a good review comment:
Prefer
&strover&String, and&[T]over&Vec<T>, in parameters. The coercion means you lose nothing and accept more callers.
Traits must be in scope
This is the other half of “where did that method come from”, and it is the cause of a very specific frustration.
use std::io::Read; // ← without this line, `read_to_string` does not exist
let mut f = File::open("x")?;
let mut s = String::new();
f.read_to_string(&mut s)?;
File has the method — through the Read trait — but trait methods are only
callable if the trait is imported. Delete the use and you get “no method named
read_to_string found”, which is technically accurate and completely
unhelpful.
Modern rustc usually suggests the missing import, which has made this far less
painful than it used to be. But when reading code, the reverse question comes up
constantly: this method is not in the type’s docs, where is it from?
The procedure:
- Check the type’s own inherent methods.
- Check what the type derefs to —
String→str,Vec<T>→[T],Arc<T>→T. - Check the trait
impls listed on the docs page (docs.rs lists them all, including blanket impls, under “Trait Implementations”). - Check the file’s
usestatements for extension traits.
Step 4 is the one people forget, and it is where itertools, futures,
rayon and tokio methods come from.
use rayon::prelude::*;
Every iterator in this file just gained par_iter(). A prelude::* import
is a bundle of extension traits.
When you see one, expect methods that are not in any type’s own documentation.
rayon adds parallel iterators, futures adds stream combinators, diesel
adds query builders, tokio::io adds async read/write.
Disambiguating
When two traits in scope both define fly, x.fly() is ambiguous. The fix is
fully qualified syntax:
Pilot::fly(&person);
<Person as Wizard>::fly(&person);
The second form — <Type as Trait>::method — is the fully general one, and you
need it whenever the receiver alone does not determine the impl. The most common
real case is associated functions with no self:
let x = <i32 as Default>::default();
let y = <Vec<u8>>::with_capacity(10);
<T as Trait>::method(x) · Trait::method(&x)
Someone hit an ambiguity, or is calling a trait method that has no receiver to infer from.
Common in generic code where T could implement several traits with the same
method name, and in macro-generated code where the macro cannot know what is in
scope. Seeing it written by hand usually means there was a real collision.
The anti-pattern
Because Deref drives method resolution, people sometimes use it to fake
inheritance:
struct Admin { user: User }
impl Deref for Admin {
type Target = User;
fn deref(&self) -> &User { &self.user }
}
// now admin.email() works, "inheriting" User's methods
This is called Deref polymorphism and it is considered an anti-pattern by the
Rust community. The std::ops::Deref documentation explicitly advises against
it. The reasons are worth being able to state:
- Method resolution becomes hard to follow — methods appear from a type that is not obviously related.
- It is not real subtyping:
&Admincoerces to&User, butVec<Admin>has no relationship toVec<User>, so it breaks exactly where inheritance would have helped. - Name collisions between
AdminandUserresolve silently in favour of the inherent one, which can change behaviour when either type gains a method.
Deref is for smart pointers — types that are a pointer to something else.
Box, Rc, Arc, MutexGuard. If your type is not a pointer, use a method or
AsRef.
You are reading unfamiliar code and hit this:
let names: Vec<String> = items.iter().map(|i| i.name()).sorted().collect();sorted() is not a method on std’s Map iterator. Describe, in order, the
exact steps you would take to find out where it comes from — and say what you
expect the answer to be.
What you should now be able to say
- Method resolution: build candidates by deref, then try by value, by
&, by&mutat each step. First match wins. Inherent beats trait. - Deref coercion turns
&Stringinto&strand&Vec<T>into&[T]— the reason those are the idiomatic parameter types. - Trait methods need the trait in scope;
prelude::*imports are bundles of extension traits. x.clone()on a&TwhereT: !Clonesilently clones the reference.<T as Trait>::methodis the fully qualified escape hatch.- Deref polymorphism is an anti-pattern;
Derefis for smart pointers.