When patterns aren't enough
You have five configuration structs in your app. Each one needs a from_env method that reads environment variables and populates the fields. You write a macro_rules! macro to generate the method. It works for the first struct. Then you try to make it generic so it works for any struct. Suddenly you're juggling token trees, counting commas, and writing patterns that look like hieroglyphics. You realize you need to inspect the struct's fields to generate the code. macro_rules! hits a wall. You need a procedural macro.
Declarative vs Procedural
Rust offers two ways to write macros. macro_rules! is declarative. You define patterns that match against tokens. If the tokens fit the pattern, Rust substitutes them into a template. It's like a sophisticated find-and-replace with wildcards. The compiler expands these macros during parsing, using built-in logic. No extra crates. No separate compilation.
proc_macro is procedural. You write a function that receives the raw tokens of the macro invocation. You can parse those tokens into a data structure, run arbitrary logic, and return new tokens. It's like hiring a code-writing robot. You hand it the text of your code. It reads the text, understands the grammar, makes decisions based on the content, and writes back the generated code. The robot can do math, look up tables, and iterate over fields. The template can only match shapes.
Minimal examples
A macro_rules! macro matches token patterns. You define fragments like $expr or $ident and use repetition operators like * or +.
// Declarative macro: matches a single identifier and generates a function
macro_rules! define_greeting {
($name:ident) => {
/// Prints a greeting using the provided identifier.
fn $name() {
println!("Hello from {}", stringify!($name));
}
};
}
define_greeting!(say_hi);
fn main() {
say_hi(); // Prints: Hello from say_hi
}
The compiler matches $name:ident against say_hi. It substitutes say_hi into the template. The stringify! macro converts the identifier to a string literal. This happens entirely within the compiler's parsing phase.
A proc_macro is a function in a separate crate. It receives a TokenStream and returns a TokenStream. In practice, nobody manipulates TokenStream directly. The ecosystem convention is to use syn for parsing and quote for generating code. These libraries turn raw tokens into Rust-like data structures and back.
// lib.rs in a proc-macro crate
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput};
/// Derive macro that adds a `describe` method returning the struct name.
#[proc_macro_derive(Describe)]
pub fn describe_derive(input: TokenStream) -> TokenStream {
// Parse the input tokens into a DeriveInput data structure
let input = parse_macro_input!(input as DeriveInput);
let name = &input.ident;
// Generate the implementation block
let expanded = quote! {
impl #name {
pub fn describe(&self) -> &'static str {
stringify!(#name)
}
}
};
expanded.into()
}
The #[proc_macro_derive(Describe)] attribute tells the compiler this function implements a derive macro named Describe. When you write #[derive(Describe)] on a struct, the compiler invokes this function. The function parses the struct, extracts the name, and returns code that implements a describe method.
How the compiler handles them
macro_rules! macros are built into the compiler. They expand during the parsing phase. The compiler reads the macro invocation, matches tokens against the patterns, and produces the output. This is fast. It adds no overhead to the build process beyond the parsing itself.
proc_macro macros require a separate crate. You define the macro in a crate with proc-macro = true in Cargo.toml. This setting tells Cargo to compile the crate as a dynamic library rather than a normal library. During your build, Cargo compiles the proc-macro crate first. Then, when the compiler encounters a proc-macro invocation, it spawns the proc-macro crate as a helper process. The compiler sends the tokens to the helper via inter-process communication. The helper processes the tokens and sends back the result. The compiler inserts the result into the source code and continues.
This separation has consequences. The proc-macro crate cannot depend on the crate using it. That would create a circular dependency. If your proc-macro needs types defined in the user crate, you must re-export those types or define them in a shared dependency. Proc-macros also add build time overhead. Every invocation spawns a process. If you have hundreds of derive macros, your build slows down.
The hygiene trap
Hygiene refers to how the compiler handles variable names and scopes. macro_rules! macros are hygienic. The compiler tracks where tokens come from. If you define a variable x inside a macro_rules! macro, it doesn't clash with a variable x in the caller's scope. The compiler treats them as distinct.
proc_macro macros are not hygienic by default. The compiler treats the generated tokens as if they were written at the call site. If your proc-macro generates a variable name that exists in the caller's scope, you get a conflict.
// Caller code
let x = 10;
my_proc_macro!(); // Generates: let x = 5;
// Error: cannot assign twice to immutable variable `x`
You must manage hygiene manually in proc-macros. Libraries like quote provide helpers like format_ident! to generate unique identifiers. Always use unique names for intermediate variables in proc-macros to avoid shadowing. Hygiene is the safety net. macro_rules! gives it to you for free. proc_macro makes you build your own.
A realistic derive macro
Here's a proc-macro that implements a trait based on struct fields. This is impossible with macro_rules! because you can't iterate over fields dynamically based on the struct definition.
// lib.rs in proc-macro crate
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput, Data, Fields};
/// Derive macro that implements `Configurable` for structs with named fields.
#[proc_macro_derive(Configurable)]
pub fn configurable_derive(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as DeriveInput);
let name = &input.ident;
// Extract fields from the struct
let fields = match &input.data {
Data::Struct(s) => match &s.fields {
Fields::Named(f) => &f.named,
_ => panic!("Configurable only supports structs with named fields"),
},
_ => panic!("Configurable only supports structs"),
};
// Generate code for each field
let field_names: Vec<_> = fields.iter().map(|f| &f.ident).collect();
let field_types: Vec<_> = fields.iter().map(|f| &f.ty).collect();
let expanded = quote! {
impl #name {
/// Creates an instance from environment variables.
pub fn from_env() -> Result<Self, String> {
Ok(#name {
#(#field_names: std::env::var(stringify!(#field_names))
.map_err(|e| format!("Error reading {}: {}", stringify!(#field_names), e))?
.parse::<#field_types>()
.map_err(|e| format!("Error parsing {}: {}", stringify!(#field_names), e))?,
)*
})
}
}
};
expanded.into()
}
The macro parses the struct, checks that it has named fields, and iterates over them. It generates a from_env method that reads each field from an environment variable and parses it. The repetition syntax #(...)* in quote iterates over the vectors and generates code for each field. This pattern is common in proc-macros: parse, validate, iterate, generate.
Pitfalls and errors
Proc-macros add complexity. Errors can be harder to debug. If your proc-macro panics, the compiler reports an error pointing to the macro invocation. The error message is whatever string you passed to panic!. Make sure your error messages are helpful. Include the struct name and the specific field that caused the issue.
macro_rules! errors are often clearer. The compiler tells you which pattern failed and which tokens didn't match. Proc-macro errors depend on your implementation. If you forget to handle a case, you might get a cryptic panic.
Build time is another concern. Proc-macros spawn processes. If you use many proc-macros, your build slows down. Profile your build if you notice slowdowns. Consider caching or reducing the number of macro invocations.
Dependency management can be tricky. The proc-macro crate cannot depend on the user crate. If you need shared types, create a separate crate for them. Both the proc-macro crate and the user crate can depend on the shared crate. This avoids circular dependencies.
Compiler error E0433 (unresolved import) often appears if the proc-macro crate isn't linked correctly. Ensure the proc-macro crate is listed as a dependency in Cargo.toml and the feature is enabled. Error E0658 (feature not stable) appears if you use unstable proc-macro features like proc_macro_span. Stick to stable features unless you have a specific need.
Proc macros are powerful, but they cost build time and complexity. Use them only when macro_rules! can't do the job.
Choosing the right tool
Use macro_rules! when you need simple code generation that matches token patterns. Use macro_rules! when you want zero build overhead and no separate crate. Use macro_rules! for utilities like vec!, format!, or simple logging macros. Use macro_rules! when hygiene is important and you don't want to manage it manually.
Use proc_macro when you need to inspect the AST to generate code. Use proc_macro for custom derives, attributes, or function-like macros that require parsing logic. Use proc_macro when you need to generate code based on field names, types, or other structural information. Use proc_macro when you need to perform computations or lookups that macro_rules! can't handle.
Reach for macro_rules! first. It's built-in, faster to compile, and easier to debug. Switch to proc_macro only when you hit the pattern-matching limits. Start with macro_rules!. If the patterns start looking like a knot, reach for proc_macro.