When text substitution stops working
You are building a configuration loader. You want to write config!("database.url") and have it compile into a type-safe lookup that panics at compile time if the key does not exist. A regular function cannot inspect the string literal before the program runs. A macro_rules! macro can, but it fights you when the input syntax grows complex or when you need to generate deeply nested AST nodes. Function-like procedural macros solve this exact gap. They run as separate compile-time programs that rewrite your source code before the rest of the compiler sees it.
What a function-like macro actually is
Rust compiles in two distinct phases when procedural macros are involved. The compiler first builds your macro crate as a dynamic library. It then loads that library into the rustc process. When the main crate hits a my_macro!(...) call, rustc pauses, feeds the raw tokens inside the parentheses to your function, and waits for the output. Your function returns a new stream of tokens. rustc drops those tokens back into the source file and continues parsing.
Think of it as a custom blueprint editor. The architect hands you a rough sketch. You run it through a validation script, fix the measurements, and hand back a polished set of instructions. The construction crew never sees the original sketch. They only work with your expanded output. The macro function itself is just a pure transformer. It takes syntax in, validates it, and spits out new syntax.
The bare minimum setup
The foundation requires a dedicated crate. You cannot mix procedural macros with regular library code in the same crate. Create a new crate and flip the proc-macro flag in Cargo.toml.
[lib]
# Tells Cargo to compile this as a dynamic library
# and automatically links the proc_macro crate.
proc-macro = true
The flag tells Cargo to compile the crate as a dynamic library instead of a static archive. It also links the proc_macro crate, which provides the TokenStream type. Your macro function lives in src/lib.rs. It takes a single TokenStream and returns either a TokenStream or a Result<TokenStream, Error>.
use proc_macro::TokenStream;
/// Echoes the input tokens back unchanged.
#[proc_macro]
pub fn echo(input: TokenStream) -> TokenStream {
// The input arrives as a flat stream of lexical tokens.
// Returning it directly proves the pipeline works.
input
}
Add this crate as a dependency in your main project. Call it with the exclamation mark syntax: echo!(hello world);. The compiler replaces the call site with whatever the function returns. Keep the macro crate isolated. Mixing it with application logic breaks the compilation model.
How the compiler hands you the tokens
Compilation happens in a strict sequence. Cargo builds the macro crate first because the main crate depends on it. The output is a .so, .dylib, or .dll file. When rustc starts compiling the main crate, it encounters echo!(hello world);. It extracts hello world as a TokenStream, loads the macro library, and invokes echo. The function returns the tokens. rustc splices them back into the abstract syntax tree.
The rest of the compiler treats the expanded tokens as if you typed them manually. This means type checking, lifetime analysis, and optimization all run on the expanded code, not the macro call. The macro itself runs in a sandboxed context. It cannot access the main crate's variables, types, or Cargo.toml metadata directly. It only sees tokens. Treat the macro as a pure function that transforms syntax trees. If you try to read a file or query the network, you will slow down every developer's build. Keep the work fast and deterministic.
A realistic transformation with syn and quote
Working with raw TokenStream is painful. The stream is unstructured. You cannot easily check if the input is a valid expression or a struct definition. The community standard is to pair syn and quote. syn parses the token stream into Rust data structures. quote turns those structures back into tokens. Add them to your macro crate's Cargo.toml with the proc-macro feature enabled.
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, Expr};
/// Wraps an expression in a debug print that shows both the code and its value.
#[proc_macro]
pub fn debug_eval(input: TokenStream) -> TokenStream {
// Parse the raw tokens into a strongly typed AST node.
let expr = parse_macro_input!(input as Expr);
// Generate new code that prints the expression and evaluates it.
let expanded = quote! {
{
// Capture the expression as a string for the log message.
let val = #expr;
println!("DEBUG: {} = {:?}", stringify!(#expr), val);
val
}
};
// Hand the generated tokens back to the compiler.
TokenStream::from(expanded)
}
The #expr syntax inside quote! injects the parsed AST node as tokens. stringify!(#expr) converts it to a string literal at compile time. The block syntax ensures the macro expands to a single expression that returns the original value. This pattern lets you use debug_eval!(x + y) anywhere an expression is allowed. The community expects syn and quote in every procedural macro. Raw token manipulation belongs in niche parsers, not application code.
Error handling and diagnostics
Procedural macros introduce three common failure modes. The first is hygiene. Macros do not capture variables from the surrounding scope. If your macro generates let x = 5; and the caller already has a variable named x, the compiler treats them as separate bindings. You cannot accidentally shadow a caller's variable unless you intentionally use syn::parse to inject identifiers.
The second is error reporting. When a macro fails, the error points to the call site, not the macro definition. If you return a Result and the Err branch triggers, rustc displays the error message at the exact line where the macro was invoked. Returning TokenStream directly forces you to use panic!(), which produces a compiler crash dump instead of a clean diagnostic. Always return Result<TokenStream, syn::Error> for production macros.
The third is the E0433 unresolved import trap. If your macro generates code that references a type like std::collections::HashMap, but the caller's crate does not import it, compilation fails. The macro cannot assume the caller's dependencies. Generate fully qualified paths or document the required imports. Trust the hygiene rules. They prevent subtle bugs that macro_rules! struggles with.
use proc_macro::TokenStream;
use syn::{parse_macro_input, Expr, LitStr};
/// Validates that the input is a string literal.
#[proc_macro]
pub fn expect_string(input: TokenStream) -> Result<TokenStream, syn::Error> {
// Parse expecting a string literal, fail fast if it is not.
let lit = parse_macro_input!(input as LitStr);
// Return the literal wrapped in a block to ensure expression context.
let output = quote::quote! { #lit };
Ok(TokenStream::from(output))
}
The parse_macro_input! macro handles the Result conversion automatically. If the input does not match LitStr, syn generates a syn::Error with a precise span pointing to the bad token. rustc renders it as a standard compiler error. Use syn::Error::new when you need custom validation messages. Point the error span to the exact token that failed. A precise span saves hours of debugging for the person calling your macro.
Testing and debugging macros
Testing procedural macros requires a different strategy than regular functions. You cannot call them directly from unit tests because they run inside rustc. The community convention is to write integration tests that compile small Rust snippets using trybuild or std::process::Command. You create a tests/ directory in your macro crate. Each test file contains a minimal main.rs that invokes your macro. The test runner compiles the file and checks the exit code or stderr output.
// tests/ui/valid_input.rs
use my_macro_crate::debug_eval;
fn main() {
// This should compile and print the debug line at runtime.
let result = debug_eval!(2 + 2);
assert_eq!(result, 4);
}
When debugging, eprintln! inside the macro prints to the terminal during cargo build. The output appears alongside compiler messages. You can also use syn::Error::new to halt compilation and inspect the parsed AST. Keep test cases minimal. A macro test should verify one expansion path at a time. Isolate edge cases like empty input, nested parentheses, or missing commas. Treat macro tests as contract verification. If the expansion changes, the tests catch it before downstream crates break.
When to reach for function-like macros
Use macro_rules! when you need simple text substitution or pattern matching on a fixed number of arguments. Use function-like procedural macros when you must parse arbitrary syntax, generate complex AST nodes, or run compile-time logic that exceeds declarative macro limits. Use #[proc_macro_derive] when you want to attach custom behavior to structs or enums via #[derive(MyTrait)]. Use #[proc_macro_attribute] when you need to modify existing functions or structs with #[my_attr] annotations. Reach for regular functions when runtime evaluation is acceptable and compile-time generation adds unnecessary build complexity. Keep the macro surface small. Every procedural macro slows down your build pipeline.