When a simple loop isn't enough
You're writing a parser for a custom config file. Each line changes how you interpret the next line. Or you're building a game where entities update based on a complex simulation step that tracks collisions, timers, and cooldowns. The standard 0..n range or .iter() on a vector won't cut it. You need a sequence that remembers where it left off, tracks flags, maybe holds a buffer, and decides dynamically when to stop. That's a custom iterator with complex state.
Rust gives you the Iterator trait to build exactly this. You define a struct to hold the state, implement next to advance that state, and return values one by one. The result integrates seamlessly with .map(), .filter(), .collect(), and every other tool in the standard library. You get lazy evaluation, zero-cost abstraction, and composable logic without writing a single while loop.
The tour guide analogy
Think of an iterator as a tour guide. The guide carries a clipboard with the current location, the route map, and a list of visited spots. Every time you ask "Where's next?", the guide checks the clipboard, updates the current location, and points you to the next stop. If there are no more stops, the guide shakes their head.
In Rust, the clipboard is your struct. The "Where's next?" question is the next() method. The guide's answer is Some(value) when there's a stop, or None when the tour ends. The struct holds all the variables needed to make the next decision. The next method mutates those variables and produces the output. The loop machinery calls next repeatedly until it gets None.
Minimal example: a Fibonacci generator
Start with a struct that holds the state. Implement Iterator and define the Item type. The next method takes &mut self, which is the key: it allows you to mutate the struct's fields to advance the sequence.
/// Generates Fibonacci numbers up to a limit.
struct Fibonacci {
current: u64,
next: u64,
}
impl Iterator for Fibonacci {
// The Item type is what comes out of the iterator.
type Item = u64;
fn next(&mut self) -> Option<Self::Item> {
// Stop when the number exceeds the limit.
if self.current > 1_000_000 {
return None;
}
// Capture the value to return before mutating state.
let result = self.current;
// Advance the sequence for the next call.
let new_next = self.current + self.next;
self.current = self.next;
self.next = new_next;
Some(result)
}
}
fn main() {
let fib = Fibonacci { current: 0, next: 1 };
// Collect the first few values to verify.
let values: Vec<u64> = fib.take(5).collect();
println!("{values:?}");
}
Return None to end the sequence. The loop machinery respects it immediately.
How the compiler handles it
When you call next, Rust passes a mutable reference to your struct. You mutate the fields. You return Option<Item>. The compiler enforces that next can be called repeatedly as long as the iterator exists. The &mut self signature guarantees exclusive access to the state during each step.
This design prevents data races and aliasing issues. If you try to store a reference to an item returned by next and then call next again, you'll hit E0502 (cannot borrow as mutable because it is also borrowed as immutable). The iterator needs &mut self to advance, but your reference holds onto the previous state. The borrow checker blocks you from creating a dangling reference or corrupting the sequence.
The compiler also inlines next calls. There's no virtual dispatch. No heap allocation for the iterator protocol. The generated machine code is identical to a hand-written while loop with the same logic. You get the abstraction without the cost.
Mutate state, return value, repeat. That's the contract.
Realistic example: parsing tokens with lifetimes
Real iterators often borrow data. A parser yields substrings from a larger buffer. This requires lifetimes to tie the iterator's output to the source data. The iterator struct needs a lifetime parameter, and the Item type uses that lifetime.
/// Yields whitespace-separated tokens from a string slice.
struct TokenParser<'a> {
input: &'a str,
pos: usize,
}
impl<'a> Iterator for TokenParser<'a> {
// Items are references into the original string.
type Item = &'a str;
fn next(&mut self) -> Option<Self::Item> {
// Skip leading whitespace.
while self.pos < self.input.len() {
let byte = self.input.as_bytes()[self.pos];
if byte.is_ascii_whitespace() {
self.pos += 1;
} else {
break;
}
}
// End of input reached.
if self.pos >= self.input.len() {
return None;
}
// Find the end of the current token.
let start = self.pos;
while self.pos < self.input.len() {
let byte = self.input.as_bytes()[self.pos];
if byte.is_ascii_whitespace() {
break;
}
self.pos += 1;
}
// Return the slice and advance past it.
Some(&self.input[start..self.pos])
}
}
fn main() {
let text = " hello world rust ";
let parser = TokenParser { input: text, pos: 0 };
// Chain adapters to process tokens.
let upper: Vec<&str> = parser.map(|t| t.to_uppercase()).collect();
println!("{upper:?}");
}
Lifetimes tie the iterator to the data. If the data drops, the iterator becomes invalid.
Convention note: When an iterator returns references, the lifetime parameter on the struct is mandatory. The community convention is to name the lifetime 'a and propagate it to Item. This makes it clear that the iterator doesn't own the data; it borrows it. The lifetime ensures you can't outlive the source.
Making it work with for loops
The Iterator trait lets you call .next() manually or chain adapters. To use the struct directly in a for loop, you need IntoIterator. This trait converts a value into an iterator. It's the bridge between your type and the for syntax.
impl IntoIterator for Fibonacci {
type Item = u64;
type IntoIter = Fibonacci;
fn into_iter(self) -> Self::IntoIter {
// Consume self and return it as the iterator.
self
}
}
fn main() {
let fib = Fibonacci { current: 0, next: 1 };
// Now this works directly.
for num in fib {
if num > 100 {
break;
}
println!("{num}");
}
}
Convention aside: Most iterator structs implement IntoIterator by returning self. This is idiomatic. It allows users to write for x in my_struct without calling .into_iter() explicitly. The compiler inserts the call automatically. If your struct already implements Iterator, the IntoIterator impl is usually trivial.
Implement IntoIterator to unlock for loop syntax. It's a small addition that pays off in ergonomics.
Pitfalls and common errors
The most common bug is returning Some without changing the state. The iterator yields the same value forever. Your program hangs. Always verify state advancement on every branch. If a condition triggers Some, the struct must be in a different state for the next call.
Another trap is infinite recursion in next. If next calls itself or calls another method that calls next, you'll blow the stack. Keep next iterative. Use loops inside next if needed, but never recurse.
Type mismatches happen when the Item type doesn't match the return value. The compiler catches this with E0308 (mismatched types). Double-check that Some(value) matches type Item. If Item is &str, you can't return Some(String). Convert or adjust the type.
Lifetimes can be tricky when the iterator holds multiple borrows. If your struct borrows from two different sources, you need multiple lifetime parameters. The compiler will guide you with errors if the lifetimes conflict. Trust the borrow checker here. It prevents you from returning a reference to data that might drop before the iterator finishes.
Verify state advancement on every branch. An infinite loop is a silent killer.
When to use custom iterators
Use a custom struct iterator when your state spans multiple fields, requires invariants, or needs to be stored and resumed across function calls. Use std::iter::from_fn with a closure when the logic is short, the state fits in captured variables, and you don't need to expose the state type. Use a plain while loop when you only need to consume the sequence once and don't care about chaining adapters like .map() or .filter(). Reach for the Iterator trait when you want lazy evaluation and seamless integration with Rust's collection ecosystem.
Pick the tool that matches the complexity. Don't over-engineer a counter, don't under-engineer a parser.