When loops get in the way
You're writing a parser for a custom config file. The file could be gigabytes long. Loading it all into a Vec<String> crashes your program. You need to process lines as they arrive, one at a time, without holding onto the past. Or imagine a game loop where you generate random events: a meteor strike here, a supply drop there. You don't know how many events will happen, but you need to handle each one as it pops up.
Rust solves this with the Iterator trait. It turns your data source into a stream of values you can consume lazily. You don't manage the loop index. You don't track the boundary conditions. You define how to produce the next value, and the rest of the language composes around that. Iterators turn data sources into streams you can consume without holding the past.
The iterator contract
An iterator is a state machine that hands you values one by one. It remembers where it left off and knows how to produce the next item. Think of a ticket dispenser at a deli. You don't see the roll of tickets inside the machine. You just press a button and get the next number. The machine holds the state: the current number, the roll of paper, the logic to increment. When the roll runs out, the machine stops giving tickets.
In Rust, the Iterator trait defines this contract. Your struct holds the state. The next method is the button press. It returns Some(value) when there's data, or None when the stream ends. The trait requires two things: an associated type Item that declares what values the iterator produces, and the next method that advances the state and returns an Option.
Treat the iterator as a black box that generates values. You press next, you get a result. The internals are hidden.
Minimal implementation
Here's a counter that yields numbers from 1 up to a limit. It demonstrates the bare minimum structure.
/// A simple counter that yields numbers from 1 up to a limit.
struct Counter {
/// The current value to yield next.
current: u32,
/// The maximum value before the iterator stops.
limit: u32,
}
impl Iterator for Counter {
// Define the type of items this iterator produces.
type Item = u32;
// The core method: called to get the next value.
fn next(&mut self) -> Option<Self::Item> {
// Check if we've reached the limit.
if self.current >= self.limit {
// Return None to signal the end of the stream.
return None;
}
// Capture the value to return before incrementing.
let value = self.current;
// Advance the internal state for the next call.
self.current += 1;
// Wrap the value in Some to indicate success.
Some(value)
}
}
fn main() {
let mut counter = Counter { current: 0, limit: 3 };
// Consume the iterator manually to see the mechanics.
println!("{:?}", counter.next()); // Some(0)
println!("{:?}", counter.next()); // Some(1)
println!("{:?}", counter.next()); // Some(2)
println!("{:?}", counter.next()); // None
}
The type Item line tells the compiler what type flows out of this iterator. It's an associated type, not a generic parameter. This locks the output type for the implementation. The next method takes &mut self because the iterator must mutate its internal state to track progress. It returns Option<Self::Item>. Some(value) means "here's the next item." None means "we're done."
The Option return type is the contract. Some means go, None means stop. No exceptions, no magic values.
Anatomy of next
The signature fn next(&mut self) -> Option<Self::Item> carries weight. The &mut self is mandatory. Every call to next consumes the current position and advances to the next. If next took &self, the iterator couldn't remember where it was. You'd get the same value forever.
The return type uses Option to handle termination cleanly. In languages without Option, iterators often use sentinel values like -1 or null. This forces the iterator to choose a type that has a spare value, or to wrap the type in a box. Rust's Option works for any type. Even bool can be iterated. The stream ends when None appears.
The community convention is to treat for loops as syntactic sugar for while let Some(item) = iterator.next(). When you write for item in iterator { ... }, the compiler rewrites it to call next repeatedly until None. This means your next implementation controls the loop. If next is expensive, the loop is slow. If next panics, the loop panics.
Real-world tokenizer
A counter is simple. Real iterators often parse data or traverse structures. Here's a tokenizer that yields words from a string, skipping punctuation and whitespace. This shows how to manage a cursor and slice data.
/// An iterator that yields words from a string, skipping punctuation and whitespace.
struct WordTokenizer<'a> {
/// The source text being processed.
text: &'a str,
/// Current position in the string.
pos: usize,
}
impl<'a> Iterator for WordTokenizer<'a> {
// The iterator yields string slices tied to the source lifetime.
type Item = &'a str;
fn next(&mut self) -> Option<Self::Item> {
// Skip non-alphanumeric characters to find the start of a word.
while self.pos < self.text.len() {
let byte = self.text.as_bytes()[self.pos];
if byte.is_ascii_alphanumeric() {
break;
}
self.pos += 1;
}
// If we reached the end, there are no more words.
if self.pos >= self.text.len() {
return None;
}
// Mark the start of the current word.
let start = self.pos;
// Advance until we hit a non-alphanumeric character or the end.
while self.pos < self.text.len() {
let byte = self.text.as_bytes()[self.pos];
if !byte.is_ascii_alphanumeric() {
break;
}
self.pos += 1;
}
// Slice the word from the source string.
Some(&self.text[start..self.pos])
}
}
fn main() {
let text = "Hello, world! Rust is great.";
let tokenizer = WordTokenizer { text, pos: 0 };
for word in tokenizer {
println!("Word: {}", word);
}
}
This example introduces a lifetime parameter 'a. The tokenizer holds a reference to the source string. The words it yields are slices of that string. The lifetime 'a ties the iterator's life to the string's life. If the string drops, the iterator becomes invalid. The compiler enforces this. You get E0597 if the borrowed value doesn't live long enough.
Lifetimes keep references honest. If the data drops, the iterator dies. The compiler enforces this boundary.
Lifetimes and borrowing
When your iterator yields references, lifetimes become part of the signature. The iterator must live shorter than the data it references. The WordTokenizer example shows the pattern: the struct has a lifetime parameter, and type Item uses that lifetime.
The community prefers owned iterators when possible to avoid lifetime complexity. If you can return String instead of &str, you avoid the lifetime parameter. This is the "owning iterator" pattern. It costs allocation but simplifies the API. Many standard library iterators, like Lines from BufRead, return owned strings for this reason. The caller doesn't have to manage the lifetime of the underlying buffer.
If you stick with references, be careful about mutability. An iterator yielding &T is immutable. You can't modify the data through the iterator. If you need mutation, you need &mut T. That requires the iterator to hold &mut T and yield &mut T. This is harder to implement because you can't hold multiple mutable references at once. The borrow checker will reject attempts to yield overlapping mutable slices.
Adapters and composition
Iterators shine when composed. Methods like map, filter, take, and zip are adapters. They don't execute immediately. They return new iterator structs that wrap your original one.
A Map iterator holds the source iterator and a closure. When you call next on Map, it calls next on the source, applies the closure, and returns the result. This creates a pipeline of lazy transformations. Nothing happens until you consume the chain with .collect() or a loop.
fn main() {
let numbers = vec![1, 2, 3, 4, 5];
// Chain adapters: filter evens, square them, take first three.
let result: Vec<u32> = numbers
.into_iter()
.filter(|n| n % 2 == 0)
.map(|n| n * n)
.take(3)
.collect();
println!("{:?}", result); // [4, 16]
}
The performance cost is minimal. The compiler inlines adapter methods aggressively. The generated code is often identical to a hand-written loop with index math. Adapters express intent clearly. A chain of .filter().map().take() reads like a specification.
Pitfalls and errors
The most common mistake is forgetting to advance the internal state. If next returns a value but doesn't change self, the iterator loops forever. The compiler won't catch this. It's a logic error. Always verify that next modifies state or reaches a terminal condition.
Another trap is borrowing rules. next requires &mut self. If you hold a reference to the iterator while trying to call next, you get E0502 (cannot borrow as mutable because it is also borrowed as immutable). This happens if you try to store a reference to an item inside the iterator while iterating. The iterator owns the data or the position. You can't hold onto a piece of it while asking for the next piece.
If you need to accumulate results while iterating, use .fold() or a local variable. Don't try to mutate the iterator from outside the loop.
Don't fight the borrow checker here. If you need to hold a reference while iterating, collect the data first or restructure the state.
Choosing the right tool
Rust provides several ways to handle sequences. Pick the right one based on your needs.
Use Iterator when you need to generate a sequence of values lazily from internal state.
Use IntoIterator when you want a type to be usable in for loops, converting the value into an iterator.
Use FromIterator when you want to construct a collection from an iterator, like building a Vec from a stream.
Use std::iter::once when you have a single value and need an iterator for compatibility with methods that expect a stream.
Use std::iter::repeat when you need an infinite stream of the same value, usually combined with .take() to limit the count.
Use DoubleEndedIterator when your data supports walking from both ends, enabling .rev() and efficient palindrome checks.
Iterators compose. If you find yourself writing a loop with index math, reach for an iterator. The compiler will likely optimize it to the same machine code.