The problem with manual loops
You're parsing a log file. You need to skip blank lines, ignore comments starting with #, and extract key-value pairs. You could write a while loop with an index, manually checking bounds, updating state, and breaking when done. It works, but it's fragile. Add a new filter and the loop logic gets tangled. You want to write for entry in log_parser and let the compiler handle the iteration mechanics. That's where custom iterators come in.
Iterator as a state machine
An iterator is a state machine that produces values one at a time. It holds the current position, any temporary buffers, and the rules for generating the next value. The outside world doesn't see the state. It only calls a single method to get the next result.
Think of an iterator like a ticket dispenser at a concert. You don't see the stack of tickets inside. You press a button, and one ticket comes out. You press again, another ticket. When the machine is empty, it displays a "sold out" sign. The machine remembers how many tickets are left. You just interact with the button.
In Rust, the Iterator trait is the contract. The next method is the button. The return type is Option<Self::Item>. Some(value) is a ticket. None is the "sold out" sign. The iterator owns the state. The loop owns the flow.
Minimal implementation
Every custom iterator needs a struct to hold state and an implementation of the Iterator trait. The trait requires an associated type Item and a method next.
struct Counter {
current: u32,
}
impl Iterator for Counter {
// The type of value this iterator produces.
type Item = u32;
fn next(&mut self) -> Option<Self::Item> {
// Check if we still have values to yield.
if self.current < 5 {
self.current += 1;
Some(self.current)
} else {
// Signal that the iterator is exhausted.
None
}
}
}
fn main() {
let counter = Counter { current: 0 };
for num in counter {
println!("{}", num);
}
}
The type Item line tells the compiler what type lives inside the Option. The next method takes &mut self because it must update the internal state. It returns Option<Self::Item> to indicate success or exhaustion.
Convention aside: type Item is the standard name for the associated type. You can name it anything, but the ecosystem expects Item. Deviating breaks tooling and confuses readers. Stick to Item.
How the for loop uses your iterator
The for loop is syntactic sugar. It desugars to a loop that calls next repeatedly. The compiler rewrites your code roughly like this:
let mut iter = counter;
loop {
match iter.next() {
Some(num) => {
println!("{}", num);
}
None => break,
}
}
This reveals two critical facts. First, the iterator is lazy. Creating Counter does nothing. Values are only produced when next is called. This saves memory and computation. You can create an iterator over a billion items without allocating a billion slots.
Second, the for loop consumes the iterator. It takes ownership or a mutable reference. You cannot iterate over the same iterator twice unless you clone it or recreate it. If you need multiple passes, you either clone the iterator or implement Clone on your struct.
Keep next focused. It should advance state and return a value. Don't perform heavy I/O or complex logic inside next unless necessary. The caller expects next to be fast and predictable.
Realistic example: filtering a log
A custom iterator shines when you need to generate values based on complex rules. Here's a parser that skips comments and blank lines from a vector of strings.
struct LogParser {
lines: Vec<String>,
index: usize,
}
impl Iterator for LogParser {
type Item = String;
fn next(&mut self) -> Option<Self::Item> {
// Loop until we find a valid line or run out.
while self.index < self.lines.len() {
let line = self.lines[self.index].clone();
self.index += 1;
// Skip comments and blank lines.
if line.starts_with('#') || line.trim().is_empty() {
continue;
}
// Return the first valid line.
return Some(line);
}
// No more lines.
None
}
}
fn main() {
let logs = vec![
"INFO: Start".to_string(),
"# This is a comment".to_string(),
"".to_string(),
"WARN: Low disk".to_string(),
];
let parser = LogParser {
lines: logs,
index: 0,
};
for entry in parser {
println!("Parsed: {}", entry);
}
}
The next method contains a while loop. It scans internal state until it finds a value to return. This is a common pattern. next can do as much work as needed to produce the next item. The caller sees only the result.
Convention aside: LogParser holds Vec<String>. Cloning strings on every next call can be expensive. If performance matters, consider returning references or using indices. For a learning example, ownership is clearer. In production, profile first.
The IntoIterator bridge
If you want to use for entry in my_collection where my_collection is not an iterator, you need IntoIterator. This trait converts a type into an iterator. It's the bridge between collections and loops.
struct LogCollection {
lines: Vec<String>,
}
impl IntoIterator for LogCollection {
type Item = String;
type IntoIter = LogParser;
fn into_iter(self) -> Self::IntoIter {
LogParser {
lines: self.lines,
index: 0,
}
}
}
Now you can write for entry in LogCollection { ... }. The into_iter method consumes self and returns an iterator. This separates the collection logic from the iteration logic. It's the pattern used by Vec, HashMap, and other standard types.
Convention aside: IntoIter is the standard name for the associated type. Item is the value type. into_iter takes self by value. If you want to iterate by reference, implement IntoIterator for &LogCollection and return an iterator that borrows.
Pitfalls and compiler errors
Custom iterators introduce subtle bugs if you break the contract. The compiler helps, but not with everything.
If your next method returns u32 instead of Option<u32>, the compiler rejects you with E0308 (mismatched types). The return type must be Option<Self::Item>. This is a hard error. You can't ignore it.
If you forget to implement Iterator, you get E0277 (trait bound not satisfied) when you try to use the struct in a for loop. The compiler tells you the type doesn't implement Iterator. Add the impl and the error vanishes.
The dangerous pitfall is the None contract. Once next returns None, it must return None forever. If your iterator toggles between Some and None, the for loop breaks early. The loop assumes None means exhaustion. It doesn't call next again. You'll lose data.
Treat None as a tombstone. Once you return it, the iterator is dead. If you need to reset state, provide a method to recreate the iterator. Don't try to make next recover from None.
Another pitfall is infinite loops. If you forget to update state, next returns the same value forever. The for loop never ends. Add bounds checks. Use usize for indices and check against lengths. Rust won't stop you from writing an infinite iterator. It's your responsibility to ensure termination.
When to build a custom iterator
Custom iterators are powerful but add complexity. Use them when the benefit outweighs the cost.
Use a custom iterator when you need to generate values on demand based on complex state that doesn't fit a simple closure. Use std::iter::successors when your sequence is defined by a starting value and a function to compute the next value from the previous one. Use iterator adapters like map and filter when you can transform or filter an existing collection without managing new state. Use a simple for loop over a range when you just need to repeat an action a fixed number of times.
Don't build a custom iterator if a slice and a filter will do. The standard library provides tools for most common patterns. Reach for custom iterators when you hit the limits of adapters.