The invisible character that breaks your parser
You wrote a CSV parser on your Mac. It works perfectly. You deploy to a Windows server. The parser chokes. Every line fails validation. You stare at the log output and see "status: ok" matching "status: ok", but the check returns false. You print the bytes and find a stray \r hiding at the end of every line. The file uses \r\n endings. Your code expected \n. You spend an hour debugging before realizing the Enter key is the culprit.
Line endings are a historical accident that Rust refuses to hide from you. The language treats files as streams of bytes. It does not perform magic conversions. If you want consistent text processing, you have to handle the delimiters yourself. The standard library gives you the tools, but you need to know which tool matches your goal.
Bytes, not magic
Line endings come from mechanical typewriters. The "Enter" key did two things: it swung the carriage back to the left margin (Carriage Return, \r) and it rolled the paper up one line (Line Feed, \n). Modern computers just roll the paper up. Unix and macOS use only \n. Windows kept both habits and uses \r\n. Old Macs used only \r.
Rust exposes these bytes directly. A file is a sequence of u8 values. There is no "text mode" that auto-converts line endings like C or Python sometimes provide. This design gives you full control. You decide when to normalize, when to preserve, and when to treat the input as binary data.
Think of line endings like punctuation in a foreign language. If you are reading a book, you want the text cleaned up so you can focus on the meaning. If you are analyzing the typography, you need the raw characters. Rust assumes you might need the raw characters, so it hands them to you.
Normalizing a string blob
When you have text in memory and need to standardize the endings, the simplest approach is a replacement chain. You must replace \r\n before you replace \r. If you reverse the order, \r\n becomes \n\n because the \r gets eaten first, leaving a double newline.
fn normalize_text(input: &str) -> String {
// Replace CRLF first. If you replace CR first, CRLF becomes LF+LF.
// Then replace lone CR. This handles all three legacy formats.
input.replace("\r\n", "\n").replace("\r", "\n")
}
fn main() {
let messy = "Line1\r\nLine2\rLine3\nLine4";
let clean = normalize_text(messy);
// clean is "Line1\nLine2\nLine3\nLine4"
assert_eq!(clean, "Line1\nLine2\nLine3\nLine4");
}
This approach allocates a new String. It is fine for small configs or user input. It is terrible for streaming gigabytes of logs. The replace method scans the entire input and builds a fresh allocation. For large data, use a streaming reader instead.
Convention aside: cargo fmt formats your Rust source code to use LF endings. It does not touch data files. Your source code should always be LF. Let Git handle the conversion for Windows developers with core.autocrlf. Rust code in a repository should never contain \r.
The BufRead trait: lines versus read_line
When reading files, the std::io::BufRead trait is your primary interface. It wraps a reader and provides methods to consume text line by line. The two methods you will use most are lines() and read_line(). They behave differently in ways that trip up beginners.
lines() returns an iterator of Result<String>. It strips the newline delimiter. It handles \n, \r\n, and \r automatically. You get clean strings. read_line() appends to a buffer you provide. It keeps the newline delimiter. You get the raw bytes including the ending.
use std::io::{BufRead, BufReader};
use std::fs::File;
fn main() -> std::io::Result<()> {
let file = File::open("data.txt")?;
let reader = BufReader::new(file);
// lines() strips the delimiter. It handles \n, \r\n, and \r.
// Each item is a Result<String>.
for line_result in reader.lines() {
let line = line_result?;
// line contains "Hello" not "Hello\n"
println!("Processed: {}", line);
}
Ok(())
}
The lines() iterator is the safe default for text processing. It abstracts away the platform differences. You write code that works on Linux, Windows, and macOS without branching logic. The iterator reads from the buffer until it finds a delimiter, strips the delimiter, and yields the content.
read_line() is useful when you need to preserve the exact structure or when you are parsing a protocol where the delimiter is part of the message. It requires a mutable buffer. It returns the number of bytes read.
use std::io::{BufRead, BufReader};
use std::fs::File;
fn main() -> std::io::Result<()> {
let file = File::open("data.txt")?;
let mut reader = BufReader::new(file);
let mut buffer = String::new();
// read_line() appends to the buffer. It keeps the newline.
// It returns Ok(bytes_read).
while reader.read_line(&mut buffer)? > 0 {
// buffer contains "Hello\n" or "Hello\r\n"
// You must clear the buffer or manage it manually.
println!("Raw: {:?}", buffer);
buffer.clear();
}
Ok(())
}
The split trap is a common pitfall. If you read a whole file into a String and call text.split('\n'), you get slices that still contain \r on Windows files. split is a naive character split. It does not understand line endings. lines() understands line endings. Use lines() for text. Use split only when you are certain of the delimiter or when processing binary data.
Writing with control
Writing files requires explicit choices. Rust does not auto-convert \n to \r\n on Windows. The writeln! macro always appends \n. This is a deliberate design. Rust keeps the byte stream predictable. If you need CRLF, you write it yourself.
use std::io::Write;
use std::fs::File;
fn write_config(path: &str, entries: &[(&str, &str)]) -> std::io::Result<()> {
let mut file = File::create(path)?;
for (key, value) in entries {
// writeln! appends \n. It never appends \r\n.
// This produces LF endings on all platforms.
writeln!(file, "{}={}", key, value)?;
}
Ok(())
}
fn write_windows_style(path: &str, lines: &[&str]) -> std::io::Result<()> {
let mut file = File::create(path)?;
for line in lines {
// Manual CRLF for legacy Windows tools.
// write_all expects bytes. String does not implement AsRef<[u8]> directly.
// The compiler rejects file.write_all(line) with E0277.
file.write_all(line.as_bytes())?;
file.write_all(b"\r\n")?;
}
Ok(())
}
The writeln! macro is the convention for text files. It produces LF endings. Most modern tools on Windows handle LF correctly. Git, editors, and Rust tooling all prefer LF. Only write CRLF when you are interoperating with a legacy system that demands it.
Convention aside: let _ = file.flush(); is a signal to readers that you considered the buffer state and chose to drop the result. When writing critical data, explicit flushes or dropping the file handle ensure bytes hit the disk. The OS may buffer writes.
Pitfalls and edge cases
Line ending handling introduces subtle bugs. The compiler catches type errors, but logic errors slip through.
The replace order error is silent. If you write text.replace("\r", "\n").replace("\r\n", "\n"), you double the newlines in CRLF files. The first pass turns \r\n into \n\n. The second pass does nothing. The result is broken. Always replace \r\n first.
The write_all type error is common. write_all expects AsRef<[u8]>. String does not implement this trait directly. The compiler rejects file.write_all(line) with E0277 (trait bound not satisfied). You must call line.as_bytes() or use write!/writeln! macros.
The lines() allocation cost is real. lines() returns String values. Each call allocates memory. If you are processing millions of lines, the allocation overhead adds up. For high-performance parsing, use BufRead::read_line() with a reused buffer, or use a zero-copy parser like nom.
The binary file trap is dangerous. If you open a binary file with BufReader and call lines(), you will split the binary data on bytes that happen to match 0x0A. This corrupts the data structure. Always open binary files with File directly and read bytes. Do not use BufRead methods on binary data unless you are certain of the format.
Decision matrix
Use BufRead::lines() when you want clean strings without delimiters and the input is text. Use BufRead::read_line() when you need to preserve the delimiter or minimize allocations by reusing a buffer. Use String::replace("\r\n", "\n").replace("\r", "\n") when you have a text blob in memory and need normalization before processing. Use writeln! when writing text files; accept that it produces LF endings everywhere. Use manual \r\n writing only when interoperating with legacy Windows tools that demand CRLF. Reach for nom or regex when you need to parse complex line-based protocols with custom rules.
Trust lines() to strip the noise. Normalize early in your pipeline. Binary is the truth; text is an interpretation.