When derive isn't enough
You're parsing a configuration file. The API sends colors as hex strings like "#FF0000", but your Rust code needs a struct with separate red, green, and blue fields. You tried #[derive(Serialize, Deserialize)]. The compiler rejected the JSON. The wire format and your Rust types don't match one-to-one. You need to bridge the gap.
That's where custom serializers and deserializers come in. You keep the type safety of Rust on both ends, but you take control of the messy middle. Serde provides the machinery; you provide the rules for how your data transforms when it crosses the boundary.
The adapter pattern
Serde is a translation layer. The #[derive] macro writes a literal translator. It looks at your struct fields and maps them directly to JSON keys. It's fast and easy, but it's rigid. Custom serialization is like hiring a specialist translator who knows your specific dialect. You tell Serde exactly how to turn your Rust type into the wire format, and how to turn the wire format back into your Rust type.
The most common approach uses helper functions attached to fields via attributes. You write a function that takes your type and a serializer, and returns a result. You write another function that takes a deserializer and returns your type. Serde calls these functions at the right moments. This keeps your custom logic isolated and leaves the rest of your struct automatic.
Minimal example: Hex colors
Here is a complete example. The Color struct holds numeric components. The Theme struct uses helper functions to serialize and deserialize the primary field as a hex string.
use serde::{Serialize, Deserialize};
#[derive(Debug, Serialize, Deserialize)]
struct Color {
r: u8,
g: u8,
b: u8,
}
// Helper function for serialization.
// Takes a reference to Color and a generic Serializer.
fn serialize_color<S>(color: &Color, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
// Format the components as a hex string.
let hex = format!("#{:02X}{:02X}{:02X}", color.r, color.g, color.b);
// Delegate to the serializer's string method.
// This returns the correct Result type for the serializer.
serializer.serialize_str(&hex)
}
// Helper function for deserialization.
// Takes a generic Deserializer and returns a Color.
fn deserialize_color<'de, D>(deserializer: D) -> Result<Color, D::Error>
where
D: serde::Deserializer<'de>,
{
// Deserialize the input as a String first.
// This consumes the deserializer and gives us the raw text.
let hex = String::deserialize(deserializer)?;
// Parse the hex string into components.
// In production code, handle parsing errors properly.
let bytes = hex.strip_prefix('#').unwrap_or(&hex);
let r = u8::from_str_radix(&bytes[0..2], 16).unwrap();
let g = u8::from_str_radix(&bytes[2..4], 16).unwrap();
let b = u8::from_str_radix(&bytes[4..6], 16).unwrap();
Ok(Color { r, g, b })
}
#[derive(Debug, Serialize, Deserialize)]
struct Theme {
// Attach the helper functions to the field.
#[serde(serialize_with = "serialize_color", deserialize_with = "deserialize_color")]
primary: Color,
}
fn main() {
let theme = Theme {
primary: Color { r: 255, g: 0, b: 0 },
};
let json = serde_json::to_string(&theme).unwrap();
println!("{}", json); // {"primary":"#FF0000"}
let parsed: Theme = serde_json::from_str(&json).unwrap();
println!("{:?}", parsed);
}
Convention aside: The community prefers helper functions over implementing Serialize or Deserialize traits directly for field-level customization. Helper functions keep the logic local and avoid trait coherence issues. Save manual trait implementations for newtypes or collection types.
Keep your helper functions small. If the function grows past twenty lines, you're probably fighting the abstraction.
Why the generic signatures?
The function signatures look intimidating. S: serde::Serializer and Result<S::Ok, S::Error> appear in every custom serializer. These generics exist because Serde works with many formats. The same code can serialize to JSON, YAML, TOML, or a binary format like Bincode. Each format has its own internal representation and error type.
The S parameter represents the serializer engine. You don't know if it's writing to a JSON string or a binary buffer. You call methods on S like serialize_str or serialize_map. The compiler fills in the implementation for the specific format at compile time.
The S::Ok type is the success value. For JSON, it's often (). For a streaming serializer, it might be the writer itself. You almost never construct S::Ok manually. You call serializer.serialize_str(...) and that method returns the Result with the correct Ok type. The compiler handles the plumbing.
The S::Error type is the error type for the format. You can't return a standard std::io::Error or a custom error enum. The serializer needs its own error type so it can format the message correctly for the target format. You use helper methods like serde::de::Error::custom to report problems. This keeps your code format-agnostic.
Trust the error types. S::Error exists for a reason. Don't try to bypass it.
Realistic example: Safe IDs
A common real-world problem involves large integers. JavaScript loses precision for numbers larger than 2^53. If your Rust struct has a u64 ID, serializing it as a number breaks clients that parse the JSON in JS. The solution is to serialize the ID as a string.
use serde::{Serialize, Deserialize};
#[derive(Debug, Serialize, Deserialize)]
struct User {
// Serialize as string, deserialize from string.
#[serde(serialize_with = "serialize_id_as_str", deserialize_with = "deserialize_id_from_str")]
id: u64,
name: String,
}
fn serialize_id_as_str<S>(id: &u64, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
// Convert the number to a string and serialize it.
serializer.serialize_str(&id.to_string())
}
fn deserialize_id_from_str<'de, D>(deserializer: D) -> Result<u64, D::Error>
where
D: serde::Deserializer<'de>,
{
// Deserialize as string first.
let s = String::deserialize(deserializer)?;
// Parse the string back to u64.
// Map the parse error to the deserializer's error type.
s.parse().map_err(serde::de::Error::custom)
}
fn main() {
let user = User { id: 12345678901234567890, name: "Alice".to_string() };
let json = serde_json::to_string(&user).unwrap();
println!("{}", json); // {"id":"12345678901234567890","name":"Alice"}
}
The map_err(serde::de::Error::custom) call is crucial. The parse method returns a Result<u64, ParseIntError>. The function signature requires Result<u64, D::Error>. You use map_err to transform the parse error into the deserializer's error type. This satisfies the compiler and ensures the error message flows through Serde correctly.
Convention aside: When multiple fields share the same custom logic, put the helper functions in a module and use #[serde(with = "module_name")]. It reduces repetition and keeps your struct definitions clean.
Handling multiple formats with Visitors
Helper functions work when the wire format is predictable. What if the API sometimes sends a string and sometimes sends a number? Calling String::deserialize fails when the input is a number. You need to inspect the input and handle both cases.
This requires the Visitor pattern. A visitor is a struct that implements methods for different input types. Serde calls the appropriate method based on what it finds in the data.
use serde::{Deserialize, Deserializer, de::Visitor};
use std::fmt;
// The visitor struct. It doesn't need fields for simple cases.
struct IdVisitor;
impl<'de> Visitor<'de> for IdVisitor {
type Value = u64;
// Tell Serde what this visitor expects.
// This message appears in error output.
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a string or a number")
}
// Called when the input is a string.
fn visit_str<E>(self, v: &str) -> Result<u64, E>
where
E: serde::de::Error,
{
v.parse().map_err(E::custom)
}
// Called when the input is a u64.
fn visit_u64<E>(self, v: u64) -> Result<u64, E>
where
E: serde::de::Error,
{
Ok(v)
}
}
fn deserialize_flexible_id<'de, D>(deserializer: D) -> Result<u64, D::Error>
where
D: Deserializer<'de>,
{
// Tell the deserializer to use our visitor.
deserializer.deserialize_any(IdVisitor)
}
You use deserialize_any to let Serde decide which visitor method to call. The visitor implements visit_str and visit_u64. If the input is a string, Serde calls visit_str. If it's a number, Serde calls visit_u64. You can implement other methods like visit_i64 or visit_bytes if needed.
The visitor pattern is your escape hatch for polymorphic input. Use it when the wire format varies and you need to adapt.
Pitfalls and compiler errors
Custom serialization introduces a few common traps. The compiler usually catches them, but the error messages can be dense.
If you return a standard Result with a concrete error type, the compiler rejects you with E0308 (mismatched types). Your function signature promises Result<S::Ok, S::Error>. You can't hand back a Result<u64, std::io::Error>. The serializer needs its own error type. Use serde::de::Error::custom to wrap your error.
If you forget the generic bounds, you get E0277 (trait bound not satisfied). The compiler needs to know that S implements Serializer and D implements Deserializer. Always include where S: serde::Serializer and where D: serde::Deserializer<'de>.
If you try to deserialize a type that doesn't implement Deserialize, you get E0277 again. You can't call String::deserialize on a type that Serde doesn't know how to handle. Stick to types that implement Deserialize, or use serde_json::Value to parse into a generic structure first.
Don't construct errors manually. You might be tempted to return Err("bad input".into()). This fails because the compiler can't infer the error type. Always use the error helpers provided by Serde. They ensure the error integrates with the format's error reporting.
Treat the error types as part of the contract. If you can't satisfy the signature, your logic is likely trying to do too much.
Decision matrix
Use #[derive(Serialize, Deserialize)] when your Rust types match the wire format exactly. The macro generates efficient code with zero overhead.
Use #[serde(serialize_with = "func", deserialize_with = "func")] when a single field needs custom formatting. This keeps the rest of your struct automatic and isolates the custom logic to helper functions.
Use #[serde(with = "module")] when multiple fields share the same custom logic. You can put the helper functions in a module and apply the whole module to a field.
Use a Visitor implementation when the wire format varies and you need to handle multiple input types, like a string or a number. This gives you full control over deserialization based on the actual data.
Use #[serde(remote = "Type")] when you need to customize serialization for a type you don't own, like a third-party struct. This lets you write a custom implementation without unsafe or wrapper types.
Use manual impl Serialize only when you are building a newtype wrapper or a collection type that needs deep integration with Serde's streaming API. Helper functions cover 99% of use cases.
Reach for plain derives whenever possible. Custom serialization adds maintenance burden. Only add it when the wire format demands it.