How to Implement Connection Pooling for Network Clients
You are writing a scraper that needs to fetch a thousand URLs. You write a loop, create a new HTTP client for each request, and watch the latency spike. The remote server starts throttling you. Your code isn't slow because of the network. It is slow because you are paying the handshake tax a thousand times. Every connection opens, negotiates TLS, sends one request, and dies. You need to keep those doors open.
Connection pooling solves this. It reuses existing TCP connections for multiple requests. You open a connection once, send ten requests, and close it when you are done. The pool manages the lifecycle. It hands you a connection when you need one and takes it back when you are finished. If the pool is empty, you wait until someone returns a connection. This saves time, saves memory, and keeps remote servers happy.
The rental car counter
Think of a connection pool like a rental car counter. You do not manufacture a car when you arrive. You grab keys from the rack. You drive. You return the keys. If the rack is empty, you wait in line. The pool manages the fleet.
In Rust, the actix-web ecosystem provides this via the awc client. The client holds a pool of connections. The pool uses a semaphore to enforce limits. A semaphore is a counter with a lock. It tracks how many connections are active. If you try to open more than the limit, you block until someone returns a connection. This prevents your application from opening thousands of sockets and exhausting file descriptors.
Minimal example
The awc client enables pooling by default. You configure the limit using ConnectorConfig. The limit controls how many concurrent connections the pool allows per host.
use actix_web::awc::{Client, ClientRequest};
use actix_web::awc::client::config::ConnectorConfig;
#[actix_web::main]
async fn main() -> std::io::Result<()> {
// Build the client once.
// The client holds the connection pool internally.
let client = Client::builder()
// Set the maximum number of concurrent connections per authority.
// The pool will block if this limit is reached.
.connector_config(ConnectorConfig::default().limit(10))
.finish();
// Reuse the client for multiple requests.
// The pool reuses connections across these calls.
let resp = client.get("https://example.com").send().await?;
println!("Status: {}", resp.status());
// Read the body to return the connection to the pool.
let _body = resp.body().await?;
Ok(())
}
Reuse the client instance. Creating a new client creates a new pool. You lose the benefit of pooling if you instantiate a client inside a loop.
How the pool works
When you call client.get(url).send(), the client asks the pool for a connection. The pool checks the authority. An authority is the combination of scheme, host, and port. https://example.com:443 is one authority. http://example.com:80 is another.
The pool maintains a separate semaphore for each authority. This design isolates failures. If api.example.com is slow and fills its pool, requests to cdn.example.com continue to flow. You do not get blocked by a slow dependency.
If a connection is available, the pool hands it to you. You send the request. You read the response. When the response is dropped, the connection goes back to the pool. The semaphore releases a permit. Another waiting task can now proceed.
If no connection is available, your task suspends. It waits in a queue. The pool wakes you up when a connection is returned. This backpressure protects your system. You never open more connections than the limit allows.
Realistic concurrent usage
In real applications, you often spawn multiple tasks that share the same client. The client must be shared across tasks. Rust's ownership system requires you to clone the client. Cloning a client is cheap. It clones the handle to the pool, not the pool itself. The underlying pool is wrapped in an Arc. Cloning just bumps a reference count.
use actix_web::awc::{Client, ClientRequest};
use actix_web::awc::client::config::ConnectorConfig;
use futures::future::join_all;
#[actix_web::main]
async fn main() -> std::io::Result<()> {
// Build the client once.
// All tasks will share this pool.
let client = Client::builder()
.connector_config(ConnectorConfig::default().limit(5))
.finish();
let urls = vec![
"https://example.com/api/1",
"https://example.com/api/2",
"https://example.com/api/3",
"https://example.com/api/4",
"https://example.com/api/5",
"https://example.com/api/6",
];
// Create async tasks.
// Each task clones the client.
// The clone is cheap and shares the pool.
let tasks: Vec<_> = urls.into_iter().map(|url| {
let client = client.clone();
async move {
let resp = client.get(url).send().await?;
// Always read the body.
// Dropping the response without reading can leave the connection in a bad state.
let _body = resp.body().await?;
println!("Done: {}", url);
Ok::<_, actix_web::Error>(())
}
}).collect();
// Run all tasks concurrently.
// The pool enforces the limit across all tasks.
join_all(tasks).await;
Ok(())
}
Clone the client, not the pool. The clone is cheap. It shares the underlying resources. If you try to move the client into a closure without cloning, the compiler rejects you with E0382 (use of moved value).
Pitfalls and errors
Creating a client per request. This is the most common mistake. If you create a client inside a loop, you create a new pool every time. The pool never gets a chance to reuse connections. You pay the handshake cost for every request. Build the client once and reuse it.
Dropping the response without reading. The connection is tied to the response stream. If you drop the response without consuming the body, the stream is dropped. The connection might be reused, but if there is unread data, the TCP stream is corrupted for the next request. The pool may discard the connection. This wastes resources. Always read or drain the body. A common convention is let _ = resp.body().await; to signal that you intentionally discard the body.
Setting the limit too low. If the limit is too low, tasks spend most of their time waiting for permits. Your throughput drops. You create a bottleneck. Profile your application. Increase the limit if tasks are waiting.
Setting the limit too high. If the limit is too high, you risk exhausting file descriptors on your machine. You also risk overwhelming the remote server. The server may start rejecting connections or throttling you. A safe starting point is 10 to 50 connections per host. Adjust based on server capacity and your local ulimit -n.
Forgetting authority keying. The limit applies per authority. If you hit example.com and api.example.com, they are separate authorities. You can have 10 connections to each. If you expect a global limit, you need to use the same authority or implement a custom connector.
Read the body. Dropping a response without consuming the body can leave the connection in a bad state. Always drain or read.
When to use pooling
Use the default Client when you need standard HTTP requests with automatic reuse. The builder enables pooling by default. You get connection reuse without extra configuration.
Use ConnectorConfig::limit when you must cap concurrent connections to a single authority to respect server rate limits or save local file descriptors. This is essential for high-concurrency applications.
Use ClientBuilder::disable_keep_alive() when the remote server mishandles persistent connections and drops data on reused sockets. Some legacy servers break when clients reuse connections. Disabling keep-alive forces a new connection for every request. This is slower but more compatible with broken servers.
Use a custom Connector implementation when you need to route traffic through a proxy, use Unix domain sockets, or inject custom headers at the transport layer. The default connector handles standard HTTP and HTTPS. Custom connectors give you full control over the transport.
Pool by default. Disable pooling only when you have a specific reason. The handshake cost is rarely worth the complexity of managing connections manually.