Adding middleware support to Rust reqwest
Continuing with our open source series, we present a middleware adapter for the ubiquitous reqwest Rust crate.
This is the second post in our open source series, where we talk about engineering challenges at TrueLayer and open source our solutions to them. The first one was , in case you missed it.This post is about , a crate built on top of the reqwest HTTP client to provide middleware functionality.
The problemRunning applications at scale requires built-in resilience when communicating over the network with internal and external services, because services fail. Retries are a common strategy to improve reliability. The mechanics are fairly simple: wrap each request in a loop and retry until you get a successful response or you run out of attempts.We have tens of clients across our codebases: we don’t want to re-implement retries in an ad-hoc fashion for each of them.At the same time, we’d prefer to keep our domain code free of this network-level concern — it’d be ideal to have retries transparently implemented in the HTTP client itself. We could write a RetryHttpClient that wraps a standard client to augment it with retry functionality — but retries are not the full story. There is other functionality we want our HTTP clients to handle: propagation of distributed tracing headers, caching, logging. But we don’t want to write TracingRetryableClient, TracingRetryableCachingHttpClient, RetryableTracingCachingHttpClient (order matters!) and all the other possible combinations.We want an abstraction that is composable. All these pieces of functionality follow the same pattern:
- We want to run some arbitrary logic before and after executing a request
- The logic is completely independent to the problem domain, it is just concerned with the underlying transport and the requirements of the organisation as a whole (eg logging standards).
Rust HTTP client middlewareAt TrueLayer we use as our HTTP client for all our Rust services.We chose it because it provides an async-first API, it is compatible tokio and it has been used extensively in production.Sadly, reqwest doesn’t support middleware out of the box.What were our options?
- Use an off-the-shelf crate, either replacing reqwest or built on top of it. At the time of writing, no other well-established Rust HTTP client supporting middleware ticks the same boxes that reqwest does for us. has good mindshare and built-in middlewares, but .
- Try to get middleware support implemented upstream. reqwest maintainers have been discussing this from 2017 (see ) and still there doesn’t seem to be consensus, not even on whether such functionality belongs to the crate. So we’re unlikely to get something through anytime soon.
- The final alternative was to wrap reqwest and implement middlewares on top of that, so that’s the approach we went for. reqwest-middleware was born.
Prior artBefore we talk about our approach let’s take a look at some middleware APIs in the wild:SurfSurf is a Rust HTTP client. Here’s an example middleware from :
As you can see, it takes a request object and a next value, which can be used to forward that request into the remaining pipeline, and returns a Response. This allows us to manipulate requests by mutating them before forwarding down the chain, we could also change the res value we get back from next.run before returning.We can even use control flow around next, which allows for retries and short-circuiting:
ExpressExpress is a well-established Node.js web framework. Its middlewares are written as plain functions, here’s an example from :
This is very similar to surf’s approach, except here we take a response object and can mutate it directly: the middleware function doesn’t return anything.Towertower describes itself as a library of generic Rust components for networking applications. It’s used as a building block for many notable crates such as hyper and tonic. tower’s middlewares are a bit more involved, most likely because they did not want to force dynamic dispatch (eg async_trait) on their users. As for the other libraries, this is the example given on :
Ignoring the poll_ready method which is used for backpressure, tower's Services are defined as functions from a request to a response: call returns a Future where Future::Item is the associated Service::Response. The async middleware trait in surf is simpler because it relies on a procedural macro (async_trait) to use the async fn syntax in traits — behind the scenes it translates into futures. This is necessary because async is not supported in trait methods yet, see by Nicholas D. Matsakis for an in-depth look into why.Middlewares in tower are defined through the Layer trait which simply maps one service into another. Implementation usually involves having a generic struct wrapping some Service and delegating call to it. The wrapped service plays the same role of the next parameter in surf and express. It gives you a way to call into the rest of the middleware chain. This approach still lets us manipulate requests and responses in the same way we could with the next-based APIs.FinagleFinagle is a JVM RPC system written in Scala. Let’s take an example middleware from the as well:
Here we also have Services which are very similar to tower: a function from a request into a response.Middlewares in Finagle are called Filter. The Filter type is more complex than tower’s Layer as it doesn’t require the Req and Rep types in apply to be the same as the request and response for the service parameter.SimpleFilter, as the name implies, is a simplified version with fixed request/response types. A SimpleFilter takes a request and the wrapped Service as parameters and returns a response, so it functions like the tower API but collapses Layer::layer and Service::call into the single SimpleFilter::apply method.Middleware typesIn general you’ll find that middleware APIs fall into one of two categories: either the middleware is a function taking a request and a next parameter, like in surf and express, or a mapping from one Service into another, like tower and Finagle do.Overall both approaches give just as much flexibility. Both require at least one extra dynamic dispatch per middleware, as Rust does not support impl Trait in return types of trait methods (yet), so we went with the Next approach because that makes it easier to implement middleware, as shown by the difference between surf and tower.
reqwest-middlewareWe ended up with a pretty standard API for middlewares (see for a more detailed view of the API):
is used to get arbitrary information across middlewares in a type-safe manner, both from an outer middleware into deeper ones and from an inner middleware out to previous ones.For demonstration purposes, here’s a simple logging middleware implementation: