Adding middleware support to Rust reqwest

This is the second post in our open source series, where we talk about engineering challenges at TrueLayer and open source our solutions to them. The first one was gRPC load balancing in Rust, in case you missed it.

This post is about reqwest-middleware, a crate built on top of the reqwest HTTP client to provide middleware functionality.

The problem

Running applications at scale requires built-in resilience when communicating over the network with internal and external services, because services fail. 

Retries are a common strategy to improve reliability. The mechanics are fairly simple: wrap each request in a loop and retry until you get a successful response or you run out of attempts.

We have tens of clients across our codebases: we don’t want to re-implement retries in an ad-hoc fashion for each of them.

At the same time, we’d prefer to keep our domain code free of this network-level concern — it’d be ideal to have retries transparently implemented in the HTTP client itself. 

We could write a RetryHttpClient that wraps a standard client to augment it with retry functionality — but retries are not the full story. There is other functionality we want our HTTP clients to handle: propagation of distributed tracing headers, caching, logging. 

But we don’t want to write TracingRetryableClient, TracingRetryableCachingHttpClient, RetryableTracingCachingHttpClient (order matters!) and all the other possible combinations.

We want an abstraction that is composable. 

All these pieces of functionality follow the same pattern:

• We want to run some arbitrary logic before and after executing a request

• The logic is completely independent to the problem domain, it is just concerned with the underlying transport and the requirements of the organisation as a whole (eg logging standards).

The good news is this is a common problem in software systems, as such there’s also a common solution for it: middlewares.

(aside: note that we’re referring to a very specific kind of middleware in this article, the term itself is more general. See the Wikipedia page on middleware for usage in other contexts)

Rust HTTP client middleware

At TrueLayer we use reqwest as our HTTP client for all our Rust services.

We chose it because it provides an async-first API, it is compatible tokio and it has been used extensively in production.

Sadly, reqwest doesn’t support middleware out of the box.

What were our options?

• Use an off-the-shelf crate, either replacing reqwest or built on top of it. At the time of writing, no other well-established Rust HTTP client supporting middleware ticks the same boxes that reqwest does for us. surf has good mindshare and built-in middlewares, but it requires pulling in async-std.

• Try to get middleware support implemented upstream. reqwest maintainers have been discussing this from 2017 (see this issue) and still there doesn’t seem to be consensus, not even on whether such functionality belongs to the crate. So we’re unlikely to get something through anytime soon.

• The final alternative was to wrap reqwest and implement middlewares on top of that, so that’s the approach we went for. reqwest-middleware was born.

With reqwest-middleware we’re able to attach middleware to a Client and then proceed to make requests as if we were using reqwest directly:

Prior art

Before we talk about our approach let’s take a look at some middleware APIs in the wild:

Surf

Surf is a Rust HTTP client. Here’s an example middleware from their own docs:

As you can see, it takes a request object and a next value, which can be used to forward that request into the remaining pipeline, and returns a Response. This allows us to manipulate requests by mutating them before forwarding down the chain, we could also change the res value we get back from next.run before returning.

We can even use control flow around next, which allows for retries and short-circuiting:

Express

Express is a well-established Node.js web framework. Its middlewares are written as plain functions, here’s an example from their docs:

This is very similar to surf’s approach, except here we take a response object and can mutate it directly: the middleware function doesn’t return anything.

Tower

tower describes itself as a library of generic Rust components for networking applications. 

It’s used as a building block for many notable crates such as hyper and tonic. tower’s middlewares are a bit more involved, most likely because they did not want to force dynamic dispatch (eg async_trait) on their users. 

As for the other libraries, this is the example given on the tower docs:

Ignoring the poll_ready method which is used for backpressure, tower's Services are defined as functions from a request to a response: call returns a Future where Future::Item is the associated Service::Response. 

The async middleware trait in surf is simpler because it relies on a procedural macro (async_trait) to use the async fn syntax in traits — behind the scenes it translates into boxing futures. This is necessary because async is not supported in trait methods yet, see this post by Nicholas D. Matsakis for an in-depth look into why.

Middlewares in tower are defined through the Layer trait which simply maps one service into another. Implementation usually involves having a generic struct wrapping some Service and delegating call to it. 

The wrapped service plays the same role of the next parameter in surf and express. It gives you a way to call into the rest of the middleware chain. This approach still lets us manipulate requests and responses in the same way we could with the next-based APIs.

Finagle

Finagle is a JVM RPC system written in Scala. Let’s take an example middleware from the finagle docs as well:

Here we also have Services which are very similar to tower: a function from a request into a response.

Middlewares in Finagle are called Filter. The Filter type is more complex than tower’s Layer as it doesn’t require the Req and Rep types in apply to be the same as the request and response for the service parameter.

SimpleFilter, as the name implies, is a simplified version with fixed request/response types. A SimpleFilter takes a request and the wrapped Service as parameters and returns a response, so it functions like the tower API but collapses Layer::layer  and Service::call into the single SimpleFilter::apply method.

Middleware types

In general you’ll find that middleware APIs fall into one of two categories: either the middleware is a function taking a request and a next parameter, like in surf and express, or a mapping from one Service into another, like tower and Finagle do.

Overall both approaches give just as much flexibility. Both require at least one extra dynamic dispatch per middleware, as Rust does not support impl Trait in return types of trait methods (yet), so we went with the Next approach because that makes it easier to implement middleware, as shown by the difference between surf and tower.

reqwest-middleware

We ended up with a pretty standard API for middlewares (see the docs for a more detailed view of the API):

Extensions is used to get arbitrary information across middlewares in a type-safe manner, both from an outer middleware into deeper ones and from an inner middleware out to previous ones.

For demonstration purposes, here’s a simple logging middleware implementation:

Conclusion

We wrapped reqwest with a middleware-enabled client that exposes the same simple API. This enabled us to build reusable components for our resiliency and observability requirements. 

On top of that we also published reqwest-retry and reqwest-opentracing which should cover a lot of the use cases for this crate.

Developers can now harden their integrations with remote HTTP simply by importing a couple of crates and adding with_middleware calls to the client set up code — no disruption to any other application code.

We're hiring! If you're passionate about Rust and building great products, take a look at our job opportunities