I’ve really enjoyed using gRPC-Gateway to build gRPC/JSON-over-HTTP (RESTful) APIs in Go for quite a few years now. However, at first glance, it appears to be incompatible with one of my favorite ways to deploy small serverless apps - using AWS API Gateway and AWS Lambda.

According to the documentation, gRPC-Gateway is architected as follows:

This means generally that gRPC-Gateway apps usually run two servers - one for gRPC and one for HTTP. The HTTP server is a reverse proxy that translates HTTP requests to gRPC calls, and actually makes local gRPC calls to the gRPC server. The servers typically listen on different ports, although you can configure them to listen on the same port using something like cmux.

Immediately you can see that this architecture is not directly compatible with AWS Lambda. Lambda is stateless by definition, and does not outwardly support the idea of running servers and listening on any port(s). Lambda provides an adapter library to support HTTP APIs via standard Go http.Handlers, but it doesn’t support running a server. In fact, if you try to listen with a net.Listener in a Lambda function, you will get an error.

Luckily, an http.Handler is all we need to run the gRPC-Gateway reverse proxy component on Lambda using the adapter, and we can use gRPC-Gateway’s runtime.ServeMux for that.

That covers the HTTP portion. However, we’re still left with the problem of running the gRPC server, which the HTTP proxy makes outbound requests to. As I just mentioned, Lambda forbids running servers. So how then, can gRPC-Gateway work?

This is where the trick comes in. We actually can run the gRPC server on Lambda, but we can’t listen on a port. Instead, we can create a Unix domain socket and listen on that1 (you may have guessed the punchline from this post’s title). You might point out that Lambda’s filesystem is readonly, and you would generally be right… but luckily, the /tmp directory is writable (up to 512 MB by default).

The plan of attack is then:

  1. Create a Unix domain socket in /tmp and create a net.Listener to listen on it.
  2. Run the gRPC server using this listener.
  3. Create the gRPC-Gateway HTTP reverse proxy, and configure its gRPC entrypoint to be the gRPC server’s Unix domain socket (see docs).
  4. Pass the HTTP reverse proxy’s runtime.ServeMux to the Lambda adapter library.
  5. Run the Lambda function with the handler function.

This ends up working surprisingly well, and I haven’t encountered any issues yet. All the interceptors and other gRPC server features work as expected. The main downside is that the gRPC server is not exposed via Lambda and API Gateway, but this is unfortunately a limitation of those platforms.

FAQ

Can I make gRPC requests to the Lambda function?

Nope.

Can’t I just use the gRPC-Gateway mux without the gRPC server?

Yes and no. If you register your services directly to the mux using the Register...Handler methods generated by gRPC-Gateway, then you can use the mux without a gRPC server. You could pass this mux directly to the Lambda adapter, and it would work as a plain HTTP server.

However, if you want to use gRPC specific features like interceptors (commonly used for auth for example), which are registered directly to the gRPC Server, you’ll be out of luck. Being able to use (auth) interceptors was actually my main motivation for writing this.

Just… why?

Good question. As I mentioned above, you still can’t make gRPC requests to the Lambda function. So why even bother, and instead why not just avoid gRPC altogether and use a regular JSON-over-HTTP API (e.g. via a http.Handler)?

If that’s all you need, then you’re right - you don’t need to go through all this trouble. However, if you already have an existing gRPC service that you want to run on Lambda, and you’re OK with exposing it as JSON-over-HTTP API, then this is a good way to do it.

If you’re starting from scratch, I do not recommend this approach. Instead, I recommend using Connect. You should be able to pass Connect’s regular HTTP multiplexer to the Lambda adapter and be done with it.

Sample Project

I’ve created a sample project that demonstrates this setup. It’s a simple gRPC service that echoes a message back to the client. The gRPC server and gRPC-Gateway reverse proxy are both run in the same Lambda function. The project contains a Terraform configuration to deploy the Lambda function and API Gateway. Please check it out for more details!

  1. Apparently this trick was pointed out years ago in an article by TJ Holowaychuk. See the “Running any web server in Lambda” heading. ↩