Adapting a gRPC service to REST with ScalaPB Gateway

Now that we have seen a Scala gRPC service and a Scala RESTful service it's only natural to wonder if there is any middle ground. You want to have a nice and performant backend but you could also  have a lot of RESTful legacy code or be really into curl. Your wish is my command.

The first piece of the puzzle is the fact that GOOG has a standard for mapping REST APIs onto protobuf declarations. With no hacks or any hard thinking you can augment your existing proto files to be REST-friendly.

The second piece is making use of such a mapping. Apparently there is a Go implementation of what is known as gRPC Gateway. It can work with any gRPC service and so could be used with the ones implemented in Scala. But there is another, arguably more appealing option. The ScalaPB library we met before can generate a Scala RESTful gateway too. I guess it's not as well tested or supported but then again there is not that much code behind it to prevent it from serious use.

So I went ahead and updated my two-service gRPC example to support ScalaPB REST Gateway generation. It intentionally borrows a lot from the pure gRPC example so that the difference can be easily seen by comparing files. Technically, you still need exactly the same gRPC service implementation as before. The proto files are almost the same.

There are two new objects. To begin with we need a new abstraction to wire together the generated Gateway code. It's essentially a gRPC client with hooks to plug the RESTful stuff into it. The other is just a typical HTTP client. I followed the path of least resistance and used Jackson for JSON marshaling and commons-http for sending requests. There is a little wiring but in general the code speaks for itself.

What I learned from this exercise is that making gRPC co-exist with REST is easy. I am not sure if running a Gateway locally for each service instance is a better idea than creating a single API Gateway service to front the rest of the backend.

As a parting note, I am surprised there is no other IDL for describing RESTful services for the purpose of code generation. With a standard AST and parser library it would be quite strait forward to generate REST endpoints for any popular service framework. It's equally surprising that the Akka team does not have a generator for akka-http endpoint from protobuf declarations. It looks like such a natural step for them and the ScalaPB project even has the foundation in place.

Asynchronous RPC server with Akka HTTP

At work my team maintains a number of RESTful services. The good news is that they are implemented with akka-http which makes things tolerable. The bad news is that it's all about REST and JSON. So we are having a lively debate about a possible migration to something more like gRPC.

Personally, I am not a fan of non-binary protocols and data formats. At least partially it is related to my experience in building analytics systems where any query routinely requires a lot of data. Yes, for the time being you have to use REST/JSON to communicate between the browser and your API Gateway. But using the same approach to make your backend services talk to each other never made sense to me. I am really looking forward to the day when WASM/HTTP2&Co finally make front end development great again.

In the mean time I would like to see if I can gather enough evidence in favor of my position. So in the spirit of my recent gRPC experiment I am going to implement basically the same API with REST. As a bonus, I want to mention a couple of observations I made using akka-http. As usual, the code is on github. I continue the tradition of implementing a service with two endpoints. I even mostly follow the same code structure as before with gRPC.

The picture above sketches high-level interactions. There are a few noteworthy Akka-related details inside but nobody will be surprised by another RESTful service implementation.

• TestServiceA - is where the actual logic would be. It also hosts a trait and case classes which would be generated from a protobuf file in case of gRPC
• HttpEndpointA - an actual HTTP endpoint that plugs service logic into HTTP network endpoint. A good example of Akka HTTP expressiveness.
• AkkaHttpServer - the service implementation entry point. In typical Scala fashion it directly implements the main method and performs basic Akka dependency wiring. It is also responsible for binding request handlers to the underlying HTTP server.
• Http - technically an object in a high-level Akka API it represents the usual functionality of request/response-style interactions over HTTP 
• AkkaHttpClient - not shown; wraps akka-http-based client logic. I spent some time getting it right because client-side examples are harder to find. In particular it illustrates how to use Spray JSON marshaller. But it could be any HTTP client capable of sending a POST request with a JSON body. 

If you are curious enough to look at the code I would like to highlight a few details useful in real life. Not all of them are prominent enough in usual examples. The HttpEndpointA class would be a good illustration of most of them.

• ExceptionHandler - you can configure a handler for exceptions uncaught otherwise. It is the last line of defense against unexpected problems. 
• RejectionHandler - such a handler is very useful if you want to explicitly deal with invalid requests (e.g. garbled JSON or more likely a JSON body representing a request to some other service)
• request timeout - even though Akka has a default global property ("akka.http.server.request-timeout") you could configure it explicitly for a service that is expected to take an unusually long time 
• implicit JSON (de)serialization with Spray - it's very convenient to use but rather puzzling initially to configure. In addition to the marshaller object holding implicit vals notice how the "import ServiceAJsonMarshaller" statement makes it usable by "as" and "complete" directives
• composing multiple Routes - different styles are possible here; if there were another URL to support in this endpoint we could (a) declare another "def process2: Route" and then compose the two with "process ~ process2" using another somewhat cryptic Akka directive

On the client side there are a few more details to mention. My first reaction was that Akka HTTP client code looks somehow less straitforward than what one would see with, say, commons-http. Any HTTP client would work for this example but I wanted to see one with akka-http to better grok the API.

• JSON marshalling with Spray was a puzzle again. It was not clear to me from the official documentation but googling helped.
• Somewhat unusually, the client requires the same ActorSystem and Materializer infrastructure as the server. 

My conclusion is that if circumstances force me to implement anything RESTful I will reach for Akka HTTP first. But in comparison with real RPC it's more work to produce less safe code. But that's a topic for another day.

gRPC in Scala with ScalaPB

Curiously enough my old post on gRPC still gets disproportionately many hits. Now that I work with Scala full time I was curious to see how my old Java example would compare with a Scala version. Spoiler alert: I don't have that much new to report. Other people covered it better before. This post probably makes more sense as a head-to-head comparison between Java and Scala versions.

From what I see Scala people tend to avoid Java libraries. I am skeptical it's a good idea especially around networking and concurrency where the Java ecosystem is at its best. But I am still learning and have no strong opinions developed yet. After all, last time I used Java j.u.c Futures, Guava Futures, and CompletableFutures were still all in the game.

In case of gRPC the unofficial Scala code generator is known as ScalaPB. Once you configure it in your SBT project it feels as natural as the official Java version. The good news is that the generated code operates in terms of Scala Futures and so feels idiomatic. But you still need to initialize the gRPC foundation and that part looks almost the same as in Java.

As before, the goal is to support two different service endpoints on the same server and to have fully asynchronous client and server implementations. As we concentrate on the request/response approach we completely ignore the question of streaming. gRPC bootstrap code is so simple that I could not find enough of it to extract anything truly generic. So even though I notionally have a server and a client they just wrap initialization logic and might be unnecessary.

There are a couple technicalities I would like to highlight:

• I had to google and experiment to find out how to configure ScalaPB for a multi-module project and a non-default proto file directory
• even though your actual logic is all about Futures and so requires ExecutionContexts, it's easier to start with an Executor to name threads and configure underlying thread pools and then wrap it into a context
• I can think of a few slightly different ways to propagate ExecutionContexts; implicit vals are idiomatic but seem to be less readable despite saving a little typing

All in all, using ScalaPB feels a little more idiomatic but not that different from how it's done in Java.

Asynchronous RPC server with GRPC

After a severe disappointment with the state of the Thrift RPC framework I was looking forward to the first stable release of GRPC. Of the original big three only the protobuf-based ecosystem seems to be still evolving enough to be a viable choice. With the arrival of v1.0 it is finally ready for production.

For better or worse despite using a binary transport protocol GRPC forces things with "HTTP" in their names upon us. That is unfortunate because a protocol designed for connecting to browsers outside of the data center will incur some penalties in complexity and probably latency on backend services not exposed to the public Internet. 

The question I want to investigate today is pretty much the same I had playing with Thrift last year. What does it take to configure and run a backend service exposing multiple end points with no blocking calls. Spoiler alert: it's so trivial I should probably double-check my understanding of the threading model to confirm GRPC is as reactive as it seems to be :)

To begin with I would like to illustrate a typical GRPC call. Let's assume we have a service API with one method. GRPC uses protobuf3 as its IDL and, other than not having required fields anymore, I find it hard to see any difference from protobuf2 or even Thrift. 

The first pleasant surprise is that a single maven plugin with a simple, mostly OS-specific configuration is all it takes to compile IDL into Java classes. For each XXX service you basically end up with two classes:

• xxxProto - request and response classes
• xxxGrpc - request handler base class and client-side factory method for different IO styles

The second pleasant surprise is how uniform server-side API is. When processing a request you are given an Observer with the same onNext/onComplete/onError methods used for all four currently supported RPC types. On the client side it's even simpler, all you need is a request instance and a guava callback instance.

where

• a Channel is created for a TCP destination
• a service stub using the new Channel is obtained
• a callback instance is created to process server response
• an RPC method is called on the stub
• a mirror copy method is called on the request handler but with an observer instead of a callback
• the handler calls either onNext/onSuccess or onError observer methods to finish processing
• once the server response is received, GRPC invokes the callback on a thread from a pre-configured executor

Server-side initialization is equally terse. You give it a handler instance for each service end point, an executor, and a port to bind to. Four lines of code and you are in business.

In my example I created two nearly identical protobuf files to describe two service end points. There are two abstractions to establish a connection on the client-side and bootstrap the server on the server side. I followed the same CompletableFuture-based approach to implementing request handlers as discussed in the Thrift post. A unit test wires everything together and calls both end points in parallel.

So far so good. But what ointment would be without a fly? There are a couple of things that I either don't understand or need more time to sort out. And at least one of them is probably necessary for production deployments. 

A trivial note is that GRPC is only a second library I am aware of that depends on JUL (Parquet being the first but only temporarily).

One odd question I still have is how asynchronous Java GRPC implementation really is. What confuses me is that I can see an "asynchronous basics tutorial" for C++ only. Is it some special kind of asynchronicity attainable only by those who manage memory manually? Or is there some blocking still left in Java GRPC library?

A real question is the complexity of GRPC security. My example follows the lead of GRPC Java tutorial when it calls "usePlaintext()" on channel builder. At first glance I am not even sure if SSL/TLS is necessary for the traffic inside of the data center or whether AWS ELB could interfere with it. A topic for another day I guess.

Asynchronous RPC server with Thrift

A. Introduction

So many systems use Protocol Buffers to define APIs and DTOs that it’s easy to forget there is also Thrift. In contrast to the PBs, Thrift has always lacked proper documentation. It is enough to say that one of the most popular guides for it was written by an outsider. 

On the other hand, Thrift comes with an actual RPC library. The PBs world will get one only once GRPC is released. For those of us who do not base our backend services on something like Finagle, libthrift could be a reasonable choice at least initially. 

B. Documentation and examples

When one considers a new RPC framework one of the first questions to ask is how well its support for asynchronous communications is. Strangely enough, in case of Thrift the official documentation is pretty silent. As a matter of fact, I was not able to google an example of an asynchronous Thrift-based service or any guidelines for using Thrift-generated classes with “Async” in their name. 

What  I found was all about synchronous calls. Strangely enough, even one libthrift alternative I found  had nothing to say about asynchronous calls to my utter surprise. 

So I had to dig a little deeper to understand how to do it. While exploring it I created a small project that I will use as a complete example. The idea was to prototype a service with two different service interfaces. Each one takes one request object and returns a response object.

C. Thrift-generated code walk-through

If you look at the file generated by the Thrift compiler for your service you’ll see a few classes with the prefix “Async”:

• AsyncIface: the asynchronous version of your service interface with additional AsyncMethodCallback argument
• AsyncClient: client-side view of your RPC service interface; it requires some configuration to actually make calls 
• AsyncProcessor: server-side intermediary between your actual request handler and the TServer you will use to wrap everything into a running process

D. Wiring up libthrift infrastructure

The generated classes need to be plugged into the libthrift infrastructure. On the server side, you will:

• create an instance of your request handler
• wrap it into an instance of Thrift-generated AsyncProcessor
• in case of multiple service interfaces, create a multiplexed processor and register each async processor with it using a unique name
• create a server configuration with typical parameters such as TCP port and a j.u.c executor to accept requests. There are a few TServer implementations to choose from
• start the server

On the client side, you need to: 

• create two reusable instances: a client manager and a protocol factory
• with those two instantiate (for every service interface) a client factory 
• make the factory create a client class instance
• call the client

Each client takes a TCP host/port pair so in real life it would take a discovery service of some kind  to find those. Notice that AWS ELB supports TCP traffic load balancing and so there would be the only URL for any number of servers in that case.

E. The curious case of multiplexed asynchronous processor

When I mentioned a multiplexed processor I lied to you. It turns out there is no such thing currently in libthrift-0.9.3.jar . So if you use a multiplexed protocol factory on the client side there will be no peer on the server to demultiplex it. As an immediate workaround, I implemented one. As a real solution we have got a pending pull request for THRIFT-2427.

At first glance a glaring hole of this size in a mature library from a big company makes me think that Thrift is an obsolete dead-end. That would explain why they never bothered to produce reasonable documentation. I have some hope for GRPC in this respect.

F. Service client and request handler ideas

Aside from wiring up the auto-generated classes with the infrastructure from the libthrift library you will also need to implement an actual request processor and to call the client class somewhere. In my project you can find both pieces in the unit test. As a working end-to-end example it mostly follows this description even though it makes some effort to represent and configure RPC service definitions in a more generic way.

On the server side it is convenient to process request by 

• supplying the corresponding job to some executor (different from the one used for networking)
• call Thrift RPC callback from a CompletableFuture listener

In a similar way, on the client side a CompletableFuture allows to receive a response either synchronously with get or asynchronously in a CompletableFuture listener. Notice how an actual response instance is wrapped into a Thrift-generated class representing an RPC method call.