gRPC is an open-source Remote Procedure Call framework created by Google. It uses Protocol Buffers to make services easy to define and consume. Built on HTTP/2, it is fast and supports bidirectional streaming.

Even with those advantages, leaving familiar HTTP/JSON behind for an unfamiliar framework is not an easy decision. You need to learn Protocol Buffers, change server configuration, rethink error logging and debugging, and much more.

After considerable thought, Buzzvil decided to adopt gRPC and now uses it in several areas. Here, I’ll share why we chose it, how we implemented it, and what we learned.

Why gRPC?

Buzzvil uses a microservices architecture, or MSA. Our reasons are discussed in https://medium.com/@ssowonny/Buzzvil’s Software Architecture Team. Building an MSA brings a variety of challenges.

One major hurdle is network latency. In a monolith, modules communicate within a single process. In an MSA, services run in separate processes and on different hosts, communicating frequently. gRPC with Protocol Buffers is generally faster than HTTP/JSON. It doesn’t save an enormous amount of time on each request—perhaps only a few milliseconds in exchange for significant learning and code. But when a single incoming request triggers many internal calls, even small reductions in network time matter.

Defining communication contracts between services is just as important. Lower coupling brings benefits, but independently changing services can introduce new problems. JSON serialization makes it easy for contracts to drift or for mistakes to slip in, and API route changes can be difficult to roll out broadly. Good documentation or tools such as OpenAPI/Swagger can help. But those also take time to learn. If we need to learn something anyway, why not choose the option with better performance?

Setting up gRPC

gRPC generates client code and server interfaces from Protocol Buffers. Compiler options determine the target language, so one proto file can generate server and client code for Go, Python, Java, Swift, and more.

We created a buzzapis repository to collect our Protocol Buffers service definitions. It contains the proto files for all Buzzvil microservices, making it easy to see which APIs each service provides.

packages
├── auth
│   ├── auth.proto
│   └── package.json
├── geo
│   ├── geo.proto
│   └── package.json
├── reward
│   ├── reward.proto
│   └── package.json
...

API changes in buzzapis are released automatically. After review through a pull request and a merge into master, continuous delivery builds the changed packages for each supported language. The resulting libraries are published to their respective package managers, such as PyPI and Bintray. It sounds simple, but every step presents issues to solve.

Compiling Protocol Buffers

Install protoc, then run the following command to compile for the target language. Easy, right?

$ protoc -I=packages/auth --go_out=build/go/auth auth.proto

Building only what changed

When did you last touch a Makefile? It’s one of those things many people rarely use after university. Yet as the number of services grows and you need to manage multiple languages and environments, Makefiles can suddenly come to mind. They are very useful for compiling only changed proto files with protoc.

build/go/%.pb.go: %.proto
    protoc -I=$(dir $<) -f $(notdir $<) --go_out=$(dir $@)

Publishing the libraries

Once Protocol Buffers makes services easy to define, setting up and consuming those services should be easy too. As mentioned earlier, we automatically publish the generated libraries to each language’s package manager. That requires us to:

1. Publish only services changed since the previous release.
2. Manage a separate version for each service.
3. Generate the files each language needs for publication, such as setup.py, .gemspec, and build.gradle.

lerna

We use lerna to simplify these tasks. Although designed for npm, it is also useful for managing multiple projects in a monorepo.

lerna manages versions using the package.json file in each directory.

$ npx lerna list -l
lerna notice cli v3.13.3
lerna info versioning independent
auth          v0.1.2 packages/auth
config       v0.1.12 packages/config
...

It can also increment versions only for packages that have changed.

$ npx lerna version patch --no-push --yes

A templating tool such as Gomplate can generate language-specific publication files from package.json. A Makefile can coordinate these tools to build changed packages, generate the required files, and publish libraries to each language’s repository.

Keeping internal libraries private

Internal service definitions, server interfaces, and client libraries shouldn’t be exposed publicly, so we need private package management.

• For Python, we run a private PyPI server. Pip can be directed to it with an external index option. Controlling access to that server lets us manage dependencies without publishing libraries externally.
• For Java, Bintray handles access control for us.
• For Go, we use go mod. Since it uses source repositories such as GitHub as its package manager, we only need to control access to those repositories.

Most other package managers also support private libraries, so they can be configured according to their respective guides.

We still need HTTP/JSON

RESTful APIs remain mainstream. A service that doesn’t offer a JSON API can even feel behind the times. Even if internal services use gRPC, we still want to expose the same functionality externally over HTTP/JSON. Good things are more fun when shared.

Running a separate HTTP server, however, adds maintenance costs. Keeping two implementations synchronized when logic changes is painful. Even with Clean Architecture or a similar approach separating reusable business logic, it still takes work.

Fortunately, we can transcode gRPC to HTTP/JSON! With the appropriate HTTP/JSON declarations in Protocol Buffers, a single gRPC server can support both. Getting HTTP/JSON support without a separate service implementation was another major reason we chose gRPC.

rpc GetShelf(GetShelfRequest) returns (Shelf) {
  option (google.api.http) = { get: "/v1/shelves/{shelf}" };
}
message GetShelfRequest {
  int64 shelf = 1;
}

With this configuration, clients can use either the GetShelf RPC or the /v1/shelves/{shelf} REST API. The server implementation hasn’t changed at all.

There is still no free lunch

Nothing is quite that simple. Smooth development and deployment across this process requires a well-designed CI/CD setup. At Buzzvil, we configured the transcoder with Kubernetes and Istio, and continuous delivery with Helm and Spinnaker. If this post gets enough likes, I’ll explore that setup in more detail next time.

It was worth trying

Change always comes with apprehension. Even so, working in an organization that lets us move through that fear and try something new is deeply rewarding. Adopting gRPC was a good decision that changed more than our protocol: it also influenced our development culture and practices, including an API-first approach. There are still many challenges ahead, but I believe working through them will make our system more reliable, faster, and more adaptable.