Home > C++, Networking, NymphRPC, Protocols, RPC > NymphRPC: my take on remote procedure call libraries

NymphRPC: my take on remote procedure call libraries

Recently I open-sourced NymphRPC [1], which is a Remote Procedure Call (RPC) library I have been working on for the past months. In this article I would like to explain exactly why I felt it to be necessary to unleash yet another RPC library onto the world.

The past years I have had to deal quite a lot with a particular RPC library (Etch [2][3]) due to a commercial project. This is now a defunct project, but it spent some time languishing as an Apache project after Cisco open-sourced it in 2011 and got picked up by BMW as an internal protocol for their infotainment systems [4].

During the course of this aforementioned commercial project it quickly became clear to me that the Etch C library which I was using had lots of issues, including stability and general usability issues (like the calling of abort() without recovery option when any internal assert failed). As the project progressed, I found myself faced with the choice to either debug this existing library, or reimplement it.

At this point the C-based library was around 45,000 lines of code (LoC), with countless concurrency-related and other issues which made Valgrind spit out very long log files, and which proved to be virtually impossible to diagnose and fix. Many attempts resulted in the runtime becoming more unstable in other places.

Thus it was that I made the decision to reimplement the Etch protocol from scratch in C++. Even though there was already an official C++ runtime, it was still in Beta and it too suffered from stability issues. After dismissing it as an option, this led me to the next problem: the undocumented Etch protocol. Beyond a high-level overview of runtime concepts, there was virtually no documentation for Etch or its protocol.


Fast-forward a few months and I had reverse-engineered the Etch protocol using countless Wireshark traces and implemented the protocol in a light-weight C++-based runtime of around 2,000 LoC. Foregoing the ‘official’ runtime architecture, I had elected to model a basic message serialisation/deserialisation flow architecture instead. Another big change was the foregoing of any domain specific language (DSL) as with Etch to define the available methods.

The latter choice was primarily to avoid the complexity that comes with having a DSL and compiler architecture which has to generate functioning code that then has to be compiled into the project in question. In the case of a medium-sized Etch-based project, this auto-generated code ended up adding another 15,000 LoC to the project. With my runtime functions were defined in code and added to the runtime on start-up.

In the end this new runtime I wrote performed much better (faster, lower RAM usage) than the original runtime, but it left me wondering whether there was a better RPC library out there. Projects I looked at included Apache Thrift [5] and Google Protocol Buffers [6].

Both sadly also are quite similar to Etch in that they follow the same DSL (ISL) and auto-generated code for clients/servers path. Using them is still fairly involved and cumbersome. Probably rpclib [7] comes closest, but it’s still very new and has made a lot of design choices which do not appeal to me, including the lack of any type of parameter validation for methods being called.


Design choices I made in NymphRPC include such things as an extremely compact binary protocol (about 2x more compact than the Etch protocol) while allowing for a wide range of types. I also added dynamic callbacks (settable by the client). To save one the trouble of defining each RPC method in both the client and server, instead the client downloads the server’s API upon connecting to it.

At this point NymphRPC is being used for my file revision control project, FLR [8], as the communication fabric between clients and server.

Performance and future

Even though the network code in NymphRPC is pretty robust and essentially the same as what currently runs on thousands of customer systems around the world – as a result of this project that originally inspired the development of NymphRPC – it is still very much under development.

The primary focus during the development of NymphRPC has been on features and stability. The next steps will be to expand upon those features, especially more robust validation and ease of exception handling, and to optimise the existing code.

The coming time I’ll be benchmarking [9] NymphRPC to see how it compares other libraries and optimise any bottlenecks that show up. Until then I welcome anyone who wishes it to play around with NymphRPC (and FLR) and provide feedback 🙂


[1] https://github.com/MayaPosch/NymphRPC
[2] https://etch.apache.org/
[3] https://en.wikipedia.org/wiki/Etch_%28protocol%29
[4] http://www.bmw-carit.de/open-source/etch.php
[5] https://en.wikipedia.org/wiki/Apache_Thrift
[6] https://en.wikipedia.org/wiki/Protocol_Buffers
[7] https://github.com/rpclib/rpclib
[8] https://github.com/MayaPosch/FLR
[9] http://szelei.me/rpc-benchmark-part1/

Categories: C++, Networking, NymphRPC, Protocols, RPC Tags: , ,
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: