This tutorial assumes that you are familiar with the usage and purpose of the uwsgi fastrouter and you are facing an edge-case (like “Darth Vader wearing a t-shirt with your face”) so you have to use some kind of code-driven “configuration”. The fastrouter documentation page recommends the –fastrouter-use-code-string commandline argument of uwsgi to solve such terribly complicated routing problems by executing your own code/logic for each request to decide which gateway to send it to. The official documentation (at the previous link) shows an example where a python script provides the routing logic and the doc states that you can use any uwsgi-supported language to configure the fastrouter (although my uwsgi-2.0.3 seems to have the code_string feature only in its python and ruby plugins if I’m right...). This tutorial shows you how to write and compile a C++ plugin that contains the routing logic for the fastrouter. This document can also serve a partial/basic C++ plugin tutorial.
To make things a bit more complicated I will do the development of this plugin on windows using a cygwin environment. In case of such a simple plugin this involves only 1-2 extra steps compared to building on linux, I will comment the differences. For production I’m using “original” Debian and Ubuntu distros so my examples work there for sure.
Create a directory for your plugin somewhere in your filesystem. Note that this directory doesn’t have to be inside the extracted uwsgi source directory, put it anywhere. I will refer to this directory as $PLUGIN_DIR in this tutorial. The plugin directory will contain the following things:
Here is the structure and contents of my $PLUGIN_DIR:
$PLUGIN_DIR
my_router_config.cc
uwsgiplugin.py
my_uwsgi.ini
my_uwsgi_lib.ini (needed only on cygwin)
my_router_config.cc: The extension is .cc because uwsgiconfig.py doesn’t recognise the .cpp extension. In case of .cpp ext (and other unhandled extensions) uwsgiconfig.py appends an additional (default) .c extension and tries to compile the source as C (but it fails as it doesn’t find the my_router_config.cc.c source file).
uwsgiplugin.py:
my_uwsgi.ini:
[uwsgi]
inherit = minimal
main_plugin = corerouter, fastrouter
my_uwsgi_lib.ini: (needed only on cygwin)
[uwsgi]
inherit = minimal
main_plugin = corerouter, fastrouter
as_shared_library = true
The my_uwsgi_lib.ini file is needed only on cygwin and it is a copy of my_uwsgi.ini with an extra line appended: as_shared_library = true. You need neither my_uwsgi.ini nor my_uwsgi_lib.ini if you are working with a pre-built new uwsgi binary that supports the –build-plugin commandline parameter but only uwsgi version ~2 and newer have it.
Of course you can skip this step if you are working with a new uwsgi binary. Otherwise download the uwsgi source (uwsgi-2.0.3.tar.gz in my case) and extract it, then enter the extracted source folder.
The “build system” of uwsgi is a python script called uwsgiconfig.py and when you run it your shell’s current directory must be the extracted uwsgi source dir (where the uwsgiconfig.py is located). From now all commands will be executed in this source directory.
It is possible to build uwsgi with different configurations and its plugins can be built as either embedded plugins or external shared objects. Building external plugins for newer uwsgi releases can be done anytime and you need only an uwsgi binary and the compilers, there is no need for the uwsgi sources. (On cygwin you also need a libuwsgi.a lib file that can be built with a trick). On cygwin we first build libuwsgi.a but on linux you simply skip this step. Then we have to build the uwsgi binary (uwsgi on linux, uwsgi.exe on cygwin).
The uwsgiconfig.py script builds uwsgi on multiple threads. For some reason on my cygwin this multithreaded building fails (terminates without any error messages) and I worked this around by setting the CPUCOUNT env var to 1. You may, or may not need this workaround on cygwin... On linux multithreading build works fine. Now let’s build the cygwin specific libuwsgi.a library:
Note that these steps are needed only on cygwin. Now let’s build uwsgi:
The above command produces uwsgi on linux and uwsgi.exe on cygwin. We have used custom ini files to build a minimal uwsgi that serves only as a fastrouter that loads our fastrouter logic plugin. The use of this ini file results in an uwsgi that doesn’t have dependencies on libs like ssl, pcre and it includes only the bare minimum set of uwsgi plugins needed for the fastrouter. From now you don’t need the uwsgi sources (you can even delete them if you want). The only things we have to keep is the uwsgi binary (and libuwsgi.a on cygwin) because building an external uwsgi plugin can be done by running uwsgi with the –build-plugin parameter and the uwsgi binary has an embedded copies of the uwsgiconfig.py and uwsgi.h files needed for a plugin build.
Now if you are lucky you have both the uwsgi binary and the my_router_config_plugin.so plugin in the current directory. Building the plugin by executing the uwsgi binary is very useful because this way it automatically uses the same uwsgiconfig.py and uwsgi.h files and the same CFLAGS that were used to build the uwsgi binary itself. Unfortunately older uwsgi releases don’t have the –build-plugin commandline parameter and in that case you have to build the plugin with the uwsgiconfig.py script:
If you have a newer uwsgi that supports the –build-plugin option then I recommend using that to build your plugin.
I assume that you more or less know about the usage/purpose of uwsgi fastrouter so I only show you how to start and parametrize uwsgi with our newly built plugin:
The above command starts the fastrouter that listens on loopback 9000 for incoming requests and the –fastrouter-use-code-string commandline parameter instructs the fastrouter to ask plugin modifer=251 (our plugin) for the target gateway for each incoming request. I think the –plugin and –my-router-cfg-file commandline arguments speak for themselves...
The extra argument of the –fastrouter-use-code-string is “251::”. This is basically 3 strings separated by two ‘:’ characters but our plugin doesn’t need (ignores) the second and third string so I provided there empty strings. If you take a look at the linked Darth Vader example it solves the problem using the python plugin that actually utilizes these strings: the –fastrouter-use-code-string commandline argument of uwsgi
Note that I’ve chosen 251 as the modifier of my plugin because based on my research modifier 1 has a lot to do with The uwsgi Protocol and moreover if you take a look at the plugins/example or plugins/cplusplus example plugins in the uwsgi source dir then you will see that those are using modifier1=250 and 251 seems to be a free id. Note that I’ve also tried 0 as the modifier1 that is the default modifier1 used by uwsgi and its very first plugin: the python plugin. This seems to work and it seems that this registers our plugin with modifier1=0 by “overriding the python plugin” but I wanted to be polite so I’ve chosen modifier=251.
We started the fastrouter with the “–fastrouter 127.0.0.1:9000 –fastrouter-use-code-string 251::” commandline arguments so it will be listening on loopback port 9000 for incoming requests and it will ask plugin modifier1=251 (our plugin) for the route for each request. I will use nginx to bomb requests on port 9000 of the fastrouter. Here is the location block from my nginx config:
location /test {
include uwsgi_params;
uwsgi_pass 127.0.0.1:9000;
uwsgi_param UWSGI_FASTROUTER_KEY $request_uri;
}
So nginx will route all requests coming to url path /test to the fastrouter by setting UWSGI_FASTROUTER_KEY (basically a “cgi variable”) to a user defined string. UWSGI_FASTROUTER_KEY can be anything, you have put something into it that you can use in your plugin to decide where (which gateway) to send the request. In this case I’ve decided to send the $request_uri to my plugin but you can really put there anything you want. If you don’t specify the UWSGI_FASTROUTER_KEY in the nginx config then the fastrouter will use something else instead of it as the fastrouter key (but I think specifiying the UWSGI_FASTROUTER_KEY is highly recommended), more on that in the Notes section of the fastrouter docs.
With the above fastrouter + nginx config when the fastrouter receives a request from nginx it calls the CodeString() function of our plugin to ask for the gateway address to use for that request.
When the fastrouter calls your CodeString() function the values of the function parameters are the following:
The function must return with a pointer to a string that contains the gateway address, for example: “127.0.0.1:8001”. On that gateway there must be another uwsgi instance listening on an uwsgi protocolled socket. The pointed string must be valid until the next call to the CodeString function. This is usually critical only if you are using extra threads in your plugin because otherwise the fastrouter itself is single threaded async stuff.
We have reached the end of the tutorial. Now you know how to handle in C/C++ a complex routing problem where Darth Vader wears a t-shirt with your face and you have also learnt how to build a C++ plugin using the uwsgi build system.