You are here

Performance regression analysis with Gauger

Primary tabs

To help avoid performance regressions, GNUnet uses Gauger. Gauger is a simple logging tool that allows remote hosts to send performance data to a central server, where this data can be analyzed and visualized. Gauger shows graphs of the repository revisions and the performace data recorded for each revision, so sudden performance peaks or drops can be identified and linked to a specific revision number.

In the case of GNUnet, the buildbots log the performance data obtained during the tests after each build. The data can be accesed on GNUnet's Gauger page.

The menu on the left allows to select either the results of just one build bot (under "Hosts") or review the data from all hosts for a given test result (under "Metrics"). In case of very different absolute value of the results, for instance arm vs. amd64 machines, the option "Normalize" on a metric view can help to get an idea about the performance evolution across all hosts.

Using Gauger in GNUnet and having the performance of a module tracked over time is very easy. First of course, the testcase must generate some consistent metric, which makes sense to have logged. Highly volatile or random dependant metrics probably are not ideal candidates for meaningful regression detection.

To start logging any value, just include gauger.h in your testcase code. Then, use the macro GAUGER() to make the buildbots log whatever value is of interest for you to gnunet.org's Gauger server. No setup is necessary as most buildbots have already everything in place and new metrics are created on demand. To delete a metric, you need to contact a member of the GNUnet development team (a file will need to be removed manually from the respective directory).

The code in the test should look like this:

[other includes]
#include <gauger.h>

int
main (int argc, char *argv[])
{
 
  [run test, generate data]
  GAUGER("YOUR_MODULE", "METRIC_NAME", (float)value, "UNIT");
}

Where:

YOUR_MODULE
is a category in the gauger page and should be the name of the module or subsystem like "Core" or "DHT"
METRIC
is the name of the metric being collected and should be concise and descriptive, like "PUT operations in sqlite-datastore".
value
is the value of the metric that is logged for this run.
UNIT
is the unit in which the value is measured, for instance "kb/s" or "kb of RAM/node".

If you wish to use Gauger for your own project, you can grab a copy of the latest stable release or check out Gauger's Subversion repository.