You are here

libgnurl

Primary tabs

libgnurl is a fork of libcurl, which is mostly for GNUnet but it might be usable for others, hence we're releasing the code on this website to the general public. Please read the README for instructions, as you must supply the correct options to configure to get a proper build of libgnurl. In addition to the source as a TAR, we also offer the changes we made against libcurl's Git repository to create libgnurl. In the following, I will explain the motiviations behind this fork.

Motivation

cURL supports a bunch of crypto backends. GNUnet requires the use of GnuTLS, but other variants are used by some distributions. Supporting other crypto backends would again expose us to a wider array of security issues, may create licensing issues and most importantly introduce new bugs as some crypto backends are known to introduce subtle runtime issues. While it is possible to have two versions of libcurl installed on the same system, this is error-prone, especially as if we are linked against the wrong version, the bugs that arise might be rather subtle.

For GNUnet, we also need a particularly modern version of GnuTLS. Thus, it would anyway be necessary to recompile cURL for GNUnet. But what happens if one links cURL against this version of GnuTLS? Well, first one would install GnuTLS by hand in the system. Then, we build cURL. cURL will build against it just fine, but the linker will eventually complain bitterly. The reason is that cURL also links against a bunch of other system libraries (gssapi, ldap, ssh2, rtmp, krb5, sasl2, see discussion on obscure protocols above), which --- as they are part of the distribution --- were linked against an older version of GnuTLS. As a result, the same binary would be linked against two different versions of GnuTLS. That is typically a recipe for disaster. Thus, in order to avoid updating a dozen system libraries (and having two versions of those installed), it is necessary to disable all of those cURL features that GNUnet does not use, and there are many of those. For GNUnet, the more obscure protocols supported by cURL are close to dead code --- mostly harmless, but not useful. However, as some application may use one of those features, distributions are typically forced to enable all of those features, and thus including security issues that might arise from that code.

So to use a modern version of GnuTLS, a sane approach is to disable all of the "optional" features of cURL that drag in system libraries that link against the older GnuTLS. That works, except that one should then NEVER install that version of libcurl in say /usr or /usr/local, as that may break other parts of the system that might depend on these features that we just disabled. Libtool versioning doesn't help here, as it is not intended to deal with libraries that have optional features. Naturally, installing cURL somewhere else is also problematic, as we now need to be really careful that the linker will link GNUnet against the right version. Note that none of this can really be trivially fixed by the cURL developers.

Rename to Fix

At this point, developers that don't want to rebuild an entire distribution from scratch get grumpy. Grumpy developers do silly things, like forking code to fix it. I called the fork gnurl (to be pronounced with a grumpy voice and an emphasis on the R) as it is bits of cURL, a bit more GNUish, for GnuNet, and gnurl can be pronounced to indicate the grumpy origins.

How does forking fix it? Easy. First, we can get rid of all of the compatibility issues --- if you use libgnurl, you state that you don't need anything but HTTP/HTTPS. Those applications that need more, should stick with the original cURL. Those that do not, can choose to move to something simpler. As the library gets a new name, we do not have to worry about tons of packages breaking as soon as one rebuilds it. So renaming itself and saying that "libgnurl = libcurl with only HTTP/HTTPS support and GnuTLS" fixes 99% of the problems that darkened my mood. Note that this pretty much CANNOT be done without a fork, as renaming is an essential part of the fix. Now, there might be creative solutions to achieve the same thing within the standard cURL build system, but I'm not happy to wait for a decade for Daniel to review the patches. The changes libgnurl makes to curl are miniscule and can easily be applied again and again whenever libcurl makes a new release.

Summary

I want to note that the main motiviations for this fork are technical The goal of the cURL project is clearly to support many crypto backends and many protocols. That is a worthy goal, and I wish them luck with it. The goal for libgnurl is to support only HTTP and HTTPS (and only HTTP 1.x) with a single crypto backend (GnuTLS) to ensure a small footprint and uniform experience for developers regardless of how libcurl was compiled.

Using libgnurl

Projects that use cURL only for HTTP/HTTPS and that would work with GnuTLS should be able to switch to libgnurl by changing "-lcurl" to "-lgnurl". That's it. No changes to the source code should be required. Continue to read the cURL documentation --- as libgnurl strives for bug-for-bug compatibility with the HTTP/HTTPS/GnuTLS subset of cURL. However, we're happy to add new features relating to this core subset and might be easier to convince than the cURL developers. ;-)

Git repository

You can get the Gnurl Git repository using
git clone git://git.taler.net/gnurl.

AttachmentSize
Patch for gnurl-7.40.0239.51 KB
gnurl-7.40.0.tar.bz23.13 MB
gnurl-7.40.0.tar.bz2.sig543 bytes

Comments

I just noticed that Daniel decided to respond to this here.

He says a bunch of things with respect to my perceived cURL's usage in the wild that contradict what I said above, and he's likely right. None of this really matters for gnurl, other than maybe there are fewer applications for which gnurl will suffice instead of curl, but it doesn't fundamentally matter if gnurl is useful for just a few packages or ten million.

However, Daniel is wrong in one major point:

Is renaming the produced library really that hard to do without forking the project? If I want to produce a renamed output from an open source project out there, I apply a script or hack the makefile of that project and I keep that script or diff in my end. No fork needed. I think I must’ve misunderstood some subtle angle of this…

Well, applying that script or hack to the makefile is exactly what the "gnurl-diffs.tgz" contains, and I have no plans for gnurl to really do anything else. gnurl is a micro-fork, not something where anyone is expected to do some major development. Now, others will need to have an easy way to compile with this change, and so keeping that script or diff "in my end" is hardly the right spirit here; in the end, we'll need binary packages build from this, and hence having a .tar is even easier than first grabbing the cURL source, applying a patch and then building that. Overall, I actually hope that the diff is going to get smaller over time.

I should also mention that Daniel's idea of just creating a build of a special version of cURL to a particular directory (other than /usr) works in theory, but is terribly error-prone in practice. I also don't see Debian shipping non-conflicting (!) packages for libcurl build against different SSL libraries anytime soon; let me know if that does happen, because that's clearly what Daniel is proposing here.

With respect to the SSL incompatibilities, the GNU libmicrohttpd testcases ("make check") run fine with cURL linked against GnuTLS and hang with OpenSSL. Now, I also don't think there is an easy fix, so there was hardly a point to pursue the matter further.

Finally, as Daniel writes:

Why would he need to send me such patches in the first place? Why would I have to review the patches? Why would we merge them?

which just confirms my point: the gnurl patches are not something he'd want to receive, look at or merge.