Boost.Spirit + Flatbuffers + Catch + Biicode + CLion + Github

by Max Galkin

This weekend I’m test-driving several new technologies. OK, OK, maybe not so new for everyone else, but new for me. I’m trying to write a parser for the Flatbuffers IDL grammar using Boost.Spirit, I’m referencing Boost using Biicode, I’m testing the code using Catch, I’m editing the project in CLion, and I’m pushing the results to Github yacoder/flatbuffers-with-spirit!

This time I’m just going to summarize my impressions, and I’ll dive deeper into details in future posts.

CLion 1.0 RC. I very much like the quality of all products from JetBrains that I used, and I’m excited about CLion. CLion is a cross-platform IDE for C and C++, which understands CMake files directly as “project model”. It was available for free during the “alpha” development stages, but they’ve just shipped their first RC on April 1st (not a joke), so you can only use it for a 30-day trial now.

Unfortunately, my experience with it so far for this particular project wasn’t great. They warn about known perf and memory issues when working with large codebases (“especially using Boost”!). Though my project isn’t large per se, but it uses probably some of the nastiest templates and macros from Boost, which are difficult to parse, I can imagine. And it drives CLion crazy. Not only it shows false positive red squiggles, it also tries to enforce some weird tab indentation in some places, and from time to time freezes for 10-20 seconds, especially when I’m changing some includes or identifier namespaces… Again, CLion is a great direction overall for JetBrains, and I know they’ve improved CLion a lot during the alpha preview period, but for this project VS 2015 CTP experience was better so far.

CLion doesn't approve of Boost.Fusion macros

CLion doesn’t approve of Boost.Fusion macros

Biicode. Biicode is a cross-platform C/C++ dependency manager. If you don’t know what a dependency manager is for, watch talk #5 from my post “5 most interesting talks from Meeting C++ 2014”. C++ open source authors seem to be notoriously eager to implement every piece of functionality from scratch with “zero dependencies”, but, for example, for my project I don’t want to implement my own generic parser, I want to use Boost.Spirit, just without including Boost sources into the repository. Biicode lets me do just that. I’ve verified it works when I clone the repo on another Windows machine, but I expect it to work on OS X and Linux too. You’d only need to install biicode, and run bii init, bii configure, bii build after cloning the repository, maybe with a couple extra parameters (I give examples in cmd files in yacoder/flatbuffers-with-spirit).

In general, to depend on something using Biicode you need to first let Biicode know about it, they call it “publishing a block”. But Boost and some other frameworks are already published, and in most cases all you need to do is to add an include in your cpp file pointing to one of the blocks, and biicode will configure the rest for you, download the source code, adjust CMake files, etc. With Boost (since it it so large), you need to add bii_find_boost() in one of CMake files and specify which boost libs you need, if they aren’t header-only, see blog post here.

Actually, Biicode worked out quite nicely for me. They are still changing some syntax in CMake files and the usage in general, so some of their older documentation pages are slightly outdated, but they quickly follow up on comments, stack overflow discussions, and email questions, so I’ve had it running pretty quickly.

Building with bii

Building with bii

Flatbuffers. I blogged about this library earlier, and that’s when I got the idea to try to write a parser for its grammar using some generic framework.

Boost.Spirit. Boost.Spirit is not “new” really, it’s 15 years old at least. It’s a template library for describing and applying grammars in C++. At the moment, it consists of 3 major parts: Boost.Qi (for writing parsers), Boost.Karma (for generators) and Boost.Lex (for lexers). For now I’ve only written a Boost.Qi grammar which can parse a simple struct in Flatbuffers IDL, but I have to say I spent probably 90% of the time reading Qi manuals and stack overflow and watching Youtube talks:

It’s a 3-hour-long talk, and as an extra hurdle, there is no sound during the third hour of the recording. I guess one just have to gaze in awe. By the way, even searching for talks about Boost.Qi is a challenge in itself, because most of Youtube results for “boost qi” are videos of Chinese people doing gymnastics, rubbing feet to strengthen kidney health, etc. Not particularly relevant.

The thing about Boost.Qi, and just about pretty much any other C++ template library, is that one does not simply include it and start happily writing the code using Intellisense hints about the API. No, you have to stop and read the manual first, and the examples, and all the explanations before you even write anything remotely advanced, because any typo even in a trivial expression produces dozens of pages of incomprehensible template type deduction spew… But then, after this initial investment of time, I think next steps will be easier, overall Boost.Spirit grammar language is quite beautiful, it more or less resembles formal notation for grammars in BNF/PEG. Sort of. I’m still struggling to understand the difference between "operator =" and "operator %=" when applied to grammar rules.

Catch. I blogged about this library earlier too. It’s just a simple single-header test framework for C++. Here I just want to remark that it sort of conflicted with some of biicode rules: when Biicode scans through your source code it tries to find all main() functions, and based on that it names the binaries and makes a list of the targets to build. With Catch the main() function is actually not in your “cpp” file, but it is included from the catch.hpp, and it confused Biicode quite a bit. To restore the harmony of the Universe I had to add two lines in the [mains] section in biicode.conf:

[mains]
    # Manual adjust of files that define an executable
    # !main.cpp  # Do not build executable from this file
    # main2.cpp # Build it (it doesnt have a main() function, but maybe it includes it)
    main.cpp
    !catch.hpp

Github. Y’all know what Github is, am I right? But what you don’t know, or at least what I didn’t know, is what to push to my Git repo from the files produced by Biicode. After consulting with Biicode folks, and looking at some examples, I’ve excluded all the unnecessary stuff via the .gitignore file as follows:

bii/
bin/
build/
cmake/
deps/
.idea/

I’m still not sure how to properly handle some CMake-related files in the repository, e.g. the CMakeLists.txt contains some absolute paths that you will need to adjust in your clone (maybe I should just exclude it and see if Biicode can re-generate it during the configuration step?), but other than that everything else seem required, and it’s really just a couple extra files aside from the code, which is good.

I’m also not 100% sure on whether I should be pushing all the Biicode “blocks” from this project into one repo, or should I have a repo per “block” or what’s the best arrangement there, but my project is not big enough to worry about this just yet.

UPDATE 2015-04-21: this topic continued in “Mini-update on flatbuffers-with-spirit”.