The Waffleshop

This is a follow-up to my previous posting about using Flex and Bison together in C++ mode. This new example actually does something useful: parses an INI file and gives easy access to the values. Just like the last example, this is not a tutorial. Download the source code and give it a read – I tried to make it as straightforward as possible.

Download the source code.

This example uses the parser and the scanner to extract data from a stream into data structures. Once the stream has been parsed, there is no more need for the parser / scanner. Instead of having them as member variables of the class, they have been moved to local variables in the function that parses the stream so that we aren’t wasting memory keeping those class instances in memory when they no longer serve a purpose.

This example also keeps track of locations using the yylloc variable so we can tell where an error occurs. If an error occurs, an exception (of type std::string) is thrown that says what the error is and where it occured (line number, character number).

The Makefile for this example is greatly improved; builds are now incremental (files that have not changed are not recompiled).

I am NOT an expert on Bison, Flex, C++, Makefiles, etc. and have coded this as a learning experience. It is provided in the hopes that others will be able to learn from it. If you notice anything that can be improved on, please leave a comment so that other visitors as well as myself can benefit. This code is provided under the WTFPL license with no warranty.

Flex and Bison are used together to create parsers, usually for programming-related tasks like parsing source files or SQL statements. While not as easy on the programmer as newer libraries like Spirit OR ANTLR, they are more efficient since they generate raw C code that can be compiled into your application. They are also capable of outputting C++ code, but there is a lack of clear documentation / examples available demonstrating this.

What I present here are my efforts over the last couple of days towards creating a program that uses the two together in C++ mode. The result is trivial – numbers are recognized and divided by five – since the focus of this program is to demonstrate using Flex and Bison. If you do not know Flex and Bison, I recommend Lex & Yacc by Tom Niemann (Flex is a clone of Lex, Bison is a clone of Yacc).

Download the example source code here.

This is an example, not a tutorial – I’m not going to take you through the code line by line. I do have some notes for you, though:

• waffleshop.y
  • The require line tells Bison that this is meant for Bison 2.4.1. While it might work on other versions of Bison, the C++ code generated is ‘experimental’ and subject to change between versions.
  • The skeleton line specifies an alternate template to use – lalr1.cc is the C++ version.
  • The parse-param options tell Bison that we want the class to have an additional member variable – the scanner. Since Bison calls the scanner to get a token, the Bison class needs to have a reference to the scanner. The lex-param option tells Bison that when it calls yylex, it should pass the scanner as an additional argument. Our implementation of yylex invokes the passed scanner and returns the result.
  • There are two sections of code: the first is ‘%code requires’. The code inside of this block is put in the generated header file as well as the c file. The other %code block contains code that only goes in the c file, and since we don’t want anything else calling the yylex global function we make it static to the file.
• WaffleshopParser.h
  • This is a convenience class that bundles the parser and the scanner together; it is good Object-Oriented design to do this, and is recommended in the Bison documentation.
• WaffleshopScanner.h
  • FlexLexer.h is provided by Flex, and defines the base class that generated Flex scanner classes inherit from. The preprocessor directive surrounding it is necessary because FlexLexer.h is a mess (it says so right in the Flex generated code on line 16).
  • The yylex function has to be overloaded because Bison passes a pointer to the yylval variable. It would be nice if we could just specify this in YY_DECL, but the base Flex class has yylval with no arguments defined as pure virtual, forcing us to implement it anyway; I just went ahead and used it.

I used the following resources to construct this example:

• Flex Documentation
• Bison Documentation
• Timo Bingmann’s example

Please post any suggestions or questions – I am no expert on either Flex or Bison and am always interested in improving my code!

Update: After you have gleaned all you can from this example, check out this followup about creating an INI file parser using Flex and Bison in C++.