Steve

Thanks,
Michael

Thanks for the detailed comment. Using EBNF and looping sounds like a good way to simplify expression parsing. I’ve looked at TDOP before, and will look at it again, though this article was intended as a review of the basics.

Finally, you meant “for LL(1)” in the first paragraph of the comment, right ? If not, in what sense do you mean “one token peek-ahead” ?

Your right-recursive calculator grammar will create parse trees that have incorrect operator association, BTW. You can see this trivially in your provided source - the example ‘5 - 3 - 5′ evaluates to the incorrect answer 7. Note that the only point in distinguishing between expr and term is for operator precedence, as both rules simply parse binary operators.

EBNF maps closer to hand-written LL grammars than does plain BNF, because loops are easily written in LL, and are thus best represented explicitly rather than by recursion. So, for example, your expr production would look better like this:

expr ::= term { “+” term | “-” term } ;

The natural way to write a parser for this production will result in correct operator associativity.

(Angle-bracketed text is generally reserved in grammars for terminals produced by the lexer; I am aware that you are including the lexical structure in the grammar too, though using regular grammar rather than the context-free BNF grammar.)

Finally, let me point out that unmodified recursive descent is not an efficient way to parse expressions, particularly for languages with many precedence levels. There are simple precedence parser idioms which integrate well with recursive descent, see e.g.:

http://javascript.crockford.com/tdop/tdop.html

The Delphi compiler, on which I work, uses a similar mechanism to the one Doug describes for its expression parser. Similarly, Google Chrome’s JavaScript engine, V8, also uses a precedence parser for expressions. (It also uses hand-written recursive descent.)