Java Tree Builder (JTB) 1.4.11

Last update : apr xxth, 2016 - version 1.4.11

Overview

JTB (Java Tree Builder) is a syntax tree builder and visitors generator to be used in front of JavaCC (Java Compiler Compiler).
It takes a JavaCC grammar file as input (you just have to change the extension from ".jj" to ".jtb" and add a few options) and it automatically generates the following:

a set of syntax tree classes based on the productions in the grammar, utilizing the Visitor design pattern;
four interfaces: IVoidVisitor, IVoidArguVisitor, IRetVisitor, IRetArguVisitor;
four depth-first visitors: DepthFirstVoidVisitor, DepthFirstVoidArguVisitor, DepthFirstRetVisitor, DepthFirstRetArguVisitor, whose default methods simply visit the children of the current node;
a JavaCC grammar ".jj" file (jtb.out.jj by default), with the proper annotations to build the syntax tree during parsing (which then must be compiled with JavaCC).

New visitors, which subclass any generated one, can then override the default methods and perform various operations on and manipulate the generated syntax tree.

JTB can be compared to JJTree in the following way:

in JJTree you explicitly define the nodes you want to build; in JTB it builds by default all of them (but you can tell not to build some of them),
JJTree generates only interfaces for visitors and you have to code the visitors, so you "fill" them; JTB generates interfaces and "full" visitors (i.e. with all the code to visit the tree), so you just "modify" them, sometimes just "reducing" them (for not visiting branches), so in a sense it is faster to develop on an already generated - bullet proof - code,
in JJTree, you are tempted to tune the tree, i.e. define only the nodes you need, but if you want to develop different visitors, you have to think about all the nodes you want to traverse for the different purposes and so you may find yourself sometimes having to enrich the grammar and defeat what you have previously done; in JTB you have all the tree right from the beginning so adding a new visitor is not disruptive,
JTB can "inline" the accept methods, i.e. it generates for a node the code to work directly on its children, which is usually the right place to take decisions and to reduce the control logic (otherwise you usually have to access the fathers and check their types),
JJTree may be more adequate when only small trees and few visitors are needed; JTB may be more adequate when you have to work on the entire tree and build many visitors.

History

04/2016 - 1.4.11 : fixed (added) spaces between the '<' and '>' and the token labels in the generated javadoc comments; fixed the conditions on displaying the "warning: Empty choice : a NodeChoice with a 'null' choice member ..."; fixed column numbers in messages

11/2015 - 1.4.10 : added JTB.do_main() returning an int return code

08/2015 - 1.4.9 : same as 1.4.8.fix_a

01/2015 - 1.4.8.fix_a : fixed regression in jtb.out.jj generation

10/2012-12/2014 - 1.4.8 : added generation of JavaCodeProductions ('%' syntax); fixed bug #3164860; fixed some messages; fixed some problems (null choice for an empty choice, prefix / suffix management in lookahead); checked duplicated Javacode production names; fixed the Java grammar (in ExplicitConstructorInvocation & ClassOrInterfaceBodyDeclaration); added annotations in the generated classes; fixed some cases and improved printing specials (comments); improved short names generation for generated intermediate variables; modified the warning message on ignored LOOKAHEAD (as it can actually be not ignored); fixed java blocs generation; some internals improvements

09/2012 - 1.4.7 : added sub comments in the generated visitors java code; did some optimizations; fixed bugs JTB-1 & JTB-2; fixed some generation errors; added a check when 'n = prod()' and 'void prod()'; added the '!' syntax for not generating a node creation (in a BNFProduction, RegExprSpec and ExpansionUnit) ; fixed some Java grammar issues and did some refactoring; added column numbers in messages; performed tests on more limit cases; fixed missing in JavaCCInput() and missing parentheses in Annotation()

01/2011 - 1.4.6 : JTB hosted at http://java.net/projects/jtb; new contributor (Francis Andre Kampbell); added -va and -npfx and -nsfx options

12/2010 - 1.4.5 : Convert nodes and visitors output directories to absolute paths

7/2010 - 1.4.4 : Fixed missing space after throws issue in visiting MethodDeclaration & ConstructorDeclaration of JavaPrinter; Fixed index issue in visiting TypeArguments of JavaPrinter

04/2010 - 1.4.3.2 : bug #2991455

04/2010 - 1.4.3.1 : bugs #2990962 and #2989497

03/2010 - 1.4.3 : Fixed wrong conditionals generation and TreeFormatter generation issues; added node declarations initialization

02/2010 - 1.4.2 : Fixed missing java.util package in generated JTBToolkit class, fixed accept inlining issue in visiting ExpansionChoices

02/2010 - 1.4.1 : Fixed wrong conditionals generation and missing static modifier for generated return types issues (bug 2945965), fixed hard error in semantic checker, fixed command line options overwrite issue

11/2009 - 1.4.0 : New version delivered / integrated under the JavaCC Eclipse Plug-in

xx/2005 - 1.3.2 : Last (known) version from UCLA Compilers Group : see JTB 1.3.2 home page

New Features in version 1.4.0+ / 1.4.7

Here are the following enhancements that have been added to JTB 1.3.2:

Internal code

Internal Java code extensively refactored (performances, namings, JDK 1.5 syntax, numerous fixes, ...)
Internal Java code Javadoc fully revised
Internal parser upgraded to version 4.2 of JavaCC, with performance enhancements
Added Ant build files

Generated code

Renamed syntax tree and visitors interfaces and classes
Generated Java code extensively refactored (performances, namings, JDK 1.5 syntax, numerous fixes, ...)
Generated Java code Javadoc fully revised, specially with more detailed fields comments
Special tokens fully printed in the output file (with the -tk option)

User enhancements

Removed "only void BNFProduction result type" limitation
Removed "only non void JavaCodeProduction" limitation
Added proper handling of "PrimaryExpression() = Identifier() ... | RegularExpression() ..." syntax
Added a -ia option for inlining visitor accept methods on base nodes
Changed behavior of option -jd: now turns on printing classes, methods and fields javadoc comments
Added a -dl option for depth level management generation
Added -d, -nd & -vd options, modified -p, -pd & -vp options (to separate directories from packages)
Added input file options "JTB_XX" (in the "options" section of the ".jtb grammar file) corresponding to the command line options "-xx"
i.e: JTB_O = "myout.jj" <-> -o myout.jj, JTB_TK = true <-> -tk
(1.4.7) Added ability to not generate a (JTB) node creation, with the '!' special character:
- either in the BNFProduction declaration : e.g. : void MyProd() ! : { ... }
- or in the RegExprSpec declaration : e.g. : TOKEN : { < MyToken : "foo" > ! ... }
- or in ExpansionUnits where a BNFProduction or a RegExprSpec is used : e.g. : void MyProd() : { ... MyProd() ! ... < MyToken> ! ...}

How to use

Environment:

JTB 1.4.7 is meant to be used as a front end for JavaCC 5.0, with a 1.6.0 JDK, and so is command line by nature.
But it has been integrated within the JavaCC Eclipse Plugin , which embeds the latest JTB version each time is is released, and avoids running the tool on the command line.

On the command line:

java -jar jtb-1.4.x.jar -h : to view all available options
java -jar jtb-1.4.x.jar [CL_Option] -si : to use system.in as the input file
java -jar jtb-1.4.x.jar [CL_Option] [input-file] : to use input-file as the input file

In the input file options section:

options {(JTB_BOOL_OPT=(true|false); | JTB_STR_OPT="str";)*}

Options are the following:

CL_Option JTB_Option Description

-cl JTB_CL=(true|false) Print a list of the classes generated to standard out

-d dir JTB_D="dir" Short for (and overwrites) "-nd dir/syntaxtree -vd dir/visitor"

-dl JTB_DL=(true|false) Generate depth level info

-e JTB_E=(true|false) Suppress JTB semantic error checking

-f JTB_F=(true|false) Use descriptive node class field names

-h N/A Display this help message and quit

-ia JTB_IA=(true|false) Inline visitors accept methods on base classes

-jd JTB_JD=(true|false) Generate JavaDoc-friendly comments in the nodes and visitor

-nd dir JTB_ND="dir" Use dir as the package for the syntax tree nodes

-np pkg JTB_NP="pkg" Use pkg as the package for the syntax tree nodes

-ns class JTB_NS="class" Use class as the class which all node classes will extend

-o file JTB_o="file" Use file as the filename for the annotated output grammar

-p pkg JTB_P="pkg" Short for (and overwrites) "-np pkg.syntaxtree -vp pkg.visitor"

-pp JTB_PP=(true|false) Generate parent pointers in all node classes

-printer JTB_PRINTER Generate a syntax tree dumping visitor

-si N/A Read from standard input rather than a file

-scheme JTB_SCHEME Generate Scheme records representing the grammar and a Scheme tree building visitor

-tk JTB_TK=(true|false) Generate special tokens into the tree

-vd dir JTB_VD="dir" Use dir as the package for the default visitor classes

-vp pkg JTB_VP="pkg" Use pkg as the package for the default visitor classes

-w JTB_W=(true|false ) Do not overwrite existing files

Support

Feel free to report any bug or suggest enhancement at the JavaCC Eclipse Plug-in Sourceforge site.

Download

JTB download page.

License

JTB 1.3.2 is licensed under the BSD license.
JTB 1.4.x adds the GPL license and the CECILL license.

CL_Option	JTB_Option	Description
-cl	JTB_CL=(true\|false)	Print a list of the classes generated to standard out
-d dir	JTB_D="dir"	Short for (and overwrites) "-nd dir/syntaxtree -vd dir/visitor"
-dl	JTB_DL=(true\|false)	Generate depth level info
-e	JTB_E=(true\|false)	Suppress JTB semantic error checking
-f	JTB_F=(true\|false)	Use descriptive node class field names
-h	N/A	Display this help message and quit
-ia	JTB_IA=(true\|false)	Inline visitors accept methods on base classes
-jd	JTB_JD=(true\|false)	Generate JavaDoc-friendly comments in the nodes and visitor
-nd dir	JTB_ND="dir"	Use dir as the package for the syntax tree nodes
-np pkg	JTB_NP="pkg"	Use pkg as the package for the syntax tree nodes
-ns class	JTB_NS="class"	Use class as the class which all node classes will extend
-o file	JTB_o="file"	Use file as the filename for the annotated output grammar
-p pkg	JTB_P="pkg"	Short for (and overwrites) "-np pkg.syntaxtree -vp pkg.visitor"
-pp	JTB_PP=(true\|false)	Generate parent pointers in all node classes
-printer	JTB_PRINTER	Generate a syntax tree dumping visitor
-si	N/A	Read from standard input rather than a file
-scheme	JTB_SCHEME	Generate Scheme records representing the grammar and a Scheme tree building visitor
-tk	JTB_TK=(true\|false)	Generate special tokens into the tree
-vd dir	JTB_VD="dir"	Use dir as the package for the default visitor classes
-vp pkg	JTB_VP="pkg"	Use pkg as the package for the default visitor classes
-w	JTB_W=(true\|false )	Do not overwrite existing files