Jemos Clanker is a Java source analysis framework for the auto-generation of Java code, starting from the analysis of Javadoc tags and/or Annotations. Its goals are similar to XDoclet but the similarities end there. The reason why I wrote Clanker is that I couldn't find anywhere a Java product which would make easy for me the simple programmatic management of Javadoc tags and Annotations. Have you ever tried to define your own tags with XDoclet? I tried with EJB 2.1 for the auto-generation of Business Delegates, and I was descouraged by the learning curve before me. In February 2006 I participated to JavaUK06 where I was introduced to EE5 and the fantastic world of the Java Persistence Framework. I then decided to try and use XDoclet to auto-generate the XML file needed when deploying the persistence framework in a J2SE environment. After about 5 minutes I realized that I was wasting my time. I also tried to look on the net for a tool which would allow a programmatic approach to Annotation and Javadoc processing and I found the Annotation Processing Tool (APT). Again, after few minutes I felt descouraged, so I decided to write my own framework with these simple goals in mind:

  • Ease of programmatic use
  • Easy access/management of Javadoc tags
  • Easy access/management of Java Annotations
  • The information extracted from Java sources should have been available in a script-friendly environment (like Jelly)
In few words I wanted something that I could configure programmatically and that could easily answer to needs like:
  • I want to be able to have access in a JavaBean-like form to all classes which have got the following Javadoc tag(s)
  • I want to be able to have access in a JavaBean-like form to all classes which have got the following Annotation(s)
  • I want to be able to use all the above information in a script-like environment (like from within an XML file, a Jelly script, etc.)

Why the name Clanker?

When I decided to write Clanker I was reading a series of books from Ian Irvine, which you can find by clicking here. Most of the names used by Clanker have been taken from those books (with the author's permission), especially the Well of Echoes series. The only thing the author asked was that Clanker had to be used as open-source product and that if used commercially then explicit permission should have been requested to the publisher. A bit of terminology from these books: a Clanker is a war machine that humans built in order to fight the alien 'Lyrinx', ex-humans that thanks to the use of the 'Secret Art' (call it magic) went to live in a place amongst the three worlds amongst which this saga is ambiented. Two hundreds years before the tale of the 'Well of Echoes' begins (i.e. the tale written in the previous series 'The View from the Mirror'), the main character of that tale opened the way between the worlds (a 'gate') and the Lyrinx came on the Earth in order to start a new life. Humans, frightened by those beasts, started a war against them and this is what is written in the 'Well of Echoes' series. It was so much fun reading these books that I decided to use some of the names contained in them for some of the objects in Clanker. So the analysis processors are called 'Lyrinx', filters are part of a 'Crystal' (a powerful object in the above tale), the main analysis object is called a 'Thapter' (a machine which represents possibly the only chance that humans have to win the war). I hope I haven't messed things too much :)

High level overview

Back to the top

Jemos Clanker is currently composed by two main modules:

  • Analysis engine. This module is responsible for performing the source analysis and to fill in a bean-like object with results.
  • Result engine. The result engine defines a library of Jelly tags than can be used from within Jelly scripts in order to auto-generate code.
There are two kind of users who can benefit from Jemos Clanker: those who want to use only XML-like syntax to auto-generate code, and those who want to use it programmatically. The two ways (XML-like/Programmatic), although different in the form, are identical when comes down to what really happens behind the scenes, since the XML-like structure is just a user-friendly window towards the programmatic approach.

The steps to use Clanker are briefly listed below and these are the same both for XML-based or programmatic approach:

  • Users create a configuration object
  • Users initialize a Controller with the above configuration
  • Users create a Thapter object with the controller created above
  • Users launch the analysis process
  • Users use the results either programmatically or from within a script framework which works well with Java-bean structures
Since I wanted to create a framework, each major component in Jemos Clanker is defined by an interface for which there is a default implementation, but nothing avoids you from creating your implementation. The different method contracts all use interface types, so customization shouldn't be an issue.

Analysis Engine: main components

Back to the top

At high level, there are few major components, shown in the picture below and about which follows a description.

  • Thapter. This is the engine of the analysis process. Its responsibilities are to coordinate the analysis activities and to maintain an object (the Amplimet, see below), which contains the analysis results. A Thapter needs to be initialized with a Controller which in turn needs to be created with a Configuration object. Referring to the 'Well of Echoes' series, from which the Thapter takes the name, a Thapter is an alien construct that the main character of this series made fly and then ever since became an important anspect in winning the war for humans.
  • Configuration. A configuration object is created as first step when using Jemos Clanker. It can be initialized with different parameters in order to process only determined Javadoc tags and Annotations.
  • Controller. A controller in the book series is a special artifact that artisans can manufactur and which can move things thanks to the power of crystals, by using the 'aura' manifested by those crystals. In this framework as in the book, a controller is used to give 'life' to a Thatper (and Clankers for that matter) and in order to be of any use, it must be created using a configuration object.
  • Filter. Filters are used by the analysis engine to filter information on the Java sources being analyzed. Currently only three types of filters are supported which allow you to select classes with specific Javadoc tags and/or to select only classes with specific Annotations.
  • Crystal. A crystal is an object containing all possible filters that Jemos Clanker uses in order to 'filter' analysis results. The approach to set up filters is to create the desired filters first, to set those filters in a Crystal object and to set the Crystal object in a Configuration object.
  • Component. A component represents a logical part in a Java source program. The logical parts currently supported are: Packages, Classes, Methods, Instance Variables, Local variables, Annotations, Javadoc tags. When the analysis engine runs, it fills one or more of these components with the analysis result. The components are related in a parent-child structure, so that Package and Classes can have children but local variables can't. The final artifact of the analysis process is an Amplimet, a JavaBean containing the root of the hierarchy (the list of Packages) from which is possible to retrieve all the children, in a recursive way. This makes very easy processing the results in a script-based framework also for the non most technical user.
  • Amplimet. The Amplimet is the final artifact of the analysis engine. In the book series, an Amplimet is an extremely precious crystal, which has been in a mine for thousands of years waiting for someone to discover it and which, apparently, contains its own will. Similarly, in this framework an Ampliment is the most precious object available at the end of the process, since it's possible to retrieve the list of analyzed packages from it and then, from each package is possible to retrieve the list of classes and so on. The Amplimet maintains a registry of Components and offers methods to manage this registry.

Analysis engine: starting the analysis process

Back to the top

Here follows a brief overview of the activities necessary to start the analysis process. In the case of filters used, users can specify whether they want the analysis engine to return only the classes matching those filters, or all the classes containing at least one of the filter matches.

Result engine: main components

Back to the top

The result engine is the second component of Jemos Clanker. Its role is to offer users functionalities for auto-generation of Java code. This goal is achieved through two strategies:

  • To offer users XML tags to trigger the analysis process.
  • To offer users an easy framework to write formatters.
The result engine is composed mainly by two sets of functionalities:

  • Java classes which offer the backend for XML Tag processing, using the Jelly framework
  • A set of standard formatters for auto-generation of Java classes, such as interfaces, EJB3 xml configuration file for entity beans deployed on a J2SE environment, etc. (currently only interfaces are supported, but other formatters are being developed).
The cool thing about the result engine, is that you can extend it to create your own tags and templates. The result engine offers a set of core tags to launch the analysis process (HeartBitTag), to provide file selection functionalities (FileSetTag, IncludeTag), and to provide Filter, Annotations, and Javadoc functionalities (FilterTag, AnnotationTag, JavadocTag). Here follows an high level overview of the result engine tags' structure:

The tags structure is as follows:

  • HeartBitTag. This is the top level tag used to configure and lauch the analysis engine.
  • FilterTag. Child of HeartBitTag. This tag can be used to setup a filter on the java sources being analyzed. It is possible to ask the analysis engine to return only some Javadoc tags and/or annotations and/or only the classes containing some Javadoc tags and/or annotations. The filters on Javadoc and/or Annotations can be specified at global, class, method or instance variable level.
  • FileSetTag. Child of HeartBitTag. This tag allows users to specify, with a syntax very much like the one Ant uses, the list of files the analysis engine will need to analyze.
  • AnnotationTag. Child of FilterTag. This tag allows users to specify annotation filters at global, class, method and instance variable level.
  • JavadocTag. Child of FilterTag. This tag allows users to specify Javadoc filters at global, class, method and instance variable level.
  • IncludeTag. Child of FilesetTag. This tag allows users to specify which files to include in the analysis process, very much like the Ant include tag does, allowing for pattern-like definitions (i.e. '**', '**/*', '**/*java', etc)

The list of tags shall be declared in the jelly-config.xml file under the Jemos Clanker root installation folder. The list of tags defined in this file is automatically processed by the class, which loads and registers them with the Jelly engine.

Here follows an example of the jelly-config.xml file with the declaration of the core tags:

<?xml version="1.0" encoding="ISO-8859-1"?>

  <!--  Tags -->
  <tag workName="heartbit" class="" />
  <tag workName="fileset" class="" />
  <tag workName="include" class="" />
  <tag workName="filter" class="" />
  <tag workName="docTag" class="" />
  <tag workName="annTag" class="" />
  <!--  Formatters -->
  <tag workName="interfaces" 
       class="" />


Generating plain Java interfaces

Back to the top

From the user perspective, using the power of Jemos Clanker is as easy as writing a normal maven.xml goal. Let's have a look at an example where a user wants Jemos Clanker to auto-generate Java interfaces for a set of classes containing the @clanker1 and the @interface-extract-me Javadoc tags. This example is contained in the Jemos Clanker Test Project, which can be downloaded from the download area. The classes analyzed by the analysis engine are part of the Jemos Clanker distribution and are located under %CLANKER_HOME%/analysis-engine/src/test/uk/co/jemos/clanker/test/dummyclasses/classes. In a real-scenario, obviously users would ask the analysis engine to analyze Java classes located somewhere else. The cool thing about Jemos Clanker is that in order to add a new Javadoc to be processed, you don't have to write any new class (as opposed to XDoclet) but you just have to specify your new Javadoc tag in the XML file which starts the analysis process. Let's say that you want the analysis process to generate Java interfaces for all classes containing a @user-defined-interface Javadoc tag. All you'll have to do is to declare this Javadoc tag in all the classes for which you want Jemos Clanker to auto-generate a Java interface, and then change the xml file which triggers the analysis engine (in our example the maven.xml file below) indicating to use this tag. Let's go a bit further...Let's say that you want to create a new functionality for all classes containing certain Javadoc tags and/or Annotations. All you have to do is to define these Javadoc tags/Annotations in your classes and to configure an XML file to process these in your classes. This already works natively! The only part you'd need to write a new is an extension to the result engine to do something with the data returned by the analysis engine. That's right: the analysis engine would return you an object (the famour Amplimet) containing all the classes matching your filters, then you should write probably some Java tag back-end classes and/or XML files to do something with these data. The interface formatter explained below is an example of such extension.

Users can create their own formatters, to auto-generate any kind of Java source. The files needed to auto-generate Java code are:

  • A maven.xml file, from which starting the analysis engine through the result engine, specifically through the HeartBit Tag. This is if you want to launch the analysis process through Maven. You can decide to launch the analysis process in other ways, even programmatically if you want.
  • An interceptor file which would be invoked by HeartBit and which would be given a JavaBean-like object containing all classes returned by the analysis engine. This is a simple XML file (in the test project called interfaces-proxy.xml)
  • One or more Formatters, which would be invoked by the interceptor file above and which would then ask the Jelly engine to run the jelly skeleton file.
  • A jelly file containing the skeleton of the Java source for which is required the auto-generation. This script would be invoked by any formatter which has been invoked by the interceptor file above (in the test project this is called interfaces.jelly)
Let's have a look at the different files involved in the test project. The first one showed is maven.xml, which simply triggers the analysis process, indicating to the analysis-engine which files should be analyzed, and the interceptor file to invoke once the analysis has completed. The files are contained in the Jemos Clanker distribution, and in this case the only one containing the Javadoc tag @clanker1 is the %CLANKER_HOME%/analysis-engine/src/test/uk/co/jemos/clanker/test/dummyclasses/classes/TestClass

whereas the only one containing the @interface-extract-me tag is %CLANKER_HOME%/analysis-engine/src/test/uk/co/jemos/clanker/test/dummyclasses/classes/AnotherTestClass


  <ant:property environment="env" />

  <preGoal name="java:compile">
    <jms:heartbit type="class" 
      processJavadocs = "true" 
      processAnnotations = "false" 
    <jms:fileset dir="${env.JEMOS_CLANKER_HOME}/analysis-engine/src/test/uk/co/jemos/clanker/test/dummyclasses/classes/">
      <jms:include name="**" />
    <jms:filter type="javadoc">
      <jms:docTag name="@clanker1" scope="all" />
      <jms:docTag name="@interface-extract-me" scope="all" />

The above XML snip shows the following information:

  • We want only Java class types matching our filter (attribute type = "class"), as opposite to every class even if not containing the @clanker1, @interface-extract-me tags. If users want all classes indistinctly, the attribute type should be set to "all".
  • We don't want Annotations to be processed (attribute processAnnotations = "false")
  • We set the debug level to INFO. Please note that the logging activity inside the tags uses the JDK logging API, since when using log4j no output was produced. This may be something that will change if I'll discover why log4j doesn't work from within the Jelly engine
  • The script to invoke from the HeartBitTag class once the analysis engine has terminated (the interceptor file described above) is interfaces-proxy.xml. The flow here is simple: HeartBitTag invokes the analysis engine which, once finished, produces an Amplimet object with all the required information; HeartBitTag then, acting as a router, invokes Jelly programmatically (i.e. it invokes a jelly script) passing the relevant information (in this case the bean-like objects encapsulating classes information) along the stack, which then will use those information to do something.
  • The analysis process needs to process only the java sources under the {env.JEMOS_CLANKER_HOME}/analysis-engine/src/test/uk/co/jemos/clanker/test/dummyclasses/classes/ folder, and not in the subdirectories. The xml elements used to indicate source selections is very similar to the one that Ant uses (fileset, include).
  • We want only the classes with the Javadoc tags @clanker1 and @interface-extract-me at any scope (classes, method, instance variable), identified by the scope = "all" attribute, that is to say that the analysis engine will need to look for the @clanker1 and @interface-extract-me Javadoc tags at class, method and instance variable level. The test project will extract and but you can play around with the maven.xml file and the dummy sources (by adding and selecting different Javadocs and/or Annotations) to obtain a different number of classes and therefore interfaces.

Here follows a snippet of few method from the HeartBitTag class. The entry point is the doTag() method:

	public void doTag(XMLOutput output) throws MissingAttributeException,
			JellyTagException {
		buff.append("[HeartBit]Validating Heartbit...\n");
		buff.append("[HeartBit]Invoking the body...\n");
		if (sources.isEmpty()) {
			LOG.warning("No sources have been selected. Please see heartbit.log for more info");
			buff.append("[HeartBit]No sources have been selected. \n");
		} else {
			buff.append("[HeartBit]The following sources will be analyzed:\n");
			for (String s: sources) {
				buff.append("[HeartBit]" + s + "\n");
			try {
				//This is where the magic happens: it invokes the analysis engine, 
				//sets the obtained classes in the context, and runs the script
			} catch (ConfigurationException e) {
				throw new JellyTagException(e);
			} catch (ThapterException e) {
				throw new JellyTagException(e);
			} catch (JellyException e) {
				throw new JellyTagException(e);
			} catch (IOException e) {
				throw new JellyTagException(e);
			} finally {
This method validates the parameters, invokes the body (i.e. it executes the fileset, include, filter and docTag elements) to set up the analysis process configuration object and then launches the analysis-process.

Here follows a snippet of the method which launches the analysis engine:

	public void launchAnalysisEngine(XMLOutput output)  throws ConfigurationException, 
	  		 							 JellyException, IOException {
		//Prepares the configuration object
		ConfigurationImpl config = new ConfigurationImpl();
		//Sets the filters if any were set up by the children
		Crystal crystal = new Crystal();
		if (null != javadocFilter) {
		if (null != annotationFilter) {
		if (getType().toLowerCase().equals("class")) {
		} else {
		//Sets the sources to analyze
		//Sets the flag whether to process Javadoc and/or Annotations
		try {
			//Creates the controller with the configuration object
			Controller controller = new ControllerImpl(config);
			//Prepares the Thapter with the controller (Analysis Engine)
			Thapter thapter = new ThapterImpl(controller);
			buff.append("[HeartBit]Launching the thapter...\n");
			//Launches the analysis engine
			buff.append("[HeartBit]Thapter ended the execution. Retrieving the amplimet...\n");
			//The Amplimet contains the results passed back by the analysis engine
			Amplimet amplimet = thapter.getAmplimet();
			//Prepares an object containing all classes returned by the analysis engine
			Package p = null;
			for (Component pkg: packages) {
				p = (Package)pkg;
				buff.append("[HeartBit]Package: " + p.getPackageName());
			//Sets the data in the context and invokes the output script
			getContext().setVariable("classes", this.getClasses());
			//Routes the process to the file identified by the processingScript
			//element passed as parameter by the maven.xml file
			getContext().runScript(outputF, output);
		} catch (ConfigurationException e) {
			throw e;
		} catch (ThapterException e) {
			throw e;
		} catch (IOException e) {
			throw e;
The method creates a configuration object, based on the information contained in the heartbit tag and sub-tags (see maven.xml above) and with this initializes a Thapter object (the analysis engine entry point). It then launches the analysis process which returns, in the Amplimet object, the analysis results. The method then retrieves all Package objects returned by the analysis engine, from these it retrieves all classes, sets a Jelly context variable classes with all classes, and proxies the continuation of the process to the xml file identified by the processingScript parameter (see maven.xml file above).

Here follows the content of interfaces-proxy.xml. (the processing script described in the previous paragraph.). The formatter is the class, located in the result engine subproject. Its role is to validate the parameters passed as argument, to create the output directory structure, to set the Jelly context variables and to trigger, for each Java class, the execution of the jelly skeleton file to create the Java interfaces. The jelly skeleton file will use the variables set in the context by the Formatter. Please note in the following file the declaration of the ClankerJellyLibrary class which allows the application to use the jms: tag (you could have named whatever you liked in the namespaces declaration). Since this file received from HeartBitTag a context object called 'classes' containing a java.util.List of Clazz objects, now it can pass it to other objects (like in this case, where the application passes it to the Interfaces formatter).

<?xml version="1.0"?>
   <j:jelly trim="false" 
       <-- Invokes the 'interfaces' formatter which will create, 
           for each class contained in 'classes' a java interface,
           based on the skeleton defined in the 'interfaces.jelly' 
           script -->
         	classes = "${classes}"
         	jellyScript = "interfaces.jelly"
         	outDir = "${}/jemos-clanker/intf"
         	suffix = "Intf" />

This is the meaning of the above tag attributes:

  • classes (Required). Contains a java.util.List of Clazz types passed back by HeartBitTag (which in turn received them by the analysis engine)
  • jellyScript (Required). This attribute contains the full path to the jelly script containing the skeleton of the Java source which we want to generate.
  • outDir (Required). This attribute defines the full path of the output directory where the auto-generated code will be placed. Future releases may provide a default to ${}/classes/jemos-clanker/intf
  • . The interface formatter will create a folder structure matching outDir.
  • packageName (Required). This defines the package name where the auto-generated classes will be placed. The result engine will create a folder structure matching the package name. This folder structure will be placed under outDir.
  • suffix (Not Required). This attribute defines the suffix to append to each class name to create the interface name. It defaults to Intf.
Below is shown the content of interfaces.jelly, which is a skeleton to create interfaces, with all the power and flexibility offered by Jelly. This script is invoked by the Interface formatter for each class returned by HeartBitTag. In the example, the script simply invokes the getInterfaceSignature() method for each method of the class identified by the clazz object. This method is responsible to return an interface-friendly signature of itself.

<?xml version="1.0"?>
<j:jelly trim="false" 
   //Auto-generated by Jemos Clanker. Do not edit
   package ${packageName};
    * Defines interface for class ${clazz.fullName}
   public interface ${}${suffix} {
     <j:forEach items="${clazz.methods}" var="method">

Here are shown snips of the Interface formatter class:

	public void doTag(XMLOutput output) throws MissingAttributeException,
			JellyTagException {"Classes size: " + classes.size());"Jelly script: " + this.getJellyScript());"Output Directory: " + this.getOutDir());"Package Name: " + this.getPackageName());"Suffix: " + this.getSuffix());
		//Creates the full path where to store the files under
		String destinationDir = PathUtils.resolvePackageNameToPath(this.getPackageName());
		destinationDir = this.getOutDir() + File.separatorChar + destinationDir;
		destinationDir.replace('\\', File.separatorChar);
		destinationDir.replace('/', File.separatorChar);"The Java interfaces will be created at: " + destinationDir);
		//Generates the Java interfaces
		try {
		} catch (FormatterException e) {
			throw new JellyTagException(e);
The above snips is the entry point of the formatter, and actually of every Tag class. In this case, after a big of debugging, the method invokes the validate() method to validate the parameters and create the output folder structure; it then combines a unique output path by concatenating the output dir path to the package name path (resolved as file path) and then invokes the method responsible to trigger the generation of the Java interfaces.

The method which triggers the Java interface generation is shown below:

	private void generateInterfaces(String outDir) throws FormatterException {
		//A logging buffer
		StringBuffer buff = new StringBuffer();
		//The Java interface name
		StringBuffer fileName = new StringBuffer(outDir);
		//The Output Stream
		OutputStream os = null;
		//The jelly context
		JellyContext context = new JellyContext();
		//Sets the variables common to all 
		context.setVariable("suffix", this.getSuffix());
		context.setVariable("packageName", this.getPackageName());
		//For each class, it sets the context variables and runs the
		//jelly script which contains the interfaces skeleton
		for (Clazz c: this.getClasses()) {
			if (!outDir.endsWith("/")) {
					.append(".java");"Creating: " + fileName.toString());
			try {
				os = new FileOutputStream(new File(fileName.toString()));
				//Sets the context variables for each class
				context.setVariable("clazz", c);
				XMLOutput output = XMLOutput.createXMLOutput(os); 
				context.runScript(new File(this.getJellyScript()), output);
			} catch (FileNotFoundException e) {
				buff.append("An error occurred while trying to create")
				  .append(" the file output stream. The following file name: ")
				  .append(" wasn't found.");
				throw new FormatterException(buff.toString(), e);
			} catch (UnsupportedEncodingException e) {
				buff.append("An error occurred while creating the XMLOutput")
				  .append(" object to write the Java interface to. This")
				  .append(" is the exception thrown by the application:")
				throw new FormatterException(buff.toString(), e);
			} catch (JellyException e) {
				buff.append("An error occurred while running the jelly script: ")
				throw new FormatterException(buff.toString(), e);
The snip above shows that the method, after validating the required parameters and creating the necessary folder structure, it sets the Jelly context variables (the ones common to each class and the ones particular to each class) and then triggers the execution of the interfaces.jelly script through the context.runScript() method. This will cause the Jelly engine to execute the interfaces.jelly method which will use the context variables to generate a Java interface.

The results are shown below:

   //Auto-generated by Jemos Clanker. Do not edit
    * Defines interface for class
   public interface TestClassIntf {
     public String getFirstName() 
     public void setFirstName(String firstName) 
     public void testMethodForInterfaceFormatting(String param1, String param2, String param3) 
     public String testMethodForInterfaceWithReturnType(String param1, String param2, String param3) 
     public testMethodForInterfaceWithComplexReturnType(String param1, 
     	String param2, 
     	String param3) 


   //Auto-generated by Jemos Clanker. Do not edit
    * Defines interface for class
   public interface AnotherTestClassIntf {
     public void businessMethod1() throws Exception;
     public void businessMethod2() throws Exception;
     public void businessMethod3() throws Exception;