Jemos Clanker is a Java source analysis framework for the auto-generation
of Java code, starting from the analysis of Javadoc tags and/or Annotations.
Its goals are similar to
XDoclet but the similarities end there.
The reason why I wrote Clanker is that I couldn't find anywhere a Java product
which would make easy for me the simple programmatic management of Javadoc
tags and Annotations. Have you ever tried to define your own tags with
XDoclet? I tried with EJB 2.1 for the auto-generation of Business
Delegates, and I was descouraged by the learning curve before me.
In February 2006 I participated to JavaUK06 where I was introduced
to EE5 and the fantastic world of the Java Persistence Framework.
I then decided to try and
use XDoclet to auto-generate the XML file needed when deploying the
persistence framework in a J2SE environment. After about 5 minutes
I realized that I was wasting my time. I also tried to look
on the net for a tool which would allow a programmatic approach
to Annotation and Javadoc processing and I found the
Annotation
Processing Tool (APT). Again, after few minutes I felt descouraged,
so I decided to write my own framework with these simple goals in mind:
When I decided to write Clanker I was reading a series of books from Ian Irvine, which you can find by clicking here. Most of the names used by Clanker have been taken from those books (with the author's permission), especially the Well of Echoes series. The only thing the author asked was that Clanker had to be used as open-source product and that if used commercially then explicit permission should have been requested to the publisher. A bit of terminology from these books: a Clanker is a war machine that humans built in order to fight the alien 'Lyrinx', ex-humans that thanks to the use of the 'Secret Art' (call it magic) went to live in a place amongst the three worlds amongst which this saga is ambiented. Two hundreds years before the tale of the 'Well of Echoes' begins (i.e. the tale written in the previous series 'The View from the Mirror'), the main character of that tale opened the way between the worlds (a 'gate') and the Lyrinx came on the Earth in order to start a new life. Humans, frightened by those beasts, started a war against them and this is what is written in the 'Well of Echoes' series. It was so much fun reading these books that I decided to use some of the names contained in them for some of the objects in Clanker. So the analysis processors are called 'Lyrinx', filters are part of a 'Crystal' (a powerful object in the above tale), the main analysis object is called a 'Thapter' (a machine which represents possibly the only chance that humans have to win the war). I hope I haven't messed things too much :)
Jemos Clanker is currently composed by two main modules:
The steps to use Clanker are briefly listed below and these are the same both for
XML-based or programmatic approach:
At high level, there are few major components, shown in the picture below and about which follows a description.
Here follows a brief overview of the activities necessary to start the analysis process. In the case of filters used, users can specify whether they want the analysis engine to return only the classes matching those filters, or all the classes containing at least one of the filter matches.
The result engine is the second component of Jemos Clanker.
Its role is to offer users functionalities for auto-generation
of Java code. This goal is achieved through two strategies:
The tags structure is as follows:
The list of tags shall be declared in the jelly-config.xml file under the Jemos Clanker root installation folder. The list of tags defined in this file is automatically processed by the uk.co.jemos.clanker.jelly.libs.ClankerJellyLibrary class, which loads and registers them with the Jelly engine.
Here follows an example of the jelly-config.xml file with the declaration
of the core tags:
<?xml version="1.0" encoding="ISO-8859-1"?>
<tags>
<!-- Tags -->
<tag workName="heartbit" class="uk.co.jemos.clanker.jelly.tags.HeartBitTag" />
<tag workName="fileset" class="uk.co.jemos.clanker.jelly.tags.FileSetTag" />
<tag workName="include" class="uk.co.jemos.clanker.jelly.tags.IncludeTag" />
<tag workName="filter" class="uk.co.jemos.clanker.jelly.tags.FilterTag" />
<tag workName="docTag" class="uk.co.jemos.clanker.jelly.tags.JavadocTag" />
<tag workName="annTag" class="uk.co.jemos.clanker.jelly.tags.AnnotationTag" />
<!-- Formatters -->
<tag workName="interfaces"
class="uk.co.jemos.clanker.jelly.tags.formatters.InterfacesTag" />
</tags>
From the user perspective, using the power of Jemos Clanker is as easy
as writing a normal maven.xml goal. Let's have a look at an example where
a user wants Jemos Clanker to auto-generate Java interfaces for a set
of classes containing the @clanker1 and the
@interface-extract-me Javadoc tags. This example is contained
in the Jemos Clanker Test Project, which can be downloaded from the
download area. The classes analyzed by the analysis engine are part of
the Jemos Clanker distribution and are located under
%CLANKER_HOME%/analysis-engine/src/test/uk/co/jemos/clanker/test/dummyclasses/classes.
In a real-scenario, obviously users would ask the analysis engine to analyze Java classes located somewhere else.
The cool thing about Jemos Clanker is that in order to add a new Javadoc
to be processed, you don't have to write any new class (as opposed to XDoclet)
but you just have to specify your new Javadoc tag in the XML file which
starts the analysis process. Let's say that you want the analysis process
to generate Java interfaces for all classes containing a
@user-defined-interface Javadoc tag. All you'll have to
do is to declare this Javadoc tag in all the classes for which
you want Jemos Clanker to auto-generate a Java interface, and then
change the xml file which triggers the analysis engine (in our example
the maven.xml file below) indicating to use this tag. Let's go a bit
further...Let's say that you want to create a new functionality for
all classes containing certain Javadoc tags and/or Annotations. All
you have to do is to define these Javadoc tags/Annotations in your
classes and to configure an XML file to process these in your classes.
This already works natively! The only part you'd need to write a new
is an extension to the result engine to do something with the data
returned by the analysis engine. That's right: the analysis engine
would return you an object (the famour Amplimet) containing all the
classes matching your filters, then you should write probably some
Java tag back-end classes and/or XML files to do something with these
data. The interface formatter explained below is an example of such
extension.
Users can create their own formatters, to auto-generate any kind
of Java source. The files needed to auto-generate Java code are:
<project
xmlns:j="jelly:core"
xmlns:u="jelly:util"
xmlns:ant="jelly:ant"
xmlns:jms="jelly:uk.co.jemos.clanker.jelly.libs.ClankerJellyLibrary"
default="jar:install"
>
<ant:property environment="env" />
<preGoal name="java:compile">
<jms:heartbit type="class"
processJavadocs = "true"
processAnnotations = "false"
debugLevel="INFO"
processingScript="interfaces-proxy.xml">
<jms:fileset dir="${env.JEMOS_CLANKER_HOME}/analysis-engine/src/test/uk/co/jemos/clanker/test/dummyclasses/classes/">
<jms:include name="**" />
</jms:fileset>
<jms:filter type="javadoc">
<jms:docTag name="@clanker1" scope="all" />
<jms:docTag name="@interface-extract-me" scope="all" />
</jms:filter>
</jms:heartbit>
</preGoal>
</project>
The above XML snip shows the following information:
Here follows a snippet of few method from the HeartBitTag class. The entry
point is the doTag() method:
public void doTag(XMLOutput output) throws MissingAttributeException,
JellyTagException {
buff.append("[HeartBit]Validating Heartbit...\n");
this.validate(output);
buff.append("[HeartBit]Invoking the body...\n");
this.invokeBody(output);
if (sources.isEmpty()) {
LOG.warning("No sources have been selected. Please see heartbit.log for more info");
buff.append("[HeartBit]No sources have been selected. \n");
this.logActivity();
} else {
buff.append("[HeartBit]The following sources will be analyzed:\n");
for (String s: sources) {
buff.append("[HeartBit]" + s + "\n");
}
try {
//This is where the magic happens: it invokes the analysis engine,
//sets the obtained classes in the context, and runs the script
this.launchAnalysisEngine(output);
} catch (ConfigurationException e) {
throw new JellyTagException(e);
} catch (ThapterException e) {
throw new JellyTagException(e);
} catch (JellyException e) {
throw new JellyTagException(e);
} catch (IOException e) {
throw new JellyTagException(e);
} finally {
this.logActivity();
}
}
}
Here follows a snippet of the method which launches the analysis engine:
public void launchAnalysisEngine(XMLOutput output) throws ConfigurationException,
ThapterException,
JellyException, IOException {
//Prepares the configuration object
ConfigurationImpl config = new ConfigurationImpl();
//Sets the filters if any were set up by the children
Crystal crystal = new Crystal();
if (null != javadocFilter) {
crystal.setCommentFilter(javadocFilter);
}
if (null != annotationFilter) {
crystal.setAnnotationFilter(annotationFilter);
}
if (getType().toLowerCase().equals("class")) {
crystal.setReturnType(ReturnTypes.CLASSES);
} else {
crystal.setReturnType(ReturnTypes.ALL);
}
config.setCrystal(crystal);
//Sets the sources to analyze
config.getSources().addAll(this.getSources());
//Sets the flag whether to process Javadoc and/or Annotations
config.setProcessComments(this.isProcessJavadocs());
config.setProcessAnnotations(this.isProcessAnnotations());
try {
//Creates the controller with the configuration object
Controller controller = new ControllerImpl(config);
//Prepares the Thapter with the controller (Analysis Engine)
Thapter thapter = new ThapterImpl(controller);
buff.append("[HeartBit]Launching the thapter...\n");
//Launches the analysis engine
thapter.takeOff();
buff.append("[HeartBit]Thapter ended the execution. Retrieving the amplimet...\n");
//The Amplimet contains the results passed back by the analysis engine
Amplimet amplimet = thapter.getAmplimet();
//Prepares an object containing all classes returned by the analysis engine
packages.addAll(amplimet.getComponents(COMPONENT_TYPE_PACKAGE));
Package p = null;
for (Component pkg: packages) {
p = (Package)pkg;
buff.append("[HeartBit]Package: " + p.getPackageName());
classes.addAll(p.getChildren(COMPONENT_TYPE_CLASS));
}
//Sets the data in the context and invokes the output script
getContext().setVariable("classes", this.getClasses());
//Routes the process to the file identified by the processingScript
//element passed as parameter by the maven.xml file
getContext().runScript(outputF, output);
output.flush();
} catch (ConfigurationException e) {
throw e;
} catch (ThapterException e) {
throw e;
} catch (IOException e) {
throw e;
}
}
Here follows the content of interfaces-proxy.xml.
(the processing script described in the previous paragraph.).
The formatter is the class
uk.co.jemos.clanker.jelly.formatters.InterfacesTag,
located in the result engine subproject. Its role is to validate
the parameters passed as argument, to create the output directory
structure, to set the Jelly context variables and to trigger,
for each Java class, the execution of the jelly skeleton file
to create the Java interfaces. The jelly skeleton file will
use the variables set in the context by the Formatter.
Please note in the following file the declaration of
the ClankerJellyLibrary class which allows the application
to use the jms: tag (you could have named whatever
you liked in the namespaces declaration). Since this file
received from HeartBitTag a context object called 'classes'
containing a java.util.List of Clazz objects, now it can
pass it to other objects (like in this case, where
the application passes it to the Interfaces formatter).
<?xml version="1.0"?>
<j:jelly trim="false"
xmlns:j="jelly:core"
xmlns:x="jelly:xml"
xmlns:html="jelly:html">
<-- Invokes the 'interfaces' formatter which will create,
for each class contained in 'classes' a java interface,
based on the skeleton defined in the 'interfaces.jelly'
script -->
<jms:interfaces
classes = "${classes}"
jellyScript = "interfaces.jelly"
outDir = "${maven.build.dir}/jemos-clanker/intf"
packageName="uk.co.jemos.clanker.intf"
suffix = "Intf" />
</j:jelly>
This is the meaning of the above tag attributes:
<?xml version="1.0"?>
<j:jelly trim="false"
xmlns:j="jelly:core"
xmlns:x="jelly:xml"
xmlns:html="jelly:html"
xmlns:jemos="jelly:uk.co.jemos.clanker.jelly.libs.ClankerJellyLibrary">
//Auto-generated by Jemos Clanker. Do not edit
package ${packageName};
/**
* Defines interface for class ${clazz.fullName}
*/
public interface ${clazz.name}${suffix} {
<j:forEach items="${clazz.methods}" var="method">
${method.interfaceSignature}
</j:forEach>
}
</j:jelly>
Here are shown snips of the Interface formatter class:
public void doTag(XMLOutput output) throws MissingAttributeException,
JellyTagException {
LOG.info("Classes size: " + classes.size());
LOG.info("Jelly script: " + this.getJellyScript());
LOG.info("Output Directory: " + this.getOutDir());
LOG.info("Package Name: " + this.getPackageName());
LOG.info("Suffix: " + this.getSuffix());
this.validate(output);
//Creates the full path where to store the files under
String destinationDir = PathUtils.resolvePackageNameToPath(this.getPackageName());
destinationDir = this.getOutDir() + File.separatorChar + destinationDir;
destinationDir.replace('\\', File.separatorChar);
destinationDir.replace('/', File.separatorChar);
LOG.info("The Java interfaces will be created at: " + destinationDir);
//Generates the Java interfaces
try {
this.generateInterfaces(destinationDir);
} catch (FormatterException e) {
throw new JellyTagException(e);
}
this.invokeBody(output);
}
The method which triggers the Java interface generation is shown below:
private void generateInterfaces(String outDir) throws FormatterException {
//A logging buffer
StringBuffer buff = new StringBuffer();
//The Java interface name
StringBuffer fileName = new StringBuffer(outDir);
//The Output Stream
OutputStream os = null;
//The jelly context
JellyContext context = new JellyContext();
//Sets the variables common to all
context.setVariable("suffix", this.getSuffix());
context.setVariable("packageName", this.getPackageName());
//For each class, it sets the context variables and runs the
//jelly script which contains the interfaces skeleton
for (Clazz c: this.getClasses()) {
if (!outDir.endsWith("/")) {
fileName.append(File.separatorChar);
}
fileName.append(c.getName())
.append(this.getSuffix())
.append(".java");
LOG.info("Creating: " + fileName.toString());
try {
os = new FileOutputStream(new File(fileName.toString()));
//Sets the context variables for each class
context.setVariable("clazz", c);
XMLOutput output = XMLOutput.createXMLOutput(os);
context.runScript(new File(this.getJellyScript()), output);
} catch (FileNotFoundException e) {
buff.append("An error occurred while trying to create")
.append(" the file output stream. The following file name: ")
.append(fileName)
.append(" wasn't found.");
LOG.severe(buff.toString());
throw new FormatterException(buff.toString(), e);
} catch (UnsupportedEncodingException e) {
buff.append("An error occurred while creating the XMLOutput")
.append(" object to write the Java interface to. This")
.append(" is the exception thrown by the application:")
.append("\n")
.append(e.getLocalizedMessage());
LOG.severe(buff.toString());
throw new FormatterException(buff.toString(), e);
} catch (JellyException e) {
buff.append("An error occurred while running the jelly script: ")
.append(this.getJellyScript())
.append(".");
LOG.severe(buff.toString());
throw new FormatterException(buff.toString(), e);
}
}
}
The results are shown below:
//Auto-generated by Jemos Clanker. Do not edit
package uk.co.jemos.clanker.intf;
/**
* Defines interface for class uk.co.jemos.clanker.test.dummyclasses.classes.TestClass
*/
public interface TestClassIntf {
public String getFirstName()
throws uk.co.jemos.clanker.exceptions.ConfigurationException,
uk.co.jemos.clanker.exceptions.GateException;
public void setFirstName(String firstName)
throws uk.co.jemos.clanker.exceptions.ConfigurationException,
uk.co.jemos.clanker.exceptions.GateException;
public void testMethodForInterfaceFormatting(String param1, String param2, String param3)
throws uk.co.jemos.clanker.exceptions.ComponentException,
uk.co.jemos.clanker.exceptions.ConfigurationException,
uk.co.jemos.clanker.exceptions.FilterException;
public String testMethodForInterfaceWithReturnType(String param1, String param2, String param3)
throws uk.co.jemos.clanker.exceptions.ComponentException,
uk.co.jemos.clanker.exceptions.ConfigurationException,
uk.co.jemos.clanker.exceptions.FilterException;
public uk.co.jemos.clanker.components.Clazz testMethodForInterfaceWithComplexReturnType(String param1,
String param2,
String param3)
throws uk.co.jemos.clanker.exceptions.ComponentException,
uk.co.jemos.clanker.exceptions.ConfigurationException,
uk.co.jemos.clanker.exceptions.FilterException;
}
//Auto-generated by Jemos Clanker. Do not edit
package uk.co.jemos.clanker.intf;
/**
* Defines interface for class uk.co.jemos.clanker.test.dummyclasses.classes.AnotherTestClass
*/
public interface AnotherTestClassIntf {
public void businessMethod1() throws Exception;
public void businessMethod2() throws Exception;
public void businessMethod3() throws Exception;
}