Gradle AntBuilder

Every Gradle Project includes an AntBuilder instance, making any and all of the facilities of Ant available within your build files. Gradle provides a simple extension to the existing Groovy AntBuilder which adds a simple yet powerful way to interface with existing Ant build files: the importBuild(Object antBuildFile) method. Internally this method utilizes an Ant ProjectHelper to parse the specified Ant build file and then wraps all of the targets in Gradle tasks making them available in the Gradle build. The following is a simple Ant build file used for illustration which contains some properties and a couple of dependent targets.

<?xml version="1.0"?>
<project name="build" default="all">
    <echo>Building ${ant.file}</echo>

    <property file="build.properties"/>
    <property name="root.dir" location="."/>

    <target name="dist" description="Build the distribution">
        <property name="dist.dir" location="dist"/>
        <echo>dist.dir=${dist.dir}, foo=${foo}</echo>
    </target>

    <target name="all" description="Build everything" depends="dist"/>
</project>

Importing this build file using Gradle is a one-liner.

ant.importBuild('src/main/resources/build.xml')

And the output of gradle tasks –all on the command line shows that the targets have been added to the build tasks.

$ gradle tasks --all
...
Other tasks
-----------
all - Build everything
    dist - Build the distribution
...

Properties used in the Ant build file can be specified in the Gradle build or on the command line and, unlike the usual Ant property behaviour, properties set by Ant or on the command line may be overwritten by Gradle. Given a simple build.properties file with foo=bar as the single entry, here’s a few combinations to demonstrate the override behaviour.

Command line invocation	Gradle Build Config	Effect	Result
gradle dist	ant.importBuild('src/main/resources/build.xml')	build.properties value loaded from ant build is used	foo=bar
gradle dist -Dfoo=NotBar	ant.importBuild('src/main/resources/build.xml')	command line property is used	foo=NotBar
gradle dist -Dfoo=NotBar	ant.foo='NotBarFromGradle' ant.importBuild('src/main/resources/build.xml')	Gradle build property is used	foo=NotBarFromGradle
gradle dist -Dfoo=NotBar	ant.foo='NotBarFromGradle' ant.importBuild('src/main/resources/build.xml') ant.foo='NotBarFromGradleAgain'	Gradle build property override is used	foo=NotBarFromGradleAgain

How to deal with task name clashes

Since Gradle insists on uniqueness of task names attempting to import an Ant build that contains a target with the same name as an existing Gradle task will fail. The most common clash I’ve encountered is with the clean task provided by the Gradle BasePlugin. With the help of a little bit of indirection we can still import and use any clashing targets by utilizing the GradleBuild task to bootstrap an Ant build import in an isolated Gradle project. Let’s add a new task to the mix in the Ant build imported and another dependency on the ant clean target to the all task.

<!-- excerpt from buildWithClean.xml Ant build file -->
    <target name="clean" description="clean up">
        <echo>Called clean task in ant build with foo = ${foo}</echo>
    </target>
    <target name="all" description="Build everything" depends="dist,clean"/>

And a simple Gradle build file which will handle the import.

ant.importBuild('src/main/resources/buildWithClean.xml')

Finally, in our main gradle build file we add a task to run the targets we want.

task importTaskWithExistingName(type: GradleBuild) { GradleBuild antBuild ->
    antBuild.buildFile ='buildWithClean.gradle'
    antBuild.tasks = ['all']
}

This works, but unfortunately suffers from one small problem. When Gradle is importing these tasks it doesn’t properly respect the declared order of the dependencies. Instead it executes the dependent ant targets in alphabetical order. In this particular case Ant expects to execute the dist target before clean and Gradle executes them in the reverse order. This can be worked around by explicitly stating the task order, definitely not ideal, but workable. This Gradle task will execute the underlying Ant targets in the way we need.

task importTasksRunInOrder(type: GradleBuild) { GradleBuild antBuild ->
    antBuild.buildFile ='buildWithClean.gradle'
    antBuild.tasks = ['dist', 'clean']
}

Gradle Rules for the rest

Finally, you can use a Gradle Rule to allow for calling any arbitrary target in a GradleBuild bootstrapped import.

tasks.addRule("Pattern: a-<target> will execute a single <target> in the ant build") { String taskName ->
    if (taskName.startsWith("a-")) {
        task(taskName, type: GradleBuild) {
            buildFile = 'buildWithClean.gradle'
            tasks = [taskName - 'a-']
        }
    }
}

In this particular example, this can allow you to string together calls as well, but be warned that they execute in completely segregated environments.

$ gradle a-dist a-clean

Source code

All of code referenced in this article is available on github if you’d like to take a closer look.

Related posts:

1. Why do I Like Gradle?
2. A Groovy/Gradle JSLint Plugin
3. Five Cool Things You Can Do With Groovy Scripts

Gradle is a build system in which builds are described using a declarative and concise DSL written in the Groovy language. This article describes how you can wrap proven Apache Ant Tasks in a Gradle Plugin to make using them as effortless as possible. We’ll also go over some of the tools Gradle provides for building and testing robust Plugin functionality following some easy patterns.

Creating new custom Plugins for Gradle is a relatively straightforward and easy process. Within a Plugin it’s possible to configure a Gradle Project with new properties, dependencies, Tasks – pretty much anything that you can configure in a build.gradle file can be encapsulated into a Plugin for abstraction, portability and reuse. One of the easier ways to add functionality through a Plugin is to encapsulate an existing Ant Task and enhance it by providing the ease-of-use and configuration that Gradle users have come to expect. Recently, I’ve been writing a lot more JavaScript and was looking for static analysis tools to help guide me away from ‘bad habits’. The popular choice for static analysis of JavaScript code seems to be JSLint, so here’s an example of providing that functionality for a Gradle build by wrapping an existing JSLint task and making it easier to work with.

Anatomy of a Gradle Plugin

Gradle plugins can most easily be built using Gradle itself. There is a conveniently available gradleApi() method you can call to include the required framework classes, demonstrated in the dependencies section of a build.gradle file shown in Listing 1. For this example we’re also using the Groovy Plugin and JUnit for testing, so we will include those dependencies as well.

dependencies {
    compile gradleApi()
    groovy group: 'org.codehaus.groovy', name: 'groovy', version: '1.7.6'
    testCompile group: 'junit', name: 'junit', version: '4.8.2
}

Listing 1: The dependencies portion of a build.gradle file for building our Plugin

Creating a new Gradle Plugin is a simple matter of implementing the Gradle interface and its single required method, the skeleton of which is shown in Listing 2. Within the apply method, the Plugin can configure the Project to add Tasks or properties as needed.

class JSLintPlugin implements Plugin<Project>
{
	void apply(Project project)
	{
		//configure the Project here
	}
}

Listing 2: Skeleton of the Plugin implementation

Integrating with Ant

I’ve never been overly fond of Ant, mostly due to the extremely verbose and repetitive nature of the xml declaration. But the fact remains that Ant has been a primary and well-used build tool for years, and the Tasks written for it have been tried and tested by many developers. The Groovy AntBuilder, in combination with the facilities Gradle provides for dependency resolution and classpath management, makes it very easy to incorporate existing Ant functionality into a build and abstract most of the details away from the end user. For this plugin, we add the library containing the Ant Task to a custom configuration so that we can have it automatically downloaded and easily resolve the classpath.
Listing 3 shows how the configuration for the JSLint4Java Task could appear in an Ant build.xml file. Note that you’re on your own here to provide the required library for the classpath.

<taskdef name="jslint"
         classname="com.googlecode.jslint4java.ant.JSLintTask"
         classpath="/path/to/jslint4java-1.4.jar"/>
<target name="jslint">
    <jslint options="undef,white" haltOnFailure="false">
        <formatter type="xml" destfile="${build.dir}/reports"/>
        <fileset dir="." includes="**/*.js" excludes="**/server/"/>
    </jslint>
</target>

Listing 3: Configuring the Ant target in a build.xml file

Gradle makes it easy to separate the configuration and execution phases of the build, allowing for a Plugin to add an Ant Task to the Gradle Project and expose the (optional) configuration in a build script. In addition, Gradle encourages a pattern of providing a ‘convention’ object alongside of a Plugin to clearly separate the concerns.
Listing 4 demonstrates some code from the Plugin implementation that adds a ‘jslint’ Task to a Gradle Project, setting the specific options based on a convention object. Note how we extract the classpath using the notation project.configurations.jslint.asPath.

private static final String TASK_NAME = 'jslint'
private Project project
private JSLintPluginConvention jsLintpluginConvention
...
// some of the code in the apply method
this.jsLintpluginConvention = new JSLintPluginConvention(project)
project.convention.plugins.jslint = jsLintpluginConvention
project.task(TASK_NAME) << {
    project.file(project.reportsDir).mkdirs()
    logger.info("Running jslint on project ${project.name}")
    ant.taskdef(name: TASK_NAME, classname: jsLintpluginConvention.taskName,
        classpath: project.configurations.jslint.asPath)
    ant."$TASK_NAME"(jsLintpluginConvention.mapTaskProperties()) {
        formatter(type: jsLintpluginConvention.decideFormat(),
                destfile:  jsLintpluginConvention.createOutputFileName())
        jsLintpluginConvention.inputDirs.each { dirName ->
            fileset(dir: dirName,
			 includes: jsLintpluginConvention.includes,
			 excludes: jsLintpluginConvention.excludes)
        }
    }
}

Listing 4: Adding a jslint Task to a Gradle Project

If you’re not already familiar with the Gradle syntax for creating new Tasks inline, the project.task(String taskName) method is called to instantiate a new Task, and the ‘< <' syntax is used to to push the Task activities into the 'doLast' Task lifecycle phase.
Allowing for configuration of the Task in a build script is as simple as exposing a method named the same as the Task that takes in a Closure parameter and applies that Closure to set properties on the convention object, as shown in Listing 5.

/**
 * Perform custom configuration of the plugin using the provided closure.
 * @param closure
 */
def jslint(Closure closure)
{
    closure.delegate = this
    closure()
}

Listing 5: A method to allow for configuration of the jslint Task from a build script

The simple use-case

As per usual when working with Gradle, using this plugin in the most basic case requires only these things:

• Declare a dependency on the Plugin source. This can be either a released jar to be downloaded from a repository by Gradle or you can download a jar manually from pretty much anywhere and add it to the classpath directly.
• Apply the Plugin to a Gradle build.
• Call the jslint Task as part of the build.

The entire configuration and usage looks something like Listing 6, assuming that the gradle-jslint-plugin jar is found in /usr/home/gradlelibs.

/* In a build.gradle file */
buildscript {
	dependencies {
		classpath fileTree(dir: '/usr/home/gradlelibs', include: '*.jar')
	}
}
apply plugin: org.kar.jslint.gradle.plugin.JSLintPlugin

/* and on the command line... */
gradle jslint

Listing 6: Configuring the jslint Plugin in a build script and calling it from the command line

By default, this is enough to scan for all .js files under the directory where the build script is located and create a JSLint text report using the basic settings.

The not-so-simple case

Of course in the real world the defaults aren't always what we need, so being able to easily configure the Task is essential. Fortunately, Gradle makes extending a custom Plugin to allow for configuration by a simple Closure, so we can exercise the code from Listing 5 in a build script with the Closure definition in Listing 7.

jslint {
	haltOnFailure = false
	excludes = '**/metadata/'
	options = 'rhino'
	formatterType = 'html'
}

Listing 7: Departing from the default jslint Task configuration

Extending Ant Task capabilities

The Ant Task as is can produce either a plain text document or an xml report, but transforming the results into a more consumable html format is easy to do using an Ant xslt Task. Having Gradle wrap the Task definition allows for simply adding a new formatter type to the configuration and abstracting away the details from the end user. A copy of an xsl file available online is easy to incorporate with the plugin and can be used to transform the xml output into a nicely formatted web page. Being able to program around Ant Tasks like this is a great way to enhance their value in your build. An example of simple output from the test cases included in the Plugin is shown in Figure 1.

Figure 1: Example html formatted output from jslint

Testing using ProjectBuilder

In order to facilitate testing custom Tasks and Plugins, the Gradle framework provides the ProjectBuilder implementation to handle most of the heavy lifting. This gives you a mocked out instance of a Gradle Project that builds into a temporary directory; very handy for testing how Tasks behave under real working conditions. Having tools like this available directly from the framework removes a lot of potential barriers that might otherwise discourage testing of custom components. The source code that accompanies this article uses the ProjectBuilder to achieve 100% code coverage of the project and is available on github at https://github.com/kellyrob99/gradle-jslint-plugin if you'd like to look closer for some ideas on how to test your own Gradle Plugins. An already built version of the jar is also available if you'd like to try without having to build it yourself: https://github.com/kellyrob99/gradle-jslint-plugin/blob/master/releases/gradle-jslint-plugin-0.1-SNAPSHOT.jar.

How could we improve this Plugin?

This implementation represents the 'brute-force' method of wrapping an Ant Task, and there are several ways to enhance its function, if we wanted to spend some additional time and effort on the Plugin. It would actually be far more flexible if we extended the Gradle DefaultTask to provide the actual functionality; this would allow for the possibility of executing the Task separately against different sets of JavaScript in the same project with different configurations of JSLint. In the case of an application with both client and server-side JavaScript, for instance, you might want to apply different rules. In that event you'd probably also want to have the capability to aggregate multiple result sets, which would be a relatively easy feature to add.
Having a separate Task implementation defined would also make it easier to clearly define the inputs and outputs of the process using one of the @Input or @Output Gradle annotations, allowing for incremental builds including JSLint execution. The full set of annotations available from the org.gradle.api.tasks package allow for combining File and/or simple property types to make your Tasks smarter regarding whether or not running again would produce a different result than the last execution.
This article was created using the recently released 0.9 version of Gradle and version 1.4.4 of jslint4java.

Learn more

If you'd like to find out some background or more about Gradle and how to create your own custom Plugins and Tasks, you can find some good information on these sites:

• The Gradle documentation at http://gradle.org/documentation.html
• In particular, the Project API is a great starting point for getting to know Gradle: http://gradle.org/latest/docs/javadoc/org/gradle/api/Project.html
• The excellent examples of Mr. Haki at http://mrhaki.blogspot.com/search/label/Gradle%3AGoodness
• Published plugins on github can be found with this search: https://github.com/search?type=Repositories&language=&q=gradle+plugin

Related posts:

1. Using Gradle to Bootstrap your Legacy Ant Builds
2. Why do I Like Gradle?
3. Groovy inspect()/Eval for Externalizing Data

    +----------------------+-------+-------+
    | Name                 | Files |  LOC  |
    +----------------------+-------+-------+
    | Controllers          |     1 |    42 |
    | Groovy Helpers       |     1 |    47 |
    +----------------------+-------+-------+
    | Totals               |     2 |    89 |
    +----------------------+-------+-------+

Please note that this is about as minimal as you could reasonably get away with(and kinda ugly to boot), but nevertheless it does manage to implement a UI and backend for exercising the entire API in a total of 2 gsp files and 1 controller action. What’s noticeably missing are some tailored views for each of the different JSON responses and the implementation of additional query parameters on the calls.

StackExchange API

This is really pretty well documented and “consistent”. Calls in some cases require an {id} in the url and that’s about it. Each of the individual calls has its own help page that describes the options, like the badges page for instance. To make things a little more helpful while exploring the API, each of the implemented endpoints in this app are also hyper-linked to the corresponding manual page on api.stackoverflow.com.

RESTful Access

Couldn’t be a whole lot easier than using HttpBuilder, which provides nice and concise ways to execute the GET request, inspect the response and deal with success/failure of the request. Aside from assigning the root domain and method using variables, this is little different from the canonical example on codehaus.

def http = new HTTPBuilder("http://api.$domain")
http.request(GET, JSON) {
    uri.path = "/${VERSION}$method"
    headers.'User-Agent' = 'Mozilla/5.0 Ubuntu/8.10 Firefox/3.0.4'
     response.success = { resp, json ->
        answer = json
    }
     response.failure = { resp ->
        answer = "Unexpected error: ${resp.statusLine.statusCode} : ${resp.statusLine.reasonPhrase}"
    }
}

The syntax here could probably get even Groovier using the Grails REST client plugin wrappers, and indeed incorporating that plugin is how the HttpBuilder dependency is being provided in this app, but I hit on a working implementation first time out so we’ll leave that for another day, shall we?

What’s it Look Like?

Did I mention it was ugly? Where the API call requires a parameter, you get a text field. Each domain gets its own submit button, officially representing my first attempted usage of g:actionSubmit in Grails. Doesn’t work quite the way I expected, but it does certainly work. Click a button and you’re shown the raw JSON result, along with the call you made. In an ugly fashion, or did I say that already? Still, for a couple of hours of hacking, I’m not unhappy with the result.

StackApps

This is where the apps in the contest get shown off and, in a wonderful display of dogfooding, it’s guess what – another StackOverflow clone! The page for this application can be found here. And the source code is up on github if you’d like to take a look.

Next Steps?

Toying with the idea of porting this to appengine, which would unfortunately mean replacing HttpBuilder and the underlying Apache HttpClient in favor of something that didn’t use a threaded approach, most likely using URL fetch. That and some slightly less ugly tabular and/or tree views of the JSON responses would be kinda nice to have.

Related posts:

1. Hooking into the Jenkins(Hudson) API
2. Grails-UI DataTable using XML for a model
3. Gource Visualizations of the Groovy/Grails/Griffon Projects

Where Hibernate Validator Shines

Annotations on domain classes allow for easily validating object state at the time of persistence. Excellent integration with frameworks like JBoss Seam allow this same ability to be utilized for validating web forms on the client-side with little more than an <s:validateAll/> tag. Seam practically hides the entire interaction with validation components from the developer. Since validation rules are defined directly in the domain class, you can (almost) guarantee that no objects with inconsistent state will ever end up being saved in your database. There are certain validations that aren’t possible to verify without actually looking in the database, unique constraints for example, but generally in my experience hibernate validator is extremely easy to configure and work with. Implementing CRUD functionality is pretty trivial, and UIs can achieve consistency since all validations are applied equally.
Alternatively, if you can’t or don’t want to use annotations for some reason, you can specify your validation rules in an xml file. Usually a singleton validation.xml file is made available on the classpath and picked up automagically when a ValidationFactory is created. A simple xml configuration looks like this:

<constraint-mappings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemalocation="http://jboss.org/xml/ns/javax/validation/mapping validation-mapping-1.0.xsd" xmlns="http://jboss.org/xml/ns/javax/validation/mapping">
    <default-package>org.kar.test</default-package>
    <bean class="ValidateTestableClass">
        <field name="name">
            <constraint annotation="javax.validation.constraints.NotNull">
        </constraint>
    </field>
</bean>

and is meant to be applied to this simple class:

class ValidateTestableClass
{
    int id
    String name
    String description
    boolean enabled
}

Comparing with Grails Validation

Grails automatically provides validation capabilities for domain classes and command objects, and enables adding the same behavior to any pogo through a combination of the @Validateable annotation and a static constraints closure. Adding validation support to arbitrary classes also requires specifying which packages to scan for the annotation.
Plugin support from projects like bean-fields simplifies the handling of client-side validation and rendering error markers in the UI, an ability which the Grails framework provides natively by adding an ‘errors’ field directly onto the domain or command object class instances bound to a web form.

Dynamic Constraints

Both the hibernate validator and the Grails strategies for applying validation described here have the same limitation: both are universally applied to all instances of a class. There’s no easy apparent way to override those constraints at runtime, although I suspect that some fancy MOP’ing or configuration could probably be used to accomplish overrides at runtime.
Hibernate validator also supports creation of ad hoc validators by seeding with one or more xml documents. Or if you’re like me and hate hand editing xml, you can leverage Groovy to take a bit of the pain away. Here’s the same xml snippet from above in a Groovy Closure, generated simply by turning the DomToGroovy class loose on the raw xml:

looseConstraint = {
        mkp.declareNamespace(xsi: 'http://www.w3.org/2001/XMLSchema-instance')
        'constraint-mappings'(xmlns: 'http://jboss.org/xml/ns/javax/validation/mapping',
                'xsi:schemaLocation': 'http://jboss.org/xml/ns/javax/validation/mapping validation-mapping-1.0.xsd') {
            'default-package'('org.kar.test.objects')
            bean('class': 'ValidateTestableClass') {
                field(name: 'name') {
                    constraint(annotation: 'javax.validation.constraints.NotNull')
                }
            }
        }
    }

Losing all the angle brackets is a good start, but we really haven’t saved a lot of typing. Until you start taking advantage of the ability to define more complicated structures. Note the use of a list structure here to apply NotNull constraints to multiple fields.

strictConstraint = {
        mkp.declareNamespace(xsi: 'http://www.w3.org/2001/XMLSchema-instance')
        'constraint-mappings'(xmlns: 'http://jboss.org/xml/ns/javax/validation/mapping',
                'xsi:schemaLocation': 'http://jboss.org/xml/ns/javax/validation/mapping validation-mapping-1.0.xsd') {
            'default-package'('org.kar.test.objects')
            bean('class': 'ValidateTestableClass') {
                ['name', 'description'].each {
                    field(name: it) {
                        constraint(annotation: 'javax.validation.constraints.NotNull')
                    }
                }
                field(name: 'id') {
                    constraint(annotation: 'javax.validation.constraints.DecimalMin') {
                        element(name: 'value', '2')
                    }
                }
                field(name: 'enabled') {
                    constraint(annotation: 'javax.validation.constraints.AssertTrue')
                }
            }
        }
    }

Applying Dynamic Constraints

Applying constraints is as simple as converting Closures to xml and mapping them to a Configuration object, which then supplies a Validator to use. StreamingMarkupBuilder is utilized to create the xml behind the scenes.

    /**
     * Create a configuration object passing closures as validation mapping documents.
     * @param closures closures to render into validation mapping documents
     * @return config
     */
    public Configuration createConfig(Closure... closures)
    {
        Configuration config = Validation.byDefaultProvider().configure()
        closures.each {
            config.addMapping(new ByteArrayInputStream(GroovyXmlConversionUtil.convertToXml(it).bytes))
        }
        config
    }

I haven’t tested the use of multiple mappings extensively, but minimally each class you’re configuring must be confined to a single mapping – you can’t extend the validations by layering configurations on top of one another. You should however be able to map constraints for different classes in separate Closures.

Crying out for a Builder!

Going from Closures to xml is a quick and dirty way to test out this functionality, but what would really be nice is a Builder that could create an appropriate validation environment more directly. At the least it would allow for removing the namespace declarations and explicit package naming that make up the bulk of the content.

So what do you get?

1. 1. Ability to declare validations against any existing Java or Groovy class without changing the source code
2. 2. Programmatic ability to create the configuration of validations
3. 3. A choice of which validations to apply at runtime
4. 4. Consistency with the behavior of domain class validation

And what’d I get? A fun bit of quick coding on a Saturday afternoon. Nice! Source code is available on git-hub if you want to check it out.

Related posts:

1. Groovy and CSV: How to Get Your Data Out?
2. Jira Grails Plugin
3. Breaking Weak CAPTCHA in… slightly more than 26 Lines of Groovy Code

The Pretty Pictures

Here are the three images to test against. In order to put them in a suitable format for the open source tesseract-ocr program to process we need to make them bigger, remove the background noise and transform them into a ‘tif’ format. The python program we’re porting utilizes the PIL library for image handling and the pytesseract library for wrapping tesseract; I didn’t look very hard for java equivalents and just coded the required functions directly.

Reading in the Image

The python code for this is three lines, one to load the image and a couple more to convert it into a format suitable for directly manipulating pixel color through RGB values. Groovy takes a bit more to do the same, but being able to use a ‘with’ block makes interacting with the Graphics object a lot cleaner than the same Java code

//python
from PIL import Image
img = Image.open('input.gif')
img = img.convert("RGBA")
pixdata = img.load()

//Groovy
BufferedImage image = ImageIO.read(new File(fileName))
BufferedImage dimg = new BufferedImage(image.width, image.height, BufferedImage.TYPE_INT_ARGB)
dimg.createGraphics().with {
    setComposite(AlphaComposite.Src)
    drawImage(image, null, 0, 0)
    dispose()
}

Removing the Background Noise

In both cases we’re doing essentially the same thing: finding all non-black pixels and setting them to white. This leaves only the actual embedded text to stand out. Being able to utilize the Java Color constants makes the Groovy version a little more readable, IMO, but otherwise the two pieces of code are generally equivalent.

//python
for y in xrange(img.size[1]):
    for x in xrange(img.size[0]):
        if pixdata[x, y] != (0, 0, 0, 255):
            pixdata[x, y] = (255, 255, 255, 255)

//Groovy
(0..<dimg.height).each {i="" -="">
    (0..<dimg.width).each {j="" -="">
        if (dimg.getRGB(j, i) != Color.BLACK.RGB)
        {
            dimg.setRGB(j, i, Color.WHITE.RGB)
        }
    }
}

Resizing the Image

Python‘s library usage really shines here, making this a one line call. Not quite the same in Java-land, although again there’s probably a better way to do this(I just don’t know it offhand).

//python
big = im_orig.resize((116, 56), Image.NEAREST)

//Groovy
dimg = resizeImage(dimg, 116, 56)
...
def resizeImage = {BufferedImage image, int w, int h ->
    BufferedImage dimg = new BufferedImage(w, h, image.type)
    dimg.createGraphics().with {
        setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR)
        drawImage(image, 0, 0, w, h, 0, 0, image.width, image.height, null)
        dispose()
    }
    return dimg
}

By this point the original images now look like this, and are almost ready for OCR.

Converting to a tif File

This one turns out to be a bit of a PITA in Java and particularly on a Mac, and represents the bulk of the Groovy code. Unfortunately it is also the only format that tesseract appears to accept ‘out of the box’. After googling the fun that is JAI and working with the .tif(f) format with it on a Mac, I ended up taking the code kindly provided in this blog post and Groovified it a bit to make a working transformation. Thanks very much to Allan Tan for that. One more time, there’s likely a better/easier way to do this, but honestly it’s more effort than I’m willing to put in on a weekend afternoon just to satisfy my curiosity.

//python
ext = ".tif"
big.save("input-NEAREST" + ext)

//Groovy
void convertToTiff(String inputFile, String outputFile)
{
    OutputStream ios
    try
    {
        ios = new BufferedOutputStream(new FileOutputStream(new File(outputFile)))
        ImageEncoder enc = ImageCodec.createImageEncoder("tiff", ios, new TIFFEncodeParam(compression: TIFFEncodeParam.COMPRESSION_NONE, littleEndian: false))
        RenderedOp src = JAI.create("fileload", inputFile)

        //Apply the color filter and return the result.
        ColorConvertOp filterObj = new ColorConvertOp(ColorSpace.getInstance(ColorSpace.CS_sRGB), null)
        BufferedImage dst = new BufferedImage(src.width, src.height, BufferedImage.TYPE_3BYTE_BGR)
        filterObj.filter(src.getAsBufferedImage(), dst)

        // save the output file
        enc.encode(dst)
    }
    catch (Exception e)
    {
        println e
    }
    finally
    {
        ios.close()
    }
}

OCR with Tesseract-OCR

Finally we need to pass the processed image to tesseract so it can ‘read’ it for us. Again, the python library makes this a breeze, but calling out to a command line program with Groovy is so simple that it ends up being about the same. Tesseract itself is available as a macport, as well in downloadable unix binaries and a windows executable so installing the software is a breeze.

//python
from pytesser import *
image = Image.open('input-NEAREST.tif')
print image_to_string(image)

//Groovy
def tesseract = ['/opt/local/bin/tesseract', tmpTif, tmpTesseract].execute()
tesseract.waitFor()
return new File("${tmpTesseract}.txt").readLines()[0]

Testing it out

To test it out I implemented the code in a maven project, iterate over the images and write out intermediate results to a temp directory. And it only works on two out of three of the cases. For some reason tesseract insists on consistently seeing ‘e4ya’ as ‘e4ga’. I tried to see if I could get it working by tweaking the image manipulation parameters and the order of operations(resizing before removing the background noise for instance) but that just caused the other cases to fail as well. Since in the final image the ‘y’ seems pretty clear, it’s more likely that tweaking tesseract configuration might yield better results.

public void testPrintImage()
{
    def breaker = new CaptchaBreaker()
    /* tesseract interprets "e4ya" as "e4ga" unfortunately */
    ['9koO', 'jxt9'/*,'e4ya'*/].each {String imageName ->
        def fileName = "src/test/resources/${imageName}.gif"
        assertEquals("Testing $imageName",imageName, breaker.imageToString(fileName))
    }
}

C’est Finis

I had some fun playing with areas of Java that I don’t usually interact with, and gained some appreciation for the diversity and ease-of-use exposed by just a couple of python libraries. It’s comforting to note that I was able to implement all of the required functionality from those libraries in < 90 lines of Groovy. With a little more effort I think the final product could be tweaked to avoid the intermediate file system reads/writes as well, but that’s for another day.
Source code is available on github if you’d care to take a look, and thanks for stopping by!

Related posts:

1. Different Flavors of Embedded Groovy in Java Apps or “How To Make your Java Groovier!”
2. Achieving Groovy-like Fluency in Java with Google Collections
3. Groovy and Hibernate Validator for Dynamic Constraints

The slideware plugin provides a framework for creating presentations with a little twist – you can execute code live from the presentation software. If you’ve ever given a presentation about programming you probably are fully aware of the transition from presentation software to your preferred environment for demonstrating code samples. Well, now you can stay entirely within the same application, editing and running code live.

The plugin itself isn’t available from the Griffon repository but source code and a built 0.2 version can be found on github. Considering both the youth of the plugin and the beta status of the framework, it worked impressively well AND had a rich feature set. All of the expected “powerpoint” features are there: themes, layout control, styling, slide transitions and export are all pretty easy to incorporate and configure. The code editor view also works very well. A great variety of additional plugins are harnessed to put all the pieces together and as a result building one of these applications is a great way to tour the platform.

Theming

Applying a theme to the presentation is simply selecting a Java Look and Feel to apply in Initialize.groovy. The Substance jar is included with the plugin so I test drove a few of the nice setups in there and finally settled on the SubstanceMagmaLookAndFeel. There is definitely a wide variety of L&F’s to choose from in that bundle alone and, although I haven’t done it myself, they seem pretty tweakable as well. Plus any old L&F should plug in nicely, I would imagine.

//Initialize.groovy
SwingBuilder.lookAndFeel('org.jvnet.substance.skin.SubstanceMagmaLookAndFeel',
       'mac', 'nimbus', 'gtk', ['metal', [boldFonts: false]])

Layout

Controlling the page composition is a standard Swing Layout or, in the case of the default slide you get with the included “create-slide” script, a MigLayout. Framing the standard variety of slides is very simple. Bulleted pages, title slides, code slides and custom layouts are very easy to accomplish. I don’t have a lot of experience using this particular layout but the presentations Andres has made available on github have a good diversity of examples of how they look in practice.

//the default create-slide generated template
import net.miginfocom.swing.MigLayout

slide(id: "slide0", layout: new MigLayout("fill","[center]","[center]")) {
    label("Insert your text here")
}

Styling

Styling is supplied by the css plugin, on which slideware has a dependency. The default style.css file sets out just some reasonable defaults for the fonts used in different parts of the app, and I didn’t see any real need to fiddle with it. Especially happy to see the nice monospace code font. On a totally related note I recently installed the Inconsolata monospace font to try for development and I’ve been very happy seeing it in my editor, but it’s still nice to see this kind of polish applied to the presentation. The code editor view even includes syntax highlighting! More on that coming right up…

Slide Transitions

Moving between slides with style is the responsibility of the transitions plugin. You can see see all of the animations this plugin enables over here at this page describing Transitions and Transition2Ds. Pretty slick stuff and defined as simply as a parameter to each “slide” node in a script.

slide(id: "slide3", layout: new MigLayout("fill","3%[center]3%","3%[center]3%"),
        title: "Junit3",
        transition: new FlurryTransition2D(Transition2D.OUT)) {
    scrollPane(constraints: "grow") {
        widget(createEditor(text: script))
    }
}

Code Editor

Code slides embed an editable widget and allow for composing and executing Groovy scripts or classes of arbitrary complexity. You can execute the code with a keyboard shortcut and a window will open displaying the console output. The code itself is executed through a GroovyShell, enabling pretty much anything you might want to do. If you have an internet connection and the required repositories configured, Grapes simplifies packaging the dependencies for the application as your code samples can directly Grab the jars they need. Basically, the first time you execute a code slide you’ll have to put up with a small pause while Ivy downloads to your local repository, unless of course the required dependencies are already there. In my case I changed strategies from including jars in the application lib directory to a Grapes approach and I think it’s a better way to go.

I was surprised to find that the editor even included undo functionality. It’s definitely not close to a full blown IDE, and there’s absolutely no reason that it should be. For the task of demoing simple code examples it’s more than up to the task, even to the point of maintaining your edits between slide transitions, allowing you to move back and forth through the slide deck without any state problems.

Export

The application includes a “Print” feature which iterates through the entire slide deck and renders it to a pdf. Distribution of one of these presentations is really very easy, including the ability to create installers for all major platforms simply by adding the packaging plugin.

Test Application

The test app I built while looking into this is pretty simple. It’s got a title slide, a bullet slide and 4 code slides. The code simply demonstrates how you can create test classes and make assertions for Junit 3(with GroovyTestCase), Junit 4 and TestNG. GroovyShell recognizes all 3 of these test files by interface or annotation and executes them appropriately. In each case the console output of the test framework is the result, including the new Spock inspired ascii art assert failure renderings. The Test result files are also written to disk, and that TestNG html output is what I’m used to looking at anyhow. Show me the Green!

The last code slide is just a slightly updated version of an example script on the Grapes page, using the current version of Google Collections and intentionally introducing a failure- mostly just to show off that new assert rendering I mentioned a moment ago. VERY helpful at highlighting the exact nature of a failure. It also encourages me to pay more attention to how I name variables, something I’m sure every developer that has ever worked with me will cheer at. Man, I suck at naming things.

I also developed a brute force test that loads each slide and executes scripts if finds embedded there. Failures are hard to detect since the direct output is simply a text block, but some fairly simple regex’s applied to the output make me at least moderately confident that the code won’t fail at show time.

Overall Impression

It took a couple of afternoons(six hours or so total) to download the source code from github, explore it, create a simple presentation and document the experience. I won’t begin to suggest that I’m fully aware of all the details happening behind the scenes, but the end user experience is pretty fluid: create a slide, tailor the layout, add content and then repeat. The included examples were more than enough documentation on how to hit the ground running. The plugin code itself is a great example of the MVC nature of Griffon, not a whole lot of code, but a great deal of power and expressability. There were a couple of glitches happening in the background, mostly just logging to the console with no visible effect to the application, but overall it functioned as well as (not) advertised. For publicly unreleased software it was an absolute pleasure to work with and I plan on continuing with the development of this particular presentation.

Everything you need to build and run this stuff yourself is publicly available. In my case, Griffon generally has a recent Macport available for both the released(griffon @0.2.1) and development versions(griffon-devel @0.3-BETA-2). Switching versions is relatively painless and, for applications this simple, testing out upgrades is basically just going through the presentation once in a functional test. Versions for other platforms can be downloaded from the Griffon download page.

Deliverables

Source code for the sample application is available on github here. Please just leave a comment on this page if you have any problems running it.

Here’s the pdf produced by the application ‘Print’ feature: Groovy Testing Presentation (251)

And if you don’t feel like downloading anything, here’s how it all looks in pretty pictures.

[Show as slideshow]

[View with PicLens]

Related posts:

1. VIJUG Griffon/Groovy Presentation
2. Achieving Groovy-like Fluency in Java with Google Collections
3. Fun day playing with new Stuff

Groovy on the command line

The quickest and simplest way to run a Groovy Script or Class, command line arguments are automatically marshalled into an ‘args’ String array. Please note that due to a problem I’m having with my syntax highlighter plugin the process execution is shown here in single quotes; the actual code requires a GString(double quoted) in order to do the replacement for the inline variable. " THAT WordPress!

//the script
myArgs = args
result = args.join(' ')
println result
println myArgs

//...and the test
    void testGroovyCall()
    {
        def proc = 'groovy $groovyScriptOne Hello World'.execute()
        proc.waitFor()
        def result = proc.text.split()
        assert result[0] == 'Hello'
        assert result[1] == 'World'
    }

GroovyShell

This is the basis of Groovy script execution. The GroovyShell allows for executing scripts, passing in a particular Binding context that allows for bi-directional communication between the script and the calling code. Parameters can be passed into the executing script in the Binding and results can be stored there to be returned to the calling context. GroovyShell also allows for running a class from the ‘main’ method, passing in String arguments. It will also execute implementers of Runnable and test files for JUnit or TestNG. Script text can also be declared inline and executed in the same way as files on disk. All in all, pretty bloody handy. Here’s a straightforward example of running a dirt simple Groovy script and inspecting the results. Note that this isn’t executable as shown, but I’ll provide the full source code on github for anyone who wants a closer look. Note that I’m also passing in an ‘out’ variable in the Binding, which effectively redirect System.out to a specified Writer implementation – a nice touch for inspecting output.

//the script
myArgs = args
result = args.join(' ')
println result
println myArgs

 //...and the test
    void testGroovyShell()
    {
        Binding binding = helper.createBinding()
        def shell = new GroovyShell(binding)
        shell.evaluate(new File(groovyScriptOne))
        helper.assertBinding(binding)
    }

//...and the Binding creation/assertion
     def static args = ['Hello', 'World'].asImmutable()
     /**
     * Create a Binding with a single parameter to be passed to scripts and an 'out' Writer to redirect console output.
     */
    private Binding createBinding()
    {
        Binding binding = new Binding()
        def sWriter = new StringWriter()
        def pWriter = new PrintWriter(sWriter)
        binding.setVariable ('args', new ArrayList(args))
        binding.setVariable ('out', pWriter)
        return binding
    }

    /**
     * Assert that the expected 'common' actions are done with the Binding by each of the use cases.
     * The original 'args' should be as expected.
     * A copy of 'args' should have been placed in the Binding during execution.
     * The 'result' should be the concatentation of 'args' separated by spaces.
     */
    private def assertBinding(Binding binding)
    {
        assert binding.variables.size() == 4
        assert binding.variables.args.value[0].toString() == args[0]
        assert binding.variables.args.value[1].toString() == args[1]
        assert binding.variables.result.value.toString() == args.join(' ')
        assert binding.variables.myArgs.value[0].toString() == args[0]
        assert binding.variables.myArgs.value[1].toString() == args[1]
    }

GroovyScriptEngine

The GroovyScriptEngine enables dynamically running Groovy sources located in a fixed set of content roots, complete with reloading modified scripts in between executions. Running a Groovy script this way is essentially the same as using GroovyShell.

    void testGroovyScriptEngine()
    {
        Binding binding = helper.createBinding()
        def gse = new GroovyScriptEngine(new File('.').toURL())
        gse.run(groovyScriptOne, binding)
        helper.assertBinding(binding)
    }

GroovyClassLoader

An extension to URLClassLoader that enables parsing Groovy sources into Class representations. Once a Class object is created, instances of the class can be created easily and either cast to a known type or manipulated through convention by use of the standard Groovy ‘invokeMethod’. This works equally well on Groovy and Java btw. Here’s an example of running a Java class using GroovyClassLoader. In this case the Java file has a field called ‘binding’ and implements a ‘run’ method.

    /**
     * Dynamically compile, instantiate, inspect and call methods on a POJO.
     */
    void testGroovyClassLoaderOnJava()
    {
        GroovyClassLoader loader = new GroovyClassLoader();
        Class javaClass = loader.parseClass(new File(javaFileOne));

        def groovyObject = javaClass.newInstance();
        def binding = helper.createBinding()
        groovyObject.binding = binding
        if(groovyObject.metaClass.respondsTo(groovyObject, 'run'))
        {
            groovyObject.invokeMethod('run', null);
            helper.assertBinding(binding)
        }
        if(groovyObject.metaClass.respondsTo(groovyObject, 'main'))
        {
            groovyObject.invokeMethod('main', new ArrayList(helper.args) as String[]);
        }
    }

(Groovy)Console

The Console can be embedded in Java or Groovy code to provide a dynamic interactive Swing environment. This is the same UI spawned from the command line invocation of ‘groovyConsole’. Internally it uses GroovyShell for actual execution, and so can do everything that GroovyShell can do – plus a couple of additions. For one, you can add jars and/or directories to the classpath used when executing your scripts.

The Best of Both Worlds – at Least for my use case

In actual practice these patterns can be used a lot more successfully by observing standard Java practices, like casting classes parsed using GroovyClassLoader to a known interface before interacting with them, or by using Classes to organize business logic inside of a Script that essentially functions as a ‘main’ method. This example defines two dependent internal classes, marshals parameters to them and then returns the results attached to the originally passed in Binding.

/**
 * Classes inside of a Script.
 */
class TestableClass
{
    Binding binding

    def run()
    {
        binding.with
        {
            setVariable('myArgs', getVariable('args'))
            setVariable('result', getVariable('args')?.join(' '))
        }
        return binding
    }
}

class TestableClass2
{
    Binding binding

    public TestableClass2(Binding binding)
    {
        this.binding = binding;
    }

    def run()
    {
        return new TestableClass(binding: binding).run()
    }
}

if (args)
{
    def internalBinding = new Binding()
    internalBinding.setVariable('args', new ArrayList(args))
    internalBinding = new TestableClass2(internalBinding).run()
    args = internalBinding.args
    myArgs = internalBinding.myArgs
    result = internalBinding.result  //return value from script
}
else
{
    println 'no args!!'
}

Summary of Groovy Execution Methods

Related posts:

1. Achieving Groovy-like Fluency in Java with Google Collections
2. Groovy CliBuilder with multiple arguments
3. Breaking Weak CAPTCHA in… slightly more than 26 Lines of Groovy Code