I manage a large number of builds at work, spread across several build servers. When we release a new version all of the builds need to be updated to point to new working branches. This script takes advantage of the fact that our branches all end in the version number to quickly check that all of the last builds were on the expected version.
Using the Jenkins API is very easy and the new capabilities of Groovy introduced by JSONSlurper make it easier than ever to consume.

import groovy.json.JsonSlurper

assert args && args.size() == 2
def urls = args[0].split(',')
def expectedVersion = args[1]
urls.each { url ->
    println "examining url $url"
    def json = new JsonSlurper().parseText(url.toURL().text)

    json.jobs.each {
        try
        {
            def job = new JsonSlurper().parseText("${it.url}/api/json?depth=1".toURL().text)
            if (!job.builds[0].changeSet.revisions[0].module.endsWith(expectedVersion))
            {
                println "$it.url fails!"
            }
            else
            {
                println "$it.url passes!"
            }
        }
        catch (e)
        {
            println "Exception thrown processing $it.url : $e"
        }
    }
}

2. Look up the artifacts from the last successful Jenkins build

This is very handy if you want to automate deployment of software from your build system and can replace a lot of error prone manual updating of scripts. In this particular example you can link up quickly with the latest, greatest build of JRuby from their public CI server.

import groovy.json.*
def HOSTNAME = 'http://ci.jruby.org'
def JOBNAME = 'jruby-dist'
def JOB_URL = "$HOSTNAME/job/$JOBNAME/lastSuccessfulBuild"
def text = "$JOB_URL/api/json".toURL().text
println JsonOutput.prettyPrint(text)
def json = new JsonSlurper().parseText(text)
json.artifacts.each{
    println it
}

[relativePath:dist/jruby-bin-1.7.0.dev.tar.gz, fileName:jruby-bin-1.7.0.dev.tar.gz, displayPath:null]
[relativePath:dist/jruby-bin-1.7.0.dev.tar.gz.md5, fileName:jruby-bin-1.7.0.dev.tar.gz.md5, displayPath:null]
[relativePath:dist/jruby-bin-1.7.0.dev.tar.gz.sha1, fileName:jruby-bin-1.7.0.dev.tar.gz.sha1, displayPath:null]
[relativePath:dist/jruby-bin-1.7.0.dev.zip, fileName:jruby-bin-1.7.0.dev.zip, displayPath:null]
[relativePath:dist/jruby-bin-1.7.0.dev.zip.md5, fileName:jruby-bin-1.7.0.dev.zip.md5, displayPath:null]
[relativePath:dist/jruby-bin-1.7.0.dev.zip.sha1, fileName:jruby-bin-1.7.0.dev.zip.sha1, displayPath:null]
[relativePath:dist/jruby-complete-1.7.0.dev.jar, fileName:jruby-complete-1.7.0.dev.jar, displayPath:null]
[relativePath:dist/jruby-complete-1.7.0.dev.jar.md5, fileName:jruby-complete-1.7.0.dev.jar.md5, displayPath:null]
[relativePath:dist/jruby-complete-1.7.0.dev.jar.sha1, fileName:jruby-complete-1.7.0.dev.jar.sha1, displayPath:null]
[relativePath:dist/jruby-jars-1.7.0.dev.gem, fileName:jruby-jars-1.7.0.dev.gem, displayPath:null]
[relativePath:dist/jruby-jars-1.7.0.dev.gem.md5, fileName:jruby-jars-1.7.0.dev.gem.md5, displayPath:null]
[relativePath:dist/jruby-jars-1.7.0.dev.gem.sha1, fileName:jruby-jars-1.7.0.dev.gem.sha1, displayPath:null]
[relativePath:dist/jruby-src-1.7.0.dev.tar.gz, fileName:jruby-src-1.7.0.dev.tar.gz, displayPath:null]
[relativePath:dist/jruby-src-1.7.0.dev.tar.gz.md5, fileName:jruby-src-1.7.0.dev.tar.gz.md5, displayPath:null]
[relativePath:dist/jruby-src-1.7.0.dev.tar.gz.sha1, fileName:jruby-src-1.7.0.dev.tar.gz.sha1, displayPath:null]
[relativePath:dist/jruby-src-1.7.0.dev.zip, fileName:jruby-src-1.7.0.dev.zip, displayPath:null]
[relativePath:dist/jruby-src-1.7.0.dev.zip.md5, fileName:jruby-src-1.7.0.dev.zip.md5, displayPath:null]
[relativePath:dist/jruby-src-1.7.0.dev.zip.sha1, fileName:jruby-src-1.7.0.dev.zip.sha1, displayPath:null]

3. Read in a CSV file, filter it, and write out the result

Groovy Grapes and one of my favorite libraries, OpenCSV, make it trivial to script ‘one off’ solutions for removing work that would normally take place manually in Excel. For me it comes up most often with data import/export from systems I’m working on and the need to create particular test environments using those import/export mechanisms. For example, using a scripted solution like this you can quickly:

• Export users from one test system into a CSV file, along with local phone number contact data
• Replace the existing phone numbers with a mix of local and international numbers
• Import the new data into a different test system
• Verify that the system properly deals with international calling concerns

This example simply trims out all of the rows where the second column of input contains values less than 100. It’s also nice to note that OpenCSV is smart enough to let you ignore one or more ‘header rows’, a common attribute of CSV files.

@Grab(group = 'net.sf.opencsv', module = 'opencsv', version = '2.3')
import au.com.bytecode.opencsv.CSVReader
import au.com.bytecode.opencsv.CSVParser
import au.com.bytecode.opencsv.CSVWriter

def TEST_FILE_NAME = 'test.csv'
def TEST_OUTPUT_FILE_NAME = 'testOut.csv'

List<String[]> rows = new CSVReader(new FileReader(new File(TEST_FILE_NAME)), CSVParser.DEFAULT_SEPARATOR, CSVParser.DEFAULT_ESCAPE_CHARACTER, CSVParser.DEFAULT_QUOTE_CHARACTER, 1).readAll()
def rowsOver100 = rows.findAll {it[1].toInteger() > 100}

File output = new File(TEST_OUTPUT_FILE_NAME)
if (output.exists())
{
    output.delete()
}
output.createNewFile()
output.withWriter { writer ->
    new CSVWriter(writer).writeAll(rowsOver100)

}

//println rows

4. Download, install and run a stand-alone webserver

This example comes courtesy of a recent article by Andrew Glover about Gretty, a lightweight http server. I don’t have a concrete use-case for this right now, but it’s easy to note that in this case you can code a large part of the server behaviour directly in the script, unlike, for instance, if you were to use a script to stand up a Jetty server. This should let you publish a script once in a central location and stand up as many Gretty servers providing the declared services as you like. This particular example simply echos back ‘Hello’ plus whatever path you append to localhost:8080.

import org.mbte.gretty.httpserver.* 

@GrabResolver(name='gretty', 
  root='http://groovypp.artifactoryonline.com/groovypp/libs-releases-local')
@Grab('org.mbte.groovypp:gretty:0.4.279') 

GrettyServer server = [] 
server.groovy = [ 
    localAddress: new InetSocketAddress("localhost", 8080), 
    defaultHandler: { 
        response.redirect "/" 
    }, 
    "/:name": {
        get {
            response.text = "Hello ${request.parameters['name']}"
        } 
    } 
] 
server.start()

5. Remotely execute any of these scripts by url

As of Groovy 1.8.3/1.9-beta-4 you can refer to a Groovy script hosted at a remote url and execute it locally. If you work on different machines a lot(as I do) this can really help keep your toolbox of scripts handy, regardless of where you are. Since these scripts are publicly hosted on GitHub(thanks guys!), you can simply copy the url from the ‘view raw’ link at the bottom and go to your console to execute them like this:

$ groovy https://gist.github.com/raw/1430856/cea0186a862217015fb4fc2b63eb0ad3575c06fc/jenkinsBuildArtifacts.groovy
{
    "actions": [
        {
            "causes": [
                {
                    "shortDescription": "Started by timer"
                }
            ]
        },
        {
            "buildsByBranchName": {
                "origin/master": {
                    "buildNumber": 684,
                    "buildResult": null,
                    "revision": {
                        "SHA1": "3419fee4bb9436d3222ff3df8dd7fcf2308a6919",
                        "branch": [
                            {
                                "SHA1": "3419fee4bb9436d3222ff3df8dd7fcf2308a6919",
                                "name": "origin/master"
                            }
                        ]
                    }
                }
            },
... and so on

Final note

Being able to share scripts across a variety of machines has definitely made my professional life a lot easier and I’m very glad to see the ability to execute code from a URL in Groovy. But what really turned me onto the idea was its usage in Gradle, which supports loading remote Groovy/Gradle scripts using the “apply from: {url or file}” syntax. I find that more and more I reference particular tasks remotely in a build, especially for static analysis tools that don’t necessarily need to be run on a regular basis.
A great example of this is the recently released Gradle Templates Plugin, which is basically the Gradle answer to the Maven Archetype plugin. Create a build.gradle with the following content:

apply from: 'http://launchpad.net/gradle-templates/trunk/1.2/+download/apply.groovy'

And we have the following tasks available to create new Gradle projects and objects with conventional structure:

Template tasks
--------------
createGradlePlugin - Creates a new Gradle Plugin project in a new directory named after your project.
createGroovyClass - Creates a new Groovy class in the current project.
createGroovyProject - Creates a new Gradle Groovy project in a new directory named after your project.
createJavaClass - Creates a new Java class in the current project.
createJavaProject - Creates a new Gradle Java project in a new directory named after your project.
createScalaClass - Creates a new Scala class in the current project.
createScalaObject - Creates a new Scala object in the current project.
createScalaProject - Creates a new Gradle Scala project in a new directory named after your project.
createWebappProject - Creates a new Gradle Webapp project in a new directory named after your project.
exportAllTemplates - Exports all the default template files into the current directory.
exportGroovyTemplates - Exports the default groovy template files into the current directory.
exportJavaTemplates - Exports the default java template files into the current directory.
exportPluginTemplates - Exports the default plugin template files into the current directory.
exportScalaTemplates - Exports the default scala template files into the current directory.
exportWebappTemplates - Exports the default webapp template files into the current directory.
initGradlePlugin - Initializes a new Gradle Plugin project in the current directory.
initGroovyProject - Initializes a new Gradle Groovy project in the current directory.
initJavaProject - Initializes a new Gradle Java project in the current directory.
initScalaProject - Initializes a new Gradle Scala project in the current directory.
initWebappProject - Initializes a new Gradle Webapp project in the current directory.

Related posts:

1. Using Gradle to Bootstrap your Legacy Ant Builds
2. Hooking into the Jenkins(Hudson) API
3. Groovy inspect()/Eval for Externalizing Data

• Groovy syntax and a very terse and descriptive dsl that makes build scripts easily comprehensible
• flexibility of layout, configuration, organizing build logic – pretty much everything
• incremental builds, based on an easily implementable pattern
• convention over configuration paradigm, thank you very much Maven
• clear separation of build configuration from execution
• extensibility at every level

The most basic Java build

apply plugin: 'java'

That’s it. One line of Groovy in a file called ‘build.gradle’ and you can build a Java project with a Maven-standardized project layout.

├── build.gradle
└── src
    ├── main
    │   ├── java
    │   └── resources
    └── test
        ├── java
        └── resources

Included with the Java plugin are tasks to compile, package, test and javadoc your code. You also get configuration objects to describe the artifacts that your build depends on and those that it produces. Of course, with this most basic setup these configurations don’t yet have anything in them, but in a complex build they’re very handy for isolating the responsibilities of each task. Because you can both configure the existing configurations and add custom ones yourself, it’s very easy to accommodate a project that follows a different structure, whether that is just differently named source directories or multiple directories that need specific processing. This is VERY handy for legacy builds.

Incremental builds

Gradle provides a very easy way to create tasks that are able to execute only if their declared input and/or output artifacts have changed. This makes it trivial to incorporate your custom build behaviour into an incremental build. As an example I’d like to expand upon something I read by Etienne Studer in this month’s JAXmag. It’s a great example of developing an incremental task. First here’s the task implementation almost verbatim from the article. I’ve updated it slightly to make the output more readable using FileUtils, and you can examine the file it spits out from the build/reports/size directory that will be automatically created when it executes.

class Size extends DefaultTask
{
    @InputFiles FileTree inputDir
    @OutputFile File outputFile

    @TaskAction
    void generate()
    {
        def totalSize = inputDir.files.inject(0) { def size, File file -> size + file.size()
        }
        outputFile.text = FileUtils.byteCountToDisplaySize(totalSize)
    }
}

If you simply include this class definition in a build file, it will be automatically compiled and available for use elsewhere in the script. It could just as easily be defined in a separate file(local or remote), in a jar on the classpath or in the buildSrc directory of your Gradle project. This flexibility enables developing and evolving tasks in a very agile fashion, encouraging you to publish the results for reuse instead of re-implementing the logic in other projects.
In order to execute this task as part of a build, we first need to configure it. In this case, I’m configuring it to work on all declared configurations and all source, both code and associated resources. In order to do so, I’m using a couple of Gradle internal classes, FileTree and SourceSet, which actually sound pretty self-explanatory to me. The key part is the assignment of the ‘inputDir’ and ‘outputFile’ properties on the task.

task size(type:Size){
    def filetree = sourceSets.inject(new UnionFileTree()) { FileTree total, SourceSet sourceSet ->
        total += sourceSet.allSource
        total
    }
    inputDir = filetree
    outputFile = file("$reportsDir/size/size.txt")
}

Just to prove that it’s working incrementally, here’s the output from successive invocations. Note that ‘UP-TO-DATE’ in the output that indicates the task was skipped the second time around, because none of the inputs changed and the output file hasn’t been deleted.

gradle-intro$ gradle size
:size

BUILD SUCCESSFUL

Total time: 2.963 secs

gradle-intro$ gradle size
:size UP-TO-DATE

BUILD SUCCESSFUL

Total time: 2.912 secs

This task does actually have a concrete dependency on the Java plugin, since without that convention applied neither the ‘sourceSets’ or ‘reportsDir’ objects would be present. Here’s the complete version of the build file that’s been built so far, 28 lines including imports and spacing.

import org.apache.commons.io.FileUtils
import org.gradle.api.internal.file.UnionFileTree
import org.gradle.api.tasks.SourceSet

apply plugin: 'java'           

task size(type:Size){
    def filetree = sourceSets.inject(new UnionFileTree()) { FileTree total, SourceSet sourceSet ->
        total += sourceSet.allSource
        total
    }
    inputDir = filetree
    outputFile = file("$reportsDir/size/size.txt")
}

class Size extends DefaultTask
{
    @InputFiles FileTree inputDir
    @OutputFile File outputFile

    @TaskAction
    void generate()
    {
        def totalSize = inputDir.files.inject(0) { def size, File file -> size + file.size()
        }
        outputFile.text = FileUtils.byteCountToDisplaySize(totalSize)
    }
}

Sharing your new Task

The simplest way to share build logic is to ‘apply’ it. This simple shorthand covers everything from plugins to local files to remotely hosted resources. Here’s how easy it is to incorporate this particular task implementation and configuration into a build using a relative file path.

apply from : '../gradle-intro/build.gradle'

And because only a simple http connection is required to share it to a broader base, here’s how you can reference a copy on this site. Sorry about the .txt extension, but I didn’t feel like editing php to allow a new filetype today

apply from : 'http://www.kellyrob99.com/blog/wp-content/uploads/downloads/2010/11/gradleSizeTask.txt'

Some other reading

If you’re still not sold on Gradle, here’s some articles I’ve seen recently that at the very least will give you a better perspective.
Hibernate – Gradle why?
Maven VS Gradle VS Ant
Maven to Gradle: Part 1, Part 2, Part 3
A comparison of build script length
Ant/Gradle/Maven comparison
DZone article
OpenMRS Mailing list on Maven VS Gradle

Related posts:

1. Using Gradle to Bootstrap your Legacy Ant Builds
2. A Groovy/Gradle JSLint Plugin
3. Groovy inspect()/Eval for Externalizing Data

CSV is Not XML

Or any other suitably constrained data format for that matter. There are some pretty clear ‘rules’ out there for transporting data in this format, but of course hardly anybody listens to them. I found this and this in the first page of a quick Google search.
However, while looking for available sample material on the web to demonstrate with, I found many great examples of how to screw it up: not quoting empty strings, not escaping embedded line breaks, commas embedded in unquoted text, etc. All of these are things that prevent automation from properly interacting with the data. There are also, of course, the CSV ‘formats’ that insist on embedding comments and descriptions, trying to present more than one matrix of information or any of the other crazy things you can do when writing to a format with no real metadata layer.

< \rant>

Unfortunately, there’s nothing as convenient as an xml schema available for describing what information is encoded in the document and, in general, the only potentially useful metadata in an individual file is the header line. The good news is that all of these deficiencies can be overcome if you know they exist. And there’s some pretty nifty tools you can use to take out much of the guesswork. I’ll suggest a couple of ideas for how to deal with files that don’t adhere to the ‘rules’, but the main example I’ve decided on is a straightforward and well formed set representing the periodic table of elements, provided generously by Akiscode. This file is directly machine readable(has no headers or embedded data), quotes all values and just follows all the ‘rules’. Here’s the first five lines as an example.

Atomic Number	Atomic Mass	Name	Symbol
1	1.00794	Hydrogen	H
2	4.002602	Helium	He
3	6.941	Lithium	Li
4	9.012182	Beryllium	Be
5	10.811	Boron	B

     The simple format of the example CSV file.

Groovy All by Itself

Groovy makes dealing with String objects pretty painless. By adding facilities like easy casting and find()/findAll() for quickly turning raw Strings into real data, you can program some fairly complex questions. In this simple example I’m depending on all values being quoted and I am not protecting against casting problems. I’m running this through Maven which really helps to keep test data organized; all you have to do is drop your test file in /src/test/resources and it’s automagically available on the test classpath.
No specific handling is done for different types here; everything is read in as a String and cast to a more specific type as needed. In a ‘real’ application you’re very likely going to encounter mixed alphanumeric data in any given column of data – ‘UNKNOWN’ instead of a number in a column labeled ‘Quantity’ for instance – and be more careful about casting.

//load and split the file
InputStream inputFile = getClass().classLoader.getResourceAsStream(TEST_FILE_NAME)
String[] lines = inputFile.text.split('\n')
List<String[]> rows = lines.collect {it.split(',')}

/**
 * A little helper method to get rid of the quotes in the input
 * and cast values so they can be compared properly.
 */
private double castToDouble(string)
{
    return string.replaceAll('"', '').toDouble()
}

//OK, it's parsed - let's ask some questions
private static final int ATOMIC_MASS = 1;

def elementsOver200Mass = rows.findAll {castToDouble(it[ATOMIC_MASS]) > 200}
def elementsBetween10And20 = rows.findAll { row ->
            double mass = castToDouble(row[ATOMIC_MASS])
            mass <= 20 && mass >= 10
}

So this approach works fine for well formed input, but falls apart quickly in other cases. For instance, if any columns in the file are missing values the split() function treats them as nulls, leading to different size arrays being stored in the list. Similarly you need to deal with quoted VS non-quoted content, embedded line breaks and other issues individually. Fortunately, some nice guys have done that for us.

OpenCSV

Glen Smith’s OpenCSV library nicely abstracts away a lot of the minutiae of dealing with CSV. It properly deals with escaped characters, multiline input and a host of other issues for you. Give it pretty much anything that satisfies the Reader interface as an input and it’s good to go. The return type of the CSVReader.readAll() method also satisfies the same contract as the plain Groovy version shown above, so we can interact with the parsed results in exactly the same way as in the previous example.
All values are still typed as Strings, so we need to cast in order to compare numeric values.

import au.com.bytecode.opencsv.CSVReader

List<String[]> rows = new CSVReader(
        new InputStreamReader(getClass().classLoader.getResourceAsStream(TEST_FILE_NAME)))
            .readAll()

//same finders as in the Groovy version
def elementsOver200Mass = rows.findAll {it[ATOMIC_MASS].toDouble() > 200}
def elementsBetween10And20 = rows.findAll { row ->
    double mass = castToDouble(row[ATOMIC_MASS])
    mass <= 20 && mass >= 10
}

HsqlDB

HsqlDB allows for using flat files as Text Tables, effectively turning a CSV file into a database table. Data values are cast to the types specified in your table definition, so any potential type cast failures happen right up front when loading the data. The benefit is that once the data is read in no further manual processing is necessary. Note: by default the csv file and the database file must be located in the same directory, as a security precaution (can be overridden through configuration).

In addition, leveraging sql makes querying the data extremely easy. Groovy brings in the Sql class to abstract away a lot of the normal Java boilerplate you encounter when working with a ResultSet. I didn’t do any in-depth testing to prove it out, but this is also the only one of the strategies described here that doesn’t require holding the entire data in memory in order to do arbitrary queries. That can be mitigated in the previously shown methods by processing files line by line rather than in bulk if memory usage is a concern.

//create the table definition to insert
String tableName = 'elements'
String tableDefinition = """CREATE TEXT TABLE $tableName (
    atomic_number INTEGER PRIMARY KEY,
    atomic_mass NUMERIC,
    name VARCHAR(255),
    symbol VARCHAR(3)
);"""

//create a new file database and a table corresponding to the csv file
Sql sql = Sql.newInstance("jdbc:hsqldb:file:${testdbDir.absolutePath}/testdb", 'sa', ''
    ,'org.hsqldb.jdbcDriver')
sql.execute(tableDefinition)

//set the source to the csv file
sql.execute("SET TABLE elements SOURCE '${TEST_FILE_NAME};all_quoted=true'".toString())

//querying the database that's wrapping our CSV file
def elementsOver200Mass = sql.rows("SELECT * FROM $tableName WHERE atomic_mass > ?", [200])
def elementsBetween10And20 = sql.rows(
    "SELECT * FROM $tableName WHERE atomic_mass <= ? AND atomic_mass >= ?", [20, 10])

//simple db aggregates
def count = 0
sql.eachRow("SELECT count(1) FROM $tableName WHERE atomic_mass <= ?", [20]){row->
    count = row[0]
}
def avg = 0
sql.eachRow("SELECT avg(atomic_mass) FROM $tableName".toString()){row->
    avg = row[0]
}

Which One do I Use???

Pick the one that fits best for your use case is the real answer. I didn’t know about the HsqlDB option until recently, but OpenCSV has been a personal standby for years. Then again, if you’re writing something quick and dirty at the script level, the simplicity of just applying a couple of split() operations is pretty appealing. Here’s how I generally decide.

Related posts:

1. Groovy inspect()/Eval for Externalizing Data
2. Five Cool Things You Can Do With Groovy Scripts
3. Achieving Groovy-like Fluency in Java with Google Collections

Creating Collections

Groovy really shines for this one, making the creation of Lists and Maps, empty or populated, an absolute breeze. Java has some help for creating basic Lists but begins to struggle when creating maps. This is an area that google-collections can help in, especially in regards to creating immutable Maps.

//Empty Lists
        List<String> groovyList = []
        List<String> javaList = new ArrayList<String>()
        List<String> googleList = Lists.newArrayList()  //can omit generics

//Populated Lists
        List<String> groovyList = ["1", "2"]
        List<String> javaList = Arrays.asList("1", "2")
        List<String> googleList = Lists.newArrayList("1", "2")

//Immutable Lists
        List<String> groovyList = ["1", "2"].asImmutable()
        List<String> javaList = Collections.unmodifiableList(Arrays.asList("1", "2"))
        List<String> googleList = ImmutableList.of("1", "2")

//Empty Maps
        Map<String, String> groovyMap = [:]
        Map<String, String> javaMap = new LinkedHashMap<String,String>()
        Map<String, String> googleMap = Maps.newLinkedHashMap()

//Immutable Maps
        Map<String, String> groovyMap = ["a":"1", "b":"2"].asImmutable()

        Map<String, String> javaMap = new LinkedHashMap<String,String>()
        javaMap.put("a", "1")
        javaMap.put("b", "2")
        javaMap = Collections.unmodifiableMap(javaMap)

        //OR(works only in Java, will not compile in Groovy)
        Map<String, String> javaMap = new LinkedHashMap<String, String>()
        {
            {
                put("a", "1");
                put("b", "2");
            }
        };

        Map<String, String> googleMap = ImmutableMap.of("a", "1", "b", "2")  //clunky syntax but it works

Filtering Collections

Groovy provides the very handy ‘findAll’ method that allows you to filter a Collection by applying a Closure. Google-collections provides similar facilities using the Predicate interface and two filter methods available statically on Collections2 and Iterables. This would also be a lot more readable if the Predicate definition were extracted but I wanted to show that it’s still possible to create them in-line quickly.

import static com.google.common.collect.Collections2.*

List<Integer> toFilter = [1, 2, 3, 4, 5]
List<Integer> groovyVersion = toFilter.findAll{ it < 3}
List<Integer> googleVersion = filter(toFilter, new Predicate<Integer>()
    {
        public boolean apply(Integer input)
        {
            return input < 3;
        }
    };)

Joining Collections into a String Representation

This one has come up pretty often over the years, and it’s not surprising that where the JDK hasn’t provided, enterprising developers have added support through libraries. The problem is: given a Collection of objects, create a String representation of that Collection suitable for view from a consumer of the system. And admit it – the first time you hand-coded the algorithm you left a trailing comma, didn’t you? I know I did.
Groovy has fantastic support for this use-case by simply including the ‘join’ method on Lists. Google-collections utilizes static calls on the Joiner class along with a simple DSL-like syntax to achieve the same effect. Throw in a static import to make it even more concise and it does a fine job. These two examples yield the same result.

import static com.google.common.base.Joiner.*
def toJoin = ['a', 'b', 'c']
def separator = ', '

//groovy version
def groovyJoin = toJoin.join(separator)

//google-collections version
def googleJoin = on(separator).join(toJoin)

And google-collections also supports join for Maps, something missing from Groovy(although not very hard to implement).

import static com.google.common.base.Joiner.*
def toJoin = [1: 'a', 2: 'b', 3: 'c']
def separator = ', '
def keyValueSeparator = ':'

//results in '1:a, 2:b, 3:c' which is essentially what is returned from Groovy map.toMapString()
def googleVersion = on(separator).withKeyValueSeparator(keyValueSeparator).join(map)

//results in '1 is a and 2 is b and 3 is c'
googleVersion = on(" and ").withKeyValueSeparator(" is ").join(map)

//doing the same in Groovy is slightly more involved, but really not that bad
def groovyVersion = toJoin.inject([]) {builder, entry ->
            builder << "${entry.key} is ${entry.value}"
            builder
        }.join(' and ')

Multimaps

Multimaps are one of the more interesting parts of google-collections, at least to me. Have you ever found yourself writing code to create a Map of keys to Lists? Well the various Multimap implementations in google-collections mean you never have to write that boilerplate kind of code again. Here’s a “first-stab” effort to simulate a fairly generic Multimap with pure Java code.

public class JavaMultimap
{
    private Map<Object, List<Object>> multimap = new LinkedHashMap<Object, List<Object>>();

    public boolean put(Object key, Object value)
    {
        List<object> objects = multimap.get(key);
        objects = objects != null ? objects : new ArrayList<object>();
        objects.add(value);
        multimap.put(key, objects);
        return true;
    }
}

And the same thing in Groovy, achieving a slightly smaller version.

class GroovyMultimap
{
    Map map = [:]

    public boolean put(Object key, Object value)
    {
        List list = map.get(key, [])
        list.add(value)
        map."$key" = list
    }
}

I did some primitive timings comparing Java, Groovy and google-collections Multimaps implementations and, as you’d pretty much expect, google clearly takes the lead. Where things really start to get interesting though is when you start using the Multimap in Groovy code. Imagine if you will that you want to iterate over a collection of Objects and map some of the properties to a List. Here’s a contrived example of what I’m talking about, but applying this same pattern to domain objects in your application or even a directory full of xml files is pretty much the same. If you look closely you’ll notice that Groovy actually makes this a one liner to extract all values for a property across a List of Objects(used in the assertion), but I imagine that Multimap is probably a better alternative for large data sets.

class GoogleCollectionsMultiMapTest
{
    private Random random = new Random()

    @Test
    public void testMultimap()
    {
        def list = []
        10.times {
            list << createObject()
        }
        List properties = ['value1', 'value2', 'value3']
        Multimap multimap = list.inject(LinkedListMultimap.create()) {Multimap map, object ->
            properties.each {
                map.put(it, object."$it")
            }
            map
        }
        properties.each {
            assertEquals (multimap.get(it), list."$it")
        }
    }

    Object createObject()
    {
        Expando expando = new Expando()
        expando.value1 =  random.nextInt(10) + 1
        expando.value2 =  random.nextInt(100) + 100
        expando.value3 =  random.nextInt(50) + 50
        return expando
    }
}

So Where Does This Get Us?

Between google-collections and the newer guava-libraries that contain it, there’s lots of help available for simplifying programming problems and making your code more readable. I haven’t even touched on the newly available support for primitives, Files, Streams and more, but if you’re interested in reducing the amount of code you write AND simultaneously making it more readable you should probably take a look. There’s a very nice overview available in part one and part two by Aleksander Stensby. And here’s a closer look at what google-collections can do for you.

Source Code

As per usual, source code is available at github as a maven project. Big thanks to the Spock team for sharing how they configure GMaven to properly utilize Groovy 1.7. Please feel free to take a look and comment here. Thanks!

Related posts:

1. Groovy and Glazed Lists with Grape
2. Different Flavors of Embedded Groovy in Java Apps or “How To Make your Java Groovier!”
3. My favorite new Groovy trick

Where Hibernate Validator Shines

Annotations on domain classes allow for easily validating object state at the time of persistence. Excellent integration with frameworks like JBoss Seam allow this same ability to be utilized for validating web forms on the client-side with little more than an <s:validateAll/> tag. Seam practically hides the entire interaction with validation components from the developer. Since validation rules are defined directly in the domain class, you can (almost) guarantee that no objects with inconsistent state will ever end up being saved in your database. There are certain validations that aren’t possible to verify without actually looking in the database, unique constraints for example, but generally in my experience hibernate validator is extremely easy to configure and work with. Implementing CRUD functionality is pretty trivial, and UIs can achieve consistency since all validations are applied equally.
Alternatively, if you can’t or don’t want to use annotations for some reason, you can specify your validation rules in an xml file. Usually a singleton validation.xml file is made available on the classpath and picked up automagically when a ValidationFactory is created. A simple xml configuration looks like this:

<constraint-mappings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemalocation="http://jboss.org/xml/ns/javax/validation/mapping validation-mapping-1.0.xsd" xmlns="http://jboss.org/xml/ns/javax/validation/mapping">
    <default-package>org.kar.test</default-package>
    <bean class="ValidateTestableClass">
        <field name="name">
            <constraint annotation="javax.validation.constraints.NotNull">
        </constraint>
    </field>
</bean>

and is meant to be applied to this simple class:

class ValidateTestableClass
{
    int id
    String name
    String description
    boolean enabled
}

Comparing with Grails Validation

Grails automatically provides validation capabilities for domain classes and command objects, and enables adding the same behavior to any pogo through a combination of the @Validateable annotation and a static constraints closure. Adding validation support to arbitrary classes also requires specifying which packages to scan for the annotation.
Plugin support from projects like bean-fields simplifies the handling of client-side validation and rendering error markers in the UI, an ability which the Grails framework provides natively by adding an ‘errors’ field directly onto the domain or command object class instances bound to a web form.

Dynamic Constraints

Both the hibernate validator and the Grails strategies for applying validation described here have the same limitation: both are universally applied to all instances of a class. There’s no easy apparent way to override those constraints at runtime, although I suspect that some fancy MOP’ing or configuration could probably be used to accomplish overrides at runtime.
Hibernate validator also supports creation of ad hoc validators by seeding with one or more xml documents. Or if you’re like me and hate hand editing xml, you can leverage Groovy to take a bit of the pain away. Here’s the same xml snippet from above in a Groovy Closure, generated simply by turning the DomToGroovy class loose on the raw xml:

looseConstraint = {
        mkp.declareNamespace(xsi: 'http://www.w3.org/2001/XMLSchema-instance')
        'constraint-mappings'(xmlns: 'http://jboss.org/xml/ns/javax/validation/mapping',
                'xsi:schemaLocation': 'http://jboss.org/xml/ns/javax/validation/mapping validation-mapping-1.0.xsd') {
            'default-package'('org.kar.test.objects')
            bean('class': 'ValidateTestableClass') {
                field(name: 'name') {
                    constraint(annotation: 'javax.validation.constraints.NotNull')
                }
            }
        }
    }

Losing all the angle brackets is a good start, but we really haven’t saved a lot of typing. Until you start taking advantage of the ability to define more complicated structures. Note the use of a list structure here to apply NotNull constraints to multiple fields.

strictConstraint = {
        mkp.declareNamespace(xsi: 'http://www.w3.org/2001/XMLSchema-instance')
        'constraint-mappings'(xmlns: 'http://jboss.org/xml/ns/javax/validation/mapping',
                'xsi:schemaLocation': 'http://jboss.org/xml/ns/javax/validation/mapping validation-mapping-1.0.xsd') {
            'default-package'('org.kar.test.objects')
            bean('class': 'ValidateTestableClass') {
                ['name', 'description'].each {
                    field(name: it) {
                        constraint(annotation: 'javax.validation.constraints.NotNull')
                    }
                }
                field(name: 'id') {
                    constraint(annotation: 'javax.validation.constraints.DecimalMin') {
                        element(name: 'value', '2')
                    }
                }
                field(name: 'enabled') {
                    constraint(annotation: 'javax.validation.constraints.AssertTrue')
                }
            }
        }
    }

Applying Dynamic Constraints

Applying constraints is as simple as converting Closures to xml and mapping them to a Configuration object, which then supplies a Validator to use. StreamingMarkupBuilder is utilized to create the xml behind the scenes.

    /**
     * Create a configuration object passing closures as validation mapping documents.
     * @param closures closures to render into validation mapping documents
     * @return config
     */
    public Configuration createConfig(Closure... closures)
    {
        Configuration config = Validation.byDefaultProvider().configure()
        closures.each {
            config.addMapping(new ByteArrayInputStream(GroovyXmlConversionUtil.convertToXml(it).bytes))
        }
        config
    }

I haven’t tested the use of multiple mappings extensively, but minimally each class you’re configuring must be confined to a single mapping – you can’t extend the validations by layering configurations on top of one another. You should however be able to map constraints for different classes in separate Closures.

Crying out for a Builder!

Going from Closures to xml is a quick and dirty way to test out this functionality, but what would really be nice is a Builder that could create an appropriate validation environment more directly. At the least it would allow for removing the namespace declarations and explicit package naming that make up the bulk of the content.

So what do you get?

1. 1. Ability to declare validations against any existing Java or Groovy class without changing the source code
2. 2. Programmatic ability to create the configuration of validations
3. 3. A choice of which validations to apply at runtime
4. 4. Consistency with the behavior of domain class validation

And what’d I get? A fun bit of quick coding on a Saturday afternoon. Nice! Source code is available on git-hub if you want to check it out.

Related posts:

1. Groovy and CSV: How to Get Your Data Out?
2. Jira Grails Plugin
3. Breaking Weak CAPTCHA in… slightly more than 26 Lines of Groovy Code

The Pretty Pictures

Here are the three images to test against. In order to put them in a suitable format for the open source tesseract-ocr program to process we need to make them bigger, remove the background noise and transform them into a ‘tif’ format. The python program we’re porting utilizes the PIL library for image handling and the pytesseract library for wrapping tesseract; I didn’t look very hard for java equivalents and just coded the required functions directly.

Reading in the Image

The python code for this is three lines, one to load the image and a couple more to convert it into a format suitable for directly manipulating pixel color through RGB values. Groovy takes a bit more to do the same, but being able to use a ‘with’ block makes interacting with the Graphics object a lot cleaner than the same Java code

//python
from PIL import Image
img = Image.open('input.gif')
img = img.convert("RGBA")
pixdata = img.load()

//Groovy
BufferedImage image = ImageIO.read(new File(fileName))
BufferedImage dimg = new BufferedImage(image.width, image.height, BufferedImage.TYPE_INT_ARGB)
dimg.createGraphics().with {
    setComposite(AlphaComposite.Src)
    drawImage(image, null, 0, 0)
    dispose()
}

Removing the Background Noise

In both cases we’re doing essentially the same thing: finding all non-black pixels and setting them to white. This leaves only the actual embedded text to stand out. Being able to utilize the Java Color constants makes the Groovy version a little more readable, IMO, but otherwise the two pieces of code are generally equivalent.

//python
for y in xrange(img.size[1]):
    for x in xrange(img.size[0]):
        if pixdata[x, y] != (0, 0, 0, 255):
            pixdata[x, y] = (255, 255, 255, 255)

//Groovy
(0..<dimg.height).each {i="" -="">
    (0..<dimg.width).each {j="" -="">
        if (dimg.getRGB(j, i) != Color.BLACK.RGB)
        {
            dimg.setRGB(j, i, Color.WHITE.RGB)
        }
    }
}

Resizing the Image

Python‘s library usage really shines here, making this a one line call. Not quite the same in Java-land, although again there’s probably a better way to do this(I just don’t know it offhand).

//python
big = im_orig.resize((116, 56), Image.NEAREST)

//Groovy
dimg = resizeImage(dimg, 116, 56)
...
def resizeImage = {BufferedImage image, int w, int h ->
    BufferedImage dimg = new BufferedImage(w, h, image.type)
    dimg.createGraphics().with {
        setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR)
        drawImage(image, 0, 0, w, h, 0, 0, image.width, image.height, null)
        dispose()
    }
    return dimg
}

By this point the original images now look like this, and are almost ready for OCR.

Converting to a tif File

This one turns out to be a bit of a PITA in Java and particularly on a Mac, and represents the bulk of the Groovy code. Unfortunately it is also the only format that tesseract appears to accept ‘out of the box’. After googling the fun that is JAI and working with the .tif(f) format with it on a Mac, I ended up taking the code kindly provided in this blog post and Groovified it a bit to make a working transformation. Thanks very much to Allan Tan for that. One more time, there’s likely a better/easier way to do this, but honestly it’s more effort than I’m willing to put in on a weekend afternoon just to satisfy my curiosity.

//python
ext = ".tif"
big.save("input-NEAREST" + ext)

//Groovy
void convertToTiff(String inputFile, String outputFile)
{
    OutputStream ios
    try
    {
        ios = new BufferedOutputStream(new FileOutputStream(new File(outputFile)))
        ImageEncoder enc = ImageCodec.createImageEncoder("tiff", ios, new TIFFEncodeParam(compression: TIFFEncodeParam.COMPRESSION_NONE, littleEndian: false))
        RenderedOp src = JAI.create("fileload", inputFile)

        //Apply the color filter and return the result.
        ColorConvertOp filterObj = new ColorConvertOp(ColorSpace.getInstance(ColorSpace.CS_sRGB), null)
        BufferedImage dst = new BufferedImage(src.width, src.height, BufferedImage.TYPE_3BYTE_BGR)
        filterObj.filter(src.getAsBufferedImage(), dst)

        // save the output file
        enc.encode(dst)
    }
    catch (Exception e)
    {
        println e
    }
    finally
    {
        ios.close()
    }
}

OCR with Tesseract-OCR

Finally we need to pass the processed image to tesseract so it can ‘read’ it for us. Again, the python library makes this a breeze, but calling out to a command line program with Groovy is so simple that it ends up being about the same. Tesseract itself is available as a macport, as well in downloadable unix binaries and a windows executable so installing the software is a breeze.

//python
from pytesser import *
image = Image.open('input-NEAREST.tif')
print image_to_string(image)

//Groovy
def tesseract = ['/opt/local/bin/tesseract', tmpTif, tmpTesseract].execute()
tesseract.waitFor()
return new File("${tmpTesseract}.txt").readLines()[0]

Testing it out

To test it out I implemented the code in a maven project, iterate over the images and write out intermediate results to a temp directory. And it only works on two out of three of the cases. For some reason tesseract insists on consistently seeing ‘e4ya’ as ‘e4ga’. I tried to see if I could get it working by tweaking the image manipulation parameters and the order of operations(resizing before removing the background noise for instance) but that just caused the other cases to fail as well. Since in the final image the ‘y’ seems pretty clear, it’s more likely that tweaking tesseract configuration might yield better results.

public void testPrintImage()
{
    def breaker = new CaptchaBreaker()
    /* tesseract interprets "e4ya" as "e4ga" unfortunately */
    ['9koO', 'jxt9'/*,'e4ya'*/].each {String imageName ->
        def fileName = "src/test/resources/${imageName}.gif"
        assertEquals("Testing $imageName",imageName, breaker.imageToString(fileName))
    }
}

C’est Finis

I had some fun playing with areas of Java that I don’t usually interact with, and gained some appreciation for the diversity and ease-of-use exposed by just a couple of python libraries. It’s comforting to note that I was able to implement all of the required functionality from those libraries in < 90 lines of Groovy. With a little more effort I think the final product could be tweaked to avoid the intermediate file system reads/writes as well, but that’s for another day.
Source code is available on github if you’d care to take a look, and thanks for stopping by!

Related posts:

1. Different Flavors of Embedded Groovy in Java Apps or “How To Make your Java Groovier!”
2. Achieving Groovy-like Fluency in Java with Google Collections
3. Groovy and Hibernate Validator for Dynamic Constraints

The slideware plugin provides a framework for creating presentations with a little twist – you can execute code live from the presentation software. If you’ve ever given a presentation about programming you probably are fully aware of the transition from presentation software to your preferred environment for demonstrating code samples. Well, now you can stay entirely within the same application, editing and running code live.

The plugin itself isn’t available from the Griffon repository but source code and a built 0.2 version can be found on github. Considering both the youth of the plugin and the beta status of the framework, it worked impressively well AND had a rich feature set. All of the expected “powerpoint” features are there: themes, layout control, styling, slide transitions and export are all pretty easy to incorporate and configure. The code editor view also works very well. A great variety of additional plugins are harnessed to put all the pieces together and as a result building one of these applications is a great way to tour the platform.

Theming

Applying a theme to the presentation is simply selecting a Java Look and Feel to apply in Initialize.groovy. The Substance jar is included with the plugin so I test drove a few of the nice setups in there and finally settled on the SubstanceMagmaLookAndFeel. There is definitely a wide variety of L&F’s to choose from in that bundle alone and, although I haven’t done it myself, they seem pretty tweakable as well. Plus any old L&F should plug in nicely, I would imagine.

//Initialize.groovy
SwingBuilder.lookAndFeel('org.jvnet.substance.skin.SubstanceMagmaLookAndFeel',
       'mac', 'nimbus', 'gtk', ['metal', [boldFonts: false]])

Layout

Controlling the page composition is a standard Swing Layout or, in the case of the default slide you get with the included “create-slide” script, a MigLayout. Framing the standard variety of slides is very simple. Bulleted pages, title slides, code slides and custom layouts are very easy to accomplish. I don’t have a lot of experience using this particular layout but the presentations Andres has made available on github have a good diversity of examples of how they look in practice.

//the default create-slide generated template
import net.miginfocom.swing.MigLayout

slide(id: "slide0", layout: new MigLayout("fill","[center]","[center]")) {
    label("Insert your text here")
}

Styling

Styling is supplied by the css plugin, on which slideware has a dependency. The default style.css file sets out just some reasonable defaults for the fonts used in different parts of the app, and I didn’t see any real need to fiddle with it. Especially happy to see the nice monospace code font. On a totally related note I recently installed the Inconsolata monospace font to try for development and I’ve been very happy seeing it in my editor, but it’s still nice to see this kind of polish applied to the presentation. The code editor view even includes syntax highlighting! More on that coming right up…

Slide Transitions

Moving between slides with style is the responsibility of the transitions plugin. You can see see all of the animations this plugin enables over here at this page describing Transitions and Transition2Ds. Pretty slick stuff and defined as simply as a parameter to each “slide” node in a script.

slide(id: "slide3", layout: new MigLayout("fill","3%[center]3%","3%[center]3%"),
        title: "Junit3",
        transition: new FlurryTransition2D(Transition2D.OUT)) {
    scrollPane(constraints: "grow") {
        widget(createEditor(text: script))
    }
}

Code Editor

Code slides embed an editable widget and allow for composing and executing Groovy scripts or classes of arbitrary complexity. You can execute the code with a keyboard shortcut and a window will open displaying the console output. The code itself is executed through a GroovyShell, enabling pretty much anything you might want to do. If you have an internet connection and the required repositories configured, Grapes simplifies packaging the dependencies for the application as your code samples can directly Grab the jars they need. Basically, the first time you execute a code slide you’ll have to put up with a small pause while Ivy downloads to your local repository, unless of course the required dependencies are already there. In my case I changed strategies from including jars in the application lib directory to a Grapes approach and I think it’s a better way to go.

I was surprised to find that the editor even included undo functionality. It’s definitely not close to a full blown IDE, and there’s absolutely no reason that it should be. For the task of demoing simple code examples it’s more than up to the task, even to the point of maintaining your edits between slide transitions, allowing you to move back and forth through the slide deck without any state problems.

Export

The application includes a “Print” feature which iterates through the entire slide deck and renders it to a pdf. Distribution of one of these presentations is really very easy, including the ability to create installers for all major platforms simply by adding the packaging plugin.

Test Application

The test app I built while looking into this is pretty simple. It’s got a title slide, a bullet slide and 4 code slides. The code simply demonstrates how you can create test classes and make assertions for Junit 3(with GroovyTestCase), Junit 4 and TestNG. GroovyShell recognizes all 3 of these test files by interface or annotation and executes them appropriately. In each case the console output of the test framework is the result, including the new Spock inspired ascii art assert failure renderings. The Test result files are also written to disk, and that TestNG html output is what I’m used to looking at anyhow. Show me the Green!

The last code slide is just a slightly updated version of an example script on the Grapes page, using the current version of Google Collections and intentionally introducing a failure- mostly just to show off that new assert rendering I mentioned a moment ago. VERY helpful at highlighting the exact nature of a failure. It also encourages me to pay more attention to how I name variables, something I’m sure every developer that has ever worked with me will cheer at. Man, I suck at naming things.

I also developed a brute force test that loads each slide and executes scripts if finds embedded there. Failures are hard to detect since the direct output is simply a text block, but some fairly simple regex’s applied to the output make me at least moderately confident that the code won’t fail at show time.

Overall Impression

It took a couple of afternoons(six hours or so total) to download the source code from github, explore it, create a simple presentation and document the experience. I won’t begin to suggest that I’m fully aware of all the details happening behind the scenes, but the end user experience is pretty fluid: create a slide, tailor the layout, add content and then repeat. The included examples were more than enough documentation on how to hit the ground running. The plugin code itself is a great example of the MVC nature of Griffon, not a whole lot of code, but a great deal of power and expressability. There were a couple of glitches happening in the background, mostly just logging to the console with no visible effect to the application, but overall it functioned as well as (not) advertised. For publicly unreleased software it was an absolute pleasure to work with and I plan on continuing with the development of this particular presentation.

Everything you need to build and run this stuff yourself is publicly available. In my case, Griffon generally has a recent Macport available for both the released(griffon @0.2.1) and development versions(griffon-devel @0.3-BETA-2). Switching versions is relatively painless and, for applications this simple, testing out upgrades is basically just going through the presentation once in a functional test. Versions for other platforms can be downloaded from the Griffon download page.

Deliverables

Source code for the sample application is available on github here. Please just leave a comment on this page if you have any problems running it.

Here’s the pdf produced by the application ‘Print’ feature: Groovy Testing Presentation (252)

And if you don’t feel like downloading anything, here’s how it all looks in pretty pictures.

[Show as slideshow]

[View with PicLens]

Related posts:

1. VIJUG Griffon/Groovy Presentation
2. Achieving Groovy-like Fluency in Java with Google Collections
3. Fun day playing with new Stuff

Since neither Grails nor Git are by default supported by Bamboo, I grabbed a couple of plugins to enable my home continuous integration system. Bamboo has a very sensible plugin model that allows for adding new capabilites with only a jar file in the right place and a server restart, and a fairly rich library of plugins. It’s also encouraging to see a company that is committed to eating their own dog food – recently they’ve moved their plugin resources to Jira studio as you can see on the Grails plugin home page. This gives you as the consumer of the software access to the wiki’d documentation, any open issues, and even visibility into when source code changes are committed. I’m assuming that last is subject to whether or not the plugin source code is open source or not, but still – pretty nifty.

The Grails plugin instructions speak for themselves in the 3 screen-shots on the one and only wiki page. Anything else you need to know is summarized in a one liner on the “Builder” tab when you select the Grails builder: “Use a new line to seperate Grails commands.” Gotta love clear, simple instructions. Drop the plugin jar into $BAMBOO_INSTALL_DIR/webapp/WEB-INF/lib, restart Bamboo and configure a new builder for Grails. Took a couple of minutes and knowing where my Grails install was located.

A quick Google for “Bamboo git plugin” led to a bit of a hypertext chase that eventually ended up here. Looks like this particular plugin has passed through the hands of at least a couple of maintainers, but so far it’s worked exactly as advertised. And again, loving the dog food analogy, it’s nice to see a project enabling Git hosted on github. Thanks very much to Kristian Rosenvold for running with the ball on this one! Again, drop the jar into $BAMBOO_INSTALL_DIR/webapp/WEB-INF/lib and restart Bamboo. When creating a plan you will now have Git as a choice for a Source Repository. In my case I’m just using a plain file repository that resides on the build box, but it should work just as well with a remote repository.

Setting up the Grails targets is similarly straightforward. Thanks to this article, which describes the Grails setup on Hudson, and reminded me about the –non-interactive flag since obviously the build server isn’t going to be able to interact with the console. I’m also using a couple of Grails plugins to provide static source code analysis so my parameters also include -coverage and a call to the codenarc script.

I had a little trepidation about how this would all turn out after reading this blog post, but I’ve had much better results so far in my personal experience. It’s entirely possible that the key differentiator is wrapping Grails with Maven. It’s definitely not perfect, but it is getting better all the time. Hopefully you have better luck in the future Neil!

[Show as slideshow]

[View with PicLens]

Related posts:

1. Jira Grails Plugin
2. Developing Faster with the Atlassian IntelliJ Connector
3. Gource Visualizations of the Groovy/Grails/Griffon Projects

Nothing terribly complicated going on for this plugin. Aside from the XMLRPC library and one more jar it depends upon there’s a service which implements calls for all of the methods on the XmlRpcService interface and a ‘helper’ class that makes using it a little easier(removes the need to pass the host name into each method). And any controller injected with the service gets a withJira(hostname, Closure) method which wraps the ‘helper’ class. This might not be the optimal way to do things, but I was looking for some better experience with MOP’ing in Groovy along with this little project so it worked out perfectly for me.

The method added to Controller classes just creates a ‘helper’ and sets it as the delegate for the closure.

        cClass.metaClass.withJira = {String hostname, Closure closure ->
            def helper = delegate.jiraXmlRpcService.createHelper(hostname)
            closure.setDelegate(helper)
            closure.setResolveStrategy(Closure.DELEGATE_FIRST)
            closure()
        }

Calling the withJira method looks like this, along with minimal exception handling:

            try
            {
                def projects, priorities, serverInfo
                withJira(jiraInstance.baseUrl) {
                    def token = login(jiraInstance.userName, jiraInstance.password)
                    serverInfo = getServerInfo(token)
                    priorities = getPriorities(token)
                    projects = getProjectsNoSchemes(token)
                    logout(token)
                }
                [jiraInstance: jiraInstance, projects: projects, priorities: priorities, serverInfo: serverInfo]
            }
            catch (e)
            {
                flash.message = "Failed to load data from ${jiraInstance.baseUrl}"
                redirect(action: "list")
            }

The ‘helper’ class itself represents an experiment with the Groovy ‘methodMissing’ behaviour. The idea is to store the ‘hostname’ for a series of calls against the stateless service and add that as the first parameter for all wrapped methods. It was simple and quick, but right off the bat you lose the ability to use IDE syntax completion support on this object. I thought another possibility here would be to use the @Delegate annotation on the service and/or Closure currying to add the additional ‘hostname’ parameter to methods.

class JiraXmlRpcServiceHelper
{
    def hostname
    def JiraXmlRpcService service

    def methodMissing(String name, args)
    {
        assert hostname && service
        def method = service.metaClass.methods.find {it.name == name}
        if(!method)
        {
            throw new NotImplementedException("The $name method is not implemented for ${this.getClass()}")
        }

        def newArgs = [hostname]
        args.each{newArgs << it}
        return method.invoke(service, newArgs as Object[])
    }
}

This plugin works with Grails 1.2.0.RC1 and represents about 8 hours of work all told – and a lot of fun to make! Jira 4 is not too shappy either, especially with the GreenHopper plugin. But that’s a post for another day.

A special thank you to Grails in Action and this article by Ted Naleid – both were extremely helpful putting this together. If you care to download it and take a spin here it is. Feedback is always welcome!

Related posts:

1. Bamboo, Grails and Git for Continuous Integration
2. A Groovy/Gradle JSLint Plugin
3. Grails-UI DataTable using XML for a model

Groovy on the command line

The quickest and simplest way to run a Groovy Script or Class, command line arguments are automatically marshalled into an ‘args’ String array. Please note that due to a problem I’m having with my syntax highlighter plugin the process execution is shown here in single quotes; the actual code requires a GString(double quoted) in order to do the replacement for the inline variable. " THAT WordPress!

//the script
myArgs = args
result = args.join(' ')
println result
println myArgs

//...and the test
    void testGroovyCall()
    {
        def proc = 'groovy $groovyScriptOne Hello World'.execute()
        proc.waitFor()
        def result = proc.text.split()
        assert result[0] == 'Hello'
        assert result[1] == 'World'
    }

GroovyShell

This is the basis of Groovy script execution. The GroovyShell allows for executing scripts, passing in a particular Binding context that allows for bi-directional communication between the script and the calling code. Parameters can be passed into the executing script in the Binding and results can be stored there to be returned to the calling context. GroovyShell also allows for running a class from the ‘main’ method, passing in String arguments. It will also execute implementers of Runnable and test files for JUnit or TestNG. Script text can also be declared inline and executed in the same way as files on disk. All in all, pretty bloody handy. Here’s a straightforward example of running a dirt simple Groovy script and inspecting the results. Note that this isn’t executable as shown, but I’ll provide the full source code on github for anyone who wants a closer look. Note that I’m also passing in an ‘out’ variable in the Binding, which effectively redirect System.out to a specified Writer implementation – a nice touch for inspecting output.

//the script
myArgs = args
result = args.join(' ')
println result
println myArgs

 //...and the test
    void testGroovyShell()
    {
        Binding binding = helper.createBinding()
        def shell = new GroovyShell(binding)
        shell.evaluate(new File(groovyScriptOne))
        helper.assertBinding(binding)
    }

//...and the Binding creation/assertion
     def static args = ['Hello', 'World'].asImmutable()
     /**
     * Create a Binding with a single parameter to be passed to scripts and an 'out' Writer to redirect console output.
     */
    private Binding createBinding()
    {
        Binding binding = new Binding()
        def sWriter = new StringWriter()
        def pWriter = new PrintWriter(sWriter)
        binding.setVariable ('args', new ArrayList(args))
        binding.setVariable ('out', pWriter)
        return binding
    }

    /**
     * Assert that the expected 'common' actions are done with the Binding by each of the use cases.
     * The original 'args' should be as expected.
     * A copy of 'args' should have been placed in the Binding during execution.
     * The 'result' should be the concatentation of 'args' separated by spaces.
     */
    private def assertBinding(Binding binding)
    {
        assert binding.variables.size() == 4
        assert binding.variables.args.value[0].toString() == args[0]
        assert binding.variables.args.value[1].toString() == args[1]
        assert binding.variables.result.value.toString() == args.join(' ')
        assert binding.variables.myArgs.value[0].toString() == args[0]
        assert binding.variables.myArgs.value[1].toString() == args[1]
    }

GroovyScriptEngine

The GroovyScriptEngine enables dynamically running Groovy sources located in a fixed set of content roots, complete with reloading modified scripts in between executions. Running a Groovy script this way is essentially the same as using GroovyShell.

    void testGroovyScriptEngine()
    {
        Binding binding = helper.createBinding()
        def gse = new GroovyScriptEngine(new File('.').toURL())
        gse.run(groovyScriptOne, binding)
        helper.assertBinding(binding)
    }

GroovyClassLoader

An extension to URLClassLoader that enables parsing Groovy sources into Class representations. Once a Class object is created, instances of the class can be created easily and either cast to a known type or manipulated through convention by use of the standard Groovy ‘invokeMethod’. This works equally well on Groovy and Java btw. Here’s an example of running a Java class using GroovyClassLoader. In this case the Java file has a field called ‘binding’ and implements a ‘run’ method.

    /**
     * Dynamically compile, instantiate, inspect and call methods on a POJO.
     */
    void testGroovyClassLoaderOnJava()
    {
        GroovyClassLoader loader = new GroovyClassLoader();
        Class javaClass = loader.parseClass(new File(javaFileOne));

        def groovyObject = javaClass.newInstance();
        def binding = helper.createBinding()
        groovyObject.binding = binding
        if(groovyObject.metaClass.respondsTo(groovyObject, 'run'))
        {
            groovyObject.invokeMethod('run', null);
            helper.assertBinding(binding)
        }
        if(groovyObject.metaClass.respondsTo(groovyObject, 'main'))
        {
            groovyObject.invokeMethod('main', new ArrayList(helper.args) as String[]);
        }
    }

(Groovy)Console

The Console can be embedded in Java or Groovy code to provide a dynamic interactive Swing environment. This is the same UI spawned from the command line invocation of ‘groovyConsole’. Internally it uses GroovyShell for actual execution, and so can do everything that GroovyShell can do – plus a couple of additions. For one, you can add jars and/or directories to the classpath used when executing your scripts.

The Best of Both Worlds – at Least for my use case

In actual practice these patterns can be used a lot more successfully by observing standard Java practices, like casting classes parsed using GroovyClassLoader to a known interface before interacting with them, or by using Classes to organize business logic inside of a Script that essentially functions as a ‘main’ method. This example defines two dependent internal classes, marshals parameters to them and then returns the results attached to the originally passed in Binding.

/**
 * Classes inside of a Script.
 */
class TestableClass
{
    Binding binding

    def run()
    {
        binding.with
        {
            setVariable('myArgs', getVariable('args'))
            setVariable('result', getVariable('args')?.join(' '))
        }
        return binding
    }
}

class TestableClass2
{
    Binding binding

    public TestableClass2(Binding binding)
    {
        this.binding = binding;
    }

    def run()
    {
        return new TestableClass(binding: binding).run()
    }
}

if (args)
{
    def internalBinding = new Binding()
    internalBinding.setVariable('args', new ArrayList(args))
    internalBinding = new TestableClass2(internalBinding).run()
    args = internalBinding.args
    myArgs = internalBinding.myArgs
    result = internalBinding.result  //return value from script
}
else
{
    println 'no args!!'
}

Summary of Groovy Execution Methods

Related posts:

1. Achieving Groovy-like Fluency in Java with Google Collections
2. Groovy CliBuilder with multiple arguments
3. Breaking Weak CAPTCHA in… slightly more than 26 Lines of Groovy Code

DISCLAIMER: yes, this is heavily geared towards(but not exclusive to) Groovy-esque sources. What can I say – it’s where I spend most of my time.

Google… in General

If you want to find something – well let’s just say any service that is so pervasive it becomes a verb in the language might be useful.  Learning how to harness the power of the Google architecture can greatly enhance your learning experience. And I can’t see myself Bing’ing anytime soon.

1. Google Search: Knowing and using the Google search syntax properly can be a very powerful tool for finding specific tech-geekie knowledge.
2. Google Reader: Listening to the right content can save you a lot of time when it comes to getting the most out of your code. Even if you don’t read it everyday, combined with Gears and searchability this provides another tool in the fight against techno-ignorance that doesn’t rely on an internet connection.
3. Google Mail: I combine mailing list subscriptions with filters to automatically organize content. Incoming messages from lists are archived and labeled accordingly by filter rules. With Gears installed I’ve got an off-line searchable database of grassroots questions and answers, similar to what an on-line service like MarkMail provides. It’s hard to be the ongoing discussion of community members sharing their best tips, tricks and gotchas.
4. Google code: A great place to find open source projects that can assist you at work and provide valuable  learning materials. And yes, the link targets Groovy .

Aggregators

Some kind people do most of the work for you – taking themed content and putting it all together in one place.

1. DZone: One of my favorite sources of  techie style news.
2. GroovyBlogs: A great example of ‘eating your own dog food’, this is a Grails powered site courtesy of one of the authors of Grails in Action. Most of the Groovy related blog articles I read come from this source.
3. Grails Tutorials: A nice collection of tutorial articles related to Grails and Groovy.
4. GR8Forums: Brand new, this site has the potential to be a great discussion spot.

Community Sharing

There are certain places that encourage community and knowledge sharing more than others.

1. Groovy Console in The Cloud: This is an amazing example of an application exposing a Groovy console on the Google AppEngine, complete with community support for running, saving and commenting example code.  Since the code is freely available, there’s also already a Spock console here in the cloud forked/cloned from it.
2. Github and Gists: More places to share code in project and/or snippet form.
3. StackOverflow: A community effort to put together almost 300,000(and rising) questions and answers related to programming. If you have a question to ask, head over here – chances are someone else has already given an answer. It really is impressive how well this still evolving site functions.
4. Podcasts: JavaPosse and the Grails podcast are on top of my list and usually get listened to during my commute to work.There are hundreds of tech related podcasts out there; pick one or two and give them a listen,  you won’t regret it.
5. mrhaki : OK, this guy gets special mention just for busting out code examples for pretty much every Groovy feature you can possibly imagine. Seriously the guy posts 20+ times a month on average.

Home Pages

These aren’t necessarily up to date at all times, but are great places to start looking for generalized knowledge and additional resource material.

1. Groovy: Pay particular attention to the ‘Cookbook Examples’ and the ‘Modules’ available to quickly add functionality.
2. Grails: A particularly rich and well organized site – with the latest update it’s also very pretty.
3. Griffon: The builder documentation linked from here, combined with the example scripts included with SwingPad, get you up to speed quickly on all of the buildery goodness available in Griffon.

Useful and Inspiring

These are resources I use to step out of the box and to remind myself that it’s not only in elegant code that geekiness lies.

1. Lifehacker: I get into trouble a lot for this one; in my house whenever someone asks me “How do I do {fill in the blank}?” I invariably respond with “Well what does Lifehacker say?” My personal favorite is the bundle wrapping method. With a nice mix of techie and common sense goodness, I love to follow this site.
2. Make magazine: In need of a DIY project to fill up the weekend? From the Minty Boost for a $20 portable USB charger to robots this place takes geeking out entirely to the next level.
3. Jonathon Coulton: With eye-popping geekiness, Jonathon brings us such wonderful classics as “Code Monkey” and “Re Your Brains” – and under a Creatives Commons license as well!

Twitter

Use with caution. More than any other source of data, this one can end up being seriously distracting. I’m careful about who I listen to, while simultaneously trying to remind myself that nobody cares what I had for breakfast or how much I enjoyed it. Have you tried Twitter yet? Not as easy as it sounds buster, believe you me. Just wait and see. If you can keep yourself from jumping every time a new Tweet lands at your desktop, this can be the freshest, fastest way to satisfy your tech cravings.

Books

OK, so these aren’t free, but if you want to jumpstart your knowledge on a particular piece of technology, software or framework, picking up a recent book on the subject can truly help. I try to add a new book to the library every month, but the hard part is really finding the time to read and apply the new knowledge.

I don’t expect that these suggestions are for everyone, but hopefully a few of you out there in the interwebs will find some of them helpful. And feel free to share what works for you if I’ve missed anything.

Related posts:

1. Gource Visualizations of the Groovy/Grails/Griffon Projects
2. A Grails App Demoing the StackExchange API
3. A One Day Griffon Application/Presentation

The venue was great – very nice that the Vancouver Island Tech Park is kind enough to host meetings. Next time I’ll have to come early enough to figure out how to make the projector do better than 800X600 resolution, however. Hard to show much other than a slide deck presentation with that little space. Then again I may be a little spoiled when my usual is 1920X1200.

I love using Keynote for presentations with the Apple infrared remote. Between the presenter view with notes to guide what I wanted to say(and to remind me when to step out and demo some code) and the timer to keep me on track, it went really pretty smooth. That’s my story, and I’m sticking to it. The software, both Griffon and Keynote, worked flawlessly so if things weren’t perfect, there’s only me to blame.

SwingPad seemed to make quite an impression, which is not hard to understand considering how little effort it takes to start showing results on the screen. Put that together with the recent CSSBuilder integration and the rich samples for other builders and I think more than a couple of Java developers will be installing it soon.

The question came up enough that I want to directly reference this blog post from Andres Almiray regarding GSQL in Griffon. Apparently people like to have databases handy at all times

And a big thanks to Andres for sharing two free copies of the upcoming Griffon In Action that he is co-authoring for raffle prizes tonite. Through the magic of Twitter he was very supportive about 20 seconds after the original announcement went out. Now if only they can kill all those nasty spam-bots, etc that are messing with my Twittiness!

I’ve uploaded the demonstration application at github. It shows a simple Griffon app with three tabs. The first is a minor port of Andres’ Trident/GfxBuilder example to show animation. The second is a simple demo of the @Bindable AST transformation along with the SwingBuilder syntax for accesing SwingUtilities; it comes along with an accompanying Fest test. The third requires an internet connection as it parses out the DZone rss feed and delivers it into a GlazedList, demonstrating usage of SwingWorker courtesy of the SwingXBuilder. I showed those samples along with installer plugin built versions of some of the included Griffon sample applications. WeatherWidget is a one fine looking translucent UI example!

Please ping me if you have any problems running this Griffon app, as I did trim out a bunch of the extraneous project files and test reports right before uploading. This was my first time ‘really’ using Git for source control so it’s bound to be a bit messy – nevertheless the Git experience has been great so far. I also tried to get the slides up on SlideShare, but that appears to… not be happening(been uploading now for over 20 minutes this time- about to kill it, again). So here’s the pdf.
Source code download here.

Slidedeck download here.

Related posts:

1. A One Day Griffon Application/Presentation
2. Griffon SwingX Fest testing
3. Presenting a Groovy/Griffon talk

Links on the IntelliJ site:

1. 1. The main documentation page
2. 2. Demos and tutorials including a couple of new Groovy support demonstrations
3. 3. 25 things you can do with IntelliJ 8 you can’t do with 7
4. 4. Mac Keymap reference
5. 5. Windows/Linux keymap reference
6. 6. The Jetbrains IntelliJ blog. A great way of getting tips from the guys that wrote it, and keeping up to date with improvements.

DZone: Intellij in 7 pages

1. 7. The DZone refcard for Intellij

My personal faves

1. 8. Command Shift-A to bring up a search window for actions. This one I think of as Quicksilver for my IDE.
2. 9. Alt-Enter to trigger intentions. Problem with your code? Let the IDE suggest a refactor to correct it.
3. 10. Ctrl-n for auto-generation of Java code. Getter and setter boilerplate getting you down(and you can’t switch from Java to Groovy)? Want a nice compact toString() on a class with 20 member variables? Let the IDE do the heavy lifting.
4. 11. The Javadoc Sync plugin. This goes well with #10 to javadoc all those getters and setters auto-magically.

Add logging using the debugger

1. 12. Printing to the console is a basic tool for debugging code. Everybody’s done it – because it’s easy and it works. But you often come across scenarios that disallow this quick and satisfying hackish behaviour. Namely when you are debugging into code in a library, this is not necessarily an easy option. Fortunately IntelliJ provides a nice easy way to add console logging through the debugger. You can access the breakpoint configuration dialog either by right clicking on an existing line breakpoint and selecting ‘Properties’ or by pressing Command Shift-F8 from the Debug view. From there you configure your line or Exception breakpoint with a ‘Suspend policy’ of ‘None’ and check the boxes under ‘Actions’ to enable logging. A full description of the features can be found here.

   I find this especially handy when there’s a problem iterating over a Collection and only one or some of the items in that Collection lead to an error condition during processing. Using this strategy I can provide context as to which cases passed and which failed without having to stop at a bunch of breakpoints and inspect the variables on the stack; I can just concentrate on the failure point and look back at the console output to compare against previous invocations.
   Here’s a couple of screencaps to show configuration for logging the size of a list on a line breakpoint. Note that the Groovy requires a little bit of extra magic, since the debugger gets passed a Reference object instead of the raw List.

[Show as slideshow]

[View with PicLens]

Bonus from the Community

1. 13. Grails returning the favor for the great IntelliJ support.

How about you?

1. 14. Do you know any other good places to learn about IntelliJ and how to get the most out of it? Please don’t keep it to yourself! I managed to line up a baker’s dozen, now you’re on the hook for number 14 smart guy.

Related posts:

1. Griffon support in latest Intellij EAP
2. Developing Faster with the Atlassian IntelliJ Connector
3. My favorite new Groovy trick

It appeared on the JetBrains blogroll a couple of weeks ago, and indeed may have showed up in an earlier release, but today was the first time I saw it in action. I couldn’t find anything about this new feature mentioned explicitly in the release notes but sure enough, it’s there. Given the similar project structure between Grails and Griffon, and the first class Grails support already available in IntelliJ, it’s really not too hard to see how this platform is in an ideal position to expand Griffon support. Beyond that the general Groovy support seems to get better in every release. The EAP is free to try for any holder of an IntelliJ 8.x license and seems to be pretty stable despite the disclaimers – I highly recommend taking it for a spin, especially if you’re already using this IDE. You get properly configured source directories and dependency paths, access to Griffon’s command line scripts, and even that awesome new Griffon icon shown to mark out the project. What more could you ask for?

JetBrains was beaten to the punch by a NetBeans plugin for Griffon that was released in its initial version before this year’s Java One conference. I’m not a NetBeans user myself, but I am a frequent reader of Geertjan Wielenga’s blog – he’s published some great Groovy examples. Even the Eclipse community has finally gotten around to putting out some real Groovy support, largely due to SpringSource‘s commitment.

Hopefully their recent acquisition by VMWare won’t distract them from developing it too much

Regardless of which development environment you feel most comfortable with, there’s no questioning this is a great time to be a Groovy/Grails/Griffon developer so enjoy!

Related posts:

1. My Favorite IntelliJ Resources, Tips and Tricks
2. Developing Faster with the Atlassian IntelliJ Connector
3. A One Day Griffon Application/Presentation

A DataProvider method in TestNG can return either a two dimensional Object array or an Iterator over each of the test parameters. A consumer of that method will be triggered once time for each set of parameters from the provider. Parameters are injected into the consumer method at execution time.
This greatly eases testing various expectations that run through essentially the same path of execution.

Here’s the simple method under test, courtesy of this example on Groovy Almanac

    /**
     * Use the Groovy added List.inject() method to sum a list of numbers.
     */
    def sum(list)
    {
        def sum = list.inject(0) { sum, item -> sum + item }
    }

And here’s the corresponding DataProvider test harness. The test is injected with a List of numbers to sum and the associated total expected for each case.

    @DataProvider (name = "test1")
    public Object[][] createListInjectSumData() {
        def array = new Object[3][]
        array[0] = [[1, 2, 3], 6] as Object[]
        array[1] = [[2, 4, 6], 12] as Object[]
        array[2] = [[3, 6, 9], 18] as Object[]
        return array
    }

    @Test (dataProvider = "test1")
    void testListInjectSummation(list, expectedSum) {
        Assert.assertEquals(new ListInjectExample().sum(list), expectedSum)
    }

I’ve put the maven project I used to demo this up on github here. Mostly just because I’m having a great time using Git lately.

Related posts:

1. Achieving Groovy-like Fluency in Java with Google Collections
2. Breaking Weak CAPTCHA in… slightly more than 26 Lines of Groovy Code
3. Groovy and Bash – can scripting get much easier?