BlackBox

First, let me say, I like Maven - a lot. I’ve liked the idea, if not the execution, since 1.0 (yes, I know there is some serious bad blood for the implementation of 1.0, and yes I still liked it). But I’ve recently run into some irritating problems with Maven that would again, I think, require a major reworking of what it does.

Here is the main issue Maven is designed to handle one language only for its compilation - primarily Java. This is rapidly starting to fall apart now that the JVM has some serious contenders besides Java, namely Groovy, JRuby, Jython, and Scala. There are some hackish things for getting Maven to deal with these, however it still only has one “true” source directory for compilation. A case in point is a mixed Java/Groovy project - trying to get IntelliJ to respect the fact that I want not one, but two main source directories is a pain in the ass as I have to either tell it not to sync everytime I launch or I have to refix the source directories each time I update the project.

This is what triggered this post - what happens when I want to do some Flex programming with some Java, ActionScript and MXML, all of these are true source files (and in Maven specfic directories). I’m sure some Mavenite is going to say that these should be separate projects, but in alot of cases they aren’t and it doesn’t make sense to actually separate them into sub-projects. And I’m equally as sure that some Ant-ite is going to come along and tell me to use Ant - I won’t. Ant is stuck in the past and I won’t use it unless I absolutely have to. I like the “imposed” structure that Maven offers. I can walk up to any project and know where to find stuff instead of blindly hunting around for hours trying to piece it together like some giant source-code jigsaw puzzle, blech.

So, I’m at cross-roads, what should I do? There is Ivy (for dependency management -which they got from Maven natch), but that requires Ant, blech. There’s also Gant, ok, this is better, but still Ant. GMaven, but I’m right back where I started.

Enter, Gradle, a new Groovy-based build system that offers alot of the benefits of Maven without locking you in and the flexibility of Ant. I’m not 100% sold on it yet, but I’m going to give it a shot for a relatively complex project that will have a fair amount of Java, Groovy and other fun resource files. It looks interesting in that all the build files are actually Groovy scripts that are a DSL (like Gant files). Here’s a short example with some dependencies (taken directly from their examples in the download).

import org.gradle.api.tasks.util.FileSet

usePlugin('groovy')

group = 'org.gradle'
version = '1.0'

sourceCompatibility = 1.5
targetCompatibility = 1.5

dependencies {
   clientModule(['groovy'], ":groovy-all:1.6-beta-1") {
       dependency(":commons-cli:1.0")
   }
   compile project(':groovycDetector')
   testCompile ":junit:4.4"
}

compile {
   exclude('**/Exclude.java')
   groovyExclude('**/ExcludeGroovy.groovy')
   groovyJavaExclude('**/ExcludeGroovyJava.java')
}

manifest.mainAttributes(myprop: 'myvalue')
metaInf << new FileSet(new File(srcRoot, 'metaInfFiles'))

test.options.systemProperties['org.gradle.integtest.buildDir'] = buildDir.absolutePath

I will try and keep everyone up to date as I continue to investigate this new build system. They also state that they are going to be supporting polyglot programming in the near future which I’m looking forward to.

I’ve been working on a project for a family member and need a good Content Management System for them to use. Here are my requirements:

• Easy to understand interface
• Generates URLs that are human readable - no blah.php?page_id=123 crap
• Easily supports internationalization as the site itself will need to be translated into multiple languages
• Preferrably Java (but open to Rails, and not really interested in PHP)
• Doesn’t take a rocket scientist to create and edit site templates

What I’ve looked at so far has been:

• Radiant CMS - too simple, with code embedded and no Rich Text editor that they won’t understand
• Magnolia - Waaaay too complex, nice for an enterprise solution but not for what they need it for

Just downloaded Liferay and will be taking a look at that, Daisy and JBoss Portal. Another suggestion made to me was to possibly use Contribute, which I think I’ll have them download and try out to see if it works for them (but then I’ll have to do the whole locale thing through Apache).

I’ve looked at the CMSes listed here.

• Jahia - ok, after relooking at it I may download it and try it out
• Magnolia - too complex for my needs
• InfoGlue - looks complex just like Magnolia
• Apache Lenya - had a very bad experience with this in the past and the interface looks just as awkward to use as it did before - pass
• Daisy - looks interesting
• MMBase - no online demo and I wasn’t wild about the screenshots I did see, passing for now

What I’ve found so far with alot of these (just through using their online demo) is that the interfaces are very complex. I don’t know if this is supposed to go along with their “Enterprise” tag or not, and of course a fair number of them tout their JSR-170, 168 compliance. Ok, JSR-168 might be important to know, but do I really care if they are a fully compliant Java Content Repository? Nah, not really, I just want it to work and have a usable interface. Some may care about that but to me it isn’t a selling point. I respect the CMSes who put up an online demo, it gives me a chance to play with it first to see if it’s even close to something I want, those that don’t I look for screenshots, and most likely move on anyways.

Have you ever wondered how to find the Class that has just called your class? Well, now you can (or rather you always could) by using this little trick that I got from my friend and former co-worker Leo.

Class.forName(new Throwable().getStackTrace()[2].getClassName()));

This will retrieve the caller name from the stacktrace and then you can get the actual class using Class.forName(). He uses this in an interesting way for logging.

private static final Log getLog() {
    try {
        return LogFactory.getLog(Class.forName(new Throwable().getStackTrace()[2].getClassName()));
    }
    catch (Exception e) {
        return LogFactory.getLog(FormattedLogger.class);
    }
}

Which he then imports as static methods so that he can just do:

info("Some intereting INFO logging going on here");

Pretty slick and an interesting use of the Java libraries.

Well we got the first round of reviews back from the technical reviewers and there’s a lot of work to be done. So much so that we’ve decided to put the book on hold while we regroup and figure out if/how we are going to address the issues that the reviewers found with the book. In a lot of cases it is going to require a complete rewrite of the example (and hence the chapter itself). Personally I feel the core is there, but maybe the examples weren’t as fleshed out as they could have been.

Now of course I’m a little embarrassed that I announced the book before it was done, but that’s what happens sometimes. Not sure at this point whether or not the book will continue, taking some time to regroup/rethink things and see if it is something that I want to push forward with.

I will keep everyone up to date here on the progress of things once I’ve decided what to do. In the meantime I am going to get back to some of my back-burner projects (a few of which have some DSLs in them ;-).

Now that things feel a little bit more final to me I thought I would take this opportunity to announce the book that’s been keeping me from blogging. The tentative title is Creating DSLs using Java and Groovy and it will be published by Pragmatic Programmers. I started working on this idea after last years No Fluff Just Stuff where I saw Neal Ford speak on DSLs and I talked to him briefly about a project I was working on. His talk inspired me to push forward with using DSLs for code generation specifically using Groovy.

The book will cover both Java and Groovy techniques for writing DSLs, as well as have some general guidelines for writing DSLs. We also have a chapter on ANTLR and JavaCC for writing external DSLs using Java. I tried to cover as much as I could of Groovy meta-programming and some of the additional capabilities of Groovy that make it a good language to write an internal DSL on top of, but there’s only so much space ;-).

We just sent off the book for peer review, which means its about half-way finished, maybe a little more. We are currently shooting for a release date sometime early next year, which looks very doable at this point, but I expect a ton of edits to come back from this first review.

On a final note I just want to say what a pleasure it has been working with the PragProg crew. I love, love, love their book build system and my editor Susannah rocks. I have had a blast working with them so far on this book and am looking forward to finishing this up and getting it published.

What are some of the basic steps that I’ve followed as I create my DSLs (especially with respect to Groovy)?

1. What are your spoken and unspoken requirements?
2. Who is the intended user? (this will help you decide some of the other questions later)
3. Has this kind of problem been solved before?
4. How did they solve it? (Language, Toolset, Techniques?)
5. How closely does that solution match our requirements?
6. What can we take away from that solution (if it doesn’t match the requirements)?
7. What kind of DSL do you want overall? (Internal or External)
8. Prototype, prototype, prototype - even if it is an Internal DSL you will still want to start out with a working prototype that your intended audience can play with and give you feedback on. This should be an iterative process to refine the language.
9. External DSL - how to implement?

I will cover the first 8 of these, and I will leave the External DSL for a later post (sorry ;-). For this I’ll use the steps that I’m currently going through on my own code to illustrate my own thought process.

I’ve been working on Chama recently and have been making some good progress. One of the last big hurdles before getting all of the Model, View, Controller generation working has been:

1. Connecting to a database and gathering metadata - 90% complete (just have the relationship, primary key, foreign key stuff to work out)
2. Getting Chama packaged up as a real distributable app that you can download and get up and going quickly

Rather than go through a whole bunch of Ant copy scripts and integrating that with my Maven build I decided to write a plugin that would copy everything over into a specific directory structure and zip up for me (and I can also add in the tar.gz stuff real easy now too). I gotta say it wasn’t too difficult, although the docs leave a lot to be desired in this department. With all the plugins out there you would think this would be more fleshed out, especially since they want everyone to move away from the old Jelly scripts that people wrote in Maven 1. Anyways, it’s fairly simple to do with some caveats.

• There appear to be no docs on what all the custom expressions are such as:
  • expression="${project}"
  • expression="${project.build.directory}"
  • expression="${component.org.codehaus.plexus.archiver.manager.ArchiverManager}"
• Or if you can actually use expressions like these when setting default values of properties (you can, at least that’s what I’ve gathered by looking at other plugins)
• And if you do or do not need to have getters and setters for your variables that you’ve “annotated” (I say that in quotes because it doesn’t act like true annotations, or at least there isn’t a package in Maven called annotations where you can find docs on all of these)
  • My assumption is that at least the setters are necessary so that it can use those to populate the variables you want Maven to populate your plugin with)

All that said it was mostly easy to write my little packaging plugin as I was willing to dig through the source code of some of the existing plugins that were doing similar things (maven-assembly-plugin and maven-war-plugin). That isn’t to say that I’m 100% sure I understand exactly how everything works (good idea but not 100% ;-).

If I get a chance I’ll see if I can contribute something a little more to the plugin docs than what is there currently because others shouldn’t have to do what I did to get through this and it isn’t that difficult once you know the secret handshake.

I first read about Processing in an old Wired issue (wow, finally found it, do you know how hard it is to search on “processing” on the Web? Uggh). When I first found Processing I wasn’t even thinking about DSLs (this was 3 years ago), but I know that one of my first thoughts was “Why not just use Java directly?”, but now I definitely see the light. I’ve been following the amazing work that others have been doing with Processing who have very little Java programming experience, but they know 2d and 3d graphics.

The cool thing is, is that while Processing is a DSL it is built on top of Java and can leverage any of the packages that Java has (much like Groovy can) which leads to some very interesting hardware projects. And of course, I’m also interested in Processing because I love 2d and 3d graphics and finding interesting ways to interact with them.

In short, I guess, I just wanted to point out that DSLs are everywhere, even if we didn’t see them at first, they aren’t a new idea, they’ve just been given power by naming them (much in the way that Ajax was).

Anyone have some other interesting DSLs that they’ve found?

So, where has all this DSL stuff been leading? One of my projects that has been sitting on the back-burner for a while has been Chama. Chama was born out of the frustration of writing the same boiler-plate code over and over for database-backed sites that are run on Tapestry. While you can certainly use something like Trails I was not happy with the additional amount of configuration I needed to do in order to get the forms to look and behave the way I wanted them to.

Why not use Ruby on Rails then, or even Grails if I want to stick with the Java theme?

A variety of reasons:

• I'm familiar with Tapestry, Hibernate, etc.
• I'm very familiar with Java
• I want to learn Groovy as a language
• I don't want to switch out my deployment options to have to deal with deploying my apps to a new server and having to configure it all over again so that it'll run RoR
• I like generators ;-), no really this is an exercise that I want to undertake and if others find value in it that's great
• Mostly to make it easier for me to get some app ideas I've had off of the ground

What will Chama use:

• Groovy as the primary language, with some Java thrown in where needed
• Most likely Yaml for configuration with JYaml for dealing with it
• Quite possibly several DSLs for each area (Model, View, Controller)

I don't know yet exactly how far I'm going to take Chama, mostly I just want to get a tool together that will help me build stuff. If others find value in it and it gets extended from there and heads in a direction I never thought about, great! All the code will be available under an Apache License once I start working on stuff. I have some tentative dates setup in my Trac instance (what?! using Python for a Java app?). Comments are welcome, and I will use this blog to announce releases as they happen, the first milestone is set for 12/22/06 (oops was tomorrow, moving to two weeks, wow it's been two weeks since I first set this up already).