BlackBox

Requirements

So, what are my requirements for my main two DSLs - QueryBuilder and ViewBuilder (this doesn’t exist yet, but starting on the planning right now). For QueryBuilder I wanted a clean syntax that makes it easy for developers to create constraints/restrictions on a database query. Ultimately the query could be executed by Hibernate, JPA, iBatis, whatever, so it should be ORM-agnostic. This actually isn’t too hard as they all have this concept of a criteria on an object/table and restrictions for that query. And the same goes for ViewBuilder, I want an HTML-like tree syntax that I can specify components in as well as attributes that will be applied to the view when it is built for me.

User

This one is easy, I am targeting Web application developers, initially I am targeting Tapestry Web application developers, but since the framework will be pluggable (probably around the 2.0 time) you can specify what specific pieces you want to use (Tapestry/JSF + Hibernate/JPA + Spring 1.x/2.x/Guice, etc.).

Has this problem been solved before?

In the case of the QueryBuilder, yes. Grails currently has an implementation called HibernateCriteriaBuilder. This has a nice syntax for specifying criteria and restrictions on that criteria. As for the ViewBuilder I don’t think this has been solved yet, but I will be looking. There is something similar in Groovy’s core for creating arbitrary HTML, which is close but not exactly what I’m looking for as I will need to generate more than just HTML from it.

How did they solve it?

Both of these were solved using Groovy’s BuilderSupport object which allows you to build up arbitrary “nodes” from a collection of method, property and closure calls. Very powerful and a little difficult to wrap your head around the first 10 times you look at it ;-) (alright, maybe it was just me, but that took a bit to fully understand what the hell it was doing).

How closely does that solution match our requirements?

First, for the HibernateCriteriaBuilder it actually will build up a real, live Hibernate criteria and use it, which unfortunately is not what I want. I need to generate the Java code that will come out of it. And the MarkupBuilder gets me part of the way there but not quite what I’m looking for.

What can we take away from that solution?

I decided to take away the syntax from HibernateCriteriaBuilder and looked at how they implemented it to see what I could do with that. And for the ViewBuilder I am going to extend BuilderSupport itself and use that to build up my nodes of whatever objects (so more than just plain HTML text, actually objects that represent things).

What kind of DSL do you want overall?

Definitely shooting for internal I think, but that doesn’t preclude having some kind of external DSL for the class creation stuff (that was a giant pain to do in Groovy/Java - just not all of the information I actually needed). So, having some kind of custom language for defining models could definitely come in handy down the road, but I haven’t decided anything yet. Plus, as I learn more Groovy and get feedback/suggestions I may stumble across something that simplifies my design greatly.

Prototype, prototype, prototype

This, aside from requirements and research is probably the most valuable step. Once you decide what you are going to do, how in the world do you do it? Especially if you’re dealing with Meta-programming (which can be a hard concept to wrap your head around)?

Here is where my experience will hopefully prove valuable to someone out there, here are the steps that I took when developing my QueryBuilder:

1. Write the Unit tests first, start with the simplest test you can think of for how your language should work
2. Run it - yeah I know its going to fail, but prove it to yourself ;-)
3. Make it go green - whether you go ahead and create a simple class that has one method in it that does what you want do that
4. Write another 2 tests - think about them, how are they going to change that class you just wrote, how much more complex is it going to make the logic you need to write to make sure you do what you expect.
5. Fix the first test - is it time to extend DelegatingMetaClass yet? Or maybe try something simpler like a Category? Maybe the new ExpandoMetaClass will work. Try a few of them out, but I don’t recommend you dive right into extending BuilderSupport, not yet, not unless you have already used it before and understand fully what it does. Throw in lots of println "calling so and so with these arguments". These help identify that what you are expecting is actually happening while you write your DSL.
6. Did that work? Good, write it so the next test succeeds.
7. Continue doing 4, 5, and 6 until you are happy and ready to show it to someone
8. Let them play with it, get their feedback. Rinse, repeat, refactor. (Here is where I would suggest you revisit the BuilderSupport idea, does it make sense? Remember not everything is an arbitrary nested tree, so not everything will benefit from using this structure.)

Well, I hope that this is helpful to someone ;-). I really have gotten the DSL bug and (mostly) enjoy writing them. If you run into a weird error, throw it out to the Groovy list, someone should be able to help you out.

Addendum

I meant to add this in when I first started thinking about this post. When I started writing my QueryBuilder I didn’t start out with it generating anything but simple select strings. This helped me visualize the proper nesting of things without getting too complex. Once that was working right I changed my unit tests to expect the actual Java code I wanted back and changed the logic of what the DSL output.