Crazy? Working on CouchDB Grails OR plugin 4

Posted by Warner Onstine on September 16, 2009

Not sure if I’m crazy or not but after getting an initial setup with jcouchdb I am getting tired of manually mapping my class fields with the Couch fields so I’m starting to work on a gorm-couchdb plugin.

Pouring over Grails code (and the gorm-jpa plugin code). Chime in if you’re interested in helping as this is secondary to what I want to get done. I’ll throw some code up to github soon (maybe tomorrow with an early, early version).

Posted via web from BlackBox Post

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Is Maven going away? 12

Posted by Warner Onstine on July 06, 2008

First, let me say, I like Maven – a lot. I’ve liked the idea, if not the execution, since 1.0 (yes, I know there is some serious bad blood for the implementation of 1.0, and yes I still liked it). But I’ve recently run into some irritating problems with Maven that would again, I think, require a major reworking of what it does.

Here is the main issue Maven is designed to handle one language only for its compilation – primarily Java. This is rapidly starting to fall apart now that the JVM has some serious contenders besides Java, namely Groovy, JRuby, Jython, and Scala. There are some hackish things for getting Maven to deal with these, however it still only has one “true” source directory for compilation. A case in point is a mixed Java/Groovy project – trying to get IntelliJ to respect the fact that I want not one, but two main source directories is a pain in the ass as I have to either tell it not to sync everytime I launch or I have to refix the source directories each time I update the project.

This is what triggered this post – what happens when I want to do some Flex programming with some Java, ActionScript and MXML, all of these are true source files (and in Maven specfic directories). I’m sure some Mavenite is going to say that these should be separate projects, but in alot of cases they aren’t and it doesn’t make sense to actually separate them into sub-projects. And I’m equally as sure that some Ant-ite is going to come along and tell me to use Ant – I won’t. Ant is stuck in the past and I won’t use it unless I absolutely have to. I like the “imposed” structure that Maven offers. I can walk up to any project and know where to find stuff instead of blindly hunting around for hours trying to piece it together like some giant source-code jigsaw puzzle, blech.

So, I’m at cross-roads, what should I do? There is Ivy (for dependency management -which they got from Maven natch), but that requires Ant, blech. There’s also Gant, ok, this is better, but still Ant. GMaven, but I’m right back where I started.

Enter, Gradle, a new Groovy-based build system that offers alot of the benefits of Maven without locking you in and the flexibility of Ant. I’m not 100% sold on it yet, but I’m going to give it a shot for a relatively complex project that will have a fair amount of Java, Groovy and other fun resource files. It looks interesting in that all the build files are actually Groovy scripts that are a DSL (like Gant files). Here’s a short example with some dependencies (taken directly from their examples in the download).

import org.gradle.api.tasks.util.FileSet

usePlugin('groovy')

group = 'org.gradle'
version = '1.0'

sourceCompatibility = 1.5
targetCompatibility = 1.5

dependencies {
   clientModule(['groovy'], ":groovy-all:1.6-beta-1") {
       dependency(":commons-cli:1.0")
   }
   compile project(':groovycDetector')
   testCompile ":junit:4.4"
}

compile {
   exclude('**/Exclude.java')
   groovyExclude('**/ExcludeGroovy.groovy')
   groovyJavaExclude('**/ExcludeGroovyJava.java')
}

manifest.mainAttributes(myprop: 'myvalue')
metaInf << new FileSet(new File(srcRoot, 'metaInfFiles'))

test.options.systemProperties['org.gradle.integtest.buildDir'] = buildDir.absolutePath

I will try and keep everyone up to date as I continue to investigate this new build system. They also state that they are going to be supporting polyglot programming in the near future which I’m looking forward to.

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Groovy DSL roundup

Posted by Warner Onstine on April 24, 2008

This is my first attempt at categorizing some of the DSLs I’ve run across into some kind of logical grouping. First I’ll list out some Groovy DSL examples (sound off in the comments for those I’m not aware of, the more the better). I’ll go through the DSLs first and then look at the possible categories after the break. And the first two are mine that I created

I know there are more out there but this is a good starting point for sure.

Now, on to the actual categories of DSL “types”

I have been paying attention to Martin Fowler’s new DSL book in which he specifies several types of Internal DSLs. I’m not 100% on board with all of these yet, but here they are (of course these may change as this is a work in progress, so if he changes them it isn’t my fault :P ):

  • Expression Builder: A layer that provides a fluent interface over a regular API
  • Function Sequence: A combination of function calls as a sequence of statements.
  • Nested Function: Compose functions by nesting function calls as arguments of of other calls.
  • Method Chaining: Make modifier methods return the host object so that multiple modifiers can be invoked in a single expression.
  • Object Scoping: Put DSL code in a subclass of a class that provdes the DSL vocabulary.
  • Closure:
    • Nested Closure: Express statement sub-elements of a function call by putting them into a closure in an argument.
  • Literal Collection Expression: Form language expressions using literal collection syntax
  • Dynamic Reception: Handle messages without defining them in the code
  • Annotation: Data about program elements, such as classes and methods, which can be processed during compilation or execution.
  • Macro: Define DSL expressions as macro definitions
  • Parse Tree Manipulation: Capture the parse tree of a code fragment to manipulate it with DSL processing code.

Ok, so these are the basic categories as Martin has defined them. How do these map up to the tools and techniques we have in Groovy?

  • Method/Property Missing – a special method that is called when a method can not be found allowing a developer to intercept the call
    • Maps to Dynamic Reception fairly well
  • Categories – allows new methods to be added to any class at runtime
    • Could fit into Object Scoping or maybe a new one, Duck Typing? – not sure
  • ExpandoMetaClass – a dynamically expandable bean
    • Again Dynamic Reception
  • Closures/Nested Closures
    • Closures/Nested Closures
  • Relaxed rules on the use of parentheses
    • Hmm, not sure where this would fit
  • Built-in list and map syntax
    • I believe this is Literal Collection Expression
  • Ability to dynamically add methods to an interface and have concrete classes also have the methods
    • Again, this sounds like Dynamic Reception
  • Operator overloading
    • Hmm, again I’m not sure where this fits
  • Annotations
    • Annotation

First up we have our builders (AntBuilder, XMLBuilder, Criteria Builder, etc.) these essentially mimic some other existing structure (an Ant build file, XML file, or SQL syntax). What patterns do these use? Just off the top of my head I think I see:

  • Expression Builder
  • Nested Closures
  • Dynamic Reception
  • Literal Collection Expression

JSON-lib’s Groovy interface has the following:

  • Literal Collection Expression
  • Dynamic Reception
  • Operator Overloading (as)

My Groovy Higher Order Messaging DSL has:

  • Dynamic Reception
  • Nested Closures

My Groovy Math DSL has:

  • Dynamic Reception
  • Nested Closures (or possibly what would be considered Function Sequence in some cases, even though they aren’t functions but it might fit)
  • Operator Overloading

GroovyLab has:

  • Expression Builder
  • Probably has more just don’t have time to dig into the code – I’m just looking at the quick examples)
  • Operator Overloading

Guillaume’s Unit Manipulation DSL has:

  • Expression Builder
  • Dynamic Reception
  • Closure/Nested Closure
  • Operator Overloading

Ken Barclay’s GParsec has:

  • Dynamic Reception
  • Parse Tree Manipulation? (not sure if this qualifies for this or not)

GroovyRestlet DSL has:

  • Literal Collection Expression
  • Closure/Nested Closure
  • Function Sequence

GSpec has:

  • Closure/Nested Closure
  • Function Sequence
  • Dynamic Reception

Grails’ Domain Class Validation DSL has:

  • Literal Colleciton Expression
  • Closure
  • Macros?

Tiago Antão’s Malaria Drug DSL has:

  • Object Scoping
  • Dynamic Reception

Human readable date modification has:

  • Object Scoping
  • Operator Overloading

Now, one of the big things that I think Fowler is missing here (and one that you can see several DSLs take advantage of) is Operator Overloading/Overriding. This, I think, is a key technique in DSLs and one that can make a DSL go from good to great. It’s all about letting the user think in the context they are in – be it math matrices, sql queries, file and string manipulation (overriding the << or >> operators). It’s all about the user’s context and making it easy for them to “think” in their domain and not have to worry about the language implementation (I use it quite heavily in my Math DSL to implement matrix operators for example).

I know I didn’t hit on all the DSLs I mentioned, but will try and come up with a wiki or something that people can contribute back to. In the mean time sound off in the comments if you have a DSL that I didn’t mention, or you feel my categorization is wrong (and why). I’m also curious if there are examples of some of the ones I didn’t touch on such as Annotations. This is an initial stab at this and as I said earlier I’m not 100% on board with how Fowler has organized things but I think it’s a good start.

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Thinking about picking the Groovy DSL book up again 6

Posted by Warner Onstine on March 26, 2008

But before I do I thought I’d ask everyone out there (who are interested in such a book), what would you like to see in it?

One possibility I’ve been tossing around is to turn it into a techniques or cookbook-style book. I’ve been reading through “Designing Interfaces” by Jennifer Tidwell. In this book she organizes the interface solutions into different sections and goes through each one with an example and what it solves and when to use it. I really like this idea (but others might not). Originally my book was a “How do I get started designing DSLs?” and “What in the world are they and why should I care?”.

Another thing I’ve been thinking about is ditching the Java part and making it Groovy only. This appeals to me because I feel that Groovy is much cleaner to implement an Internal DSL than Java is. Can you do DSLs in Java? Of course you can, but I’m not sure that I can give it the best treatment because it isn’t what I’m passionate about.

So, sound off in the comments, let me know what topics you’d like to see covered, what challenges have you faced in writing a DSL (or API/Framework) that you would have liked someone to trailblaze for you to show the dead-ends first?

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Groovy Roadmap

Posted by Warner Onstine on December 20, 2007

So Guillaume posted the current road map to the list a couple of days ago and I put in my .02 for fixing the (in my eyes) broken operator overloading. And the bug I really wanted is now marked to be fixed in 1.6!

I honestly can’t say how happy I am at this and will definitely test this functionality out with my Math DSL that I am working on.

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Overriding the ‘as’ operator 2

Posted by Warner Onstine on December 06, 2007

A friend of mine sent me a link to Charles Nutter’s blog on implementing interfaces using the as operator. While I was explaining how it worked to him I remembered that like many operators you can also override this one using asType(). This got us to thinking when would you use this functionality?

I have one project I’m working on right now (not ready to talk yet about as I want more code in place before I unveil it, :-P ) but this might be useful. Say that you wanted to duck-type your object for specific class instances. Overriding as may be one place to do it.

When would this make sense? Personally I think something like this makes the most sense in a library that provides some kind of functionality to a bunch of classes (rather than a one-off case). But I’m curious to see how others have used this functionality, would help to shed some light on other uses I hadn’t thought of.

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Math DSL Part Deux – Elementary Row Operations (EROs)

Posted by Warner Onstine on November 26, 2007

After sending out my query to the groovy user list I got back some ideas (no offense to Yann, just something I need to tackle on my own, so I didn’t look at your solution). What I ended up doing was two-fold:

  • Create a new MetaClass using ExpandoMetaClass
  • Define a new class that understood what row and matrix it was going to operate on (so it can have its own operator overloading)

Yes it was a little bit of extra work, but worth it. I also implemented my or operator overloading so that I can specify augmented matrices in three formats:

  • A|I for the identity matrix (only works on square matrices)
  • A|0 for the zero column, used for solving the set of linear equations
  • A|b for a vector column, also used for solving the set of linear equations

As always it helps to learn from others’ mistakes (and oddities) so I’ll share mine along the road:

  • When dealing with EMC (ExpandoMetaClass) watch out for the use of the this keyword
  • Closure delegates don’t always act the way you would hope when extending GroovyObjectSupport (at least I couldn’t get it to work right which is one reason why I switched to EMC)
  • Overriding the or operator works fine until you get to an object that has overridden the equals operator. In this case Groovy kept trying to find out if my Matrix was equal to my MatrixColumn. Once I figured this out I just returned false if the MatrixColumn was being compared against a Matrix then it worked as expected.

The next part I want to add is the ability to track the EROs through Elementary Matrices. These are matrices that start out as 1s along the diagonal and they get the same EROs applied to them (this will be useful further down the line for other operations).

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Math DSL beginning 1

Posted by Warner Onstine on November 23, 2007

I decided to start working on this taking one of the examples I worked out for the book for math Sets (partially incomplete due to Groovy’s inability to directly override lessThan, lessThanEquals, greaterThan, greaterThanEquals) and to put into code some of the techniques I was learning in my Linear Algebra class.

So far I’ve implemented the following into my matrix math DSL:

  • Matrix/Row equality – allows you to test if two matrices or rows are equal (i.e. – contain the same ordered set of numbers)
  • Matrix addition/subtraction – You can add two matrices together if they are the same dimensions (2×2, 2×3, whatever, as long as the match in row and columns)
  • Matrix/Row scalar multiplication – You can multiply a scalar value against a row or matrix
  • Matrix multiplication – You can multiply two matrices together if the first matrix A’s (say 2×3) columns match B’s rows of the matrix you are multiplying against (say 3×2) which will produce a 2×2 matrix. If you did BxA then you would end up with a new matrix of 3×3
  • Matrix powers – If the matrix is square (say 3×3) you can multiply it by itself to get the power of the matrix
  • Left-shift operation – be able to dynamically create a Row or Matrix by adding new elements to the end

I still have a lot of work which is laid out on my Trac site for GroovyMath which leads me to my next conundrum.

When dealing with matrices you need to do a lot of Reduced Echelon Form (REF) or Row Reduced Echelon Form (RREF) this consists of taking one of two Elementary Row Operations (EROs) and applying it to a row to get a matrix in the form of a diagonal (all 1s down the upper-left to lower-right diagonal with 0s everywhere else) or getting it into an upper-diagonal form (0s below the diagonal and integers everywhere else). The two EROs that you can consist of a) row interchange, where you can exchange one row with another; and b) row addition, where you can add one row to another multiplying that row by a scalar. What I would like to be able to do is something like this:

def A = new Matrix(//rows go in here)
A.r1 + A.r2*2 //which will add 2 times the second row and add it to the first and replace it
A.r1 % A.r2 //which will exchange rows 1 and 2 with each other

Another thought I had was to do this in some kind of “rowOperations” closure:

def A = new Matrix(//rows go in here)
A.rowOperations {
    r1 + r2*2
    r1 % r2
}

Which definitely reads better than the first (and of course that’s all part of a DSL how well does it translate the native tongue into code). But the issue that I have with either of these approaches (maybe, still ironing out the second in my head) is that the properties r1 and r2 don’t exist. These property accessors will need to be intercepted. Ok, not a big deal. But if I choose to implement this in MatrixRow (i.e. – actually return the row itself) it doesn’t allow me to modify the originating matrix (A in this case) only change the row. For the addition problem I think I can get around it as it is executing in a closure, but the row interchange is an issue. The individual rows don’t know they’re a part of a matrix and I’m not sure that knowing it allows them to do anything.

Anyways, just wrestling with this concept right now as it is key to moving on to the more advanced functionality in my matrix math. Any suggestions are more than welcome. One new feature that I really like will be overriding the or operator to provide Augmented Matrices (another key part). Normally you define an augmented matrix in one of three ways:

  • A|0 – add a column of all zeros
  • A|b – add a vector as the augmented matrix
  • A|I – add an Identity matrix

The first two are used to solve the set of linear equations and a bunch of other things. The second is used to find the inverse of the matrix as you transform through EROs the matrix A so that it equals I (the identity matrix) and then the side that “I” was on now equals the inverse of A. I would say stop me if I’m boring you but it’s probably too late for that. I just wanted to point out how cool it was I can override the or operator to do something completely different that makes perfect sense in my given domain.

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

DSL book on hold…unfortunately

Posted by Warner Onstine on October 30, 2007

Well we got the first round of reviews back from the technical reviewers and there’s a lot of work to be done. So much so that we’ve decided to put the book on hold while we regroup and figure out if/how we are going to address the issues that the reviewers found with the book. In a lot of cases it is going to require a complete rewrite of the example (and hence the chapter itself). Personally I feel the core is there, but maybe the examples weren’t as fleshed out as they could have been.

Now of course I’m a little embarrassed that I announced the book before it was done, but that’s what happens sometimes. Not sure at this point whether or not the book will continue, taking some time to regroup/rethink things and see if it is something that I want to push forward with.

I will keep everyone up to date here on the progress of things once I’ve decided what to do. In the meantime I am going to get back to some of my back-burner projects (a few of which have some DSLs in them ;-) .

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

DSL book in progress 6

Posted by Warner Onstine on October 10, 2007

Now that things feel a little bit more final to me I thought I would take this opportunity to announce the book that’s been keeping me from blogging. The tentative title is Creating DSLs using Java and Groovy and it will be published by Pragmatic Programmers. I started working on this idea after last years No Fluff Just Stuff where I saw Neal Ford speak on DSLs and I talked to him briefly about a project I was working on. His talk inspired me to push forward with using DSLs for code generation specifically using Groovy.

The book will cover both Java and Groovy techniques for writing DSLs, as well as have some general guidelines for writing DSLs. We also have a chapter on ANTLR and JavaCC for writing external DSLs using Java. I tried to cover as much as I could of Groovy meta-programming and some of the additional capabilities of Groovy that make it a good language to write an internal DSL on top of, but there’s only so much space ;-) .

We just sent off the book for peer review, which means its about half-way finished, maybe a little more. We are currently shooting for a release date sometime early next year, which looks very doable at this point, but I expect a ton of edits to come back from this first review.

On a final note I just want to say what a pleasure it has been working with the PragProg crew. I love, love, love their book build system and my editor Susannah rocks. I have had a blast working with them so far on this book and am looking forward to finishing this up and getting it published.

Share and Enjoy:
  • Print
  • Digg
  • Reddit
  • del.icio.us
  • Twitter
  • Facebook
  • Google Bookmarks
  • DZone

Easy AdSense by Unreal