The slides of the BoF on Elastic Grid and EC2 are finally available!

Thanks for all of you who could come. We had some interesting discussions and feedback after the talk.
This is the first time I had a chance to go to JavaZone and I must say it’s been quite interesting. A few friends and I had the opportunity to discuss with Tom J. Bang who gave us the opportunity to taste local stuff.

I also had the opportunity to discuss with a lot of people there: my friends from our local user group (Guillaume Laforge, Julien Viet, Alexis Moussine-Pouchkine, Jerome Lacoste) but also Scott Davis and Eitan Suez. (Sorry for all the other ones I forgot to cite.)

For those of you who could not make it for our BoF session, the organizers of the JavaZone event will provide a video. I’ll update the blog when the video is available.

That XML format, although quite extensible is also quite verbose, so a Domain Specific Language (DSL for short), means a lot of sense in that context in order to reduce verbosity.

In order to ease the code for such a DSL, Rio deployment descriptors (called OpStrings) parsing was rewritten using Groovy. This is something which turned out to be quite easy actually and is mostly done. I spent about 4 or 5 days on this task, including writing some Groovy Tests in order to reduce the risk of regression. This work is interesting because I was able to at least cut the code to half its initial size and it is now more easy to read, hence maintain.

Armed with this test suite, I started to work on the DSL. The option I have taken is to transform the DSL syntax into the current XML file and then use the usual XML parser. The trade off is that it minimize the work needed to be done but of course it would be better to use the DSL in order to create the Java model already using when parsing the XML format (this is still being discussed).

So here is how it looks like. Below is the XML format taken from one of Rio examples:
[sourcecode language='xml']





rio

calculator/lib/calculator.jar

calculator/lib/calculator-dl.jar

calculator.Calculator

1

calculator.Add





1

calculator.Subtract





1

calculator.Multiply





1

calculator.Divide





1

[/sourcecode]

Here is now the DSL in action for the same example:
[sourcecode language='java']
opstring(name:’Calculator’) {
groups(‘rio’)

resources(id: ‘impl.jars’, ‘calculator/lib/calculator.jar’)
resources(id: ‘client.jars’, ‘calculator/lib/calculator-dl.jar’)

service(name: ‘Calculator’) {
interfaces {
classes(‘calculator.Calculator’)
resources(ref: ‘client.jars’)
}
implementation(class: ‘calculator.service.CalculatorImpl’) {
resources(ref: ‘impl.jars’)
}
associations {
association(name: ‘Add’, type: ‘requires’, property: ‘add’)
association(name: ‘Subtract’, type: ‘requires’, property: ’subtract’)
association(name: ‘Multiply’, type: ‘requires’, property: ‘multiply’)
association(name: ‘Divide’, type: ‘requires’, property: ‘divide’)
}
maintain 1
}

service(name: ‘Add’) {
interfaces {
classes(‘calculator.Add’)
resources(ref: ‘client.jars’)
}
implementation(class: ‘calculator.service.AddImpl’) {
resources(ref: ‘impl.jars’)
}
maintain 1
}

service(name: ‘Subtract’) {
interfaces {
classes(‘calculator.Subtract’)
resources(ref: ‘client.jars’)
}
implementation(class: ‘calculator.service.SubtractImpl’) {
resources(ref: ‘impl.jars’)
}
maintain 1
}

service(name: ‘Multiply’) {
interfaces {
classes(‘calculator.Multiply’)
resources(ref: ‘client.jars’)
}
implementation(class: ‘calculator.service.MultiplyImpl’) {
resources(ref: ‘impl.jars’)
}
maintain 1
}

service(name: ‘Divide’) {
interfaces {
classes(‘calculator.Divide’)
resources(ref: ‘client.jars’)
}
implementation(class: ‘calculator.service.DivideImpl’) {
resources(ref: ‘impl.jars’)
}
maintain 1
}
}
[/sourcecode]

What’s interesting is that if you have a close look at the above syntax, you will notice that the Add, Sutract, Divide and Multiply services are configured quite the same. Wouldn’t it be nicer to define the configuration once for all those services?

This is something which is accomplished for free because the DSL is a Groovy script, so any Groovy syntax is valid in it:
[sourcecode language='java']
opstring(name:’Calculator’) {
groups ‘rio’

resources id:’impl.jars’, ‘calculator/lib/calculator.jar’
resources id:’client.jars’, ‘calculator/lib/calculator-dl.jar’

service(name: ‘Calculator’) {
interfaces {
classes ‘calculator.Calculator’
resources ref:’client.jars’
}
implementation(class:’calculator.service.CalculatorImpl’) {
resources ref:’impl.jars’
}
associations {
association name:’Add’, type:’requires’, property:’add’
association name:’Subtract’, type:’requires’, property:’subtract’
association name:’Multiply’, type:’requires’, property:’multiply’
association name:’Divide’, type:’requires’, property:’divide’
}
maintain 1
}

['Add', 'Subtract', 'Multiply', 'Divide'].each { s ->
println “Service is $s”
service(name: s) {
interfaces {
classes “calculator.$s”
resources ref:’client.jars’
}
implementation(class: “calculator.service.${s}Impl”) {
resources ref:’impl.jars’
}
maintain 1
}
}
}
[/sourcecode]

Our OpString file has now been reduced from 71 lines to 34 ones!

The only drawback in that case is that using the DSL you loose the autocompletion provided by any good XML editor.

So how does the code look like for that DSL:
[sourcecode language='java']
class GroovyDSLOpStringParser implements OpStringParser {
def OpStringParser xmlParser = new XmlOpStringParser()
def static final Logger logger = Logger.getLogger(GroovyDSLOpStringParser.class.name);

public List parse(Object source, ClassLoader loader, boolean verify, String[] defaultExportJars,
String codebaseOverride, String[] defaultGroups, boolean processingOverrides,
Object loadPath) {
logger.info “Parsing source $source”
ExpandoMetaClass.enableGlobally()

def tempFile = File.createTempFile(‘rio-dsl’, ‘xml’)
def writer = new FileWriter(tempFile)
def builder = new groovy.xml.MarkupBuilder(writer)
Script dslScript = new GroovyShell().parse(source)

dslScript.metaClass = createEMC(dslScript.class, {
ExpandoMetaClass emc ->
emc.opstring = { Map attributes, Closure cl ->
builder.printer.println(‘‘)
builder.opstring {
OperationalString(Name: attributes.name) { cl() }
}
writer.close()
xmlParser.parse(tempFile, loader, verify, defaultExportJars, codebaseOverride, defaultGroups,
processingOverrides, loadPath)
}
emc.groups = { String… groups ->
builder.Groups {
groups.each { Group(it) }
}
}
emc.cluster = { String… machines ->
builder.Cluster {
machines.each { Machine(it) }
}
}
emc.service = { Map attributes, Closure cl ->
builder.ServiceBean(Name: attributes.name) { cl() }
}
emc.serviceExec = { Map attributes, Closure cl ->
builder.ServiceExec(Name: attributes.name) { cl() }
}
emc.spring = { Map attributes, Closure cl ->
builder.SpringBean(Name: attributes.name, config: attributes.config) { cl() }
}
emc.interfaces = { Closure cl ->
builder.Interfaces { cl() }
}
emc.implementation = { Map attributes, Closure cl ->
builder.ImplementationClass(Name: attributes.class) { cl() }
}
emc.execute = { Map attributes ->
builder.Exec(nohup: attributes.nohup ? ‘yes’ : ‘no’) {
if (attributes.inDirectory)
WorkingDirectory(attributes.inDirectory)
def String[] cmd = attributes.command.split()
CommandLine(cmd[0])
if (cmd.size() – 1 > 0)
cmd[1..cmd.size() - 1].each { InputArg(it) }
}
}
emc.classes = { String… interfaceClasses ->
interfaceClasses.each { builder.Interface(it) }
}
emc.resources = { String… resources ->
builder.Resources {
resources.each { JAR(it) }
}
}
emc.resources = { Map attributes ->
builder.Resources(attributes)
}
emc.resources = { Map attributes, String… resources ->
builder.Resources(id: attributes.id) {
resources.each { JAR(it) }
}
}
emc.configuration = { String configuration ->
builder.Configuration(configuration)
}
emc.associations = { Closure cl ->
builder.Associations { cl() }
}
emc.association = { Map attributes ->
builder.Association(Name: attributes.name, Type: attributes.type,
Property: attributes.property, MatchOnName: attributes.matchOnName ? ‘yes’ : ‘no’)
}
emc.maintain = { Integer maintain ->
builder.Maintain(maintain)
}
emc.maxPerMachine = { Integer max ->
builder.MaxPerMachine(max)
}
emc.maxPerMachine = { Map attributes, Integer max ->
if (attributes.type)
builder.MaxPerMachine(max, type: attributes.type)
else
builder.MaxPerMachine(max)
}
emc.software = { Map attributes ->
builder.SystemRequirements {
SystemComponent(Name: ‘SoftwareSupport’) {
Attribute(Name: ‘Name’, Value: attributes.name)
Attribute(Name: ‘Version’, Value: attributes.version)
}
}
}
emc.software = { Map attributes, Closure cl ->
builder.SystemRequirements {
SystemComponent(Name: ‘SoftwareSupport’) {
Attribute(Name: ‘Name’, Value: attributes.name)
Attribute(Name: ‘Version’, Value: attributes.version)
builder.SoftwareLoad(removeOnDestroy: attributes.removeOnDestroy ? ‘yes’ : ‘no’) { cl() }
}
}
}
emc.download = { Map attributes ->
builder.Download(InstallRoot: attributes.installRoot,
Unarchive: attributes.unarchive ? ‘yes’ : ‘no’, Source: ”) {
Location(attributes.source)
}
}
emc.postInstall = { Map attributes, Closure cl ->
builder.PostInstall(RemoveOnCompletion: attributes.removeOnCompletion ? ‘yes’: ‘no’) { cl() }
}
emc.serviceLevelAgreements = { Closure cl ->
builder.ServiceLevelAgreements { cl() }
}
emc.sla = { Map attributes, Closure cl ->
builder.SLA(ID: attributes.id, Low: attributes.low, High: attributes.high) { cl() }
}
emc.policy = { Map attributes ->
builder.PolicyHandler(type: attributes.type, max: attributes.max,
lowerDampener: attributes.lowerDampener, upperDampener: attributes.upperDampener)
}
emc.logging = { Closure cl ->
builder.Logging { cl() }
}
emc.logger = { String name, Level level = Level.INFO, Closure cl ->
builder.Logger(Name: name, Level:level.toString()) { cl() }
}
emc.handler = { String name, Level level = Level.INFO ->
builder.Handler(ClassName: name, Level:level.toString())
}
})
List opstrings = dslScript.run()
return opstrings
}

public parseElement(Object element, GlobalAttrs global, ParsedService sDescriptor, OpString opString) {
throw new UnsupportedOperationException()
}

static ExpandoMetaClass createEMC(Class clazz, Closure cl) {
ExpandoMetaClass emc = new ExpandoMetaClass(clazz, false)
cl(emc)
emc.initialize()
return emc
}

}
[/sourcecode]

As you can see, the above code uses the XML MarkupBuilder which makes it very easy to generate the XML format. After that, it’s just a matter of delegating the real work to the XML parser!