Monday, February 25, 2008

Integrating scripting languages into your DSL using JSR-223

I've been building tools for Model-Driven Engineering (sometimes known as Model-Driven Development) now for about 11 years, but this is the first time that I've integrated scripting capabilities into a DSL using Java.  The opportunity came up when creating a DSL for modernizing legacy data for my company MAKE Technologies Inc.  Following is a summary of how this was done.

The idea is that the functional capabilities of the DSL can be extended by scripting the behaviour in the DSL itself.  By integrating with JSR-223 APIs we can support as many scripting languages as are JSR-223 compliant.  By the looks of scripting.dev.java.net, there are many including Java itself, Groovy, JRuby, Python, BeanShell, ECMAScript (JavaScript) and PNuts.

The DSL defines the concept of a 'caster', which can cast an arbitrary string value to some other value.  For example, a trivial boolean caster might consist of a function that can cast a 'Y' or 'N' value to a boolean true or false.  So we define it as follows:

<caster language="groovy" name="YesNoToBooleanCaster">
String cast(text) {
if (text == 'Yes') {
return true
} else if (text == 'No') {
return false
} else {
throw new Exception("Unexpected value '$text'");
}
}
</caster>
Elsewhere in the DSL we can refer to the new caster by it's name YesNoToBooleanCaster.

The code to make the new caster work is relatively simple using the javax.script APIs:



String languageName = "groovy"; // get the language from the DSL instance
String script = ""; // get the script from the DSL instance

ScriptEngine scriptEngine = new ScriptEngineManager().getEngineByName(languageName);
Invocable invocable = (Invocable) scriptEngine.eval(script);

Object value = invocable.invokeFunction("cast", "Yes"); // test it out
assert Boolean.TRUE.equals(value);
With a little more work, I was able to hook up auto-suggest for the language attribute in the DSL editor (ScriptEngineManager can tell you which languages are supported) and provide validation of the scripting (ScriptEngine.eval can be used to detect script syntax errors).

All of this and the script runs very quickly at runtime, thanks to bytecode compilation by most supported scripting languages!

The only real issues that I encountered were:
  • extracting useful error messages from exceptions thrown by ScriptEngine.eval can be tricky.  For example, JRuby provides the information in a getException() method on the Exception object.  Some reflection and chained exception unraveling was used to overcome this problem in a language-independent manner.
  • Older Java 6 VMs (developer preview versions) have old preview versions of the JSR-223 APIs... stick to a non-beta version of the Java 6 VM if you can, or if you're developing for Macs then you may have to catch NoSuchMethodError and use some reflection magic.


All in all I was very impressed with the ease of using JSR-223.  The end result when used with DSLs is that it is very easy to provide an extensible DSL API with a very low barrier to entry.

No comments: