December 05, 2002

Extending the Java language: hygienic macros

The Missing Sock - San Francisco. Carlos Perez - ::Manageability:: writes: The same thing was alluded too by Guy Steele in his paper, he suggested that the Java compiler be open sourced. Nevertheless, it didn't happen, however all is not lost. There's another Java compiler out there that's open source and extremely robust, its the Jikes parser from IBM. However, if you don't like hacking in C, you can also use the compiler that comes in Eclipse, it uses the same Jikes Parser Generator that Jikes uses, but its better its all written in Java! Furthermore, it has the cross JVM capability you are looking for, just look at the preferences in Eclipse. Also, the generics stuff is already in there, it was turned off when Sun decided to remove it from JDK 1.4. I also wrote earlier that the AspectJ compiler extends from the Eclipse compiler. Now, all you need now is a Hygenic Macro facility and you can make up any language construct you like!

The idea of hygienic macros like in Scheme for language with "non-regular", Lisp-like syntax is something which had me thinking for a long time. The challenge is how to provide not only semantics to new language constructs through macros, but also syntactic extensions to the language. Macros are possible in Scheme because of its uniform syntax, the new language constructs introduced by macros are just conventions translated to lower level constructs using the same regular syntax.

In Scheme you don't need any special support in the parser, macros are built entirely within the language. This is possible because of the simple syntax (or lack of it I should say ;) In Java this is not possible, you need compiler support for this to happen.

One idea would be to implement the parser for the Java language in an extensible way, which could allow new syntax extensions to be added at runtime. The only thing we would need to add new to the language is the ability to define "hygienic macros", e.g. ways to extend the syntax of the language understood by the compiler. The compiler's parser would need to be able to read these macros and record their definitions. In the process of doing this it has a chance to verify the new syntax described in the macro is still a valid grammar. During the syntax tree construction, it could invoke the defined macros to do tree transformation, which would transform the extended language into a pure Java syntax tree.

These Java macros would do nothing else than manipulate the syntax tree produced by the parser. Now while this would not be as powerful as Scheme's hygienic macros, it would still be a good alternative for all these language extensions. We won't need JSRs to extend the language in different ways, the only needed is a JSR to define the Java hygienic macro.

I wouldn't expect all the developers to start extending the Java syntax in their own way. Only experienced developers will do it. We could quickly have a library of such useful extensions which become used a lot.

Posted by ovidiu at December 05, 2002 12:25 PM |
Copyright © 2002-2016 Ovidiu Predescu.