Ovidiu Predescu's Weblog

Extending the Java language: hygienic macros

The idea of hygienic macros like in Scheme for language with "non-regular", Lisp-like syntax is something which had me thinking for a long time. The challenge is how to provide not only semantics to new language constructs through macros, but also syntactic extensions to the language. Macros are possible in Scheme because of its uniform syntax, the new language constructs introduced by macros are just conventions translated to lower level constructs using the same regular syntax.

In Scheme you don't need any special support in the parser, macros are built entirely within the language. This is possible because of the simple syntax (or lack of it I should say ;) In Java this is not possible, you need compiler support for this to happen.

One idea would be to implement the parser for the Java language in an extensible way, which could allow new syntax extensions to be added at runtime. The only thing we would need to add new to the language is the ability to define "hygienic macros", e.g. ways to extend the syntax of the language understood by the compiler. The compiler's parser would need to be able to read these macros and record their definitions. In the process of doing this it has a chance to verify the new syntax described in the macro is still a valid grammar. During the syntax tree construction, it could invoke the defined macros to do tree transformation, which would transform the extended language into a pure Java syntax tree.

These Java macros would do nothing else than manipulate the syntax tree produced by the parser. Now while this would not be as powerful as Scheme's hygienic macros, it would still be a good alternative for all these language extensions. We won't need JSRs to extend the language in different ways, the only needed is a JSR to define the Java hygienic macro.

I wouldn't expect all the developers to start extending the Java syntax in their own way. Only experienced developers will do it. We could quickly have a library of such useful extensions which become used a lot.