Ovidiu Predescu's Weblog

Convert iTunes playlists to Slimserver m3u playlists

I own few Slimdevices Squeezeboxes, which provide a great way to listen to your digital music. Through a regular remote control and the VCD screen you can choose the album, song or playlist that you want to listen to.

I rip all my music from CDs and organize and store it in iTunes. (By the way, the reason I never buy music from the iTunes Music Store is that I cannot play the DRM crippled songs on my squeezeboxes.) I then copy all the music to the hard disk of a small machine, which runs slimserver that serves the music to my squeezeboxes.

One thing that kind of bothered me for a long time was that iTunes playlist support for the slimserver software was somewhat broken. So I wrote a small script to convert the iTunes playlists into m3u playlists, the format used by slimserver. Below are the details.

XSLT whines

Bertrand writes:

I cannot agree more, although I very much like the data-driven execution model in XSLT. I also like the XPath data accessing model.

The top two things I don't like about XSLT are:

• The XML syntax, which makes programs very verbose.
• The extensibility mechanism is language agnostic, but it makes it difficult to write complex XSLT/Java programs that integrate nicely.

From this perspective, the SXML processing tools available for Scheme are a very nice integration of XML in a programming language. In SXML the XML processing is described as a real Scheme program that has access to all the library functions for traversing and accessing the XML tree. I wish Java had something of a similar simplicity, but instead it got all the DOM and SAX crap, and no portable XPath library implementation.

What is even scarier is that XSLT 2.0 is moving towards becoming a full programming language with an XML syntax.

One thing I started to pay more attention to is XQuery, and the excellent implementation done in Qexo by Per Bothner. XQuery promises to solve some of the XSLT verbosity I mentioned above. The processing model is closer to functional programming, although it does appear to loose the data-driven processing model I like in XSLT.

More on advanced XSLT usage

I found this old article by Jason Diamond via Leigh Dodds. The article describes the same technique I was referring to in a previous entry. The article refers to an even older article by Eric van der Vlist describing the same technique.

The idea is to come up with your own set of XML elements that describe best your problem domain, annotate your HTML pages with these XML elements, and then use an XSLT stylesheet to transform the annotated HTML page into another XSLT stylesheet. This last XSLT stylesheet contains all the presentation information from the original HTML; it is applied on the real input data model to generate the output HTML.

A traditional user of XSLT would write this last stylesheet by hand, mixing the presentation with the actual code that traverses the data structure to be represented. To do this, the developer would first have to reverse engineer the structure of the HTML page, and embed that structure into the stylesheet. The process is time consuming, frustrating and error-prone; the result is ugly and hard to maintain.

Annotating an HTML page and generating the stylesheet automatically is a much easier solution to maintain in the long run. The HTML page designer doesn't have to know XSLT, they would follow exactly the same design process as before. The only thing they need to learn are the extension XML elements that describe where and how the dynamic data will be placed into the HTML page.

The challenge is on the developer of the extension XML elements to be able to cover all the corner cases, and make it possible for the designer to easily embed the data in the page templates.

Processing XML configuration files using XSLT

Charles Miller decided to use Scheme to describe configuration files instead of XML.

I have a different approach. Since most of the applications I write these days end up processing XML anyways, I prefer writing XML configurations file. Since I really like XSLT's power and to avoid the proliferation of tools like commons-digester and others, I write a simple XSLT stylesheet to process my configuration files. Such a stylesheet would need to receive as parameter a context Java object which needs to be initialized according to the configuration file.

The object initialization could be written in simple XSLT templates. Using Charles' example, lets say we have a hypothetic add-module XML element with a module attribute. We'd like to have a MyClass instance that gets invoked with a addModule method and passed the string value of module. The following XSLT stylesheet would do it:

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:MyClass="java:some.package.MyClass">

  <!-- This is the MyClass instance to be initialized -->
  <xsl:param name="context"/>

  <xsl:template match="add-module">
    <xsl:value-of select="MyClass:addModule($context, @module)"/>
  </xsl:template>

</xsl:stylesheet>

An instance of MyClass would need to be passed as parameter to the XSLT stylesheet from the Java side of the world. This can be set using the Transformer.setParameter(String, Object) method.

XSLT comes for free in Java 1.4, so there is no memory overhead from using XSLT. Even so, I still prefer Saxon over Xalan 2.3.1 that is distributed in JDK 1.4.

Advanced XSLT usage or using XSLT to generate XSLT

XSLT is a very powerful language. Most people use it to convert XML to HTML: they have an XML document which needs to be translated to a rich HTML document.

The simplest way to do this is to identify how the input XML document maps to the final HTML page, and come up with the XSLT stylesheet that embeds in its templates the HTML elements of the end result. While this works fine for one HTML page, it becomes really tedious if the page needs to change often or if you need to format the input XML multiple times.

A much better approach is to take the resulting HTML page and annotate it with your own special elements. These special elements are invented for your specific problem to delimit portions of the HTML page that need to be replaced with dynamic content from the XML input file.

Once you've annotated the output file, you can write an XSLT program that takes the output page and generates an XSLT stylesheet. The generated XSLT stylesheet will take the input XML document and generate an HTML page with exactly the same structure of the page you want.

There are few things that you need to worry about when writing such an XSLT program. The output you generate is not a generic XML output, it is an XSLT program which needs to follow the XSLT namespace. To generate this namespace you need to use the xsl:namespace-alias element to alias a namespace in the current document to the XSLT namespace:

<xsl:stylesheet
  version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:gen="dummy-namespace-for-the-generated-xslt"
  exclude-result-prefixes="xsl">

  <xsl:namespace-alias stylesheet-prefix="gen" result-prefix="xsl"/>

  <xsl:template match="/">
    <!-- Generate the structure of the XSL stylesheet -->
    <gen:stylesheet version="1.0">
      <gen:output method="html"/>

      <!-- put the logic for the generated XSLT here --!>
      <gen:template match="...">
        ...
      </gen:template>
    </gen:stylesheet>
  </xsl:template>
</xsl:stylesheet>

Writing such XSLT programs is not trivial, but you'll appreciate how clean they are. You will not have to embed in your stylesheet any of the output HTML that you need to generate; the generate stylesheet will have all that.

XSLT-process 2.2 for Emacs released

I'm happy to announce a new release of the XSLT-process mode for Emacs. XSLT-process is an Emacs mode for applying and debugging XSLT stylesheets from within XEmacs or GNU Emacs. You can also use this mode to do processing of DocBook documents edited with Emacs.

Tony Addyman contributed most of the changes in this release. He's using Emacs and XSLT-process to teach XSLT to students in UK!

Here are the main changes since the 2.1 release:

• XSLT-process now uses Saxon 6.5.2, Xalan 2.4.1 and FOP 0.20.4
• Debugging using Xalan is now fully supported.
• Both JDK 1.3 and 1.4 are supported.
• During debugging sessions the output produced by executing the stylesheet is serialized immediately, then displayed and incrementally updated in an Emacs window.
• Added parameter passing to the XSLT processor. These are parameters that are specified to all the stylesheets, in addition to any other parameters. They are specified through the customization menu.
• Logging output, error messages, etc. generated by the FOP processor are displayed in the *Messages* buffer. The level of logging can be customized.
• Improvements to the display of information during debugging.
• MacOS X is now a supported platform, in addition to Linux and Windows 2000.

Minor changes in XSLT-based RSS aggregator

This evening I've changed the format of the subscriptions file in the XSLT-based RSS aggregator to include both the URL of the site and its RSS feed. I found several Weblogs which don't have RSS feeds.

Since I switched to NetNewsWire for reading the news, I no longer need to aggregate news on the main page. Instead I now generate the list of subscriptions at the right of each page using this XSLT stylesheet. Quite fun!

To solve the problem of the RSS2 variable namespace, I think a good solution is to have the main stylesheet automatically generate another stylesheet which uses the RSS feed's namespace as default namespace. This will avoid using ugly hacks like local-name() and other oddities in the XSLT aggregator stylesheet. More on this later.

XML scripting and XSLT

Ugo Cei comments on my previous comparison of native XML scripting with XSLT:

Maybe the procedural model is more understood, but is this a reason to use an inferior solution? Using goto is easier to understand than structured programming, but Dijkstra showed us that goto is harmful. Structured programming is easier to understand than object oriented programming, but does anyone really advocate throwing objects out of the window? Procedural programming is easier to understand than functional programming, but would you choose a procedural solution over XSLT? I would almost never.

Then James Strachan responds:

XSLT is great for processing an XML document in a declarative way and outputting a new XML document result. If you wanna process the XML in a different way, like work with database, invoke web services, work with your business objects then using XPath with a real programming language (say a procedural OO one) is often a good idea.

I don't think there's one size fits all uses when it comes to processing XML, its quite a broad topic.

Many times I found myself as developer of various frameworks that work with XML in great need to access XML nodes. For that you definitely some ways to access and manipulate the document's data model, and integration between the programming language (Java here) and the XML representation is crucial. JXPath has been very successful in doing just that.

However I don't think a developer of an application should use such low level features. Instead, higher level frameworks should be used instead, since they hide all the low level complexities of the XML data model, and allow for nice ways to process the XML.

SOAP-RPC libraries are a good example of that, there's no need to understand the format of the messages that go back and forth. Cocoon is another example of framework which gives a wide range of options for processing various XML sources, without having to go into the details of manually parsing or explicitly manipulating XML nodes.

RSS 2.0 support in my XSLT news aggregator

I've implemented support for RSS 2.0 in my XSLT-based news aggregator. It turns out to support it I only had to copy the RSS 0.92 templates, and add the proper namespace. Really simple! Since I was here, I also added a fairly simple fallback mechanism, so if a new RSS is invented, the aggregator simply displays the information specified in the subscription.

I've placed all the files in a CVS repository, together with my MovableType templates.

Xalan debugging support

Tony Addyman managed to add debugging support for Xalan 2.4D1 in XSLT-process for Emacs! Now XSLT-process supports both Saxon and Xalan as XSLT debuggers.

Xalan debugging support in XSLT-process has been problematic because of the lack of source location information. During late June 2001, I've modified Xalan to add support for file and line information in both the source XML and in the stylesheet.

For some reasons this code was not working in the 2.2 release, which made the XSLT-process support unusable. Because of the rapid changes in Xalan and the lack of time, I've decided to drop the debugging support for Xalan in XSLT-process. Tony took up the support for this and he's doing a great job at it!

Blogroll using RSS feeds for MovableType

I've finally managed to integrate my simple Blogroll engine into the MT system. The sytem is a combination of XML/XSLT, MT regeneration of HTML pages and crontab.

RSS feed aggregator

I've started working on a simple RSS feed aggregator. It is written entirely in XSLT, and so it has a very small footprint.