Ovidiu Predescu's Weblog

Richard Stallman at Google

Last week Richard Stallman, the father of Free Software, visited Google. He gave an introduction to free software and what it means. He emphasized on the differences between free software and open source, and why Linux is really GNU/Linux.

Having followed his arguments for few years now, they sort of make sense. I can see however why people have so many problems with his personality and ideology. He tends to quickly dismiss most of the questions with a "it makes no sense" type of answer.

As a hacker however, all the respects to him, he has managed to get us a long way from where software was in 1984 to where it is now. Emacs, GCC, GDB, most of the standard Unix tools that run on Linux today are directly due to him.

More on XEmacs for MacOS X

Hooray! I managed to get XEmacs' internal buffer structure hooked up into Cocoa's text system!

The first step was to build a library from the XEmacs source code, which I could link against my Cocoa-based project. This was pretty straightforward, adding a couple of defines and ifdefs to remove the main function from the library. I also added some rules to the makefile to build the actual library.

Then I built a very simple editor application using Cocoa's default text classes. Once this worked, I replaced the default NSTextStorage class with my own subclass that uses XEmacs' buffer internal structure to hold the characters. Since NSTextStorage's API makes use of NSString to get the string to be displayed, I also created a simple subclass of it, that accesses the characters directly from the buffer structure.

I can now edit my .bashrc file, insert, replace and delete characters in a window on screen, and have these operations reflect directly in the associated buffer structure.

One problem I need to solve is with Unicode characters. MacOS X APIs use unichar, a 16 bit short integer to represent Unicode characters, while XEmacs uses IChar, a 32 bit integer. I need to find a way to normalize XEmacs' multibyte characters to MacOS X's unicode characters. I haven't poked yet in the MacOS X's Foundation header files to see if there is anything that would do the job. If you know of something, please let me know.

The next step is to hook-up the XEmacs Lisp engine in the system. This is the first real challenge, since the important stuff in Emacs is done in Lisp.

Mailing list for XEmacs on MacOS X

I've created a mailing list for discussing the port of XEmacs to MacOS X. The mailing list is hosted by the new Google Groups2. If you're interested, please visit and subscribe to the xemacs-macosx mailing list.

XEmacs on MacOS X

For the past week I've been investigating porting XEmacs over MacOS X using the Cocoa, using the not-so-new text system, which is incredibly powerful.

All the previous attempts to port GNU Emacs or XEmacs to MacOS X have used the Carbon API, a lower level API used for compatibility with the older MacOS operating systems. The approach I have in mind plans to use the powerful display framework available in Cocoa.

So far I've investigated the internal organization of XEmacs and reviewed the major data structures. I don't have a rigorous plan yet, at this point I'm experimenting with different options available.

The best options so far seem to be the following:

• provide an Objective-C wrapper on top of XEmacs' buffer structures, that make these appear as an NSTextStorage subclass.
• for the display part, the NSTextView class provides most of the methods needed to implement the right display functionality. A proper Objective-C subclass would be able to intercept the mouse and keyboard events and update XEmacs' buffer structure with point information and new content.

Cocoa's display technology makes the display code in XEmacs redundant. One thing I'm worried about is the charset management and how to deal with different (and I think incompatible) methods for entering characters in different languages.

Nice to have, but non goals for this project:

• Expose XEmacs' internals to languages other than Emacs Lisp: Python, Ruby etc. would be really nice. This way even more people would be exposed to this great editor.
• port to GNUstep. Many years ago I contributed major components to this project. I don't know what is the status of the text system port in GNUstep.

Setting up ssh-agent on Windows XP

As you probably know already, ssh-agent is an easy way to enter the passwords for your private SSH keys only once per session. On Linux and Unix systems, when using X-Windows, it is very easy to setup ssh-agent as the parent process of your window manager. In fact most of the Linux distributions start-up the window manager this way.

The way ssh-agent works is by setting up two environment variables, SSH_AUTH_SOCK and SSH_AGENT_PID. The first is used to communicate the location of the Unix socket domain on which ssh-agent is listening for requests. The second is used to identify the Unix process id of ssh-agent, so it can be killed by ssh-add -k.

These environment variables have to communicated to every process that wants to use ssh later on, so ssh can connect to the ssh-agent process and fetch the decrypted private keys. In the Unix parent-child process model, this works just fine. The ssh-agent does the work of creating the Unix socket domain and then forks a child process. In this process it first exports the two environment variables above, then exec the process - the window manager for X-Windows. This way all the processes that inherit from it will have these environment variables available.

On Windows this is not possible, since there is no way to interpose some other process before the window manager. This of course, assumes the same parent-child relationship of processes as in Unix. The alternative is to always start ssh-agent on some well-known socket. Below, I assume you use Cygwin, an excellent free-software Unix emulator for Windows.

There are few things you need to do. First in your Windows home directory (usually c:\Document and Settings\yourusername, make sure you have a .bash_profile that reads:

. ~/.bashrc

Then create a .bashrc file in your home directory, and add to it the following:

export SSH_AUTH_SOCK=/tmp/.ssh-socket

ssh-add -l 2>&1 >/dev/null

if [ $? = 2 ]; then

   # Exit status 2 means couldn't connect to ssh-agent; start one now

   ssh-agent -a $SSH_AUTH_SOCK >/tmp/.ssh-script

   . /tmp/.ssh-script

   echo $SSH_AGENT_PID >/tmp/.ssh-agent-pid

fi

function kill-agent {

   pid=`cat /tmp/.ssh-agent-pid`

   kill $pid

}

Next, go to the Start menu, "Control Panel" -> "System" -> "Advanced" -> "Environment Variables" and add a new variable SSH_AUTH_SOCK, whose value should be /tmp/.ssh-socket. Hit OK to make the change persistent.

What happens next? The first time you open a bash terminal, an ssh-agent process is going to be automatically created. This process will listen on the Unix socket domain /tmp/.ssh-socket. Run ssh-add at the prompt to enter the password for your private key(s).

Now when you open another terminal, that will share the same ssh-agent process because of the SSH_AUTH_SOCK definition. Running ssh or any other command that uses ssh underneath will work without having to enter the password for your keys.

It will also work if you run a cygwin-ified version of XEmacs. Tramp, CVS or any other Emacs package that uses ssh will work just fine now.

The only requirement is for these programs to be cygwin-ified, otherwise the sharing described above doesn't work.

Tramp with XEmacs

As I mentioned in a previous entry, Tramp is a really nice emacs package for editing remote files. On my PowerBook laptop it works out-of-the-box with the carbonized GNU Emacs on MacOS X Jaguar, there is no need to install anything.

One thing I discovered while trying to use tramp at work on XEmacs 21.4.12/Linux is that tramp version 2.0.22 shipped with XEmacs has compatibility issues with efs, another Emacs package. To fix the problem I had to grab the latest tramp package and install it in my local ~/emacs directory.

One confusing bit was how to access the CVS repository. Following the instructions from Tramp's manual does not work because Savannah's CVS repository is no longer accessible using anonymous CVS. To access the CVS, you have to follow the instructions on Tramp's Savannah CVS page instead.

With this new version, Tramp uses now a different syntax to access remote files:

• /[machine].emacs or /[machine]~/.emacs - to open the ~/.emacs file on the remote machine.
• /[machine]/full/path/name/to/file - to open a file using its full path name

Assuming you downloaded tramp in the ~/emacs/tramp directory, you need to configure and compile the tramp files:

$ autoconf
$ ./configure --with-xemacs
$ make
$ make info

After this add the following to your .emacs file:

(add-to-list 'load-path "~/emacs/tramp/list")
(add-to-list 'Info-directory-list "~/emacs/tramp/info")
(setq tramp-default-method "ssh")
(require 'tramp)

Editing remote files in Emacs

Today I was browsing some of the plugins for blosxom (an excellent blogging tool BTW), and I encountered some Emacs code for preserving the last time modified timestamp on a file. This is especially useful for blosxom which uses this timestamp as the date to show when the post was made. So having Emacs automatically do this work for you is really neat. Each time I find such a nice little tool, I love Emacs even more ;)

Trying to hook-up that code in my Emacs I discovered it was referring to functions from Tramp, a package I never heard of before. After doing a quick search on Google, I discovered it's a mode very similar to ange-ftp, which allows the editing of remote files. However it works completely transparent over ssh too, really nice!

As with ange-ftp, you specify a remote file as /machine:filename or /user@machine:filename, if you want to login as a different user on the remote machine. File name and directory completion on the remote system works as expected, really cool!

The way this mode works is by uploading some simple programs on the remote system to allow the sending and receiving of files over ssh. You can also use scp as an alternative method of uploading and downloading files, which may be faster for larger files (I still need to investigate this option).

I'm quite impressed the way tramp works, and is definitely a much better alternative to starting a remote Emacs instance, especially if you're running an X11 version.

To use tramp, add the following to your .emacs:

(require 'tramp)

If you're using XEmacs, you will need to modify the save-buffer-same-timestamp function from here to read like below instead. Upon successful completion, funcall returns t on XEmacs, and as a result (= 0 t) throws an exception. The change below simply removes this check, as well as the enclosing and and if expressions.

(defun save-buffer-same-timestamp (&optional args)
  "Save the current buffer in visited file if modified.  Versions are
controlled with ARGS as with `save-buffer'.  The difference between
this command and save-timestamp is that this command saves the
modification time of the file from disk and resets it after the file
is saved."
  (interactive)
  (message "save-buffer-same-timestamp %s" (buffer-file-name))
  (let* ((buffer (current-buffer))
	 (file-name (if (tramp-tramp-file-p (buffer-file-name))
                        (tramp-file-name-path 
                         (tramp-dissect-file-name 
                          (tramp-completion-handle-expand-file-name
			   (buffer-file-name))))
                      (expand-file-name (buffer-file-name))))
	 (file-attributes (file-attributes (buffer-file-name)))
	 (time-string (if file-attributes 
			  (format-time-string "%Y%m%d%H%M.%S" 
					      (nth 5 file-attributes))))
         (shell-cmd (if (tramp-tramp-file-p (buffer-file-name))
			'tramp-handle-shell-command
		      'shell-command)))
    (save-buffer args)
    (funcall shell-cmd 
	     (format "touch -t %s %s" time-string file-name))
    (set-buffer buffer)
    (revert-buffer nil t)))

GNU Emacs on Panther

I've been told that the Emacs binary package I packaged does not work on the new MacOS X 10.3 Panther. Unfortunately, until I get a copy of the new operating system, I won't be able to produce a new package.

More Weblogs

I just found the weblogs of Norman Walsh and Tim Bray: subscribed.

nXML - new XML editing mode for Emacs

James Clark released nXML, a new Emacs mode for editing XML documents (via Kevin Burton). The new mode provides validation and schema-sensitive editing using a Relax NG Compact Syntax (RNC) schema. You can convert DTD and Relax NG XML schemas to RNC schemas using Trang, another tool by James Clark.

Unfortunately, nXML does not seem to work with XEmacs yet. I'll take a look at it to see if it can be easily ported.

Free software Java platforms

Richard Stallman is interested whether XSLT-process, my XSLT processor and debugger Emacs interface for the Saxon and Xalan XSLT processors runs on top of a free software JDK.

As far as I know Kaffe is one such free-software implementation of a Java virtual machine. There are many other implementations listed at Kaffe's Web site.

Does anybody have any experience with any of these in a real project? How much of the JVM functionality and the associated class libraries in JDK 1.3 are implemented? Can one run an XML parser and an XSLT processor like Saxon or Xalan on top of them?

XSLT-process 2.2 for Emacs released

I'm happy to announce a new release of the XSLT-process mode for Emacs. XSLT-process is an Emacs mode for applying and debugging XSLT stylesheets from within XEmacs or GNU Emacs. You can also use this mode to do processing of DocBook documents edited with Emacs.

Tony Addyman contributed most of the changes in this release. He's using Emacs and XSLT-process to teach XSLT to students in UK!

Here are the main changes since the 2.1 release:

• XSLT-process now uses Saxon 6.5.2, Xalan 2.4.1 and FOP 0.20.4
• Debugging using Xalan is now fully supported.
• Both JDK 1.3 and 1.4 are supported.
• During debugging sessions the output produced by executing the stylesheet is serialized immediately, then displayed and incrementally updated in an Emacs window.
• Added parameter passing to the XSLT processor. These are parameters that are specified to all the stylesheets, in addition to any other parameters. They are specified through the customization menu.
• Logging output, error messages, etc. generated by the FOP processor are displayed in the *Messages* buffer. The level of logging can be customized.
• Improvements to the display of information during debugging.
• MacOS X is now a supported platform, in addition to Linux and Windows 2000.

Updated Emacs binary for Jaguar

I've updated the binary package for Jaguar with a new version from CVS which seems to fix the crash problem that appeared in the previous version on each update of the operating system. Emacs would just segfault on each startup after the operating system would be updated.

Andrew Choi pointed out to me that some time ago Steven Tamm found a fix for this problem on emacs-dev. His fix seems to indeed solve the problem: I've compiled the package on 10.2.2 and it successfully survived the upgrade to 10.2.3, recently released.

So if you're running on MacOS X Jaguar and like Emacs, download the new Emacs binary package now!

Jaguar 10.2.2 breaks Emacs package

I've just upgraded my machine to 10.2.2. After the install, I've noticed the Emacs for MacOS X package I've put together breaks with segmentation fault on startup. I have no clue why this happens, so I recompiled the binary and everything seems to be working fine again. I'm uploading the new binary as we speak, and it should be up on the Web site shortly.

Emacs addiction

I'm addicted as well. I tried many times switching the editor, but none of the tools I've used we powerful enough to justify such a move. The primary reason why I like Emacs is because it's so easy to extend, once you learn how. None of the "extensible" editors I tried are as powerful as Emacs. In Emacs Lisp you can control everything, in fact most of what you see in Emacs is written in Lisp. The rest of the extensible editors ask you to write a plugin or code in some interpreted language, but they don't expose enough of the underpinnings for you to be able to control everything.

A truly powerful environment, no matter is an editor or a Web application, should expose its most intimate internals to a scripting language. In practice this is hard to achieve, very few environments have achieved this. Emacs is one example, Smalltalk the environment, is another one. No code compilation, no code generation crap, just write scripts.

You won't believe how powerful and still modern Emacs is. PSGML for editing of XML documents according to a DTD, including complex ones like DocBook, XSLT debugging (written by me) and editing and many others.

GNU Emacs package for MacOS X 10.2 Jaguar

Since there was interest in this, I've created a binary package for GNU Emacs, ready to be installed on MacOS X Jaguar. You can find all the necessary installation bits on my Emacs on MacOS X Web page, together with some hints on how to set it up so it integrates nicely in the MacOS X environment.

Emacs customizations for MacOS X

Here are some customizations I've found necessary to make to have a nicer integration of Emacs for MacOS X.

Download the redo package and place it somewhere in the load-path of Emacs. Add the following in your ~/.emacs file:

(setq mac-command-key-is-meta nil)
(global-set-key [(alt v)] 'yank)
(global-set-key [(alt c)] 'kill-ring-save)
(global-set-key [(alt x)] 'kill-region)

(global-set-key [(alt q)] 'save-buffers-kill-emacs)

(require 'redo)
(global-set-key [(alt z)] 'undo)
(global-set-key [(alt shift z)] 'redo)

These settings essentially bind some frequently used commands on MacOS X like copy, cut, paste, undo, redo and quit to their Apple key equivalents.

If you're interested in starting to use Emacs on MacOS X, let me know. If there's enough interest, I can make a binary package of them and share my .emacs customization file.

Emacs on MacOS X

I found David Wright's Weblog doing a search on Google for how to link MacOS X and Linux via NFS. While reading it, I came across a link to a port of Emacs 21 to MacOS X. Really cool! I'm downloading it right now, and since is a bit slow I thought I might enter this note here. It would be great if I can have Emacs running natively on MacOS X, even if it's not XEmacs.

Weblog publishing using Emacs

While reading some weblogs this night, I found Kevin Burton's records-mode for Emacs to be a really promising tool to maintain a Weblog from my editor. Browsing the code did not reveal the important component of publishing the news using Blogger API, but it should be easy to add it.

Now if I can only have XEmacs running natively on MacOS X... It's so annoying to run it in X-Windows emulation mode. Maybe after I finish the Cocoon control flow layer, and implement the user feedback application for Anteater using the new Cocoon, I'll look more closely at porting XEmacs.

Upgrading the XSLT-process package in XEmacs

Ville Skytt? contacted me few days ago regarding the status of the XSLT-process in XEmacs. I've been really lazy updating this package in the XEmacs distribution, it's version 1.2.1, more than a year old. The primary reason is that the 2.1 version of XSLT-process is about 6.5Mb, with all the Java jar files for doing Docbook translation to PDF and HTML. This is really bloated by XEmacs standards, whose total size of all packages is about 20Mb. I simply cannot imagine having a sumo distribution with more than a quarter of it being my package! I would probably be spammed by all the XEmacs users...

Steve Youngs suggested a while ago to add the ability to download the optional jar files off the Net, when the user actually needs them. Today he pointed out to Ecrypto, recently added to XEmacs packages, which adds cryptographic functions to Emacs. However these functions are low level functions for encryption only, and there's no provision for verifying signatures and all the goodies tools like PGP and GnuPG provide.

In a separate email, Ville Skytt? pointed out that I can use mailcrypt to sign the jar files that need to be downloaded on demand. I think this tool is what I need, although to make sure things are secure, the user must have gpg or pgp installed on their machine. Linux users might have it if they run a recent Linux distribution, but Windows and even MacOS X, don't come with GnuPG installed.

I need to look into what means to sign a jar file using Java tools only. I strongly suspect the out-of-the-box JVM installations recognize only Sun signed files, and there are little provisions to add third party public certificates. In which case this cannot be an option.

Xalan debugging support

Tony Addyman managed to add debugging support for Xalan 2.4D1 in XSLT-process for Emacs! Now XSLT-process supports both Saxon and Xalan as XSLT debuggers.

Xalan debugging support in XSLT-process has been problematic because of the lack of source location information. During late June 2001, I've modified Xalan to add support for file and line information in both the source XML and in the stylesheet.

For some reasons this code was not working in the 2.2 release, which made the XSLT-process support unusable. Because of the rapid changes in Xalan and the lack of time, I've decided to drop the debugging support for Xalan in XSLT-process. Tony took up the support for this and he's doing a great job at it!

Weblog interface in Emacs

It would be nice to be able to create new Weblog entries and edit existing ones using (X)Emacs. Similar standalone programs that make use of the XML-RPC API already do this.

How much time would take to port XEmacs to MacOS X?

Today I spoke with Christophe de Dinechin about his work on porting GNU Emacs to the MacOS X user interface. His job was a lot easier since he started with an old NeXTSTEP port of GNU Emacs. There is no such thing for XEmacs, everything would have to be written from scratch.

Missing XEmacs

One month in MacOS X exclusively. One program I badly miss on MacOS X is Xemacs. On my Linux desktop, I used to do pretty much everything in Xemacs. My mail program was setup to compose and reply to messages in Xemacs. All my development work, C, Objective-C, Java, XML and XSLT happened in Xemacs.