Metagnostic
home

JNIPort for Dolphin Smalltalk

Overview

Contents

Players

Layers

Examples

Configuration

InFAQ

Changes

Licence


Back to Goodies

JNIPort Problems and Limitations

This section contains some notes on what I see as the weaknesses of JNIPort.


Stability issues

JNIPort is based on the “real” Java runtime. That has great advantages in terms of performance and compatibility. It brings corresponding disadvantages in the way that the Dolphin session is exposed to the whims and derelictions of the implementers of the Java runtime — which is, after all, just a DLL.

Unfortunately, that means that using JNIPort can crash a Dolphin image. Using the Sun JVMs, the problem has grown smaller and smaller with each successive release, but there still are very occasional crashes:

  • The biggest problem is, and probably always will be, java.lang.System.exit(). There are only three ways to deal with Java code that calls this pernicious method:

    1. Do nothing; just hope that nothing will call exit(). If it does then the Dolphin session will also exit. You have no chance to save data. Actually this approach does work fairly well in practice, but is obviously not a perfect solution.

    2. Use the JNI exit “hook” to trap and ignore attempts to exit(). This ought to work, but does not — at least not with any implementation I've yet tried. Sun's JVMs up to 1.4.0 used to deadlock if you provided an exit hook that didn't exit. 1.4.1 has improved the picture slightly, and it now hits an internal breakpoint (see below) instead.

    3. Use Java's internal security policies to forbid all calls to exit(); that might work, but I haven't tried it yet.

    Or, I suppose, one could take a sharp stick and go “reason” with the code's author.

  • The programmers at Sun tend to forget that their Java runtime may only be a part of a bigger application; they leave breakpoints in the code rather than returning errors. I presume that their reasoning is that the JVM is no longer in a trustworthy state, so it's best just to exit. They don't consider the possibility that the rest of the application might be able to continue, at least enough to allow you to save data, etc.

    The symptom is that you get a Windows error message telling you that a breakpoint has been reached, and giving you the option of killing the process (the default) or debugging it. I have sometimes been able to continue for a little while by letting the system start the debugger (MS VC++ in my case) and using it to “continue” the application. The Dolphin IDE's in a decidedly unstable state by that point, though, so it's better not to do more than save any work, and perhaps do a little debugging to see how the breakpoint was triggered.

  • Any complex code can have problems; the Sun JVMs have a history of problems with floating point handling, for instance. Since Dolphin and the Java DLL are sharing a process space, Dolphin is exposed to any bugs in the Java VM's implementation. In general the latest JVMs seem pretty stable, but I still see crashes every couple of months or so. Usually they don't kill Dolphin, but that can happen too.

See the notes on individual JVMs for other information about specific problems with individual vendor's products.

Most current Java VMs for Windows use operating-system threads. (They may multiplex more than one Java thread onto an OS thread, but that doesn't affect this issue, except possibly to make it worse). Dolphin executes all Smalltalk code on just one OS thread. This mismatch of models is described in the problem with threads. It causes a number of problems, most significantly it means that I've had to use some very slow and restricting workarounds to allow any callbacks from Java to Smalltalk at all. Even with the workarounds, it is possible to create situations that deadlock. They can be avoided by taking sufficient care, but it's difficult to get right (even more difficult than avoiding deadlocks in normal threaded code, which is challenging enough in itself). The simple solution is not to use callbacks and Java threads together.

One last point under this heading: I am not IBM (or Sun, or Microsoft), I do not have the resources for extensive testing. But JNIPort is a fairly large piece of software; if I'd tested to the extent that I'd wish (either out of a programmer's natural desire to do a good job, or in my other role as a user of JNIPort) then it'd still be ages away from first release.


Performance issues

I think the biggest curable issue with JNIPort performance is that creating ghost classes can be very slow. This isn't so much because it takes a long time to create a ghost class, but because creating a ghost for one class means that all other classes that it refers to (method parameters, and so on) also get imported. Since these other classes also have ghost classes created for them, the process only stops when the transitive closure of the inter-class references have all been loaded. In theory that could load every class on the planet. In practice, the worst case I've seen is Swing, where referring to any one class is likely to drag in a large fraction of Swing — several hundred classes. Switching to a less eager scheme for creating ghost classes is my single highest priority for the future of JNIPort. Stop press: actually that is available now, if you use a JVMLazyGhostClassMaker instead of an JVMGhostClassMaker — the feature is still experimental, though.

Callbacks from Java into Smalltalk could never be very convenient, since the JNI model is that all callbacks are implementations of “native” methods (so there is no instance-specific behaviour, and no natural equivalent of a function pointer). However, because the the problem with threads, JNIPort can't even make full use of that (basically because it can't pass any object references across the interface from Java up into Smalltalk). Therefore JNIPort includes a workaround that allows Java code to construct and enqueue “request” objects that hold the parameters and return value of a callback. The only data that passes from Java to Smalltalk is a notification that the queue of requests is not empty, the rest of the process of serving the requests is driven from Dolphin (calling down into Java). This works but is not convenient for the programmer; nor is it quick. In fact it is very slow: in the millisecond range and up, which is too slow for many applications. Currently, I have no ideas at all about how this could be improved.

Whenever a new reference to a Java object enters the JNIPort system, it must generate a wrapper object. The lookups involved in finding the right classes mean that the performance is less than stellar. Wrapping a reference takes (depending on the machine, and what optimisations JNIPort can use) 10 to 100 microseconds — which is quite a long time by computer standards. It might be possible to use some sort of lazy wrapping scheme to reduce the overhead, or perhaps some kind of cache, but I haven't investigated either the feasibility or the potential benefits yet. I don't see this as high priority.


Functional issues

Probably the single most irritating thing about JNIPort (in a development context) is that you can't either run different Java runtimes at the same time, or stop the current one and start a new one. This comes down to Sun's failure to implement their own published APIs (though all the other JVMs I've tested have the same failings). There's nothing that can be done about it; if you want to start a new Java runtime, then you have to quit and restart Dolphin (fortunately, that's a fast operation).

Currently there is no way to unload a reference to a Java class. Once JNIPort has seen a reference to it (say because it's seen an instance of a subclass) it will hold onto that reference until the Java runtime is shut down. This prevents the class being garbage collected by Java. The issue may be trivial for some potential applications of JNIPort, but it directly affects my own work, so it's high on the list of things to fix. Stop press: actually that is available now, if you use a “supplementary classloader” — the feature is still experimental, though.

The methods in category 'documentation' on the class-side of JVM contain some more notes about bugs and deficiencies in the current version of JNIPort.

I you happen to think that the new “generics” in Java are a good idea (I most certainly do not) then you may be disappointed to find that JNIPort doesn't support them. In fact it can't, because generics are a fiction created entirely as a gross hack in the Java compiler, and are not seen by the Java runtime at all. Since JNIPort talks to the Java runtime, and has nothing whatever to do with the Java compiler, it can only see and manipulate the real classes, objects, and methods.


Other issues

OK, yes, I admit it. The documentation's not complete. It probably never will be. Live with it.

The documentation doesn't have enough structure to the cross-links either, I do intend to fix that.

JNIPort lives in a large number of Dolphin packages. It depends on a rather large number of further packages of code I've written. Most of them are marked “unpublished” (see here for an explanation of what I mean by that). So it brings a lot of baggage with it. There's very little I can do about that: I can't move the code from the prerequisite packages into JNIPort because other of my stuff depends on it (and not on JNIPort), and anyway that would just hide the problem. I don't have a good solution, or I'd be using it. I'm afraid it's just the practical downside of code reuse…


Copyright © Chris Uppal, 2003-2005

Java, JNI (probably), JVM (possibly), and God knows what else, are trademarks of Sun Microsystems, Inc.