Closures

As an old Smalltalker the addition of closures to the Java language sounds awfully good. Not a coding day goes by or I have a tinge of frustration when I have to do extraneous typing that would have been unnecessary in Smalltalk, where closure are the bread and butter of programming. So you would expect that I would be thrilled. However, I am having some reservations because of a very non-functional aspect of the two main proposals (BGCA and FCM): class loading. Let me elaborate.

About two years ago I played with Groovy and very much liked the builder pattern. With the builder pattern you can create code that looks like HTML but has access to all of Groovy's extensive features. This was great! A friend and I created some examples from an existing web application and it looked wonderful. Unfortunately, it turned out that there was a huge penalty because every closure was compiled into a class. With thousands of closures, class loading quickly became a performance issue. Just imagine heapspaces of many gigabytes (most of them strings, perm spaces that grow to hundreds of megabytes or more).

Knowing closures, I am convinced that untold developers will enthusiastically use them and also create thousands of closures in an average application. This will add a significantly to the number of classes in an application. And though I am the first to admit that class loading is fast, (and even faster in OSGi), the cost is absolutely not zero. And this worries me because when I look at the big vendors in the J2EE application server market than I know that their code bases are already creaking at the seams. You would be impressed if you knew the tricks that were being played to keep the system from grinding to a halt.

Until now, we have been more or less saved by computers becoming faster and faster, and disks growing bigger and bigger. However, the news I read today tells me those good days are over. Yes, we will get more and more cores, but each core will become slower. Though you can do some parallelization of class loading, there are many serial bottlenecks that are unavoidable.

The worst thing of all is that I do not fully understand why the proposals could not compile to methods. Methods are much cheaper than classes because the overhead of class loading is mitigated over all the methods in a class.

To make one thing clear, OSGi class loading is much more efficient than other class loading models due to its network nature instead of hierachy. An OSGi class loader knows authoritively who to consult for a class. OSGi based systems will have the least number of problems.

I do realize that it is usually not a good idea to let a high level design be influenced by low level details like class loading. However, I am ringing the alarm bell in this case. I am seeing too many problems at the bottom of the system that are caused by the people at the top that seem oblivious of the problems they are causing downstream. When you share resources, which is what you do in an application server, you must accept some responsibility for the commons or in the end face reduced performance.

Both JRuby and Groovy now compile closures to methods. Hey! If the dynamic guys can do it, then shouldn't we be able to do this as well in Java?

Peter Kriens