OSGi Alliance at the end of 2006

The end of the year is nearing and that is a time to reflect. What did we achieve this year and where will we be heading to? Bear with me.

Evangelism
For me, this year was special because for the first time in my life I spent much of my time in non-technical work. The OSGi Alliance board had appointed me their official evangelist. That was a bit getting used to; suddenly I had to visit conferences, present OSGi technology and just talk about it instead of doing it. The hardest part was learning how to consistently use the word “OSGi” as an adjective for some trademark reason.

This evangelism thing is a bit hectic: EclipseCon in Santa Clara, ApacheCon in San Diego, ApacheCon Europe Dublin, Oredev in Malmö, Javapolis in Antwerp, CCNC in Las Vegas, OOPSLA in Portland, and BCC in Birmingham. Busy, but most of the conferences were very gratifying because I met so many people that are enthusiastic about the OSGi technology. It never ceases to amaze me that there are so many people that can enthusiastically tell me that they had been using OSGi for many, many years; most of the times from companies that I never heard of. But I love those stories and they make my work very rewarding. Next year it looks like there will be a similar schedule so do not hesitate to talk to me when you see me in my “Ask me about OSGi” shirt..

Mobile Specifications
In 2006 we had to finalize the Mobile specifications, which had some political hurdles. The project had run as a joint project between the JCP (JSR 232) and the OSGi Alliance. One of the key aspects has been the device management; a specification that uses the same concepts as the OMA DM protocol. JSR 246 had developed a similar specification for MIDP that was originally based on the work done in the OSGi but unfortunately deviated significantly. Fortunately, the spec leads Jon Bostrom (Nokia) and Jens Petzold (BenQ) realized that two similar but different specifications in the JCP would not be such a good idea. It was therefore great that the companies worked together and aligned the APIs.

There is a always a lot of work in finalizing specifications. Not only must the text be edited, reviewed, and reviewed, and reviewed … it is also necessary to ensure that the reference implementations and test suites are correct and nicely packaged. The mobile specification was truly an international effort, which did not make it easier. We had teams in Brazil, US, Hungary, Finland, and Bulgaria all working together to finalize the required artifacts. Great work and thanks all for the cooperation and patience!

After the mobile specification was finalized, I received one of the first prototypes from Nokia, an E70, that offered an OSGi framework. I remember racing to the local UPS to get it when it had arrived in Montpellier last June. The first versions needed some work but if everything works out I should get a new version today! I hope to write about many demos with this phone in the next year.

Even after working with the mobile specification for a long time, I still believe that the vision of an OSGi platform in a mobile phone opens a huge market of both niche and mass market applications.

Automotive
From the outside there was not much news from the vehicle/automotive front. Part of the reason is that many members have been too actively involved in the EU sponsored Global System for Telematics (GST) project to spent much time inside the OSGi Alliance. However, a number of companies have remained active in the OSGi vehicle expert group and are working on some very interesting projects. One good decision of this group was to align the vehicle management specification with the mobile device management specification, including diagnostics. The Mobile OMA DM based device management architecture was slightly extended in cooperation with the mobile expert group and then included wholesale. Based on this specification, the vehicle group then developed an abstraction of the vehicle data. This abstraction allows applications to use vehicle data without having to concern themselves with actual models and car types. For example, an application can close all the doors on any compliant car or find out if it is raining.

However, the most important and interesting work has been the navigation model, driven by Olivier Pavé and Rob van den Berg (both Siemens VDO). This navigation model is my personal favorite. By providing a standardized API to the on-board navigation system the OSGi specifications will enable a very large number of interesting applications. A standard navigation system allows third parties to combine their unique data with position and guidance information. These applications can leverage a highly complex navigation system for their own goals, enabling applications that today are not feasible.

For example, the Guide Michelin could offer a bundle that helps you select a restaurant, make the reservations, and guide you to the selected restaurant. Or it could even notify you for a restaurant that is worth a detour. They even could push the envelope and suggest a trip to a restaurant that is worth the trip.

I hope that we will be able to finalize the vehicle work next year. I also have good hopes that some of the results of the GST project will be merged with the OSGi vehicle specifications. Next month (Jan 11) we will have a vehicle workshop in Detroit, looking at the guest list I am sure this will be a very interesting meeting that will put the OSGi vehicle group back on track.

Enterprise
The new kid on the block is the OSGi Enterprise Group! The OSGi Alliance is gaining a momentous amount of steam in the enterprise/server world lately. Though the core technology is very useful as is, there are clearly many requirements that could improve the core and extend the set of available standardized services. Service distribution seems to be at the top of the list, but there are many more areas, as became visible in the workshop. The challenge will be to keep the core lean and mean so that its applicability for the other markets does not get lost. However, I have full confidence that we will succeed there; there are too many people involved that distinctly dislike bloat and the success of POJO’s is a clear sign that our industry is becoming acutely aware of the threats of coupling.

BJ Hargrave (IBM) and I arranged a very successful Enterprise Workshop in September in the San Francisco bay area. This workshop has resulted in an active group of members that are willing to run an OSGi EG. The board approved a new Enterprise Expert Group last month, appointing Tim Diekmann (Siemens) and Eric Newcomer (IONA) as co-chairs.

Spring-OSGi
Before 2006, OSGi technology was put on the map for most people by the Eclipse adoption of the OSGi service platform. This year, the visibility increased significantly because Interface 21, the company behind Spring, decided to support OSGi frameworks. That definitely was a good idea looking at the market response. Adrian Colyer (Interface21) took the lead but the work was shared by Hal Hildebrand (Oracle) and Andy Piper (BEA). The most interesting part for in this process was the reaction of the participants of this group to the OSGi technology. Initially the approach was to “support” OSGi frameworks. During the project, you saw the participants get more enthusiastic about the combination. It would surprise me if OSGi did not become a premier deployment platform for Spring. And vice versa. Spring has some very interesting techniques that overlap the Declarative Services from OSGi. I think it is definitely worth the effort to investigate how these two can work together. I hope Interface 21 will become an OSGi member soon so we can explore the possibilities of specifying the Spring-OSGi collaboration in a future specification.

JSR 277 and JSR 294 module systems
The dramatic highlight of this year is obviously JSR 277 and its brethren JSR 294. JSR 277 attempts to define a module system for the future Java 7 that has a significant overlap with the functionality of the OSGi core framework, JSR 294 is doing the same but restricts itself to changes to the VM and language.

I desperately tried to be on the JSR’s expert group but I was repeatedly denied, despite efforts from people like Doug Lea. Despite this rejection, I tried to evaluate the first draft from JSR 277 without too much bias in my blog. This blog was very well read by the industry. In general the reaction was that Sun should align with other standard groups if they have achieved significant results in the market and not try to reinvent the wheel. Last week I ran into Stanley Ho and Alex Buckley (JSR 277 and JSR 294 spec leads from Sun) at Javapolis an Antwerp. We had a good and pleasant discussion and I hope this will bear some fruit. However, these JSRs will be an ongoing saga so I am curious what next year will bring us.

JSR 291 OSGi Service Platform for J2SE
IBM filed JSR 291 with Glyn Normington (IBM) as lead this year. Though the OSGi specifications were already part of the JCP in JSR 232 (Mobile) it was felt that the visibility of a JME specification by the JCP SE/EE Executive Committee was too low for comfort. JSR 291 was setup as an open expert group that would provide requirements to the OSGi Alliance. All design work is done inside the OSGi Alliance by its members. Interestingly, this process was setup because of intellectual property rights but the separating the requirements gathering process from the API design worked really well in JSR 232 and now in JSR 291. Maybe this separation of concerns should be tried more often in the JCP?

Anyway, JSR 291 is moving along and will produce a specification next year. This will come out as OSGi Release 4.1, likely in April.

Apache Felix
Last year I was asked to support an incubation project at Apache: Felix. This project was based on Oscar, the first and popular open source implementation of the OSGi specifications. It was really nice to see how Richard S. Hall rallied a large group of people with enormous energy around this project. They have been working in incubation status during this year and are now ready to graduate to a top level project. I hope, and expect, that this graduation will push many more Apache projects to first enable their artifacts to run on an OSGi platform and later commit themselves to Felix. Many of Apache’s software projects scream for an underlying OSGi platform that can unify the myriad of plugin systems and class loader abuse present in Apache and other open source code.

OSGi Popularity
The best part for me this year was the tremendous increase in outside awareness of the OSGi Alliance. I remember sadly staring at 8770 Google hits for OSGi in 2005, today we have between 1.5 and 2.5 million hits. I do not really know what the number means, and it seems to depend on where you ask it, but the trend is to my liking. Same for Google alerts. In the beginning of this year I received about one Google Alert per week on the OSGi keyword (My week was good when I received two). Today I receive one every day with 5-10 links. Also the statistics of this blog look like we are on the root of the archetypical hockey stick curve. And best, the last few weeks we received a flurry of requests for membership as well as introductions to speak on conferences. If this trend continues, next year will be very busy, but I will love it!

Future
So much more has happened this year but I am already impressed that certain readers got this far. Let us close with a short outlook for next year.

I expect that 2007 will be the year of the Enterprise. Much of the OSGi Alliance’s effort will go into creating a specification for server side/enterprise applications. I think that the key parties in this market will join the work so we can have a lean mean industry wide platform for server side applications.

I als see many signs that the home automation market is making a revival. This market is where we came from and it would do me great pleasure if we could increase our activities in this market. For the mobile market I expect several vendors to provide OSGi capability. This will create a flurry of development work. However, most of this work will be in specialized applications. The popularity of the platform will then likely become visible in 2008/2009. For 2007, mobile will offer interesting opportunities that can pay off handsomely.

OSGi DevCon
At last, I’d like to put in a plug for the OSGi developers conference which is held jointly with EclipseCon 2007 in Santa Clara 5-8 March. The OSGi track will be focused on server side, embedded, desktop, mobile, and other areas where OSGi platforms are used. If you have an interest in this area, please join us and meet all those other people that are involved in creating an eco-system for universal middleware.

Ok, this blog is way too long. Let me finish by wishing all the readers a fantastic 2007 and I hope to meet many of you next year on a workshop, member meeting, a conference, or online.

Peter Kriens

PHP on an OSGi Framework?

Last week (2006/12/08) the OSGi Alliance had a webinar for their members. Last month, Susan Schwarze from marketing had asked me very sweetly to provide a demonstration for this webinar. After some soul searching I had proposed to make a demonstration using the OSGi Framework in a multi language setting. The scenario of the demo was simple: show how to start a framework, get some basic bundles installed, install some applications from the OSGi Bundle Repository (OBR) to show it all works, and then as grande finale, install a PHP interpreter and port a PHP Wiki to an OSGi bundle in real time. And this all must happen in 20 minutes.

The most interesting part of the demo was obviously the PHP part. I always thought that having a PHP interpreter on an OSGi framework would be really cool (However, hard this is to explain to my kids and wife)! With an OSGi based PHP interpreter you can deploy PHP applications as OSGi bundles. This is attractive to companies that today use PHP on remote servers as well as device vendors that want to design their web sites in PHP.

The demo was not the first time I had come up with PHP. I had searched the web for a PHP interpreter written in Java already several times but I had never found one until I discussed this demo with BJ Hargrave. He mentioned that there was an open source project/company doing a Java based PHP interpreter: Quercus. The interpreter is a part in the bigger Resin webserver. With my usual optimism, I though: “How hard can it be to port this interpreter to an OSGi bundle?”

Well, hard.

It started all very hopeful because there was a class Quercus Servlet and the OSGi HttpServer supports servlets very well. Unfortunately, the Quercus Servlet was assuming it ran inside the Resin server, making it impossible to use with another web server. Writing my own servlet was luckily only a few hours. Unfortunately, I could not resist simplifying the Quercus Servlet which added a few unnecessary hours to undo these simplifications when I understood why the complications were there in the first place.

The next problem I ran into was the file system. Normally PHP assumes that all the sources are in the file system and in our case the files had to come from a bundle. Quercus/Resin supported a virtual file system but it took some time to find out how you could provide a new source for files. It turned out you had to write your own Path sub-class and then set the root path to an instance of your type. All other files are then relative to that root. Reading was very easy to program, I just mapped all the read primitives to the resource access in the bundle. However, writing was a lot harder. At first, I decided to allow the bundle that carries the PHP files to specify its URI and a set of writable directories. That is, I architected the following headers:

PHP-Path: /mywiki
PHP-Data config, data, wiki.d

I got this to work but ran into the problem that some PHP programs actually re-write their own source files. Having special writable directories forced me to modify several PHP programs, obviously not a desirable architecture. Then it suddenly hit me (under the shower!) that I could just do the rather standard technique of copy-on-write. So I got rid of the PHP-Data header altogether. A read would now always first look on the file system (actually the bundle’s data directory) and then in the bundle’s resources. A bit tricky was the handling of directories; they should be automatically created when you wrote a file that also resided in the jar. After this (awfully late) insight, it became quite easy and I could use several PHP programs from the net without any modifications. This worked so well that I could show off this model with a real functioning Wiki that was wrapped in a bundle directly from the Internet!

However, I wanted one more splash and I found this splash in the Quercus modules. The Quercus interpreter was designed to be very modular; all its commands were written in separate modules which were loaded at startup. This model was too tantalizing to not use it to show off the dynamic capabilities of the OSGi Framework. Would it not be impressive if you could install a new Quercus Module on the fly as a bundle (Again, an excitement my family rarely shares)? So I decided to create a Service Tracker that listened to Quercus Module services. Those service objects were then added/removed as a module to the Quercus interpreter. I had to write a new remove function inside the interpreter because, like so many programs, it had no concept of modules that could go away.

Quercus Modules could now be written as OSGi bundles. This turned out very simple to do because bnd (a program/plugin that creates bundles from a recipe) supports declarative services as well. I could just create a small bnd file that builds the bundle and indicates what class should be registered under the QuercusModule interface. Like:

Private-Package: com.caucho.quercus.lib.zip
Import-Package: *
Service-Component: com.caucho.quercus.lib.zip.ZipModule;
provide:=com.caucho.quercus.module.QuercusModule

I had decided early on that I did not want to do the demo live. I guess I am becoming chicken at my old age; the chance that something fails is just a bit too high. I therefore used Instant Demo. This application can record everything that happens in a window. You can edit this recording and provide sound as well. Instant Demo then converts the total set of files to a flash file so you can play it in any browser. I must say, using such a program is a humbling experience because every little error requires a restart. It was painful to have to repeat the same actions over and over again just to get it right.

Anyway, after a lot of hard work I finally succeeded in making four short movies about this demo. If you are interested, download this presentation and play it. Let me know if you think we should have more of such demos. Should we also do this for JRuby and run Ruby-On-Rails applications on OSGi Frameworks? Hmm, sounds cool!

Peter Kriens

P.S. The Quercus interpreter is not available because the code is not industrialized. It really worked for a demo (nothing was faked) but it requires more work to harden it for real use. However, I will contact the resin people to see if they like to modularize their webserver. Their web server screams for OSGi technology!