Monday, February 27, 2006

JSR 291 Dynamic Component Support for JavaTM SE

IBM, together with the Apache Software Foundation, BEA Systems, Inc., Intel Corporation, Nokia, Nortel Networks, Richard Hall, SAS Institute, Inc., and undersigned have submitted a new JSR: JSR 291: Dynamic Component Support for JavaTM SE. This JSR will bring the OSGi dynamic component model into Java SE. At last, I have been advocating such a JSR inside the OSGi for more years than I care to remember. The OSGi Service Platform is a Java specification; not having a JSR for this crucial specification is seriously hindering its adoption

A little known fact is that the OSGi Service Platform did start out as one of the first JSRs: JSR 8: Open Services Gateway Specification. In 1999, when the JCP first got started, we were asked by SUN to run the process according to the JCP 1.0. This caused a lot of discussions because of the intellectual property rules. Some members of the Connected Alliance (the predecessor to the OSGi Alliance) felt that the JCP gave SUN unfair power over the specifications that were to be developed by the whole group. After lots of deliberations it was therefore decided to develop the specifications inside a new organization that would have equal and fair intellectual property rules: the OSGi Alliance was born. Unfortunately, this meant we had to exit the JCP, though SUN remained as a member and provided some significant contributions.

Since then, many things have changed. The JCP is now a crucial force for Java. In this cacophonic world, JCP has become for many developers a minimum requirement relevancy. The OSGi Alliance has also changed course, in the beginning we were about residential gateways. Today, we can provide a comprehensive component model that significantly reduces the complexity of application development and can therefore take applications to new levels of functionality. What also changed was the success of JSR 232. This JSR showed us that the JCP and the OSGi Alliance can successfully work together.

Back to JSR 291. Why is this JSR so relevant for Java SE? The simple answer is size. Applications get more and more bloated because of today they include so many features. Features, that are often never used by the users. A component model reduces coupling and therefore allows applications to be delivered in separate parts instead of a big blob. An interesting example is Java itself. The current evolutionary model of Java of adding more and more packages is becoming cumbersome, even for todays roomy desktops. One of the reasons for a monolithic approach has always been the portability of applications: Write Once, Run Everywhere. This model is breaking apart due to the breadth and depth of Java. Providing Swing on a headless device is a waste of precious resources. A component model could solve this problem because missing functions can be downloaded and installed on demand. Such a component model will make it easier to include a much wider range of Java standards; only standards that are actually needed are downloaded. Currently, what to include or not to include in a language release is a very serious question because of this size constraint. Just like it is for large applications.

In the past 7 years, we have shown that the OSGi specifications have tremendous value for developers, as is demonstrated by the widespread adoption from embedded, desktop, to server applications. However, wider adoption is hindered by the fact that these specifications are not part of the JCP. I think that JSR 291 will bring the OSGi specifications under the attention of a broader range of Java developers. However, best of all, the OSGi Service specifications can provide a component model to Java that it so urgently needs, making Java even more competitive with proprietary solutions than it is today. I am really looking forward to work in this JSR. This JSR will be run with an open mailing list, so please tune in.

Peter Kriens
OSGi Evangelist

Sunday, February 26, 2006

OSGi Device Management

The biggest chunk of functionality for the OSGi Mobile Service Platform is the Device Management Tree (DMT) Admin. The Dmt Admin API is a matchmaker between management applications (both remote management servers as well as local applications) and the components that are available on a device. The Dmt Admin maintains a conceptual tree structure of interior and leaf nodes. Interior nodes have children, and leaf nodes have a primitive value like an integer, float, string, time, date etc. Plug-ins map this conceptual tree to actual hardware registers or other value sources. The conceptual tree is closely modeled after the OMA Device Management specification.

In the past two days I have written my first plug-in for the Mobile Expert Group. This group is considering using the Dmt Admin for their R4 specification but they want to make the API simpler for Java applications. I had to write a prototype to make a change request.

Initially the task to write a plug-in looks daunting. The interface that must be implemented has 22 methods, and they look quite, well, not simple. Fortunately, as I found out, most methods do not require an implementation to make the plug-in work. Actually, I was quite surprised how easy it was to implement a simple leaf node. You only have to write a Plugin service that is registered with the Framework when you start up. This plug-in service is asked for a session object that you must implement. This session is used to access the plug-in.

My first attempt was to create a plug-in that maintained the position. The OSGi R4 specification has a Position object that lends itself very well to this concept. A Position object contains six fields: longitude, latitude, altitude, track, and speed. Each of the values is a Measurement object that contains three fields: value, error, and Unit. Overall, a Position is mapped to a sub-tree of eighteen nodes. Not big, but also not trivial.

A key problem of writing a plug-in is to map the URIs (which are String[] objects for the plug-in) to your function. My first attempt was hard-coding this relation, but that became very cumbersome amazingly quickly. I must admit that I am allergic to any redundancy in code, but even less sensitive people should have been affronted. I developed this allergy 10 years ago when working on an application with thousands of fields that were hardcode in hundreds of places.

So for my first attempt used some tables to minimize the redundancy, but the results were still not very satisfactory. Intricate knowledge of the tree was necessary in too many different places.

Looking at the code I remembered having a class that makes any Java object look like a Map, very similar in vein to Java beans. Could I use this? The properties of a bean also form a tree. Properties that result in primitive types like String, byte[] can be mapped to leaf nodes and the remaining properties are treated as interior nodes. This turned out to be a very interesting architecture. I threw away all the code I had written and the plug-in became almost completely generic. Virtually all the code needed for my plug-in was now provided by the Position and Measurement classes. That is the way I like it! Even better, the only change I had to make to my bean inspector was to be able to reason about the tree, even in the absence of instances. The original class only worked with instances.

What the prototype showed me was that, with a little bit of help, it is almost trivial to write a Dmt Admin plug-in. As proof, hereby a class that implements a (non-transactional) plug-in for locales. The remaining code is completely agnostic of the problem domain area.

public class TinyPlugin extends TestPlugin {
  TinyPlugin(BundleContext context) { super("./Tiny", context);}

  public Locale[] getLocales() { return Locale.getAvailableLocales(); }

  public String getDefaultLocale() {
return Locale.getDefault().toString();
  }
     
  public void setDefaultLocale(String locale) {
    String parts[] = locale.split("_");
    Locale l = new Locale(parts[0],
parts.length > 1 ? parts[1] : "",
     parts.length > 2 ? parts[2] : "");
     Locale.setDefault(l);
  }
}

Tree:
     ./Tiny
          Locales
          defaultLocale

This class (together with a library of generic code) implements a full Dmt Admin plug-in. It will be hard to make a similar plug-in with less code. And as you probably have found out in this article, I am a staunch believer in small is beautiful (and robust).

     Peter Kriens
     OSGi Evangelist

Tuesday, February 21, 2006

When adding a new portable to my home network I again realized how wide the gap is between technically unchallenged people and the rest of the world. As a technical person, I sometimes have this snug feeling of superiority because I can get things to work that the rest of the world can only look at in bewilderment. I gladly accept the compliments of our partner when we fixed the PC or connected the new stereo, as if I deserve it. When I look around, I see that many colleagues are similar.

However, sometimes I wonder if our skills are not blinding us from the complexity we put on the rest of the world. And this is not because we are so clever and our users are not, it is actually often the other way around. Users are often too clever to learn unnecessary details and complexities, they just refuse to bring up the patience.

Why is it that the whole consumer industry has been incapable of creating a viable alternative to the iPod, over a period of three years? Why do I still notice, at the worst possible moments, that my address book in my phone is not synchronized? Why does connecting a new PC to the network in a secure way feels like rocket science?

I think a reason is that technical people are not bothered with complexity, we usually do not even notice it, and often thrive on it. A friend of mine is getting started with Java and I am helping him out. Teaching him Java confronts me with the complexity that I no longer notice. It is frightening to see how complicated Java has become over the past 10 years.

I think this complexity in software development is inevitable. However, by being used to a highly abstract and complex world we sometimes do not realize how simplistic the services are that people really like and pay for. For example, a service like “Push to Talk” looks like nothing special but made a huge difference for customers and the Nextel’s bottom line. Another example is how Ericsson has made billions selling business phone systems where the primary function was to provide 3 digit short numbers instead of the normal numbers.

However, we do not have to take the full blame, in most companies the feature set of products is defined by marketing. They are in charge of the product definitions. Marketing people always love simplicity because it makes the feature list longer! However, too many features do not only make a product harder to use, they also make the product harder to develop.

This also points some of the blame to the end users themselves. Cost is the single most important aspect that people use to compare products. I can understand this, because most products are so complex that a reasonable comparison is only possible after a detailed study. And who has time for that? Except when a product is really easy to use, the market recognizes this and rewards the manufacturer with a high profit margin, see the iPod as a prime example.

So how can we build simpler products? Not every company has a Steve Jobs that uniquely combines a deep understanding (and control) of management, marketing, technology, and most important, the requirement for simplicity.

Therefore, management has to become aware that quality costs but that this cost can also result in higher profits. Marketing must realize a short feature list where all features work as advertises is better than a long feature list with a lot of cruft. We technicians can help by never underestimating the patience of our end-users to figure out our products.

Peter Kriens
OSGi Evangelist

Sunday, February 12, 2006

Personal Information Manager (PIM) and OSGi

I can hardly believe that it is already nine years ago that Ericsson Research hired me to do a project called Perspective to write a 3D user interface (VRML) to personal information store. At that time I was living on the Swedish west coast and flew once a week to Stockholm. After a few weeks the manager Anders Danne asked me to join the department for a planning meeting in the beautiful Baltic skärgarden (archipelago) and give a presentation. Surprised I asked about what subject, “anything”, he answered. “Great”, I replied, “Do I get time to prepare it?”. “No problem, take whatever time you need”, he answered, right the answer I was hoping for. So I spent a week thinking very deeply and came up with a project to create a personal assistant that read your mails, saw where you browsed, knew where you used your phone, was aware of the programs you looked at, and tracked your location. The assistant did not only track all this information, it tried to infer relations, and provide timely advice to its patron. The key driver for me was that in the long run search is just not enough; we need our assistant to tell us things at appropriate times, not just to respond to our queries.

Though I was a trifle disappointed with the lackluster response to my presentation to, I was pleasantly surprised when Anders asked me to run a follow-up project based on the ideas in the presentation. So in January ’98 I sat down to try to code “Marvin”, the name of the alter-ego that should provide all these goodies. (Yes, you are right, that is where the name came from).

Why do I tell you this nostalgic story? This weekend I started going through a pile of CACM magazines and one of them focused on Personal Information Managers. More than eight years later I now see in print many of the ideas that I had, but with so few solutions yet. In my original presentation I sketched a scenario in 2003, assuming I was pessimistic. It is kind of discouraging to see that in even in 2006 most of these concepts are seen as pure research and futuristic.

The biggest disappointment was, however, to see that the ambitions of the projects were pretty low. The all tried to find concrete solutions using XML, folders, databases, RDF, and other stuff. I always intended to use Artificial Intelligence techniques for my project because I think that we need solutions where the computer has some understanding of our daily problems, that it has at least some common sense. Without this, as users, we will spent just too much time categorizing and cataloguing our heaps of information. None of the projects in the CACM even mentioned Artificial Intelligence. This holy grail of computing seems to have completely disappeared from the researchers radar screens.

So why did I leave this exciting and interesting field of research? The reason was that I based my working ideas on small units of Java code that categorized and catalogued information in a specially developed database, called an InfoBase. Due to the diversity of the real world, I had made this a pluggable system so that it was easy to add new plugins written in Java.

Based on this work, I was asked one day to present my ideas at Ericsson in Linkoping, a little village in the middle of nowhere, eh, Sweden. When I arrived after driving 4 hours through the snow, it turned out there was a confusion; they expected me to port the Java Embedded Server to their e-box in one day, and oh yes, they also needed a cool demo. The purpose of the demo was to show to IBM, Oracle, Nortel, Deutsche Telekom and many other companies the concept of a residential gateway. I made the grave mistake of succeeding that grueling day (notice that even Java did not run on the e-box yet). Thereafter I was sucked in the vortex of what became in the end the OSGi Alliance.

Eight years later I can look back at some very interesting years developing the OSGi technology, and I feel I worked on the future. Despite this satisfaction, there is a tinge of regret I did not pursue this incredibly interesting area of personal information management. Well maybe one day a big research company sees that OSGi technology is incredibly useful in this application area.

    Peter Kriens
    OSGi Evangelist

Tuesday, January 31, 2006

OSGi and the User Interface

A problem I have been working on last week with a friend is the user interface (UI) for an embedded device (he actually works on providing personalized health information, but the problems are very similar). We did not start this field, the problem has been bugging the different expert groups since the OSGi began in 1998. Tom Idermark from Ericsson and I once wrote RFP 8 for “Client Access”. I even developed a prototype on an OSGi Release 1 framework. The RFP is now 7 years old but still reads as if it was written yesterday.

So what is the problem? In an OSGi device, the UI must be adaptive to many influences: The available services, the end users, the output device, the state. At the same time the UI must run on a device severely constrained in memory and CPU cycles, not an easy problem.

This is hard, and I mean really hard. The main reason why it is so hard is that the user interface is a highly valued piece of real estate; the user identifies all the services with what it will see on the screen. Being a good citizen and seamlessly blending your bundle in the generic UI does not always improves your relationship with the marketing department. Telling them that the logo is a bit warped because the generic version did not map well to the hardware is likely to make enemies for life; only Microsoft gets away with warped logos.

Still, from a usability point of view it is important that we find a (non exclusive) solution to the OSGi UI problem. Such an OSGi UI should address the following concerns:

  • Bundles should be able to provide information, receive information, as well as notify the end user(s).

  • Securely accessible local and remote

  • Must be able to look very good

  • The style should be consistent on a device, without requiring the bundles to be intricately aware of that style

  • The end user(s) must be able to control the UI, if applicable

Web based UIs have an advantage here because they can be accessed remotely and locally, they can offload some processing to the client, today’s HTML can be used to build very rich UIs, and there is a lot of experience. The key problem is the portability of the different web techniques. Last week I was again confronted with the surprising variations browsers display in their Javascript and HTML implementations. The OSGi specifications supports Servlets, objects that generate web content, usually HTML. Servlets have the disadvantage that they want to control a complete page, it is very hard to merge the output of multiple servlets into a single page.

Portals are software frameworks that address some of these issues. The JSR 168, which is implemented in the Apache Pluto project, addresses some of the issues. Portlets are like servlets but deliver a fragment instead of a whole page. The Portal container merges different Portlets onto a single page.

However, Portals are really oriented to heavy weight computing and not applicable for smaller devices. The Portlets approach to the site style is too lightweight; it only defines a few Cascading Style Sheet classes for this purpose. Interestingly, Google has analyzed 1 billion web pages for their HTML coding style and found a surprisingly small number of primitives. Supporting the top 20 primitives would go a long way. Kumanan Yogaratnam and Eddie Drake of Espial and Myrio proposed a solution in this direction last OSGi congress.

A significant disadvantage of HTML is that the content gets mixed with form. The alternative is XML, and I have very good experiences with bundles that provide XML instead of HTML. The required formatting, based on the output device, is then created with different or a combination of XSLT stylesheets.

I have no complete solution but it looks like the parts are coming together to provide a more complete solution. Now we only have to find a group of companies that need this desperately so we can spec it!

     Peter Kriens
     OSGi Evangelist

Tuesday, January 24, 2006

The OSGi from a Python Perspective

Years ago, in the eighties and early nineties I spent every Wednesday night from 8 PM to 3 AM hacking with a Petr van Blokland, a long time friend. We spent our time doing a virtual tour through the computing world. We used any possible language, library, or created new ones when necessary. We even wrote our own Prolog interpreter. We developed lots of programs and had even more fun. After I moved to Sweden we kept in touch but all night hack sessions were of course no longer possible.

Yesterday he was here in Beaulieu and I introduced him to the OSGi specifications. It was kind of scary to see how great the divide has become technologically. I had to move to the PC because of Java in the late nineties, he kept on buying Macs. He fell in love with Python and I had to use, well again, Java. Though we always are much at ease with each other on the semantic level, the lack of common ground is a big challenge.

For me his visit is a wonderful opportunity to see how a highly experienced and intelligent software developer looks when he is confronted with OSGi out of the blue. The biggest hurdle is of course Java. I complained before, and will complain later, but Java is a very crufty language (try using inner classes). It is amazing how clean a Python class looks in comparison to a Java class with the same functionality.

I tried to get him hooked on Python development in Eclipse (that at least would give us a Geneva where we could further negotiate). We tried PyDev and TruStudio. Though neither plugin is bad, they still pale in comparison to the Java support in Eclipse. Eclipse is the first IDE I accepted because its support for Java development is fantastic. However, I do not think I would use Eclipse if the Java support was on par with PyDev/TruStudio.

We decided to try to make a bundle that used Jython servlets. Jython is a implementation of Python on Java. Wrapping the Jython JAR turned out to causes all kinds of packaging problems because it was using lots of external links. It was kind of sobering when you realized how arcane much of the work is you have to do to make a simple bundle. We then moved on to declarative services and there I also saw the XML with the eyes of a highly skilled Python developer.

I am the first person to know that for each piece of cruft in the declarative service XML, or Java source code, or manifest, there is a good reason. The OSGi Alliance tries to solve a problem that is very hard: make independently developed applications work together. This is much harder than a monolithic application, but we should not forget that simple things should remain simple. There is an increased awareness of this, noticing the LAMP and Ruby on rails discussions that all reflect badly on Java.

Several parties are working hard on making tools that will take any pain out of bundle development. The Eclipse team is improving their plugin development tools (a plugin is a bundle, not sure why they stick to the name plugin) and I had the pleasure of testing an SDK for the next generation mobile phones. The SDK uses a different technique than the standard Eclipse Plugin Development Environment, I actually prefer theirs over the Eclipse variation. For example, the SDK allows multiple deployment artifacts from the same bundle which Eclipse does not allow.

Both environments are just not good enough yet, I feel.  Neither has a really clear UI that guides you through the process and helps you add new features, all editors are very focused on a single artifact instead of taking a more holistic approach. Then again, Eclipse has only recently adopted the OSGi specifications and it is natural that the tooling will take a couple of iterations to get it right.

On the plus side, when we had entered all the cruft and updated our first bundle, I could see his eyes light up when he realized we never had to restart the framework to get the new functionality up and running. That part always remains sooo cool.

Friday, January 20, 2006

TomTom for Sale, Trades for OSGi Phone

Yesterday I had to attend the Vehicle Expert Group meeting in Sophia-Antipolis (Nic)e at Siemens VDO. Nice is about 300 km from where I live so I decided to drive. Got up at 5.45, in the car at 6.15, and after programming the TomTom navigation unit I was on my way. It was a beautiful day so I enjoyed the stars, the twilight, the sunset, and a wonderful espresso. After 2.5 hours, Truus (as we call the TomTom) told me to take exit 42. This was not the normal route so I naïvely concluded that Truus had found a better way. After 45 minutes in traffic jams on small back-roads I ended up in the middle of nowhere: I had told Truus to go to “Lucioles”, not realizing that Sofia-Antipolis has 3 different places with Lucioles in the name. Obviously, the first line had not been the right one. Entered the right spot (I thought), went there, found that the Siemens VDO had moved. Had to dig up the new address on my computer but unfortunately, Truus feigned ignorance of “Avenue de Roumanille”, or for that matter any other road I could see in the neighborhood. I called the meeting host, Olivier Pavé, but got no answer. I then drove up to an information area where I still could not find the place on the map. Fortunately, Olivier called me and told me he would pick me up. Anybody interested in a second-hand TomTom?

At first I blamed my self for not being more prepared but then I started thinking. Why do we always blame ourselves? Isn’t it the task of the technology to handle the chores of life?
  • Why did it force me to enter an address, which is error prone (especially at 6.15 in the morning!).

  • Why did I have to rely on the last address I had, while it could have been updated over the net?

  • Why did I have stale maps (they were bought in April last year, the road clearly looked older than 2 years)?

I told the group that what I really want is to tell my navigation system to go to “Olivier Pavé”. We discussed this use case, how it could be implemented. The navigation system could send an SMS to Olivier, which then replies with its position (after confirmation of course). The navigation system then just navigates there. Nice use case! Unfortunately my TomTom is a closed system so implementing this use case is a major amount of work. Even if I could operate on Truus (she is running Linux), I would have to invest a major amount of work to only one brand.

Olvier Pavé then told me about their plan to port a Midlet based online navigation system to the OSGi Mobile Service Platform. Nokia had promised them a phone to make this prototype. They demoed the Midlet, leaving Truus in the dust, because it knew the “Avenue de Roumanille” in the smallest detail!

In the OSGi Mobile Platform architecture, they would have to split the navigation system in a Navigation Service, a Position service, and a Navigation UI. This architecture would enables me to write an application that implements my  “Go to Olivier” use case, without having to write my own navigation system (and without having to go to a standardization process). Such a use case is impossible with MIDP.

This example clearly illustrates the vision of Jon Bostrom, Nokia’s Director for Java: Provide a platform for middleware and third parties will create an avalanche of innovation. Fortunately, operators are starting to see the light: Sprint has just become an OSGi member. Welcome!

     Peter Kriens
     OSGi Evangelist