Saturday, March 25, 2006

EclipseCon 2006, OSGi on the Rise

Wow! If you want to see what is happening with OSGi technology, you should have visited the EclipseCon 2006 conference in Santa Clara! OSGi is mentioned in an awful lot of presentations! There are so many people excited about what you can do with OSGi that it was heartwarming. Despite the fact that the OSGi is only the plumbing of Eclipse, many people realize that the OSGi component model is applicable in a wide range of applications. For many, it is a genuine discovery when they realize that the IDE plug-in model can also be used on a tiny embedded device like the Slug (which was very nicely demonstrated by Thomas Watson) or on the server side, which was discussed in Corona Project from Compuware, Wolfgang Gehner with his RSP proposal, and many others. An impressive number of people want to start deploying OSGi applications that (partly) replace J2EE application servers. More about that later.

On Monday Thomas Watson and I gave a tutorial about OSGi Component Programming. This went very well (though we could only finish half of our 100 slides). The feedback we got during the rest of the conference was nice; people loved the modularity and especially the service model. This was a bit of a surprise to me; I had not realized that most people know about the OSGi modularity layer but know little about the service layer. The service model is new for many people because Eclipse mainly uses extension points where the OSGi model uses services.

The difference between extension points is subtle, though important. An extension point allows a bundle to find, and work with, implementers of its own extension points. That is, the bundle owns the extension point contract and allows other to implement it. A service is more generic because it provides an interaction point between any number of bundles; there is no bundle that owns the service, any bundle can both register (implement) and get (use) a service. This distinction nicely reflects the perspective of Eclipse community and the OSGi Alliance. Extension points allow an implementation to find implementers. For example the Eclipse UI code uses extension points to find its menu entries in different plug-ins. The extension point model is not intended for another bundle to find these menu entries. In contrast, OSGi services are usable by any bundle that has the proper permissions. The OSGi model allows substitutability of the user of a service as well as the implementer of a service. There are some more differences like dynamics, versioning (extension points mandate only a single version resident at any moment in time), and life cycle, but the ownership of the contract is the most crucial.

Anyway, lots of people are using extension points that would have been nicer (ok, in my biased opinion) with services. This difference/overlap, and when to use one or the other was a big part of the talk by Jeff McAffer and BJ Hargrave. They did a joint presentation that did an excellent job of highlighting the similarities and differences. The format of 2 very knowledgeable guys taking their turns explaining the differences worked very well. There were an impressive number of people ready to listen to the Best Practices of OSGi and Eclipse.

So what is the number one aspect that signifies this conference to me? I think that it has become clear to me why there is a need for an OSGi Alliance. There is an impressive amount of projects underway. However, all these projects are about implementations. The people you meet here are very interested in collaborating on developing implementations. This approach is orthogonal to specifying service interfaces so that different implementations can collaborate. There is a clear distinction between an extension point that is unilaterally decided by a programmer and a service that is defined by a standards body or consortium. Using well specified service interfaces will allow commercial companies to compete with Eclipse, or other, implementations; something that the Eclipse community, commercial companies, other communities like Apache, but most of all the users can benefit from in the long run. Will the different communities have the patience and motivation to go through standardization processes?

The answer to this question is opportune for what will happen on the server side. The Eclipse model has made many people realize how well the component model works and they want the same benefits for the server side. There are a large number of initiatives to standardize Eclipse for the server. However, the danger is that different parties create implementations without proper regard for how all these different parts could interact. Obviously many implementations will work but properly designing a component architecture for Enterprise applications will enable substitutability, a very desirable property in my opinion. Maybe organizing a workshop about Enterprise Component Architecture could be held. Some key people in the industry could to outline such a component architecture. Good idea?

     Peter Kriens

Sunday, March 19, 2006

The OSGi GUI: Flash/SVG?

The OSGi specifications were originally developed for headless devices and user interfaces were therefore of no concern. This changed when the vehicle and the mobile groups worked on their specifications because they clearly required some graphic user interface. Within the vehicle group many discussion have taken place but we were never able to reach a consensus. The mobile group kept standardization of the UI specifically out of the charter because it was a hard choice and there are already standards for mobile phones..

Still, the lack of a good user interface standard makes it very hard to demo OSGi applications. Few decision makers get excited by a console running some cool (in my opinion) middleware. The lack of GUI is also a significant obstacle to portability between different environments. Why is it so hard to choose a specific user interface? The key problem is the wide range of devices that need to be supported; many devices do not even support a graphic display. However, there were more problems than just technical. Member companies also had a hard time compromising on the GUI technology because they already had favorite solutions. The interoperability issue was not of a major concern to most companies.

Swing and SWT are the standard solutions for Java applications. Both are toolkits with rectangular user interface components: text editors, sliders, lists, tables, etc. Though these toolkits are useful in desktop applications they fall far short of what is needed in a car dashboard or a highly usable mobile phone. They tend to consume lots of screen real estate, but worse, they are very static. In comparison to what is visible on the web nowadays, SWT and Spring applications are rather dull.

In the past few years many companies have turned to MacroMedia Flash for dynamic content. Try refusing to download the flash plugin and the web suddenly looks a lot less dynamic. Flash can play movies. Movies are sequences of animation of vector graphics. Vector graphics have many significant advantages. They are usually smaller than the corresponding bitmaps and they can be scaled, rotated, and translated while being displayed without loss of quality. Macromedia also added a programming language call ActionScript, making it possible to turn external events into changes in the movie as well triggering external actions on internal events; ActionScript is similar to Javascript. For example, on a mobile phone an incoming call can activate the part of the “movie” that shows different buttons to the end user. If the user presses such a buttons, the calls is taken or rejected. Sounds similar to SWT/Spring, the difference is that in Flash/SVG the buttons can change place, size, rotation, shape and color only limited by the imagination of the designer. Flash Lite, a stripped version of Flash that has been adapted to many mobile phones and provides phone specific APIs.

The most powerful aspect of Flash is the availability of powerful authoring tools. Macromedia and other companies provide tools for graphic designers to author compelling user interfaces. The result is impossible to achieve with Swing and SWT. And even if one could, good programmers are notoriously bad at UI design. Believe me, I tried and tried. Every time when I got a graphic designer on my design team the User Interface improved with a magnitude. I still do not know how they do it, but it works. Fortunately, they are usually lousy programmers.

Interestingly, there is also a standardization process going on at W3C for a format that is very similar to Flash: Scalable Vector Graphics (SVG), which is based on XML. SVG is not as mature as Flash yet, but they are improving quickly. SVG also has been extended with a Basic and Tiny profile so that it is suitable for smaller devices. Actually, the Flash Lite player can also play SVG-Tiny content. The obvious advantage is of course that SVG has no license cost. The bad news is that it will be harder to get SVG movies to play as consistent on a wide range devices as it is for a single company like Macromedia.

So what is the relation to OSGi? I feel that Flash/SVG might be a solution to the GUI problem. These technologies are primarily focused on the user interaction but have a fuzzier picture when it comes to programming real functionality. ActionScript is interesting, but it is no comparison what you can do with a real Java environment.

The OSGi Alliance could therefore specify how Flash/SVG content could be included in a bundle and how during runtime the events are bound to the bundle’s code. To support different screen formats, a bundle could include multiple versions. A desktop manager could load the content from the bundles and create the necessary bindings to the Java code when an application is activated.

This combination looks awfully attractive. It looks like such a model could finally provide the OSGi with the much needed User Interface option. Are there companies out there interested in this approach? Should we start working on this? Tell us what you think on the OSGi Developer mailing list ...

Monday, March 13, 2006

Health Care and Self Care

Spring is coming. No, though the first blossoms are visible in the garden, it is not that kind of spring. It is also not the Spring Framework. It is the OSGi spring! Last week I talked to many different people and it keeps surprising me how many companies tell that they have been using OSGi for years or intend to start using it in the next project. It is like green buds appearing between the last remains of winter. There is such an increasing awareness that connected devices change the playing field; The OSGi specifications together with Java are a core part of the puzzle.

A key example of this spring was shown in a report about a large health care project. The aging of the western populations has a potentially devastating effect on the health care systems in Europe and the US. Older people (after ~ 65) need exponentially more care while at the same time the labor force is decreasing. If nothing is done, hospitals will be overwhelmed in the next two decades, therefore, the problem is urgent. We need to develop solutions that will allow the health care workers to spend their time on what only nurses and doctors can do while taking other tasks of their hands.

Automation can clearly supply parts of a solution, if applied with care. There are obvious efficiencies to be gained by further applying automation in, and between, health care institutions. However, the use of these solutions has been ongoing process. It is not likely that the required efficiency improvements (my estimate is at least a magnitude) can be achieved through administrative means.

The only solution seems to be to engage the patient: self care. Obviously this simple solution is not that simple otherwise it would already have been applied successfully on a world scale. A key problem that is hard to solve is the monitoring. A hospital monitors the patient for complications and can immediately take appropriate actions when needed. The huge challenge is to provide the means for self care as well as providing the monitoring infrastructure and inevitable emergency help.

This is a very complicated problem with an unpleasant number of aspects that can be categorized as fundamental. Solutions must be extremely reliable otherwise people will die or suffer; this kind of reliability is far away from current consumer electronics (and moving in the wrong direction). The usability of the solutions must be flawless because the users will be older people with reduced eyesight, handicaps, stiff muscles, reduced brain capacity and all those other burdens that age and sickness will put on all of us. Configuring solutions must be automated and trivial or we will have to exchange nurses for technicians. And last but not least, privacy must be maintained while at the same time allowing nurses and doctors access to the information they need.

Looking at the dramatic figures it is quite clear to me that there is no choice but to develop the technology that will enable self care on a large scale, and we better hurry. It also seems clear that domotica can play an important role. Sensors to monitor the patient and devices that can help administer the medications will require local support in the home to help providing the required reliability (e.g. network failures) as well as reduce the load on the central servers by reporting only exceptions to reduce the central load. However, also mobile phones could provide an important role so that the patient can lead an active life.

The biggest technical challenge of all will be software (and I can assure you that just the social challenge will be huge). Sensors, actuators, servers, analysis, monitoring, and managing largely consists of connected software components. We must realize that much of the required functionality is beyond the state of the art today. It is therefore paramount to develop an infrastructure that provides an even playing field for anybody that can contribute to a solution. Trying to create “the” solution is planning for failure.

The OSGi specifications provide a unique building stone for such an infrastructure. No other software technology provides so many of the crucial infrastructure aspects that are needed to implement large scale self care. The portability, security, reliability, integrity, and manageability are all prerequisites for a health care infrastructure. The tremendous challenge of large scale self care is an opportunity for the OSGi Alliance. It actually is surprising that no companies have flocked around this area and created a Self Care Expert Group. Let’s face it, what is the alternative?

Peter Kriens
OSGi Evangelist

P.S. Next week I will be at EclipseCon, hope to see you there!

Monday, March 6, 2006

POJOs

Sometimes a not too familiar word suddenly pops up in totally different places. The first one or two times you ignore it, but then you start to wonder. POJO, which stands for Plain Old Java Object, is such a word for me. The concept is used to contrast between Java objects that can only exist within a larger framework and objects that only need the standard VM as their environment. For example, J2EE Java Beans can only run in a J2EE Framework because their implementations are closely intertwined with the framework’s classes and interfaces. The concept POJO was coined because of a very fundamental problem in the object oriented paradigm: coupling.

Objects entered my life more than 20 year. I had been trying to use Structured Programming techniques (Yourdon, DeMarco, Jackson, etc) in the context of embedded and distributed system software, but I was not very successful:. It just felt plain wrong. I tried many languages Assembly, Prolog, C, Plm, Pascal, Lisp but none felt right until Smalltalk (Digitalk V86) hit me. Using objects felt so right; I developed my own object oriented derivative of the C language: CO2. At that time, I remembered telling people how objects would change the software world. One of my key selling points to management was that one day we could create complex systems by just picking some objects. A rather naïve and overly optimistic view, my only excuse is that I was still pretty young.

The key reason that we cannot easily compose systems from objects can be described with one word: coupling. Objects almost got it right. They gave us encapsulation and polymorphism but we totally forgot one of the key lessons that Structured Programming taught us: coupling. Object oriented languages lack support to decrease effect of dependencies. For example, when you create a class you have to specify the exact class name, creating a very hard coupling to the implementation of that class, worse, everything that class is coupled to ad nauseam. Over the years, our community has found many solutions around this type of coupling. A key example is the factory pattern, which has the bad side effect of making the code harder to read. The causes of coupling are usually subtle but its effects are terrible. We all know the hopeless feeling that small functional changes turn into major rewrites, or when you download an interesting looking library that turns out to depend on 76 other libraries.

So what is the relation to OSGi specifications? The OSGi Service Platform was designed to address the problem of decoupling. OSGi services are POJOs, you can use any object you like for a service. This was not without opposition, some companies wanted a “Service” marker interface, but we won in the end. The whole purpose of import- and export packages is to minimize the dependencies, and therefore coupling, between bundles. The whole purpose of the service registry is to break the coupling between bundles that provide the functionality and the bundles that use the functionality.

However, also the OSGi Service Platform does not escape of being a container and thus creating unwanted dependencies. We therefore have always advised to separate OSGi code from domain specific code and minimize coupling inside your bundles. The golden rule: if a bundle can exist without knowing a fact, make sure it does not depend on that fact (that is one of the reasons why Require-Bundle is bad). In the case that couplings are necessary, concentrate them. Strive to have as few places as possible where you connect your domain objects to other subsystems. This is exactly the purpose of the popular core of the Spring Java 2 EE Framework. It is also known as Inversion of Control (IoC) or Dependency Injection (DI) principle. The Spring framework uses XML files to configure any number of Java subsystems to work together.

My favorite solution for the coupling problem since 1985 has been scripting languages. Scripting languages are perfectly suitable to connect different subsystems and to provide the always necessary glue code. A simple script language can be used both by the developer as well as the deployer to configure a system. My first attempt with a script language was EGO, a terrible forth like language just because it was easy to implement. Today there are many off the shelf solutions, a good example is the Beanshell. Beanshell is a dream for Java based solutions because it is tiny, supports the Java syntax and will integrate with any java environment seamlessly.

However, what tool you use to minimize and concentrate your dependencies is not so relevant. Most important is to reduce and concentrate coupling in large complex systems so that your users get the flexibility they need.

Peter Kriens
OSGi Evangelist

Monday, February 27, 2006

JSR 291 Dynamic Component Support for JavaTM SE

IBM, together with the Apache Software Foundation, BEA Systems, Inc., Intel Corporation, Nokia, Nortel Networks, Richard Hall, SAS Institute, Inc., and undersigned have submitted a new JSR: JSR 291: Dynamic Component Support for JavaTM SE. This JSR will bring the OSGi dynamic component model into Java SE. At last, I have been advocating such a JSR inside the OSGi for more years than I care to remember. The OSGi Service Platform is a Java specification; not having a JSR for this crucial specification is seriously hindering its adoption

A little known fact is that the OSGi Service Platform did start out as one of the first JSRs: JSR 8: Open Services Gateway Specification. In 1999, when the JCP first got started, we were asked by SUN to run the process according to the JCP 1.0. This caused a lot of discussions because of the intellectual property rules. Some members of the Connected Alliance (the predecessor to the OSGi Alliance) felt that the JCP gave SUN unfair power over the specifications that were to be developed by the whole group. After lots of deliberations it was therefore decided to develop the specifications inside a new organization that would have equal and fair intellectual property rules: the OSGi Alliance was born. Unfortunately, this meant we had to exit the JCP, though SUN remained as a member and provided some significant contributions.

Since then, many things have changed. The JCP is now a crucial force for Java. In this cacophonic world, JCP has become for many developers a minimum requirement relevancy. The OSGi Alliance has also changed course, in the beginning we were about residential gateways. Today, we can provide a comprehensive component model that significantly reduces the complexity of application development and can therefore take applications to new levels of functionality. What also changed was the success of JSR 232. This JSR showed us that the JCP and the OSGi Alliance can successfully work together.

Back to JSR 291. Why is this JSR so relevant for Java SE? The simple answer is size. Applications get more and more bloated because of today they include so many features. Features, that are often never used by the users. A component model reduces coupling and therefore allows applications to be delivered in separate parts instead of a big blob. An interesting example is Java itself. The current evolutionary model of Java of adding more and more packages is becoming cumbersome, even for todays roomy desktops. One of the reasons for a monolithic approach has always been the portability of applications: Write Once, Run Everywhere. This model is breaking apart due to the breadth and depth of Java. Providing Swing on a headless device is a waste of precious resources. A component model could solve this problem because missing functions can be downloaded and installed on demand. Such a component model will make it easier to include a much wider range of Java standards; only standards that are actually needed are downloaded. Currently, what to include or not to include in a language release is a very serious question because of this size constraint. Just like it is for large applications.

In the past 7 years, we have shown that the OSGi specifications have tremendous value for developers, as is demonstrated by the widespread adoption from embedded, desktop, to server applications. However, wider adoption is hindered by the fact that these specifications are not part of the JCP. I think that JSR 291 will bring the OSGi specifications under the attention of a broader range of Java developers. However, best of all, the OSGi Service specifications can provide a component model to Java that it so urgently needs, making Java even more competitive with proprietary solutions than it is today. I am really looking forward to work in this JSR. This JSR will be run with an open mailing list, so please tune in.

Peter Kriens
OSGi Evangelist

Sunday, February 26, 2006

OSGi Device Management

The biggest chunk of functionality for the OSGi Mobile Service Platform is the Device Management Tree (DMT) Admin. The Dmt Admin API is a matchmaker between management applications (both remote management servers as well as local applications) and the components that are available on a device. The Dmt Admin maintains a conceptual tree structure of interior and leaf nodes. Interior nodes have children, and leaf nodes have a primitive value like an integer, float, string, time, date etc. Plug-ins map this conceptual tree to actual hardware registers or other value sources. The conceptual tree is closely modeled after the OMA Device Management specification.

In the past two days I have written my first plug-in for the Mobile Expert Group. This group is considering using the Dmt Admin for their R4 specification but they want to make the API simpler for Java applications. I had to write a prototype to make a change request.

Initially the task to write a plug-in looks daunting. The interface that must be implemented has 22 methods, and they look quite, well, not simple. Fortunately, as I found out, most methods do not require an implementation to make the plug-in work. Actually, I was quite surprised how easy it was to implement a simple leaf node. You only have to write a Plugin service that is registered with the Framework when you start up. This plug-in service is asked for a session object that you must implement. This session is used to access the plug-in.

My first attempt was to create a plug-in that maintained the position. The OSGi R4 specification has a Position object that lends itself very well to this concept. A Position object contains six fields: longitude, latitude, altitude, track, and speed. Each of the values is a Measurement object that contains three fields: value, error, and Unit. Overall, a Position is mapped to a sub-tree of eighteen nodes. Not big, but also not trivial.

A key problem of writing a plug-in is to map the URIs (which are String[] objects for the plug-in) to your function. My first attempt was hard-coding this relation, but that became very cumbersome amazingly quickly. I must admit that I am allergic to any redundancy in code, but even less sensitive people should have been affronted. I developed this allergy 10 years ago when working on an application with thousands of fields that were hardcode in hundreds of places.

So for my first attempt used some tables to minimize the redundancy, but the results were still not very satisfactory. Intricate knowledge of the tree was necessary in too many different places.

Looking at the code I remembered having a class that makes any Java object look like a Map, very similar in vein to Java beans. Could I use this? The properties of a bean also form a tree. Properties that result in primitive types like String, byte[] can be mapped to leaf nodes and the remaining properties are treated as interior nodes. This turned out to be a very interesting architecture. I threw away all the code I had written and the plug-in became almost completely generic. Virtually all the code needed for my plug-in was now provided by the Position and Measurement classes. That is the way I like it! Even better, the only change I had to make to my bean inspector was to be able to reason about the tree, even in the absence of instances. The original class only worked with instances.

What the prototype showed me was that, with a little bit of help, it is almost trivial to write a Dmt Admin plug-in. As proof, hereby a class that implements a (non-transactional) plug-in for locales. The remaining code is completely agnostic of the problem domain area.

public class TinyPlugin extends TestPlugin {
  TinyPlugin(BundleContext context) { super("./Tiny", context);}

  public Locale[] getLocales() { return Locale.getAvailableLocales(); }

  public String getDefaultLocale() {
return Locale.getDefault().toString();
  }
     
  public void setDefaultLocale(String locale) {
    String parts[] = locale.split("_");
    Locale l = new Locale(parts[0],
parts.length > 1 ? parts[1] : "",
     parts.length > 2 ? parts[2] : "");
     Locale.setDefault(l);
  }
}

Tree:
     ./Tiny
          Locales
          defaultLocale

This class (together with a library of generic code) implements a full Dmt Admin plug-in. It will be hard to make a similar plug-in with less code. And as you probably have found out in this article, I am a staunch believer in small is beautiful (and robust).

     Peter Kriens
     OSGi Evangelist

Tuesday, February 21, 2006

When adding a new portable to my home network I again realized how wide the gap is between technically unchallenged people and the rest of the world. As a technical person, I sometimes have this snug feeling of superiority because I can get things to work that the rest of the world can only look at in bewilderment. I gladly accept the compliments of our partner when we fixed the PC or connected the new stereo, as if I deserve it. When I look around, I see that many colleagues are similar.

However, sometimes I wonder if our skills are not blinding us from the complexity we put on the rest of the world. And this is not because we are so clever and our users are not, it is actually often the other way around. Users are often too clever to learn unnecessary details and complexities, they just refuse to bring up the patience.

Why is it that the whole consumer industry has been incapable of creating a viable alternative to the iPod, over a period of three years? Why do I still notice, at the worst possible moments, that my address book in my phone is not synchronized? Why does connecting a new PC to the network in a secure way feels like rocket science?

I think a reason is that technical people are not bothered with complexity, we usually do not even notice it, and often thrive on it. A friend of mine is getting started with Java and I am helping him out. Teaching him Java confronts me with the complexity that I no longer notice. It is frightening to see how complicated Java has become over the past 10 years.

I think this complexity in software development is inevitable. However, by being used to a highly abstract and complex world we sometimes do not realize how simplistic the services are that people really like and pay for. For example, a service like “Push to Talk” looks like nothing special but made a huge difference for customers and the Nextel’s bottom line. Another example is how Ericsson has made billions selling business phone systems where the primary function was to provide 3 digit short numbers instead of the normal numbers.

However, we do not have to take the full blame, in most companies the feature set of products is defined by marketing. They are in charge of the product definitions. Marketing people always love simplicity because it makes the feature list longer! However, too many features do not only make a product harder to use, they also make the product harder to develop.

This also points some of the blame to the end users themselves. Cost is the single most important aspect that people use to compare products. I can understand this, because most products are so complex that a reasonable comparison is only possible after a detailed study. And who has time for that? Except when a product is really easy to use, the market recognizes this and rewards the manufacturer with a high profit margin, see the iPod as a prime example.

So how can we build simpler products? Not every company has a Steve Jobs that uniquely combines a deep understanding (and control) of management, marketing, technology, and most important, the requirement for simplicity.

Therefore, management has to become aware that quality costs but that this cost can also result in higher profits. Marketing must realize a short feature list where all features work as advertises is better than a long feature list with a lot of cruft. We technicians can help by never underestimating the patience of our end-users to figure out our products.

Peter Kriens
OSGi Evangelist