Is 9903520300447984150353281023 Too Small?

JSR 277's Stanley Ho published a rationale why (in the so far unpublished EDR2) JAva Modules (JAM) invent a brand new version scheme. A rationale that needs a lot of text. I could go in painstaking detail, but I think the rationale derails in one of the first paragraphs where the requirements are (implicitly) described. I highlighted the part that describes in what kind of situations you use the major, minor, micro, update, and qualifier version parts:

• Major version number should be incremented for making changes that are not backward-compatible. The minor and the micro numbers should then be reset to zero and the update number omitted.
• Minor version number should be incremented for making medium or minor changes where the software remains largely backward-compatible (although minor incompatibilities might be possible); the micro number should then be reset to zero and the update number omitted.
• Micro version number should be incremented for changing implementation details where the software remains largely backward compatible (although minor incompatibilities might be possible); the update number should then be omitted.
• Update version number should be incremented for adding bug fixes or performance improvements in a highly compatible fashion.
• Qualifier should be changed when the build number or milestone is changed.

Can you spot the difference between minor and micro? I can't, and that is the reason that OSGi proposes the following convention: incompatible (major), backward compatible (minor), no API change/bugfix (micro), and builds or variations (qualifier). That is, our micro is Sun's update because Sun's minor and micro appear to have an identical purpose.

And ever thought about the concept of largely backward compatible? Isn't that something like a little bit pregnant?

There are always reason to improve on existing schemes, but any improvement should be balanced against the interests of the existing and future audiences. I am not stating that the OSGi version is the mother of all version schemes, we are as fallible as all of us. Maybe we were too rationalistic; we looked at a lot of schemes and saw how people were also looking for more room at the low-end of the version scheme and hardly ever incremented the first numbers. If you standardize there is always a tension between allowing as much freedom as possible but also minimizing the complexity of the implementations that are depending on the variations you offer. We chose simplicity.

Is the OSGi scheme usable? Well, we have no outstanding requirements or bugs in this area, nor were any proposed in JSR 291, while at the same time it is heavily used. Jason van Zyl, Maven, told me they were thinking of adopting the scheme as well. SpringSource converted almost 400 open source projects to bundles and they did not complain. Seems there is a lot of practical usage out there.

Wouldn't it be a lot more productive for all of us if Sun would just adopt the OSGi scheme? There is lots of work to do on the module API, why not reuse existing specifications where you can? And if the OSGi scheme has burning issues, why not report that as a bug or change request, after all Sun is a distinguished OSGi member.

With an infinite number of builds or variations, 9903520300447984150353281023 possible bug fixes (80.000 fixes per microsecond until the Sun implodes), 4611686014132420609 backward compatible changes, and 2147483647 incompatible releases, the OSGi spec seems to have enough room to breathe for mere mortals.

Peter Kriens

P.S. You did register for the OSGi Community Event in Berlin, June 10-11? Please do so, we have a very strong program prepared with lots of cool demos. Hope to see you there!

BUGs I Like!

Last week I blackmailed Ken Gilmer of Buglabs into finally sending me an evaluation set. He had promised me this set for some time and I wanted him to show this kit on the OSGi Community Event on June 10-11 (You did register I hope?). Ken could not come due to other obligations so I promised to make a demo if he would send me an evaluation kit.

So yesterday Fedex delivered a nice red box at my doorstep! It is always nice when a company spends a bit of effort on designing the box; this package reminded me of my first TomTom, a company that also took care in making the unwrapping experience fun.

The content was a BUG with 4 modules. The BUG itself is the core computer that drives (hopefully) cheap modules. The core contains a small LCD, a minimal joystick, a push button and 4 LEDs that can also be pressed. It is powered through a separate (universal) power supply. Unfortunately, the also present small USB connector is not used to power the device; it is only there for the communications with the development computer. A slide-switch acts as on-off button.



There are 4 modules included:

1. LCD Display
2. GPS
3. Motion Detector
4. Camera

The modules insert into one of the 4 slots of the BUG, there are no specific positions nor are you limited to a single type per BUG. That is, if you want 4 LCD displays, go ahead and do it. I think the modules should be hot swappable but that did not work for me. Hope that will work in the future.

The development environment is based on (surprise!) Eclipse, called DragonFly. They told me to use Eclipse 3.3 (Europa) but I decided to live risky and use my current 3.4 Ganymede copy of Eclipse. At least this way I can only blame myself for some of the things that did not work. DragonFly replaces PDE but there are a number of similarities. Also, the manifest file is the driver for the generation. Overall it was very simple to write a new bundle when you are familiar with Eclipse. DragonFly nicely creates a complete project for you with the classpath setup correctly.

For debugging there is a pretty good emulator that reuses the same Java code and implements the services that abstract the devices. You just ask a project to "Debug As" and the select the Virtual Bug. Very smooth. Once the Virtual BUG is running it appears as a device in the list of Bugs. You can drag and drop your project to these devices and that will install them. This drag and drop interface works very well for device management.

The runtime is of course the most interesting part of this device. It runs the (now) open source PhoneME project. This is a CDC VM that supports most of the core Java SE libraries. On top of this VM, BUGLabs uses a Concierge framework. Concierge took the decision to not move to Release 4 but stick on Release 3 because of size reasons. This makes my feelings toward concierge slightly ambivalent. Release 4 added some very important changes that I hate to miss. I know that size matters, but the difference is not that dramatic with today's memory prices. Ok, I prefer OSGi over no OSGi, but staying behind on release level is very tricky. Lots of bundles now use the Bundle.getBundleContext() method, like Declarative Services, which is not part of R3. Also, the modularity layer got an important boost in R4. However, the biggest problem is compliance. Concierge was not tested for compliance and with the OSGi's current rules we cannot certify it anymore. During the development of R4 we seriously discussed the problem of the size the new features would take. Siemens, who had their own OSGi implementation for the vehicle market, even did a study to the impact. We very consciously decided that these new features were worth the added memory. I guess there is not much we can do about this, but it is a pity that we now have a split in OSGi, a split which we tried to avoid so hard. Anyway, better an OSGi R3 than no OSGi I guess ...

I decided to build a little application to get a feel for the whole thing. I decided to build an automatic camera that would take a picture whenever the GPS indicated that you had moved more than 50 meters. The track thus created would then be available over a web interface. Formatted as a KML file, it can be displayed in Google Earth. Simple.

Well, writing the application was quite smooth and the debugger was great. Getting it to run on the device was a bit harder because it just did not take pictures. It took some time to figure out how to use the Concierge shell. Even that did not help, there was something with the GPS service. Anyway, I guess with some help this will work before the community event. See if I can get my trip to Berlin recorded!

I wish I had had this device while I was working for Ericsson Research in 1998. It is absolutely perfect for trying out automation scenarios. I remember soldering an IO board, a movement sensor and a mobile phone to demonstrate a burglar alarm system based on sending a picture to the police. This scenario would have been a snap with the BUG.

So what are the application areas for the BUG? If you are one of those lucky people that can now and then wander from the virtual world into reality, then this is abslutely your device. You get a quite powerful computer that can be extended with a snap, ehh, 4 snaps. Obviously, if you still can get excited of what you can do with computers (like me), then it is just too much fun to miss. And if you give OSGi tutorials, well, this kind of device makes OSGi shine.

This is not to say they have arrived. BUG labs still has a lot of work carved out for them. The success will depend on amount of modules they can turn out in the coming years. I wish they had developed a breadboard module first so other companies and individuals could make modules. In earlier times I played a lot with the PIC processors from Microchip and it would be quite easy to interface these into the system. Also, a 1-Wire interface would be really fantastic. 1-wire devices are relatively cheap, and very easy to interface. For a few dollars you could extend a BUG with a temperature sensor, humidity sensor, unique ids, switches, cryptography tokens, etc. Remember the famous Java ring? Well, that was a 1-wire device.

The development environment and the documentation also require some work. As a true open source project they use a Wiki for their documentation so do not hesitate to help to make the BUG more popular. There is also a need for more BUG modules as well: Wifi, audio in/out, power switches, video, bar code readers, proximity detectors, RFID readers, fingerprint readers, Zigbee I/O, Bluetooth, infra-red, geiger-tellers, etc., etc.. There is a mighty amount of work awaiting them.

I really do hope other companies will pick up this concept and start to leverage it. Again, I wish I had had this device in 1998 when we were struggling to create demos!

Peter Kriens


P.S. But what on earth is the Teleporter module going to do???

Hello World

If there is one confusing aspect to OSGi then it is probably the bewildering array of programming models. One of the most asked questions lately is: why so many? Why doesn't OSGi standardize on one programming model?

Well, I think having these diverse solutions is proving the strength of OSGi. We set out to make a system that allows you to build applications from decoupled and highly cohesive modules. Decoupling means that you have no assumptions what happens over your (hopefully) close horizon. The dynamic nature of OSGi is a result of the fact that we did not want to bake in the assumption in every module that other modules are magically bound to their life cycle. However, this also means we wanted to minimize the assumptions about the programming model; we wanted to specify the absolute minimum to get collaborations going, but not more.

This clearly opened the road for others to provide programming models. And I think this is good because OSGi is about Universal Middleware. This implies that it is used in a large number of application areas: from copiers to large enterprise applications. Programming models can optimize the model for a particular application area and make it easier to use in that area. I am sure we could have specified something that was as useful as both iPOJO and Spring to their audiences. Still, because they both work on top of OSGi they can interoperate seamlessly. Components written with SAT interact with the Felix Dependency manager without being aware that these components are written with another programming model.

However, where do you start when you like to get into OSGi. So let us have a look at the different models and see how a simple Hello and Goodbye world would look in the different models.

The Example
All the examples are based on a HelloWorld service that has a hello() and goodbye() method. There is an implementation that logs the hello and goodbye messages and there is a client that calls these methods when it is activated and deactived (or when the HelloWorld service disappears).

public interface HelloWorld {
  void hello();
  void goodbye();
}

In the style of POJOs, we define our business logic in a client class and an implementation class:

public class HelloWorldClient {
  HelloWorld hello;

  public void setHello(HelloWorld hello) {
    this.hello= hello;
    if ( hello == null )
      this.hello.goodbye();
    else
      hello.hello();
   this.hello = hello;
  }
}

And the implementation:

public class HelloWorldClient {


  
  
}

Plain Old Osgi Objects
The plain OSGi way to attack the implementation is with a simple Bundle Activator.

public class HelloWorldImpl implements BundleActivator, HelloWorld, ServiceFactory {
public void hello() {

}
void goodbye();
}

Hello World (and Goodbye)

Al Bundle was told by his manager to use OSGi in their next incarnation of their product. So he was looking at the documentation, and he just did not get it. The whole idea of breaking up the application in small modules and using these things, what were they called? Services? Why could they not leave Java alone, it is already hard enough as it is today? What is wrong the classpath? Ok, they had some issues with class cast exception, and if he was really honest their integration test was always hell because getting the class path right was a pretty much black magic, but hey! It worked ...

And why was there no proper documentation? Where is the

Java Modularity

Before JavaOne, there were clearly changes in the air with respect to JSR 277 Modules for Java and JSR 294 Superpackages. The announcement of Glassfish to support OSGi and the project Fuji (an ESB/JBI implementation based on OSGi) represent a significant mind-shift at Sun. We also saw Stanley Ho's interoperability paper on the JSR 277 mailing list. I therefore was happily surprised when the JSR 277 Java Modularity spec leads, Alex Buckley and Stanley Ho, when they asked me if I still wanted to join JSR 277.

This question posed a dilemma for me. Was I still interested? Two years ago when JSR 277 started I was rejected and I made a big fuzz about this, which obviously puts a moral burden on me now. Also, the changes around JSR 294 Superpackages definitely inclined me to join. JSR 294 was now folded back in JSR 277 and its course had changed 180 degrees. Instead of an overly redundant poorly thought out model Alex had kicked the bucket under the superpackages draft and proposed the much simpler concept of an access modifier called module. Though this idea had not been worked out in detail, it clearly was much, much, more feasible. Modules setup this way could be very usable in the OSGi world. My technical me would love to work on this aspect.

However, JSR 277 now consists of two good (Repository and Language modules) and one bad part (JAM). Just before JavaOne 2008 Stanley had published a brief OSGi interoperability document. This document was not only very thin on details, it also referred in many places to an invisible Early Draft #2 (EDR2). I asked the EG members BJ Hargrave and Richard Hall if they had seen this EDR2, but it was unknown to them as well. I had recently complained about the lack of discussions on the JSR 277 mailing list, so it seems that work had been going on stealthily.

During the JSR 277 presentations Alex and Stanley it became clear to me that a lot of work had happened behind the scenes; Without any communications on the mailing lists. For somebody in the audience it was absolutely not clear that almost everything that was said was never discussed in the EG. How could I participate in an EG if the only option would be to approve work done at Sun behind the curtains? How much change can there be if the unpublished EDR2 is presented at JavaOne as the stable result for JSR 277? Should I join an EG where most of the work was already done? How many fundamental changes can you achieve in this situation?

I wish it was differently. Bryan Atsatt did a lot of work to convince Sun to open up, and it worked better than one could have expected. So, there is a lot of moral pressure on me to compromise and accept. As several people said: "Sun will not give up their JAMs, so get over it." Now, it is in my nature to compromise; I prefer working together on technical details instead of fighting. However, I just turned 50 years old and I can recall too many painful events where I compromised against my technical instinct. Years later the forces behind the compromise were forgotten and the inferior technical design remained with my name attached to it. I really do not like that and this seems one of those cases.

So why do I not like JAMs? Well, who needs them? Do they provide something that OSGi not already provides? Alex and Stanley stressed the "fact" that JAMs are "deterministic" and "simple", the Siren's song of simplistic solutions. Only when you do not fully understand the problem is JAM an attractive solution. The truth is, JAM creates unnecessary complexity, the worst of all.

There is only one argument that makes sense for a special module layer for the VM: The boot classes. Obviously during the boot phase a full-blown OSGi framework is not feasible because OSGi is written in Java. However, JAM will have exactly the same problems loading java.lang.Object because the environment to run the framework is not loaded yet. How do you process annotations without having the Class class loaded? Write an annotation layer in C? I will not even go into initialization and custom module binding support. Clearly, a special solution is required to modularize the core language packages. This problem was also faced by Apache Harmony. They also did not want to support a full blown OSGi framework in their VM, but they did understand the importance of existing standards. Apache Harmony modularized the bootpath classes and provides them as OSGi bundles using the standardized metadata. Because headers are in the manifest and easy to parse, no complex redundancy is required as with JAM's annotation support. Using the same metadata makes a seamless transition possible from booting all the way to running one of the many application servers defined on top of OSGi. For management systems it is ideal if they only have to worry about a single format.

We are now at a crucial point in time. Though I'd love to participate in the discussions about modules in the Java language and the repositories, I cannot become responsible for a module system in Java that is creating this complexity for no rational reason. I do hope Sun will take this last step as well, and drop JAM in favor of the much simpler approach of using the OSGi metadata throughout the system. In the last months we have gotten so far! Please?

Peter Kriens

P.S. Did you register at the Community Event yet? There are only 190 places and I think they will be filled up. After Sun's embrace of OSGi, a lot more people got interested. You surely do not want to miss it. Last week we finalized the program it looks surprisingly strong.

P.P.S. Do you regularly read this blog? Do you know you can become an OSGi supporter? This free of charge and it helps us make it more clear to people that you support OSGi, sometimes numbers help, becoming a supporter does help. Of course if you can afford it, we'd love even better to have you as a full member or adopter. Take a look at the conditions. Appreciated!

SpringSource Application Platform + Bundle Repository

When SpringSource announced their Application Platform and Bundle Repository I was very excited. I had been informed something was coming and picked up rumors from the grapevine.The bundle repository was in one word: stunning. I had been looking into providing such a service but had decided that I could not afford it, despite the fact that I think that many companies would really benefit from such a service. This bundle repository must have been a terrible job to do. Kudos!

The Spring Source Application Platform was, however, a bit of a shock. Looking through the documentation I found lots of headers that felt like OSGi but that I did not recognize: Import-Library, Import-Bundle, Application, etc. It looked like SpringSource had “improved” OSGi extensively. This was a bit of a shock, especially because they looked so much like standard OSGi headers.

I learned over the past 10 years that just adding a number of headers that solve a particular problem usually backfires. Something that sounds really good, does not have to be that good at all when you inspect it from a different perspective. The problem with standards is, is that it is very easy to add something but impossible to remove. You really have to get right as much as possible the first time; it is therefore better to err on the side of leaving things out. Due to the embedded origin of OSGi we have always been extremely careful to minimize the number of feature. Each additional feature require a lot of eyes and discussions and a very thorough analysis before it makes into the specification. Especially today, with a mature specification the feature interaction can be quite painful.

Do not get me wrong, as a practitioner I do know the importance of getting things to work and standards, by definition, are not always sufficient. I also know that a standardization process feels sometimes excruciatingly slow when you live in the trenches, and I love experiments.

However, Eclipse has the same problem, but they have always been a good citizen in this respect. They always added proprietary headers in their own name space, learned from their experiences, and then proposed a new standard. In all cases so far, the specification process was able to make the concept more general and fit better in the overall OSGi model. Having a separate name space was also a convenience for their customers. A specification process will undoubtedly change the semantics, requiring awkward constructs to distinguish between the SpringSource defined meanings and the OSGi defined meanings. I do realize that OSGi does not own the name space in the manifest, and I do regret heavily that we did no better scope our header names, however, this did not seem overly important 10 years ago.

Why would the standardized version be better? Because more eyes looked at it, from different perspectives (which sometimes can be quite bad for one's ego). One man's convenience is another man's complexity. This is why so far we have always focused on providing the primitives and not convenience functions because convenience is often in the eye of the beholder and can be provided during the development process in the IDE or run-time libraries.

The biggest danger of a standard like OSGi today is feature creep. The leanness of OSGi is one of its main attractions for many newcomers. It really took real hard work and untold man hours to make it this lean, believe me. Maybe the SpringSource headers are a significant improvement over the existing specification, maybe they turn out to be superfluous. Time will tell. I guess it is a sign of becoming more popular that companies try to stretch the specification in their world view. However, I do hope that most developers realize that restricting oneself to the standard may sometimes be a little bit less convenient, but it does provide one with a more options and no vendor lock-in down the line.

So sadly, my feelings are mixed. I love what they have done with the bundle repository, and also many parts of the S2AP look very impressive, Neil Bartlett's blog provides an overview that I can very much sympathize with. I know several Spring Sourcerers and they are very clever and capable people. SpringSource undeniably is meaning a lot for the OSGi organization and the adoption of the specifications. However, multiplying bundle headers feels more like coming from an apprentice than from the experts I know them to be.

Peter Kriens

P.S. On a more positive note. We are reaching the first deadline for registration of the OSGi Community event in Berlin, June 10-11. I am part of the program committee and I can ensure you that it is worth coming. We actually had to split the second day into two tracks because we had so many good sessions submitted. Looking at the number of submissions it will look like we will sell out the places we have this year. This is the premier OSGi event in Europe, we are really expecting all the key OSGi people to be there. Take advantage of the early registration fee and register today!