BUGs I Like!

Last week I blackmailed Ken Gilmer of Buglabs into finally sending me an evaluation set. He had promised me this set for some time and I wanted him to show this kit on the OSGi Community Event on June 10-11 (You did register I hope?). Ken could not come due to other obligations so I promised to make a demo if he would send me an evaluation kit.

So yesterday Fedex delivered a nice red box at my doorstep! It is always nice when a company spends a bit of effort on designing the box; this package reminded me of my first TomTom, a company that also took care in making the unwrapping experience fun.

The content was a BUG with 4 modules. The BUG itself is the core computer that drives (hopefully) cheap modules. The core contains a small LCD, a minimal joystick, a push button and 4 LEDs that can also be pressed. It is powered through a separate (universal) power supply. Unfortunately, the also present small USB connector is not used to power the device; it is only there for the communications with the development computer. A slide-switch acts as on-off button.



There are 4 modules included:

1. LCD Display
2. GPS
3. Motion Detector
4. Camera

The modules insert into one of the 4 slots of the BUG, there are no specific positions nor are you limited to a single type per BUG. That is, if you want 4 LCD displays, go ahead and do it. I think the modules should be hot swappable but that did not work for me. Hope that will work in the future.

The development environment is based on (surprise!) Eclipse, called DragonFly. They told me to use Eclipse 3.3 (Europa) but I decided to live risky and use my current 3.4 Ganymede copy of Eclipse. At least this way I can only blame myself for some of the things that did not work. DragonFly replaces PDE but there are a number of similarities. Also, the manifest file is the driver for the generation. Overall it was very simple to write a new bundle when you are familiar with Eclipse. DragonFly nicely creates a complete project for you with the classpath setup correctly.

For debugging there is a pretty good emulator that reuses the same Java code and implements the services that abstract the devices. You just ask a project to "Debug As" and the select the Virtual Bug. Very smooth. Once the Virtual BUG is running it appears as a device in the list of Bugs. You can drag and drop your project to these devices and that will install them. This drag and drop interface works very well for device management.

The runtime is of course the most interesting part of this device. It runs the (now) open source PhoneME project. This is a CDC VM that supports most of the core Java SE libraries. On top of this VM, BUGLabs uses a Concierge framework. Concierge took the decision to not move to Release 4 but stick on Release 3 because of size reasons. This makes my feelings toward concierge slightly ambivalent. Release 4 added some very important changes that I hate to miss. I know that size matters, but the difference is not that dramatic with today's memory prices. Ok, I prefer OSGi over no OSGi, but staying behind on release level is very tricky. Lots of bundles now use the Bundle.getBundleContext() method, like Declarative Services, which is not part of R3. Also, the modularity layer got an important boost in R4. However, the biggest problem is compliance. Concierge was not tested for compliance and with the OSGi's current rules we cannot certify it anymore. During the development of R4 we seriously discussed the problem of the size the new features would take. Siemens, who had their own OSGi implementation for the vehicle market, even did a study to the impact. We very consciously decided that these new features were worth the added memory. I guess there is not much we can do about this, but it is a pity that we now have a split in OSGi, a split which we tried to avoid so hard. Anyway, better an OSGi R3 than no OSGi I guess ...

I decided to build a little application to get a feel for the whole thing. I decided to build an automatic camera that would take a picture whenever the GPS indicated that you had moved more than 50 meters. The track thus created would then be available over a web interface. Formatted as a KML file, it can be displayed in Google Earth. Simple.

Well, writing the application was quite smooth and the debugger was great. Getting it to run on the device was a bit harder because it just did not take pictures. It took some time to figure out how to use the Concierge shell. Even that did not help, there was something with the GPS service. Anyway, I guess with some help this will work before the community event. See if I can get my trip to Berlin recorded!

I wish I had had this device while I was working for Ericsson Research in 1998. It is absolutely perfect for trying out automation scenarios. I remember soldering an IO board, a movement sensor and a mobile phone to demonstrate a burglar alarm system based on sending a picture to the police. This scenario would have been a snap with the BUG.

So what are the application areas for the BUG? If you are one of those lucky people that can now and then wander from the virtual world into reality, then this is abslutely your device. You get a quite powerful computer that can be extended with a snap, ehh, 4 snaps. Obviously, if you still can get excited of what you can do with computers (like me), then it is just too much fun to miss. And if you give OSGi tutorials, well, this kind of device makes OSGi shine.

This is not to say they have arrived. BUG labs still has a lot of work carved out for them. The success will depend on amount of modules they can turn out in the coming years. I wish they had developed a breadboard module first so other companies and individuals could make modules. In earlier times I played a lot with the PIC processors from Microchip and it would be quite easy to interface these into the system. Also, a 1-Wire interface would be really fantastic. 1-wire devices are relatively cheap, and very easy to interface. For a few dollars you could extend a BUG with a temperature sensor, humidity sensor, unique ids, switches, cryptography tokens, etc. Remember the famous Java ring? Well, that was a 1-wire device.

The development environment and the documentation also require some work. As a true open source project they use a Wiki for their documentation so do not hesitate to help to make the BUG more popular. There is also a need for more BUG modules as well: Wifi, audio in/out, power switches, video, bar code readers, proximity detectors, RFID readers, fingerprint readers, Zigbee I/O, Bluetooth, infra-red, geiger-tellers, etc., etc.. There is a mighty amount of work awaiting them.

I really do hope other companies will pick up this concept and start to leverage it. Again, I wish I had had this device in 1998 when we were struggling to create demos!

Peter Kriens


P.S. But what on earth is the Teleporter module going to do???

Hello World

If there is one confusing aspect to OSGi then it is probably the bewildering array of programming models. One of the most asked questions lately is: why so many? Why doesn't OSGi standardize on one programming model?

Well, I think having these diverse solutions is proving the strength of OSGi. We set out to make a system that allows you to build applications from decoupled and highly cohesive modules. Decoupling means that you have no assumptions what happens over your (hopefully) close horizon. The dynamic nature of OSGi is a result of the fact that we did not want to bake in the assumption in every module that other modules are magically bound to their life cycle. However, this also means we wanted to minimize the assumptions about the programming model; we wanted to specify the absolute minimum to get collaborations going, but not more.

This clearly opened the road for others to provide programming models. And I think this is good because OSGi is about Universal Middleware. This implies that it is used in a large number of application areas: from copiers to large enterprise applications. Programming models can optimize the model for a particular application area and make it easier to use in that area. I am sure we could have specified something that was as useful as both iPOJO and Spring to their audiences. Still, because they both work on top of OSGi they can interoperate seamlessly. Components written with SAT interact with the Felix Dependency manager without being aware that these components are written with another programming model.

However, where do you start when you like to get into OSGi. So let us have a look at the different models and see how a simple Hello and Goodbye world would look in the different models.

The Example
All the examples are based on a HelloWorld service that has a hello() and goodbye() method. There is an implementation that logs the hello and goodbye messages and there is a client that calls these methods when it is activated and deactived (or when the HelloWorld service disappears).

public interface HelloWorld {
  void hello();
  void goodbye();
}

In the style of POJOs, we define our business logic in a client class and an implementation class:

public class HelloWorldClient {
  HelloWorld hello;

  public void setHello(HelloWorld hello) {
    this.hello= hello;
    if ( hello == null )
      this.hello.goodbye();
    else
      hello.hello();
   this.hello = hello;
  }
}

And the implementation:

public class HelloWorldClient {


  
  
}

Plain Old Osgi Objects
The plain OSGi way to attack the implementation is with a simple Bundle Activator.

public class HelloWorldImpl implements BundleActivator, HelloWorld, ServiceFactory {
public void hello() {

}
void goodbye();
}

Hello World (and Goodbye)

Al Bundle was told by his manager to use OSGi in their next incarnation of their product. So he was looking at the documentation, and he just did not get it. The whole idea of breaking up the application in small modules and using these things, what were they called? Services? Why could they not leave Java alone, it is already hard enough as it is today? What is wrong the classpath? Ok, they had some issues with class cast exception, and if he was really honest their integration test was always hell because getting the class path right was a pretty much black magic, but hey! It worked ...

And why was there no proper documentation? Where is the

Java Modularity

Before JavaOne, there were clearly changes in the air with respect to JSR 277 Modules for Java and JSR 294 Superpackages. The announcement of Glassfish to support OSGi and the project Fuji (an ESB/JBI implementation based on OSGi) represent a significant mind-shift at Sun. We also saw Stanley Ho's interoperability paper on the JSR 277 mailing list. I therefore was happily surprised when the JSR 277 Java Modularity spec leads, Alex Buckley and Stanley Ho, when they asked me if I still wanted to join JSR 277.

This question posed a dilemma for me. Was I still interested? Two years ago when JSR 277 started I was rejected and I made a big fuzz about this, which obviously puts a moral burden on me now. Also, the changes around JSR 294 Superpackages definitely inclined me to join. JSR 294 was now folded back in JSR 277 and its course had changed 180 degrees. Instead of an overly redundant poorly thought out model Alex had kicked the bucket under the superpackages draft and proposed the much simpler concept of an access modifier called module. Though this idea had not been worked out in detail, it clearly was much, much, more feasible. Modules setup this way could be very usable in the OSGi world. My technical me would love to work on this aspect.

However, JSR 277 now consists of two good (Repository and Language modules) and one bad part (JAM). Just before JavaOne 2008 Stanley had published a brief OSGi interoperability document. This document was not only very thin on details, it also referred in many places to an invisible Early Draft #2 (EDR2). I asked the EG members BJ Hargrave and Richard Hall if they had seen this EDR2, but it was unknown to them as well. I had recently complained about the lack of discussions on the JSR 277 mailing list, so it seems that work had been going on stealthily.

During the JSR 277 presentations Alex and Stanley it became clear to me that a lot of work had happened behind the scenes; Without any communications on the mailing lists. For somebody in the audience it was absolutely not clear that almost everything that was said was never discussed in the EG. How could I participate in an EG if the only option would be to approve work done at Sun behind the curtains? How much change can there be if the unpublished EDR2 is presented at JavaOne as the stable result for JSR 277? Should I join an EG where most of the work was already done? How many fundamental changes can you achieve in this situation?

I wish it was differently. Bryan Atsatt did a lot of work to convince Sun to open up, and it worked better than one could have expected. So, there is a lot of moral pressure on me to compromise and accept. As several people said: "Sun will not give up their JAMs, so get over it." Now, it is in my nature to compromise; I prefer working together on technical details instead of fighting. However, I just turned 50 years old and I can recall too many painful events where I compromised against my technical instinct. Years later the forces behind the compromise were forgotten and the inferior technical design remained with my name attached to it. I really do not like that and this seems one of those cases.

So why do I not like JAMs? Well, who needs them? Do they provide something that OSGi not already provides? Alex and Stanley stressed the "fact" that JAMs are "deterministic" and "simple", the Siren's song of simplistic solutions. Only when you do not fully understand the problem is JAM an attractive solution. The truth is, JAM creates unnecessary complexity, the worst of all.

There is only one argument that makes sense for a special module layer for the VM: The boot classes. Obviously during the boot phase a full-blown OSGi framework is not feasible because OSGi is written in Java. However, JAM will have exactly the same problems loading java.lang.Object because the environment to run the framework is not loaded yet. How do you process annotations without having the Class class loaded? Write an annotation layer in C? I will not even go into initialization and custom module binding support. Clearly, a special solution is required to modularize the core language packages. This problem was also faced by Apache Harmony. They also did not want to support a full blown OSGi framework in their VM, but they did understand the importance of existing standards. Apache Harmony modularized the bootpath classes and provides them as OSGi bundles using the standardized metadata. Because headers are in the manifest and easy to parse, no complex redundancy is required as with JAM's annotation support. Using the same metadata makes a seamless transition possible from booting all the way to running one of the many application servers defined on top of OSGi. For management systems it is ideal if they only have to worry about a single format.

We are now at a crucial point in time. Though I'd love to participate in the discussions about modules in the Java language and the repositories, I cannot become responsible for a module system in Java that is creating this complexity for no rational reason. I do hope Sun will take this last step as well, and drop JAM in favor of the much simpler approach of using the OSGi metadata throughout the system. In the last months we have gotten so far! Please?

Peter Kriens

P.S. Did you register at the Community Event yet? There are only 190 places and I think they will be filled up. After Sun's embrace of OSGi, a lot more people got interested. You surely do not want to miss it. Last week we finalized the program it looks surprisingly strong.

P.P.S. Do you regularly read this blog? Do you know you can become an OSGi supporter? This free of charge and it helps us make it more clear to people that you support OSGi, sometimes numbers help, becoming a supporter does help. Of course if you can afford it, we'd love even better to have you as a full member or adopter. Take a look at the conditions. Appreciated!

SpringSource Application Platform + Bundle Repository

When SpringSource announced their Application Platform and Bundle Repository I was very excited. I had been informed something was coming and picked up rumors from the grapevine.The bundle repository was in one word: stunning. I had been looking into providing such a service but had decided that I could not afford it, despite the fact that I think that many companies would really benefit from such a service. This bundle repository must have been a terrible job to do. Kudos!

The Spring Source Application Platform was, however, a bit of a shock. Looking through the documentation I found lots of headers that felt like OSGi but that I did not recognize: Import-Library, Import-Bundle, Application, etc. It looked like SpringSource had “improved” OSGi extensively. This was a bit of a shock, especially because they looked so much like standard OSGi headers.

I learned over the past 10 years that just adding a number of headers that solve a particular problem usually backfires. Something that sounds really good, does not have to be that good at all when you inspect it from a different perspective. The problem with standards is, is that it is very easy to add something but impossible to remove. You really have to get right as much as possible the first time; it is therefore better to err on the side of leaving things out. Due to the embedded origin of OSGi we have always been extremely careful to minimize the number of feature. Each additional feature require a lot of eyes and discussions and a very thorough analysis before it makes into the specification. Especially today, with a mature specification the feature interaction can be quite painful.

Do not get me wrong, as a practitioner I do know the importance of getting things to work and standards, by definition, are not always sufficient. I also know that a standardization process feels sometimes excruciatingly slow when you live in the trenches, and I love experiments.

However, Eclipse has the same problem, but they have always been a good citizen in this respect. They always added proprietary headers in their own name space, learned from their experiences, and then proposed a new standard. In all cases so far, the specification process was able to make the concept more general and fit better in the overall OSGi model. Having a separate name space was also a convenience for their customers. A specification process will undoubtedly change the semantics, requiring awkward constructs to distinguish between the SpringSource defined meanings and the OSGi defined meanings. I do realize that OSGi does not own the name space in the manifest, and I do regret heavily that we did no better scope our header names, however, this did not seem overly important 10 years ago.

Why would the standardized version be better? Because more eyes looked at it, from different perspectives (which sometimes can be quite bad for one's ego). One man's convenience is another man's complexity. This is why so far we have always focused on providing the primitives and not convenience functions because convenience is often in the eye of the beholder and can be provided during the development process in the IDE or run-time libraries.

The biggest danger of a standard like OSGi today is feature creep. The leanness of OSGi is one of its main attractions for many newcomers. It really took real hard work and untold man hours to make it this lean, believe me. Maybe the SpringSource headers are a significant improvement over the existing specification, maybe they turn out to be superfluous. Time will tell. I guess it is a sign of becoming more popular that companies try to stretch the specification in their world view. However, I do hope that most developers realize that restricting oneself to the standard may sometimes be a little bit less convenient, but it does provide one with a more options and no vendor lock-in down the line.

So sadly, my feelings are mixed. I love what they have done with the bundle repository, and also many parts of the S2AP look very impressive, Neil Bartlett's blog provides an overview that I can very much sympathize with. I know several Spring Sourcerers and they are very clever and capable people. SpringSource undeniably is meaning a lot for the OSGi organization and the adoption of the specifications. However, multiplying bundle headers feels more like coming from an apprentice than from the experts I know them to be.

Peter Kriens

P.S. On a more positive note. We are reaching the first deadline for registration of the OSGi Community event in Berlin, June 10-11. I am part of the program committee and I can ensure you that it is worth coming. We actually had to split the second day into two tracks because we had so many good sessions submitted. Looking at the number of submissions it will look like we will sell out the places we have this year. This is the premier OSGi event in Europe, we are really expecting all the key OSGi people to be there. Take advantage of the early registration fee and register today!

KISS Principle

Wikipedia defines this principle as:

The KISS principle (acronym for "Keep It Simple, Stupid") states that design simplicity should be a key goal and unnecessary complexity avoided. It serves as a useful principle in a wide array of disciplines, such as software development, animation, photography, engineering, and strategic planning.

Today Google Alerts dumped me into two discussions where OSGi seems to be on the wrong side of the divide regarding the KISS principle. The first is an extensive debate in the dev@tomcat.apache.org mailing list, the second is the JSR 277 mailing list.

The Tomcat developers have been asked to osgify their popular web server. This resulted in a discussion that was very interesting to read. Several people seem to have a disdain for the OSGi zealots (which I guess is me) but there seems to be a general uneasiness with this request. However, my key trigger for this blog was that in this discussion I noticed that several people regard OSGi as overly complex and heavy. The JSR 277 discussion raised the same problem from another direction. From the JSR 291 interoperation discussion: "I wanted something in Java that was simpler and cleaner that OSGi-users could leverage if they wanted.". Is OSGi already ossified?

The key problem to decide this question is, as so often the case, requirements. Is Java complex? Hard to answer, if you want to write a "Hello World" it is humongous overkill. If you are writing a large scale web application it is nice to have a solid foundation. The key phrase in the definition of KISS is unnecessary complexity. Complexity is in the eye of the beholder.

First, the OSGi API is very small by any measure. There is only one mandatory package that only has 26 class files, including the security and exception classes. And you can already leverage OSGi without writing Java code. Can you call it intrusive when you can turn a JAR into a bundle by adding one or two manifest lines? All the complexity people complain about is optional. That said, I do agree that even optional things create some kind of conceptual complexity. So maybe there is a lot of cruft in the options?

Let us take a feature like import packages which is often considered hard. Most dependency mechanisms like Maven, Ivy, and also the original JSR 277 are based on the Require-Bundle concept. I.e. you declare a dependency on the wrapper.
Wrapper based dependency models are simple to understand for us humans. It would work actually very well in practice if JARs would be highly cohesive and not change their constituent packages over time. Which raises the question: Do you want the woman, or the dress?

We developers have discovered the wonders of refactoring and things do change. OSGi is a module system, it is about being able to change software while not breaking the deployments. Though import packages might be harder to understand initially than Require-Bundle, Import-Package does create more resilient systems because it minimizes coupling. That is why people that have used both systems usually decide to standardize on Import-Package (and bnd makes this quite easy).

Similar to the uses: directive. In the Tomcat discussion it was stated that the uses directives made the manifest unreadable. It was much nicer to create a manifest by hand, it looked better. I do agree from an aesthetic point of view but who reads a manifest? The manifest is intended to be read by the OSGi Framework. The uses directive is an incredible important concept to maintain classpace consistency when the same class can be loaded multiple times through different class loaders. If you have lived in a world where this model is supported, you understand that uses have a very high value because they minimize runtime problems.

I do not say that OSGi has no unnecessary complexity at all. Obviously, the specification has been evolving over almost ten years. We have to tried to keep it is simple as possible, but we are as fallible as all of us. However, requirements have always come first in OSGi and fulfilling requirements just forbid you to make things too simple because they force you to understand the problem.

Obviously the key question is, are many of these requirements highly specialized cases not useful for the majority. Or are they just solving problems that awaits anybody that ventures into the wonderful land of modularity?

Peter Kriens

First book on OSGi at JAX 2008!

Last evening I got back from a very pleasant trip to JAX 2008 in Wiesbaden. I was asked to give the best practice and the OSGi modularity presentations. Rod Johnson had already warmed the audience for OSGi with the result that we had to close the room when people where sitting, crawling, and standing everywhere even though there were still 40-50 people standing outside. Good news for OSGi, but I had to give the presentation for a second time that evening at 21.15, where I was surprised that there were still 50-60 people present that late in the evening. The next day was even better because I got a much bigger room that was also filled. Feels good after gotten used to preaching to audiences of 10 people for so many years.

I also got handed a copy of the first official OSGi book in the world. The book, written by Gerd Wütherich, Matthias Lübken, Nils Hartmann, and Bernd Kolb.

This is a thorough book on OSGi (though Equinox biased) that I recommend to anyone who is using OSGi or wants to know more about it. You can buy it at d.punkt. I hope they translate it in English soon. I already read most of the text during my Christmas vacation to help them with a review. Gerd had already shown me on the first day the thank you section. They had been really nice, and made the thank you in Dutch! Great work! It is great to finally being able to point to a good OSGi book.

JAX is a great conference, very well organized. Well, there is so much to do that I'll keep it short. Though there is one observation, unrelated to OSGi, I can't help myself making. On the way back with Ryan Air I observed that people can be incredibly rude at jumping the queue, and not just the one that look like it, even elegantly looking older people showed an incredible rudeness. Though I admire Ryan Air for many things, their queue management is non existent, seeming to call out the worst behavior in people. I figured people behaved so badly so that to get the best seats in the plane. However, when I boarded at the end of the queue the emergency seats in the middle were still mostly free. So why are people so much in a hurry that they are jumping the queue? One day I have to debug this, but now back to the build!

Peter Kriens

P.S. Talking about books, you can now also order the specification (core + compendium) in printed form for a special price.