Complete

Every time when a new industry joins OSGi there is a lot of new people entering the specification work. These people, inevitably, bring their their own culture. Mixing cultures is not always without its problems. Having lived and worked in Holland, Sweden, and France I learned the hard way that moving to another culture can be a tricky thing; it does take some time before you realize that your absolute truth is not really shared by your new countrymen.

So the Enterprise experts bring their prevailing culture. With respect to standards, this culture is heavily influenced by JCP, specifically JEE, and the way things work in the open source world. This is not always aligned with the way OSGi has been run so far. Previous OSGi specs have always been rather rather thoroughly reviewed and picked apart. A running joke in the Alliance is that we start with a telegraph pole but we we end up with a tooth pick. But what a tooth pick!

One of the key things I always liked about the final OSGi specs is that they are complete. Virtually all service specifications and framework specification thoroughly specify edge cases, are fully introspective and provide relevant events to track what is happening inside the service. In JEE vendors have much more freedom. For example, in JEE the deployment aspects are left as an implementation detail. This completeness of OSGi specs is demonstrated by the fact that (so far) all our specs were able to run from a single test framework while only requiring vendors to have their to be tested bundles installed. It is possible to define do the test setup in the test case because the specifications are sufficiently complete. For example, we can deploy a bundle in a framework without having to know the vendor of that framework.

However, some of the EEG members complain we're too specific and do not allow enough space for implementations to do it their way. By giving more leeway to implementations, you allow more innovation and vendors can more easily fit existing products under a specification. These are not invalid arguments.

So I am struggling a bit with this issue. One of my primary roles in the OSGi Alliance is to guard the consistency between the specifications. This causes me sometimes to feel that I am fighting the whole group to guard this consistency. Despite the fact that I am really trying to run the fine line between being conservative and enabling new groups to do it their way. However, the hardest part is that I continuously have to challenge my own beliefs to try to see their point of view. Well, that is what mixing cultures does to you ...

Peter Kriens

OSGi on the Road

A few weeks ago I got a very nice thank you letter from Eric Gaignet, an employee of a regional bus company in the South-West of France called RDTL Voyages. A few years ago he asked me a few questions over email and I had replied him with some architectural advice of how to use OSGi in a vehicle. I remember that at the time I thought this was a wonderful application for OSGi. He reported now that this project had gone very smoothly.

RDTL is a small regional bus carrier (150 buses) with low overhead. A few years ago they realized that new regulations, the impact on the environment, the complexity of the many isolated black box solutions in the bus, and new business opportunities were being combined with an aging IT infra-structure in the bus. This created an interesting opportunity to design a new infra-structure virtually from the ground up, taking advantage of modern technology.

The number of IT solutions in a passenger bus is surprisingly large. Today, a bus interacts with the road systems, it provides up to date information to the electronic displays, it handles the ticketing issues, and reports about the state of the bus. Traditionally, different vendors provide their own isolated solutions.

Not so for RDTL. They started a collaboration with GeenSys, a French IT company specialized in embedded systems to create the e-nove architecture. The input was to create a system where an OSGi gateway in the bus would communicate with an OSGi server in the back office. The gateway connects all the equipment in the bus, provide local information to the driver and aggregates the information to the back-end. Instead of having a proprietary ticketing system, the OSGi gateway connects to a printer, reader, and a display. This all being controlled by OSGi bundles. In the past two years, this architecture was developed and implemented in a remarkably short time. Today, there are more than 30 buses equipped with OSGi gateways.

After I queried Eric for some more information how things had been going, he replied with the following quote:

The best positive experience we got is that we immediately needed an "small application" this summer. I asked a local software company to design this software. They spent 2 days training with Geensys engineers. Thanks to e-nove, they spent only 2 days to design this application. This is great because this was one of the key goals of the project. We can react, adapt and deploy bundles in a very short time.

The e-nove architecture is a very nice example of what you can do with OSGi. It has many of the use cases that drove the development of OSGi in the early years and it is wonderful that it not only has been developed, but it now also been proven to provide the expected advantages. Thanks Eric, for this update!

Peter Kriens

JavaBlend in Ljubljana and Belgrade

About 6 months ago Aleš Justin (JBoss/Redhat) asked me if I could come to Ljubljana for the JavaBlend conference. I told Aleš to talk to OSGi marketing and after some mail exchanges we agreed that if they paid the trip I would come. After this was agreed, I was informed it also included a presentation in Belgrade. Well, from a time perspective this wasn't too much of a difference so I agreed, under the condition I could fly back Friday from Belgrade and then forgot about it.

Normally a conference trip means flying in the night before, giving your presentation, talking to a few people and then trying to get back home as quickly as possible. However, this trip turned out to be quite different; it felt more like a 3 day adventure than work.

The Ljubljana JavaBlend day was very interesting with just very good speakers. Everything helped, the weather was good, the location was excellent (a very nice castle overlooking Ljubljana), and an intriguing number of very young fashion models.

Even my presentation went well. I was fortunate that the speaker before me, Juha Lindfors (OpenRemote), spoke about an open source project doing home automation. Exactly the area that got OSGi started! Though they had taken Tomcat as their platform, I obviously worked on him to move to OSGi. I could honestly say OSGi was made for exactly their purpose.

I learned about Yugoslavia in school and I followed the news about Tito's dead in 1980, the breakup, and the balkan wars. However, the whole geographic situation was a tad fuzzy to me. The first time in former Yugoslavia, the republic Slovenia definitely charmed me, especially because Aleš had taken me on a small sight-seeing tour. However, what I did not know was that Belgrade was more than 500 KM away from Ljubljana. We would do the trip with a bus. I had visions of a small executive bus with leather fauteuils and a lovely stewardess serving us coffee and snacks. Not too bad I thought.

The next day we would leave at 2pm so that would at least gave me some time to get some work done in the morning, which I decided to do in the bar, enjoying some latte. However, this plan was disrupted by Dalibor Topic (Sun) but he made more than up for that by providing a more than worthwhile discussion about JSR 294, the meaning of life, OSGi, and Sun. It is always nice to talk to Dalibor, whom I seem to meet on more conferences than pure chance would predict.

At 2pm we gathered in the lobby and we took a taxi to the Parsek (the conference sponsor), where the bus would leave. There my visions of spacious leather seats were rather rudely shattered. Even Michael O'Leary would've blushed about the available leg space. However, the lack of space turned out not to really matter that much because the trip went amazingly fast. In the bus I was sitting next to Manik Surtani (Redhat) and we had a long and intense discussion about lots of very interesting subjects. Manik was interested in OSGyifing his Infinispan (a data grid) project so I might have made another OSGi convert. Hey, I am the evangelist!

Though Tomaž Cerar had forgotten to bring the promised bottle of whiskey, the hosts kept us well entertained through the evening driving through Croatia. We were told scary stories about the Serbians specifically and Balkan politics in general, creating a nice suspense. The eerie mood was amplified by farmers burning their land all along the highway, supervised by a slowly rising large blood-red moon. The crossing of the Kroation-Serbian border was therefore almost a disappointment when it only took 10 minutes by civilized officers instead of the 2.5 hours and brutal interrogations we were promised. Though the very modern (well in 1971) 5-star (Serbian stars that is) hotel turned a bit scary when they insisted to take my passport and then pushed it through a hole in the wall. Fortunately, I got it back half an hour later, though it took some angry looks.

The next morning we had to travel through the middle of Belgrade to the conference hotel. I still remembered the scary stories of the night before, describing Serbian drivers and Belgrade traffic. Especially the queues over bridges were told to be notorious. Now, it was kind of important to me that we started in time because the plan was that I would be the first speaker, then hop in a taxi, and hopefully catch my flight back from Tesla Airport. After all the horror stories leaving at 7.30 sounded, well, not unimportant. However, the relaxed nature that seems to be part of of all ex-Yugoslavians made us leave well after 8. Traffic was not too bad but I am pretty sure we blatantly violated the rules of the bridges of Königsberg by crossing the same bridge multiple times. And I am sure it was the same Samsung building I saw after about 20 minutes. Thinking about my taxi ride, it was also a bit disconcerting that we seemed to move farther and farther away from the airport. We did arrive before nine, well just. Any hopes to also start at nine were shattered by two introductory speakers that took their time, and more. Then, half way through my presentation, I was told to stop because the taxi was waiting. I felt that I needed at least another 10 minutes to finish and take some questions, only to discover the taxi had disappeared when I came down! Some nerve wrecking 15 minutes later, and several organizers talking in their phones (still relaxed!), an also very relaxed taxi driver showed up. And actually they were right, the drive to the airport was a non-event, though I swear I have seen cars on those roads that I had last seen when I was 5.

Thanks Aleš for inviting me and all the others that made this a really trip wonderful!

Peter Kriens

Enterprise Expert Group

I am currently sitting in the Kempinski hotel on the Munich airport waiting for my flight home. Very tired but even more satisfied. In the last few weeks we have been struggling in the Enterprise Expert Group about core issues. The last meeting, 8 weeks ago in Dublin, was actually quite heated and a bit depressive, but this meeting turned out highly productive!

The key issue was the focus of the work we're doing in the JEE APIs. Is the focus to make JEE applications run on OSGi unchanged or do we provide a recommended way of using those APIs on OSGi? The problem is that JEE has many built-in assumptions that do not hold in an OSGi world. JEE containers have a application-thread association, a much simpler class loading model with most dependencies contained in the application, and there is no visible life cycle. Especially the creation of providers through static factories is very cumbersome in OSGi.

A good example to highlight these problems is for example the static nature of JNDI. An application performs a new InitialContext() call and expects that a proper context is returned. The environment can influence the returned context via singleton that can only be set once.

The proposal on the table is that an OSGi bundle sets this singleton and then, when called, would get a Context service from the service registry. The real proposal is a bit more complicated, but this is the principle. At first sight, this sound like a viable solution. It allows bundles deliver new Context providers, and it is compatible with normal JSE/JEE code that uses new InitialContext().

Unfortunately, this static factory pattern ignores the dynamic life cycle of bundles. It raises the following concerns:

• Ordering - When the application does new InitialContext(), is the singleton already set? Is the desired provider already registered?
• Ownership - On whom's behalf is the Context provider service checked out? It is notoriously hard to determine the caller's bundle context from the stack, if at all possible in a reliable way.
• Staleness - What happens when the Context provider is stopped? The InitialContext then has a reference to a stale object.

These solutions are solvable if one lowers the bar that we held so far. You can order bundles with start-level ordering and in most cases the stale reference will not crash the system.

The problem we now faced in the last weeks was that many, coming from the JEE world, felt that the portability aspect was more important than the issues raised by this solution. Others felt that though lowering the bar in a product might be acceptable, an OSGi specification should show the right way to do it in OSGi. They felt that expecting that a JSE/JEE application always needed some porting to make it work on OSGi. There seems to be no free lunch for modularity.

During this meeting it was clear that we reached a consensus. The design was changed to be service based. In principle, a bundle would set the singleton and then register a service, allowing bundles to properly synchronized with this singleton. It is then up to containers to implement the correct life cycle handling for OSGi challenged applications. Additionally, the spec will recommend not to use the JEE/JSE static factories but will contain a section that outlines the consequences when they are used, given a proper warning to the deployer.

It was kind of invigorating to see how we have been able to come to a consensus even though the debate was sometimes heated and messy. If we can continue in this spirit, then I think we will see some very interesting specifications coming out of this group in the coming years.

Peter Kriens

About Modularity

Despite common belief, splitting up an application into a number of bundles is not automatically introducing modularity. The almost magic benefits of modularity are caused by the particular decomposition. Many decompositions can actually increase complexity, only the right decompositions reduce complexity. This was perfectly demonstrated by the father of modularity David Parnas. In his seminal paper On the Criteria To Be Used in Decomposing Systems into Modules (PDF), written in 1972. Parnas takes a simple problem, a text indexer, and shows two different decompositions. He then added the requirement that the source text files could be very large, no longer fitting in memory. The first decomposition required changing each module, the second decomposition only had one affected module.

Parnas' paper clearly shows why creating the correct decomposition is so hard, it is about predicting the future. With this disclaimer, there are some general rules that apply. These rules are in their essence the same as we use daily in writing our object oriented code. The most important being information hiding. Information hiding works because if you do not know something, you cannot make false assumptions about it. Assumptions that could break your code when violated in runtime.

With modularity, we have the fantastic tool that the rules inside the module are different from the rules outside the module. As a module author we can control what we expose from the inside and what is hidden from the outside. We can make sure that outsiders cannot make assumptions about our internal workings because they do not have access to them. By minimizing our dependencies on other modules we can reduce our own assumptions of the outside world. The less we assume, the more resilient our module will be to changes in the outside world.

In OSGi we even quite far with this model. In the initial model, you could basically only export API, not implementation classes. Each bundle could only talk to other bundles through services. A bundle is not even allowed to make assumptions about its and other modules' life-cycle. Though for many this sounds extreme, it was put in place to minimize assumptions that are so easily broken.

But there is more to modularity than just robustness and resilience. The proper decomposition can also simplify by requiring much less code. In 1984 I was the designer for an object oriented window system used in a page makeup terminal, this was my largest object oriented design so far. This design process was accompanied with an uneasy feeling that I was not designing, I was only postponing. Every abstraction seems to just postpone the really hard problems. However, one day I realized that I was done without ever having felt that I solved those really hard problems, they seem to have disappeared. In small this effect is visible with recursive algorithms. With the right decomposition, you need very little code, with the wrong decomposition you wrestle with the special cases of start and ending. The right decomposition somehow removes the need for much code.

I do not have a good understanding where the complexity goes when you have a proper modular structure but I have seen it too many times to know it happens. The problem is that once you found a proper decomposition, it feels so natural that you can't understand why you did not see this immediately.

Modularity provides benefits when it is used in the proper way, splitting an application into parts is not guaranteed to give you these benefits. If the cohesion is low and the coupling high between modules it is likely to give you more pain than gain. But there are more ways you can kill the benefits. If you break through abstractions, you likely will be broken when the implementation behind those abstractions change. Unfortunately, there are patterns in Java, mainly class loading related, that are fundamentally not modular and can easily kill the benefits of modularity.

However, if you refactor that application and you hit that right composition of modules, I can ensure you that you will immediately know why it is worth it.

Peter Kriens

A Simple Module System

To progress the work in JSR 294, BJ Hargrave, Richard Hall and undersigned have submitted a proposal for a simple module system. The purpose of this module system is to simple enough that can be supported both by Jigsaw and OSGi. In reality, this will likely be sufficient for most people. More advanced features, for example some of the special requirements that Jigsaw has with regard to the JDK modularization, can then be supported in a module system specific way.

This proposal is based on the module accessibility keyword, as is the current proposal, and there are no additional visibility rules. Module boundaries are defined by the artifact in which the modules are delivered. Modules can require other modules that must then become completely visible to the requiree.

The approach of finding a common denominator looks very promising. I am looking forward to continue working with the JSR 294 EG to iron out all the details.

Peter Kriens

No Pain No Gain

Being in the spotlight for OSGi has many good sides but it always pains when you get heavily criticized for a presentation about OSGi & Java Modularity that you gave. However, you usually learn more from criticism than from praise. Lets see if we can digest the criticism in William's blog and learn something from it.

From the first part I can see the criticism that OSGi is old technology and we keep (re)inventing component models. I guess there is a fine line between old and mature but it seems a tad unfair to hold age, without any further qualifications, against us. On the contrary, for stability, robustness, and usability a proven technology seems hard to beat. Then again, a promise is always more luring than hard reality I guess.

With the criticism that OSGi has many component models I cannot but disagree. OSGi has a single component model, since day one, and nothing has changed since then. True, on top of the OSGi component models we have different programming models because the needs in that area are quite diverse. However, all these programming models work seamlessly together through the single and only OSGi bundle and service model.

So what's left. In the end, William Louth makes a valid and interesting point: I said in my presentation that going modular is not without pain, class loaders are the major culprits in Java. However, I do not think this is a problem with OSGi, it is a fundamental problem when you modularize applications. Modules cannot see every class in the application, this global view is the antithesis of modularity. However, almost all class loader tricks I see that cause problems require this global view of the class path. Without addressing this problem, I do not think we can reap the fruits of going modular. How can Jigsaw will be able to solve these problems without destroying the benefits of modularity? Moving us from class path hell into module path hell.

The problem we're facing is that class loaders have become a very successful way to extend applications, something they were never designed for. In OSGi, we started with an extension model that makes the coupling between the modules very explicit: services. The class loading architecture was designed to support these services. The OSGi service model allows modules (bundles) to be very self contained and, ideally, only to export packages that are used to define the service.

So the big problem the OSGi faces is how to handle legacy code that heavily depends on class loaders for what OSGi does with services.

I spent considerable time thinking about this. Together with others, we created an RFP investigating all problems we could lay our hands on. My personal conclusion is that an application that requires a global unrestricted view of the class path cannot be modular. Yes, there are hacks that allow you to find the class your class loader need, but these hacks revert to class searching which implies that valuable properties like version constraints and class space constraints are no longer taken into account. My fear is that providing easy "solutions" to these practices will destroy the modular value that OSGi provides. I have found that these visibility problems are very fundamental. Time will tell if Sun is able to solve this problem with their "... much improved replacement."

Peter Kriens