Monday, July 6, 2009

No Pain No Gain

Being in the spotlight for OSGi has many good sides but it always pains when you get heavily criticized for a presentation about OSGi & Java Modularity that you gave. However, you usually learn more from criticism than from praise. Lets see if we can digest the criticism in William's blog and learn something from it.

From the first part I can see the criticism that OSGi is old technology and we keep (re)inventing component models. I guess there is a fine line between old and mature but it seems a tad unfair to hold age, without any further qualifications, against us. On the contrary, for stability, robustness, and usability a proven technology seems hard to beat. Then again, a promise is always more luring than hard reality I guess.

With the criticism that OSGi has many component models I cannot but disagree. OSGi has a single component model, since day one, and nothing has changed since then. True, on top of the OSGi component models we have different programming models because the needs in that area are quite diverse. However, all these programming models work seamlessly together through the single and only OSGi bundle and service model.

So what's left. In the end, William Louth makes a valid and interesting point: I said in my presentation that going modular is not without pain, class loaders are the major culprits in Java. However, I do not think this is a problem with OSGi, it is a fundamental problem when you modularize applications. Modules cannot see every class in the application, this global view is the antithesis of modularity. However, almost all class loader tricks I see that cause problems require this global view of the class path. Without addressing this problem, I do not think we can reap the fruits of going modular. How can Jigsaw will be able to solve these problems without destroying the benefits of modularity? Moving us from class path hell into module path hell.

The problem we're facing is that class loaders have become a very successful way to extend applications, something they were never designed for. In OSGi, we started with an extension model that makes the coupling between the modules very explicit: services. The class loading architecture was designed to support these services. The OSGi service model allows modules (bundles) to be very self contained and, ideally, only to export packages that are used to define the service.

So the big problem the OSGi faces is how to handle legacy code that heavily depends on class loaders for what OSGi does with services.

I spent considerable time thinking about this. Together with others, we created an RFP investigating all problems we could lay our hands on. My personal conclusion is that an application that requires a global unrestricted view of the class path cannot be modular. Yes, there are hacks that allow you to find the class your class loader need, but these hacks revert to class searching which implies that valuable properties like version constraints and class space constraints are no longer taken into account. My fear is that providing easy "solutions" to these practices will destroy the modular value that OSGi provides. I have found that these visibility problems are very fundamental. Time will tell if Sun is able to solve this problem with their "... much improved replacement."

Peter Kriens


  1. This particular attitude that only new things are cool and that all old things need to be replaced by new improved cool things is childish behavior manifested in adults. Most organizations investing in software are looking for lasting value, not for an endless stream or dubious change simply for the sake of change, which in the end becomes more like fashion than technology, i.e., a cyclic process designed to extract money from the gullible.

  2. This comment has been removed by the author.

  3. Tooling Tooling and again Tooling is much required for now to make real life Enterprise OSGi programming easier.

    I'd like to see OSGi solution not being such a pain to implement when you start to deal with more than a hundred of bundles. The overhead of extra work you need to maintain packages versioning, upgrade of bundles etc is stiil too much.

    I really hope things will improve quite soon with solutions like Spring DM server.

    I dream of a full blown out-of-the-box OSGi server providing me with logging, JAAS, JMS, XA JTA and remoting services.

    If OSGi solutions do not come quickly I'm afraid it will never gain widespread acceptancy.

  4. @Frederic: It is this dream or reusable components that got me started in the eighties on this subject and that still keeps me motivated.

    The problem is that this dream requires doing it right, and that unfortunately is incredibly hard. In the 60s, Fred Brooks already told us there is no silver bullet but people still hope for "simple" solutions that provide all the benefits. Alas, it takes time and requires many people to help by supporting this vision to turn it into reality. However, from the OSGi inside I see so many hopeful signs ...

    Peter Kriens

  5. I agree with your sentiment Peter. However, when we state something like "no pain no gain," we need a good story to go along with why this is the case... along with real world examples. So I tend to view this problem as an education thing. I think people are starting to grasp the benefits of modularity and reusable components but don't understand the complexities of legacy code. For example, demonstrating how popular libraries like Hibernate hijack your classpath and how that causes problems for a modular system like OSGi. I think that's the better approach...

    Otherwise, people will label you as putting chastity belts on classloaders ;)

  6. My main problem with OSGi is that it tries to do too many things and does not end up doing a very good job of all the things it does. Part of this is legacy and part of this is just bad design. OSGi was stretched from first mostly embedded software (running on J2ME CDC even) then to desktop (Eclipse) and now finally to the server.

    I've used it on top of CDC and aborted an attempt to use it server side for lack of maturity & support and a lack of any obvious benefits for dealing with all that.

    I now use Spring and dependency injection. This is one of the things that was until recently quite cumbersome in OSGi. All the boiler plate code needed just to get an instance to some service is quite cumbersome (and a likely source of bugs). Declarative services address some of the pain here of course but it is really just hiding the crap rather than making it go away. @Component and @Autowired serve me much more nicely when doing Spring. And of course the incorporation of OSGi in the Spring platform is nice. Spring hiding most of the OSGi framework would be progress for most of us.

    The two things that OSGi does well and that so far are not really provided elsewhere is enforcing provided and required interfaces and providing sandbox for each component to run in (through a clever classpath hack). Isolation and explicit dependencies are great to have in any large software system and it is going to be a core requirement in developing large distributed systems that are supposed to share resources in the cloud with other, untrusted components. However, I never liked that OSGi relies on crappy crappy manifest files here. It was the pragmatic choice when it was invented but it would be great if we could get some first class language support for modules and provided/required. Package just is not good enough here.

    Logging, messaging, etc. are not really things that belong in OSGi, I would say. There seems to be some kind of competition going on where each application framework eventually ends up reinventing all of the Corba services. OSGi is already pretty far down that path. Having IBM involved here probably does not help. But this is part of what is keeping it from being integrated with the Java platform. There's just too much cruft and redundant stuff in OSGi. Too much stuff that really is in need of some rethinking.

  7. @chris: It is education, it is tooling, it is actually having the problem ... I think OSGi solves a problem in the large, and if you make small apps monolithic is not too bad.

    @Jilles: pretty harsh criticism. I do not agree that OSGi tries to do all things. OSGi consists of 2 parts: core and compendium. The core is doing only one thing: providing an architecture where components collaborate. The compendium is purely optional and a convenience to get more collaboration. We are not inventing services, we're providing an API that can compete with all other APIs people want to put out there, increasing the chance that things are reusable.

    I am also a bit surprised about the architecture and maturity, can you elaborate? This is an argument I have not heard so far.

    About the manifest, well, it is years ago I wrote a manifest. Tools like bnd make this file disappear and take the drudgery out of writing the manifest by calculating many of its values. Complaining about the manifest is like complaining about writing the class file format, you should never have to do that.

  8. "About the manifest, well, it is years ago I wrote a manifest. Tools like bnd make this file disappear and take the drudgery out of writing the manifest by calculating many of its values. Complaining about the manifest is like complaining about writing the class file format, you should never have to do that."

    Underlying this position is a deep belief about what a component *is*.

    If you believe that the ultimate component is a service, then a bundle is a dull unit of packaging whose production should be automated with tools like bnd. Inferring dependencies from classfiles is natural in this context; it's akin to optional static typing a la Scala.

    What if you aren't interested in services? If you believe that the ultimate component is a logical module with an exported API on which other modules should depend, then explicit declarations of module relationships, written by a programmer, are natural; it's akin to mandatory static typing in Java. Inferring dependencies with tools is not natural in this context.

    This is the heart of the conceptual difference between OSGi and Jigsaw.

  9. This comment has been removed by the author.

  10. OSGI is Best!

    Luís Carlos Moreira
    Eclipse RAP, RCP, GMF and Pentaho Developer