Scala Components vs OSGi

This week I got a question how Scala components measured up to OSGi components. Now I do not have a good understanding of Scala components and I could only find a 5 year old paper of Odersky and Zenger. From this limited understanding, Scala components seem to provide a compile time construction model. The model shields a developer of the low level details how classes interact. Such a technique can significantly increase productivity because it allows us to understand more complex systems. However, though the developer is shielded from many details, the underlying code is still the usual tangled web of classes referring directly to many other classes.

Modularity has an intriguing relation with scale. Over time we've used the same trick (modularity) to provide us with new design primitives that allows us to better understand bigger systems. We've come from digital switches, octal numbers, hex numbers, assembly instructions, macros, functions, objects, and functional objects. Each step provided us with a higher level design primitive that simplified thinking about our software problems. Each step made us significantly more productive.

However, time does not stand still and the systems we build are becoming bigger and bigger. Our generation's software problem is that today we more integrate the software from many different parties than really write code. In the eighties I've rolled my own distributed database, window system, network software, text editor etc. Today, you download big functional blobs and spent most of your time figuring out how they can be configured to work in your system. having 300 to 400 JARs on your class-path is no longer an exception and I've heard about systems with 800 JARs.

The problem with the compiler component model is that the programmer's compiler has to have full visibility to every transitive dependency. Though some people feel this is the holy grail, I fail to see how this in the long run can match the reality; systems are just getting too big. This is proven by looking closer at popular open source software. You'll see the developers struggling with creating plugin and extension architectures while heavily abusing class loaders along the way. Factories and Listener patterns are desperately applied to make the code look decoupled, but in reality only hiding the coupling from sight and not actually removing it.

I think we're at a phase in the software industry where we have to let go a bit of total control. There are just not enough programmers in the world to verify all possible combinations of 800 JAR files. We have to accept that certain dependencies cannot be verified during compile time but will have to be made during deployment time, obviously with as much verification as possible. The holy grail of software is mix and match of components. You run a program and miss a feature, well add it. You don't like a feature? Get rid of it. We no longer can have the rigid coupling caused by compile time components when a system scales to a certain level.

This is not a new idea and many engineers of my generation sigh: "Oh gosh, yet another component model. Been there, done that. Doesn't work." Strengthened by being correct many times in the past. However, I think that OSGi brings a big difference to the table that few people truly understand yet.

Our industry has reached a system scale where our previous modularity solutions are becoming part of the problem. With 800 JARs, the modules can no longer be treated as a compile time abstraction. The modules need to be reified in runtime to allow the automation of their management; 800 is just way beyond the human scale. For this reason, module-to-module dependencies are implemented in Ivy and Maven and most package managers. Though this model was a big step, it does not reduce the complexity inside the modules, it only manages that complexity in a more automated way. What we need is a solution that addresses the complexity inside our software. That complexity is largely caused by coupling between different parts of our system; reducing that coupling will simplify our systems.

Modules depending on modules is a model that increases the dependency fan-out exponentially. For example, if I use library A, and library A uses a logging library, do I then depend on the logging library? Not really, that dependency is an implementation dependency of library A, it is completely unnecessary for the API dependency I am having on library A. The dependency fan-out of module dependencies tends to be extremely large and becoming cumbersome.

The key innovation OSGi provides is its µservice architecture. A µservice is not a component model; µservices enable a true component model by reifying the coupling between components. Instead of components depending on components, components now depend on µservices. This model is based on interface based programming. Back to Scala, the service package in OSGi µservices is very much like a component in scala. The key difference is that OSGi µservices do not ignore time, and this makes all the difference in the world.

Time is an aspect that we desperately try to ignore in programming languages. We can put a program to sleep and 24 hours later continue where we left off while your program is oblivious of its long absence, well until it finds out the real world actually had changed. In nature time is crucial for almost any process but handling time is hard, just think of concurrency and date handling. Few people realize how fundamental the timing aspect is of factories, dependency injection, and listeners. One component has control and it needs to interact another component, just think how crucial timing is for these cases. Still, we tend to create solutions that mostly ignore time except when we absolutely cannot. How often can you get a factory but the factory has no way to signal it can no longer provide its functionality? The effect of ignoring time is that our code is often surprisingly brittle because it implicitly makes many assumptions that are too easily violated.

µServices do not ignore time, they embrace it. By embracing time components become robust because they cannot make assumptions about the components they interact with beyond the µservice. Though many non-OSGi users think the dynamics make OSGi applications brittle, it is my experience that µservice based designs are very robust instead. It is the lack of assumptions about the other components that makes this so.

µServices neither replace nor diminish all the previous modularity techniques, on the contrary, µservices stands firmly on their shoulders. So I think Scala and OSGi are complementary. Scala does not provide what OSGi µservices provide and OSGi does not provide what scala provides. However, combined you they stand stronger and allows us to build bigger systems with less effort.

Peter Kriens

OSGi 2010 Community Event in London: Did you submit???

The deadline for submitting a presentation to the OSGi 2010 Community is rapidly approaching. Did you already submit your presentation? Take a look at for rules and inspiration. The OSGi 2010 Community Event will be held in London, September 29-30. The deadline is July 14, so if you have anything to say about OSGi or to the community then you better submit a proposal soon ...

Peter Kriens

OSGi 2010 Community Event

I've been so busy with the work for the next specifications that I've not spent much time on the next big OSGi event: The OSGi 2010 Community Event in London, September 29-30. And then I was skyped this morning that there is only two weeks left for submitting a presentation! The deadline is July 14, so if you have anything to say about OSGi to the community then you better submit a proposal soon.

I am looking forward to this event because there is so much happening in the enterprise space, and not to forget the residential space. For the next core release we are preparing a massive number of APIs that should be (almost?) stabilized at that time. But there is also much work going on in the Residential EG and the Enterprise EG. We are working on a massive set of specifications: generics for the framework, standardized shell, byte code weaving, coordinator, thread based standard io, object conversion, object formatting, simplified and more powerful handling of configuration data, classpath scanning, parametrized services (a service factory model), blueprint update that provides configuration admin API as well as as name-spaces, generic capabilities and requirements model, nested frameworks, virtual bundles, and then some tuning of existing APIs. It is almost scary so much is in the pipeline.

So I am pretty sure there will be lots of news and exciting stuff to talk about in September, both from the Alliance and its members. However, we are also really focused on experience reports. Any technology requires a solid foundation in the community and experience reports provide feedback for the directions that the OSGi should take. So please do not hesitate to submit your experiences or (wild?) ideas. From technology to business models.

You can submit at our submission system.

Peter Kriens