Tuesday, August 31, 2010

Easier Configuration Admin

We are likely to introduce a new service that makes Configuration Admin significantly easier to use. How? By using the Java type machine. Though properties are nice and flexible, they tend to cause cluttered code because of the required checks and conversions. The new model therefore defines configuration types with interfaces. For example, this is how a simple configuration type for a web service would look like:
package com.acme.webconf;
public interface WebConf {
int port();
String host();
}
The PID is defined by the interface name, in this example it is com.acme.webconf.WebConf. This "record" has a port (an integer) and a host name (a String). A bundle that wants to receive this information, only has to register a Configurable<WebConf> service. For example:
 @Component
public Webserver implements Configurable<WebConf> {
public void setup(WebConf conf ) {
web( conf.port(), conf.host() );
}
public void deleted(WebConf conf) {
stop();
}
}
No more type error prone type conversions, no more spelling errors in property names or PIDs! And you're not restricted to a single PID, you can extend the interfaces with other interfaces and each extended interface name will automatically be used as PID, merging all underlying configuration records. Annotations in this specification will also make data validation easier.

It is also possible to use factories. In that case, register a ConfigurableFactory, which looks like:
 public interface ConfigurableFactory<WebConf> {
Configurable<WebConf> newInstance();
void noConfigurations();
}
This specification will make Configuration Admin significantly easier to use.

Peter Kriens

P.S. Did you already register for the community event?
P.S. And did you look at the Masterclass On OSGi (with certificate) in Girona?

Thursday, August 19, 2010

OSGi Community Event Sept. 29-30 in London


Just a small reminder that you still have to register for the OSGi Community Event, September 29-30 in London! We really have a very strong program this year, there are lots of very interesting presentations. From the program it is clear that OSGi is maturing in many verticals of the software industry. From the Appstore for the connected home market to the enterprise application twins Apache Aries and and Eclipse Virgo, it is clear that OSGi provides a modularity solution that transcends the silos in our industry.

OSGi provides the solid foundation that is needed when applications grow in size, as applications always do. After a certain scale, strong modularity becomes a necessity to stay on top of the complexity. The community event is an excellent opportunity to learn a software technology that will become increasingly important in the next few years.

The program at the event is interesting for many different people. For beginners because there are tutorials for building enterprise applications, handling dependencies (the hallmark of any large software system), and bndtools (an Eclipse based development tool for OSGi bundles). For experts, because some of the presentations take a deep dive in the complexity of componentizing large systems, providing insights in the problems as well as providing guidance towards solving these problems. There will also be plenty of presentations providing guidance how to modularize your existing applications. And for business people because several presentations take a good look at the business cases for OSGi. The planned sessions are:
And of course as the moderator, I have to invite you to a panel: “I Love OSGi. Now Let’s Change It!” that will host some of the key people in our industry.

This event is targeted at people that have invested in OSGi and want to meet their peers, software architects that need to know more about OSGi, developers that are using OSGi, and business people looking for new opportunities in the software world. So register quickly.

I am looking forward to meet you in London next month!

Peter Kriens

Friday, August 13, 2010

OSGi Masterclass Oct. 12-15

Neil Bartlett and Peter Kriens (that being me) have decided to hold a masterclass together. Neil and I have been working for some time on different tools and we felt that though there are are lots of introductory tutorials there are no places where you can learn in-depth about using (and designing with) OSGi. Together we have quite a bit of experience in the area and we can cover quite a lot of ground.

We intend to run this masterclass very hands on. With the group we will analyze use cases (your use cases are welcome), design a solution, and create prototypes. We intend to cover a lot of different areas: service based design, distributed OSGi, persistence, GUIs, etc. As tool we will be using Neil's bndtools. If you feel not completely sure about your basic OSGi knowledge, we have an introductory day to bring you up to speed.

To keep the cost down, we've organized this masterclass in a hotel near the Girona airport. Girona is very well connected to most European cities because it is a Ryanair and Easyjet hub. Also the price of the masterclass is kept low and the class is limited to 20 people. To top it off, there is also a discount when you register before September 15.

So, consider participating because I am sure about one thing: it will be an interesting masterclass!

Wednesday, July 21, 2010

Scala Components vs OSGi

This week I got a question how Scala components measured up to OSGi components. Now I do not have a good understanding of Scala components and I could only find a 5 year old paper of Odersky and Zenger. From this limited understanding, Scala components seem to provide a compile time construction model. The model shields a developer of the low level details how classes interact. Such a technique can significantly increase productivity because it allows us to understand more complex systems. However, though the developer is shielded from many details, the underlying code is still the usual tangled web of classes referring directly to many other classes.

Modularity has an intriguing relation with scale. Over time we've used the same trick (modularity) to provide us with new design primitives that allows us to better understand bigger systems. We've come from digital switches, octal numbers, hex numbers, assembly instructions, macros, functions, objects, and functional objects. Each step provided us with a higher level design primitive that simplified thinking about our software problems. Each step made us significantly more productive.

However, time does not stand still and the systems we build are becoming bigger and bigger. Our generation's software problem is that today we more integrate the software from many different parties than really write code. In the eighties I've rolled my own distributed database, window system, network software, text editor etc. Today, you download big functional blobs and spent most of your time figuring out how they can be configured to work in your system. having 300 to 400 JARs on your class-path is no longer an exception and I've heard about systems with 800 JARs.

The problem with the compiler component model is that the programmer's compiler has to have full visibility to every transitive dependency. Though some people feel this is the holy grail, I fail to see how this in the long run can match the reality; systems are just getting too big. This is proven by looking closer at popular open source software. You'll see the developers struggling with creating plugin and extension architectures while heavily abusing class loaders along the way. Factories and Listener patterns are desperately applied to make the code look decoupled, but in reality only hiding the coupling from sight and not actually removing it.

I think we're at a phase in the software industry where we have to let go a bit of total control. There are just not enough programmers in the world to verify all possible combinations of 800 JAR files. We have to accept that certain dependencies cannot be verified during compile time but will have to be made during deployment time, obviously with as much verification as possible. The holy grail of software is mix and match of components. You run a program and miss a feature, well add it. You don't like a feature? Get rid of it. We no longer can have the rigid coupling caused by compile time components when a system scales to a certain level.

This is not a new idea and many engineers of my generation sigh: "Oh gosh, yet another component model. Been there, done that. Doesn't work." Strengthened by being correct many times in the past. However, I think that OSGi brings a big difference to the table that few people truly understand yet.

Our industry has reached a system scale where our previous modularity solutions are becoming part of the problem. With 800 JARs, the modules can no longer be treated as a compile time abstraction. The modules need to be reified in runtime to allow the automation of their management; 800 is just way beyond the human scale. For this reason, module-to-module dependencies are implemented in Ivy and Maven and most package managers. Though this model was a big step, it does not reduce the complexity inside the modules, it only manages that complexity in a more automated way. What we need is a solution that addresses the complexity inside our software. That complexity is largely caused by coupling between different parts of our system; reducing that coupling will simplify our systems.

Modules depending on modules is a model that increases the dependency fan-out exponentially. For example, if I use library A, and library A uses a logging library, do I then depend on the logging library? Not really, that dependency is an implementation dependency of library A, it is completely unnecessary for the API dependency I am having on library A. The dependency fan-out of module dependencies tends to be extremely large and becoming cumbersome.

The key innovation OSGi provides is its µservice architecture. A µservice is not a component model; µservices enable a true component model by reifying the coupling between components. Instead of components depending on components, components now depend on µservices. This model is based on interface based programming. Back to Scala, the service package in OSGi µservices is very much like a component in scala. The key difference is that OSGi µservices do not ignore time, and this makes all the difference in the world.

Time is an aspect that we desperately try to ignore in programming languages. We can put a program to sleep and 24 hours later continue where we left off while your program is oblivious of its long absence, well until it finds out the real world actually had changed. In nature time is crucial for almost any process but handling time is hard, just think of concurrency and date handling. Few people realize how fundamental the timing aspect is of factories, dependency injection, and listeners. One component has control and it needs to interact another component, just think how crucial timing is for these cases. Still, we tend to create solutions that mostly ignore time except when we absolutely cannot. How often can you get a factory but the factory has no way to signal it can no longer provide its functionality? The effect of ignoring time is that our code is often surprisingly brittle because it implicitly makes many assumptions that are too easily violated.

µServices do not ignore time, they embrace it. By embracing time components become robust because they cannot make assumptions about the components they interact with beyond the µservice. Though many non-OSGi users think the dynamics make OSGi applications brittle, it is my experience that µservice based designs are very robust instead. It is the lack of assumptions about the other components that makes this so.

µServices neither replace nor diminish all the previous modularity techniques, on the contrary, µservices stands firmly on their shoulders. So I think Scala and OSGi are complementary. Scala does not provide what OSGi µservices provide and OSGi does not provide what scala provides. However, combined you they stand stronger and allows us to build bigger systems with less effort.

Peter Kriens

Tuesday, July 13, 2010

OSGi 2010 Community Event in London: Did you submit???

The deadline for submitting a presentation to the OSGi 2010 Community is rapidly approaching. Did you already submit your presentation? Take a look at for rules and inspiration. The OSGi 2010 Community Event will be held in London, September 29-30. The deadline is July 14, so if you have anything to say about OSGi or to the community then you better submit a proposal soon ...

Peter Kriens

Thursday, July 1, 2010

OSGi 2010 Community Event

I've been so busy with the work for the next specifications that I've not spent much time on the next big OSGi event: The OSGi 2010 Community Event in London, September 29-30. And then I was skyped this morning that there is only two weeks left for submitting a presentation! The deadline is July 14, so if you have anything to say about OSGi to the community then you better submit a proposal soon.

I am looking forward to this event because there is so much happening in the enterprise space, and not to forget the residential space. For the next core release we are preparing a massive number of APIs that should be (almost?) stabilized at that time. But there is also much work going on in the Residential EG and the Enterprise EG. We are working on a massive set of specifications: generics for the framework, standardized shell, byte code weaving, coordinator, thread based standard io, object conversion, object formatting, simplified and more powerful handling of configuration data, classpath scanning, parametrized services (a service factory model), blueprint update that provides configuration admin API as well as as name-spaces, generic capabilities and requirements model, nested frameworks, virtual bundles, and then some tuning of existing APIs. It is almost scary so much is in the pipeline.

So I am pretty sure there will be lots of news and exciting stuff to talk about in September, both from the Alliance and its members. However, we are also really focused on experience reports. Any technology requires a solid foundation in the community and experience reports provide feedback for the directions that the OSGi should take. So please do not hesitate to submit your experiences or (wild?) ideas. From technology to business models.

You can submit at our submission system.

Peter Kriens

Wednesday, June 30, 2010

Reified Types: Conversions Using Generic Types

One of the most interesting aspects of the Blueprint specification was the ReifiedType. I ran into this nifty class again today because we want create a Conversion service that is useful by the new upcoming OSGi shell service. We had decided to reuse the Blueprint conversion model, but now as a separate service, something which we probably should have done already for Blueprint.

The problem that Reified Type solved for us is how to convert an Object to a specific target type by taking any generic information into account. For example, if you convert an int[] to a Collection<Long> you want to be sure that the collection contains Long objects and not Integer objects. That is, a naive implementation would convert this to Collection<Integer>, which would miserably fail later on when someone uses the collection.

I can hear many readers thinking: "But generic information is erased, isn't it????" Yup, generics are erased, but not completely! When you have an object there is no way to know the generic parameters (the parameters between the angled brackets like T, K, V etc.). I am not sure why they decided to not provide this information with the object because it seems very doable but it clearly is not present. However, the VM maintains extensive generic information everywhere else but in the object. If you have an instance field then it can describe exactly what its generic type constraints are. For example, if you have a field numbers, declared like:

Map<String,Class<T>> numbers;

Then you can get the generic type constraints with the getGenericType method that returns a Type. Virtually all reflective methods have been duplicated to provide this Type object where they used to return a Class object.

OSGi APIs are written in a restrictive subset of Java so that they can be used in a wide variety of VMs. Though we found a solution allowing us to use generics in Java 1.4 (JSR 14), we must assume a target of Java 1.4 with the minimum execution environment for our daily work. This means there is no Type class we can rely on in our APIs.

Obviously, in enterprise scenarios the VM is Java 5 or later, and the Type class is readily available. Is there a way we can eat our cake (use the generics info in our Conversion service) and have it too (no dependency on Java 5)?

The first solution that comes to mind is to duplicate all the sub-types of the Type interface in our own name space. There are actually quite a few sub-types of the Type interface. These sub-types models (and more!) all the flexibility of the generics constraints system: type variables, arrays, wild-cards (super, extends), and parametrized types.

Hmm, not only is duplicating no fun, it also turned out that the hierarchy is not that easy to work with. For our Conversion service we only need to know the raw type of a type parameter. That is, if our target type is Collection<Long> than the we only need to know that the type parameter is Long.

After struggling with this problem we found an interesting solution: the ReifiedType class. A Reified Type collapses all the intermediate levels of wildcards, variables, and arrays and provides direct access to the most specific types that can be used. It turns out that by traversing the network of Type objects it is always possible to end in a raw class, even with variables, wildcards, and arrays.

A Reified Type is always associated with a single raw class. The raw class is the class without any generics information. It provides access to the type parameters by their position. For example, the Map<String, ? extends Number> has a raw class of Map and 2 type parameters: String and Number. For this example, the previous Refied Type has a size() of 2, and provides a ReifiedType<String> and a ReifiedType<Number> for the type parameters.

In the Blueprint specification we provide a concrete implementation for Java 1.4 VMs. In Java 1.4, there are no generics so this implementations hard codes the number of type parameters to 0 by definition. We will do the same in the Conversion service. However, users of this service will be able to pass subclasses that provide the generic information to get better conversions.

So it looks like this was a case where we could have our cake and eat it too! You can already look at Reified Type in the Blueprint specification and in the next release I hope we can have a new Conversion service based on this model.

Peter Kriens