OSGi DevCon 2011 Submissions

I know you always like to submit your talks for OSGi DevCon/EclipseCon 2 minutes before the deadline, but it helps us all if you could submit them earlier! The deadline is November 30, 2010. So you have a few days left.

We already have a few submissions but we're really looking for more. There is so much happening with OSGi at the moment so there should be plenty of material. Are you using OSGi? Are you making tools? Are you considering OSGi? Is there something you do not like about OSGi? You think µservices are the greatest thing since sliced bread? You have the greatest idea ever for OSGi?

Really, if you use OSGi in a day to day situation you likely have something interesting to say to your peers. You got something to say but need a bit of help? Do not hesitate to ask me for some assistance.

• Submission
• OSGi DevCon information
  
• EclipseCon
  

Do not forget to mark your submission as an OSGi DevCon talk. Looking forward to see your submission, kind regards,

Peter Kriens

The History of Modularity

A few years ago Bjorn Freeman-Benson (formerly from Eclipse Foundation) asked me as an old friend (we know each other from the previous century) if OSGi was really novel, like Objects were, or if it was just good engineering. I then answered it was good engineering. Bjorn, I changed my mind lately. I think the OSGi µservices are an innovation on the same level as objects were!

So when I was was asked to give a presentation in Ludwigsburg for the User Forum Germany last week I felt µservices were a suitable subject. In my opinion µservices are the next big thing after objects, a progression in a long line of software improvements. This historic perspective is important to learn the lessons of our predecessors, what worked and what did not work. Personally I am fascinated how you see that in the last 60 years our predecessors modularized the existing model and moved software to the next plane, and we did this several times in a very similar, but different way.

Anyway, the presentation is available on slideshare. It is very much a Zen presentation (not much text) but lots of nice graphics. Enjoy and let me know.

Peter Kriens

JCP In Trouble

It seems that some in the Java community are becoming impatience with Oracle's handling of the JCP. Doug Lea wrote an open letter why he did not want another term on the JCP and Stephen Colebourne, a JCP specification leader and Apache Software Foundation member, is rallying against the JCP. Obviously the lawsuit against Google and the change of heart concerning the Apache Harmony field of use restriction did not win Oracle many new friends. Neil Bartlett even blogs about OSGi taking over the role of the JCP for the Java Community ... LOL

Hmm ...

If you think about, the OSGi is the only other organization that makes specifications for the Java community. And in many ways the OSGi Alliance has an arguably much better structure for Java specifications than JCP ever had:

1. Copyrights - In the JCP the specification lead together with Sun (now Oracle) have a special status. They're not even a bit more equal than others, they are on a whole different plane concerning ownership of the created Intellectual Property (IP). The contributions of other members in the expert groups are gladly taken for free but do not provide any rights in return.
   The OSGi is different. A member is a member, regardless if this member has 100.000 employees or 3. All created IP is licensed forever to all members under exactly the same conditions. Additionally, the specifications are completely free to implement by non-members. Simple, and more important, it is fair.
2. Patents -The OSGi Alliance and some of its key members have pledged not to assert any patents that are necessary to implement the specifications. During the specification process great care is taken that no submarine patents slip in.
   
3. Process - In the JCP the process is more or less in the hands of the specification leads. Having been a member of a number of JSRs I can assure you the process differs significantly. JSR 291 was run quick and efficiently, and I guess we all know what happened to JSR 277 and JSR 294, not to forget my 2 years in JSR 198, the most futile JSR of all.
   In the OSGi there is a much more standardized process that focuses on requirements first, API later. Over the past 30 years I've worked with oh so many developers developing APIs. It is a lot easier to get an agreement on requirements than on an implementation proposal. Therefore, the OSGi EGs always first work on an Request For Proposal (RFP); a document focused on requirements. Maybe those documents are not always perfect requirement documents but the process of making this requirements provides the opportunity for the EG members to create a common vocabulary. This collaborative mood helps tremendously when it is time to create the Request For Comments (RFC).
   
4. Reference Implementations (RI) - In the OSGi process a Reference Implementation is mandated before a specification can be published. For the previous releases the OSGi worked with several open source groups to provide the RI. However, the OSGi also has their own RIs when no group steps up.
   
5. Compatibility Tests (CT) - Every specification requires a CT that runs on the OSGi test harness. These CTs are free for the members and can be obtained free of cost for qualifying open source projects.
   
6. Architecture - The quality and architectural aspects of JSRs differs wildly because there is no central gating function. The running of a JSR is more less up to the specification leads and they can basically deliver what they want.
   In the OSGi Alliance all APIs have to go through the CPEG, which acts as an architecture board. Though this can be a bottleneck and a pain, it ensures consistency between specifications which is a big benefit for the users.
7. Quality Specifications - Again, in the JCP the specification is more or less what the specification lead wants to make of it. Some are excellent, other are a bit less. In contrast, all OSGi specifications are written as data sheets in the same template. Overall releases pack up a number specifications and bundle them together in a high quality book.

So what arguments could one think against the OSGi Alliance taking over the JCP role?

1. Opennes - Though the OSGi has been releasing RFCs ahead of the specifications the name lingers that the OSGi is very closed. I do think the OSGi Alliance should try to open up more but I also believe that the EG should have a period to get their act together before they have to publish. Specification writing is hard and agreeing within an EG is even harder. Anyway, so far there has been surprisingly little feedback on our released RFCs.
   
2. OSGi is Expensive - Today, the OSGi Alliance subscription fee is $25k or (€18k). They are currently trying to change our bylaws so they can actually have different tiers that will allow the subscription cost to go down for many companies. And they have invited researchers for qualifying academics. However, a bit of a reality check here. If you want to work on specifications it will cost you time and travel; both are highly expensive. If you want to do a good job, these costs dwarf the cost of the subscription fee. If the fee is the problem, should you really be wanting to work on these specifications anyway?
   The biggest advantage of the fees is that the organization is independent of the big guys and can thus be truly democratic for its members. With different tiers the big guys will pay significantly more but will require to become more equal. And was that not one of the problems with the JCP?
3. Not everyone uses the OSGi Framework - True, our specifications take the framework into account but most service specifications are written for pure POJOs. For architectural consistency it would be necessary to consider the framework in new specifications but that would not influence their usage outside a framework.
   In practice, this means ensuring the specifications are properly cohesive and uncoupled. For example, instead of putting the Factory and Provider classes inside the package, they would have to be kept in a separate packages, but that is also good modular practice in non-OSGi Java.

I am probably day dreaming to even consider for the OSGi Alliance to take over the JCP role for the Java community. But looking at the arguments it looks like Neil has a valid point ...

Peter Kriens

Masterclass Experiences

Last week Neil Bartlett and I gave a masterclass in Girona. We had a very good crowd: 10 very clever people. The first day was an introduction day but we learned the second day that we had to repeat some of the introductory material on the first masterclass day. The key reason was that even the most experienced users were not really aware how easy it is to use µservices with the bnd component annotations in bndtools.
The days were pretty intensive, going from 9 in the morning to deep in the evening. Though we stopped around 6 in the afternoon, we then tended to just move to the lobby and continue discussing and exploring. The lobby was the only place with Internet, though this sounds limiting, the effect was that we all had a good time together instead of sitting in our rooms working alone.


One of the first things we did is play with Vaadin. We used a little bundle I wrote some time ago that made it trivial to build a Web GUI with Declarative Services. We then continued with configuration. I discovered some really cool stuff just doing these exercises. Though I was the author of the Metatype specification, I never really used it since then, though I heard that the Apache Felix Webconsole supported it very well (as long as the Metatype Service is installed). A Metatype defines a type for a configuration record with sufficient information to automatically generate a decent UI for this type. It was kind of magic how a little bit of XML in the bundle suddenly allowed you to control the configuration of the component as well its number of instances. I knew it could be done, just did not realize how powerful it is.

To work on these examples we used Neil's Auction application he used for his tutorial. In this application, an Auction is (obviously) a service. Now would it not be cool to see the auctions of the others in the room? Seems like a job for Distributed OSGi! Before the masterclass Neil had investigated the existing open source implementations for the Remote Service Admin but they were either not up to the standard yet or not robust enough. So we had asked Paremus to provide us with a copy of their Remote Service Admin (RSA) implementation. They were still in active development so there was some reluctance from Paremus but they were fortunately brave enough and it paid off, RSA worked like a charm! In the end we all got each other's auctions on our screens. Distributed OSGi looks like a clear winner.

The next thing we tried was persistence. The last enterprise release provides a way to use JPA based on services. I'd never tried it so we decided to see how it works. I guess I should have prepared this part because it initially went badly. I mistakenly used the Eclipse Link bundles because they mentioned OSGi, however, it turned out that they were not at all related to the OSGi Enterprise specification. After we wasted too much time I got the brilliant idea (not really) to check how we tested the OSGi specification and then noticed that the Eclipse Link bundles we used there did not even remotely resemble what we were using. It turns out that all this work is in the Eclipse Gemini project and this is still in incubator status. Desperately after wasting so much time I used the bundles in the OSGi build. You can find now some information at the Eclipse Wiki but this desperately needs a good example. In the end we got it work thanks to Ralf Ebert who persevered.

Three attendants came from ACA-IT and used the week to work on a little Web based message aggregator based on the technologies we explored in the masterclass. They really wanted to use websockets (two way communication between browser and server) but these were only available in Jetty 7. Using parts from Felix and Equinox they actually got this to work. It was really nice to see how they leveraged the knowledge learned during the day to add this to their app during the night.

During the days we discussed an amazing amount of information. There was a lot of knowledge in the room and we could do some sparring about subjects like testing, versioning, builds, and many of the other things that occupy us as developers.

The last day each attendant had the change to submit a question to the rest of the group. This is the part I always like best because you learn about user problems. It was clear that also that day went by amazingly fast and provided a lot of food for thought. We invited all attendants to an exclusive mailing list for ''Masterclass on OSGi'' attendants. It is already working heavily.

Neil and I were very tired after this exhausting week but also very satisfied. It had been a lot of fun to do but we also learned a lot. We went out for dinner with my wife (accidentally ending up in an Irish pub with the for Neil oh so familiar English pub food) and evaluated the masterclass. As it looks now we are going to repeat it early next year on the west coast of the US. We'll inform you about the details.

Peter Kriens

Minimal OSGi Systems

Every now and then the question pops up about OSGi in really tiny environments. The press release from ProSyst last week about the smallest OSGi stack caused this question to enter my inbox: "Can OSGi be used in constrained embedded solutions?"

Though OSGi is a no-brainer, the underlying question of the use of Java turns out to be harder, even after a decade where memory prices tumbled and processor speeds exploded. In this market the Bill Of Material (BOM) still is the all overriding concern, combined with 99% experience in C(++) coding in this world. Java is usually seen as a sidekick. Nice to have but real men use C(++).

The first OSGi app I worked on was the Ericsson e-box, a home gateway with 32 Mb of RAM and 2 Mb of flash. I always wanted to get rid of the Linux on it and run Java on the bare metal so we would have a single management model. I actually wrote Java implementations of DNS, DHCP, and even wrote a Java implementation of the TCP/IP stack to be able to communicate with SLIP devices! However, others in the project were experienced Linux people and found it a lot easier to just do everything with existing software. Fighting about this architecture wasted a lot of time and in the end we ended up with an ugly hybrid that was hard to manage.

Last week, a presentation from Andre Bottaro (Orange Labs/France Telecom) and IS2T on the OSGi community event seems to indicate that things are moving again. These companies developed a minimum OSGi stack in a really small device by removing Linux and running the VM on bare metal. However, though these tiny OSGi implementations are interesting exercises I must admit that I have mixed feelings. OSGi is about downloading modules, such a model requires some headroom to have value. Why waste resources to a mechanism that can only be used for trivial downloads? Java in the phone did not become very useful until Android provided an SE like class library that was sufficiently rich and the hardware had sufficient headroom to make downloadable apps feasible.

That said, I do believe that in the end OSGi is the way to go because providing new and updated functions become more and more a necessity. And there are more activities around this. Also at the community event Tim Ellison (IBM) demonstrated Apache Harmony and how the modular approach of Harmony (it uses OSGi metadata for its modules) can be used to create really small and efficient VMs. I am going to look more in Harmony because it looks like an ideal VM to run OSGi on. I also spoke to Marcel Offermans (Luminis) who is submitting a proposal to Apache for a C(++) based runtime that is based on the principles of OSGi and will work together with OSGi through the service registry. This seems even more ambitious than the Universal OSGi idea that we played with in the OSGi Alliance a few years ago.

I do hope we do see more efforts shrinking the OSGi stack like ProSyst, Luminis, Orange Labs, IS2T and many others are doing. One day we will be at a place where OSGi is a no-brainer, in small devices, including the Java part.

Peter Kriens

JDK 7 Delayed Again

Did I get a laugh out of Mark Reinhold's announcement that Java 7 is further delayed? No, I did not; it actually made me rather sad. Java 7 contains a lot of improvements and my life would be simpler if I had those improvements available. Java is still a tremendous platform but we're losing our advantage over other platforms and the main reason is lack of a modularity.

Java is slowly being broken by the misconception that the bigger the Java platform the better it becomes. However, bigger means more internal dependencies and these dependencies tend to increase the complexity of a platform quickly. The more rope you have, the more tangled it can get. Repeatedly slipping schedules are one of the symptoms. In the nineties I've seen Ericsson waste staggering amounts of money to create a telecom application platform only to finally succumb to defeat. The simple fact is that developer needs are way too diverse to ever be captured by a single platform. Platforms that still try to satisfy these diverse needs always get torn between features and bloat. When I explained how I loved Java to my fellow Smalltalker Ralph Johnson (around '97) because it was so small, this wise man said: "Just wait." He was right. Worse, though the Java platform obtained the kitchen sink but still always requires me to include external libraries anyway.

Modularity is the only mechanism that allows us to have our cake and eat it too. If Java consisted of a core Java platform we could easily include the JSR implementations as separate modules. I never use Swing , so why does my platform have it? And maybe I need it in another version if I use it. Modularizing the JSRs would maybe complicate the life of trivial applications but for any real application handling external dependencies is already part of their complicated life. Complicated because surprisingly Java provides no support whatsoever for handling these dependencies.

Neither JSR 277, JSR 294, Jigsaw, nor Qwylt was a proper solution for handling external dependencies. Jigsaw was too simplistic for large applications and Qwylt and JSR 294 would have given us the headache of handling multiple module systems on your compiler module path. Both would have caused a packaging nightmare due to the combinatorial explosion of module system x operating system variations.

What we need is a core platform that is minimal but understands true modularity. Instead of including all these javax packages, this minimal Java should only include well defined core packages and also a mechanism to properly handle modules from (remote) repositories that make it trivial to add libraries and applications to my desktop machine and servers. Perl can do it, Ruby can do it, Python can do it, why does Java have such a hard time doing the same thing? Or live up to its promise and do it better?

Oracle should focus their Java effort in shrinking the platform and modularizing anything that is not used by most applications. Depending on the functionality of those modules, many of them can actually be compiled for 1.4 VMs (with generics), Java 5 or Java 6, making them available today and not until 2013 or even later. This decouples the life cycle of the components from the life cycle of the platform, which can never be faster than the slowest constituent component.

We do have the technology, all the parts are there. Please Oracle, this delay provides an opportunity, can we make Java agile again?

Peter Kriens

Easier Configuration Admin

We are likely to introduce a new service that makes Configuration Admin significantly easier to use. How? By using the Java type machine. Though properties are nice and flexible, they tend to cause cluttered code because of the required checks and conversions. The new model therefore defines configuration types with interfaces. For example, this is how a simple configuration type for a web service would look like:

package com.acme.webconf;
public interface WebConf {
 int port();
 String host();
}

The PID is defined by the interface name, in this example it is com.acme.webconf.WebConf. This "record" has a port (an integer) and a host name (a String). A bundle that wants to receive this information, only has to register a Configurable<WebConf> service. For example:

 @Component
public Webserver implements Configurable<WebConf> {
 public void setup(WebConf conf ) {
   web( conf.port(), conf.host() );
 }
 public void deleted(WebConf conf) {
   stop();
 }
}

No more type error prone type conversions, no more spelling errors in property names or PIDs! And you're not restricted to a single PID, you can extend the interfaces with other interfaces and each extended interface name will automatically be used as PID, merging all underlying configuration records. Annotations in this specification will also make data validation easier.

It is also possible to use factories. In that case, register a ConfigurableFactory, which looks like:

 public interface ConfigurableFactory<WebConf> {
  Configurable<WebConf> newInstance();
  void noConfigurations();
}

This specification will make Configuration Admin significantly easier to use.

Peter Kriens

P.S. Did you already register for the community event?
P.S. And did you look at the Masterclass On OSGi (with certificate) in Girona?