Real Men Don't Use DS

In general watching Stackoverflow is a pleasant way to spent some spare time. Helping other people out feels good and it is nice to see how the number of OSGi questions, and thus adoption, is increasing. However, it can also hurt looking at how people are struggling because of their own choices. There is this idea in our industry that real men start with the command line and Bundle Activators. Bare metal! Somehow this is considered to be a better way for real men to learn a technology than using a sissy IDE. Just like real men do not learn how to drive a car until they can clean and tune the carburator! The fact that  carburators have been absent in cars since the late seventies seems irrelevant to them.
I've written forewords for numerous books about OSGi and virtually all made the same mistake that to understand the technology you should first proof your worth by struggling with a Bundle Activator and a Service Tracker. Just like cars today have electronic injection so does OSGi have dependency injection. Use it. My strong recommendation is that if you want to learn OSGi, use bndtools with a Declarative Service (DS) example project. Make sure you understand the life cycle of declarative services and their immensely powerful integration with Configuration Admin. Realize that any class can be made a service with a simple annotation and any service can be made a dependency with a another annotation. If you need something to initialize, add an activate or start method to your service. Check your imports in the content pane and adjust imports them with drag and drop to make them cohesive and simple to use modules. Visualize what and how is running with Apache Felix Webconsole and Xray. Enjoy the sub-second edit-debug cycle. That is what you should learn and fall in love with because that is what makes OSGi so powerful.
Only after your mastered this initial level start looking under the hood and find that all the goodies that real men need are really there: Continuous integration, command line stuff, low level service access, weaving, proxying, bundle activators, bundle trackers, service trackers, whatever. It is wonderful to know it is all there when you need it but it is not what OSGi is all about.

The Trouble with Objects

When I discovered objects in the early '80s my mind was blown away. This was a technology that felt right, it gave me a way to think about my software that I utterly missed in structured programming. My career in the 90's centered around helping companies getting started with objects. Objects had clearly won, it was a run race. Lately, however, I have been getting more and more doubts.

The unique selling points of objects is that the data can change while the behavior remains the same (or at least backward compatible) via its public interface. If the world is the process then this is a great model. However, with 10Gb ethernets and larger, larger problems, cheaper computers, the world of a program is no longer a process: You are building distributed systems.

Distributed systems exchange information. In Java an attempt was made to preserve the object oriented semantics by not only communicating the data, but also sending the classes. However, ensuring that each system would use the correct class version and maintaing security turned quickly into a quagmire except for the simplest of systems. And even if it does work, it excludes any non-Java participant from collaborating. It is not that the OO community did not get any warning signals. The object oriented impedance mismatch with relational databases shows, at least in retrospect, that hiding your data while persisting it is not an option. At the time, we thought this was a problem that would solve itself when object oriented databases were ready (any day now!). Alas, they largely ran into the same problem that ensuring that the classes that read/wrote the data were of the same or compatible version was hard, if not impossible in many cases.

Looking at Java (and non-Java) persistence solutions I was ran into JOOQ, a library that declares peace between Java and SQL. Instead of abstracting it, it provides a type safe, fluid builder, way to use relational databases. It gave up on abstracting the database but it seems to have won a lot in simplicity and exploiting the powerful relational model. I cannot but wonder if all those abstractions to make the life of the application developer easier are in the end worth the increased complexity. Would it be worth to create a small, pet clinic like, OSGi application that would use all these different persistence models?

Persistence

My assignment for the OSGi Alliance is to increase adoption by making it easier to get started with OSGi. So I am currently writing a Request For Proposal (RFP), the standards requirements document in the OSGi Alliance. One of the primary parts is the Application Domain. In this section you neutrally describe the current practices, it is basically used to scope the problem domain area and provide a vocabulary for the subsequent sections and documents.
So last week I started the section persistence. It is an area that I have rather little experience with so I welcomed the chance to work with them during my sabbatical. I picked the document oriented database Mongodb because it felt much easier to work with in an object oriented environment than relational databases, and I must admit that choice has made me quite happy (except for the lack of transactions). However, it is clear that relational databases are the bread and butter of web applications. I therefore had to look deeper into what's happening in this area.
I then stumbled upon the debate around Java Data Objects (JDO) and Java Persistence Architecture (JPA). I had the privilege to work with Mike Keith (Oracle, JPA spec lead) on the OSGi JPA specifications so I knew something about JPA. However, so far I'd never seriously looked at JDO, and actually thought JPA was replacing JDO. Unfortunately, life seemed to be not so simple.
Both JDO and JPA define metadata to store normal Java objects in a database. JDO focuses on any type of persistent store (even S3 seems to be supported) while JPA is only used for relational databases. JDO seems to be a real working horse that got a new lift when Google selected it for its Google App engine. Though JPA is the new kid on the block its decision to limit itself to relational databases seems a severe limitation with the increasing popularity of NoSQL databases like Mongodb, Cassandra, Neo4j, etc. So far, it also looks like JDO is more portable and flexible while JPA seems faster. However, lots of experiences seem to come from toy projects or evaluation projects. Now lets not start a flamewar but I would love to hear (neutral) experiences of the use of these technologies in real life sized projects. Obviously I am extremely interested in how they work in OSGi.

Peter Kriens

The Architecture

Last year when I started with jpm4j I spent quite a bit of time exploring the state-of the art technologies. Even before that start point I had decided that the world had changed, most Java web frameworks looked very outdated with the advent of the upcoming HTML5. Now, with 55 years I guess I am an old geezer so I remember time sharing (punch cards, ASR/Teletypes, ending in intelligent terminals), the swing to the PC applications that gave us 'fat clients' in the eighties and early nineties, then the pendulum going back to 'thin' browser based clients from 1995 and onwards; and now the back swing to fat Javascript based clients. For many Java developers that started in the late nineties, no longer being a rebel but being treated as an incumbent will come as a nasty surprise since most Java web frameworks like JSP, JSF, Vaadin, Struts, Spring MVC, Wicket, and hundreds of others will be relegated to the dustbin of history over the next decade.

What changed? What has changed is HTML5 combined with Moore's Law. For the first time, there is a software platform that allows you to develop a single application code base that will run on virtually any user facing machine. From smartphones to high-end workstations. Though the rudiments have been available for a long time it was not until the browser vendors/organizations kicked w3c's lead and decided to go on their own in the WHATWG workgroup that we got a pretty decent, fairly wide, specification for web applications that was quickly supported by virtually all browsers. We are back running our application code on the client but this time we have no deployment problem nor having to worry about the client's Operating System.

This is of course a game changer for Java web frameworks. In a Java web framework the UI is managed in the server. This model requires a round-trip from the browser to the server when the GUI needs an update; a slow process that not only places a heavy demand on the server but it can also feel sluggish. With HTML5, the server can focus on the data and becomes oblivious of the GUI. The client worries about the graphics and interaction. The biggest advantage is scaling, no longer is the server required to maintain the often large state of the GUI; instead it provides a REST interface or JSON RPC interface . So not only is the footprint reduced, distributing the workload is even easier since the  protocols can easily be made stateless. Another advantage is simplicity, in my experience the server code becomes considerably smaller and therefore much simpler.

So why am I so excited about HTML5 since it seems irrelevant to OSGi? As  Emanual Rahm said: "Never let a serious crisis go to waste". I believe that HTML5 will be a disruptive change, causing a lot of web applications to be rewritten since customers will demand the responsiveness of local applications. This offers an opportunity for OSGi; an environment that in almost all aspects seems to have been designed for this paradigm shift.

Peter Kriens

Distributed Eventing RFP 158 now available

It has been a while since the OSGi Remote Services specifications were released. The Remote Services and Remote Service Admin specifications were part of the first Enterprise OSGi release which is available from early 2010. The Remote Service specs focus on using the OSGi Service model to create and consume remotely accessible services. A number of Remote Service implementations have since been created. They use a variety of wire protocols, some of them based on industry standards such as SOAP/HTTP or JSON/REST which allows them to interact with clients written in other languages.

The Remote Services specifications focus however on synchronous interactions. While asynchronous models were indeed mentioned in the original Remote Service RFP, they were not addressed in the Remote Services specifications.

Supporting asynchronous distributed eventing in OSGi through a standard, technology independent API has been on of wishlist of many for quite some time. For certain distributed event based use-cases a people have successfully used the OSGi Event Admin service. While the Event Admin API can be connected to a remote distribution or eventing system, this only addresses a certain type of use case. Specifically, reliability and queue-based semantics are not supported in a 'Distributed Event Admin' solution. For more information on the evaluation of distributing the Event Admin Service, see Marc Schaaf's Master Thesis.

Work has now finally started on a more general Distributed Eventing RFP at OSGi. This RFP looks at introducing a general Distributed Eventing solution into OSGi. You can find RFP 158 in the OSGi buzilla system here: https://www.osgi.org/bugzilla/show_bug.cgi?id=168
I you have any feedback, just leave a comment.

Oh, and just a reminder that this is an RFP, which is a requirements document. No design is yet captured in this document. The design will be part of the RFC work that is started after the RFP is done.

I am back ...

Since last week, I am back, working for the OSGi Alliance. To some this may come as a surprise but talking to people it seems that many actually had expected this.

So what happened? In the past year I've developed a repository (https://www.jpm4j.org) from scratch. Partly because I think such a repo is absolutely necessary, partly because I felt I needed to make an enterprise like app so I knew better what I was talking about. Both succeeded; What did not work out as hoped was commercializing it. I guess business aspects have never been my strongest point :-( and there is only so much I could spent on this sabbatical.

However, this was not the only aspect that made me return. Though I wanted to keep a distance of OSGi in the beginning I developed jpm with OSGi in the way that I've always have advocated: Services first. The good news was that to my great relief, this actually worked much better than I told people. I am now more convinced than ever that the OSGi service model is a true innovation that is highly undervalued in the industry. The bad news was that it required the development of too many basic components just to get started. Though there are an amazing number of libraries and bundles out there they often look awkward  because they do not leverage the unique advantages of OSGi: services + configuration admin. For me it became clear that the threshold to start with OSGi for a real web application was way too high.

After talking to some OSGi Alliance people we proposed to create a 'starter-kit' that would enable developers to start on a much higher level than a servlet. It should provide a foundation like a Ruby-on-Rails but now based on the much more scalable OSGi technology. On top of this foundation we plan to add a real application, demonstrating all the OSGi's best practices. This was proposed to the board and they actually wanted me to start weeks earlier than I had planned. So here I am!

I will therefore  be working on a foundation for OSGi web applications. If you have ideas, let me know. I am sure to keep you posted via this blog and twitter.

Peter Kriens

RFP 154 Network Interface Information Service now publicly available

Java standard APIs (i.e. java.net.NetworkInterface, java.net.InetAddress)
provide functions that allow IP network interface information, such as the IP
address and MAC address to be obtained.

However, the bundle that wants to get network interface information has to
monitor whether the information has changed or not for a certain period of
time. Changes in network interface can be pushed to the bundles concerned, the
need for polling by bundles can be eliminated. In addition, some information cannot be obtained via Java standard APIs.

This RFP describes the need for a mechanism that notifies concerned parties of
changes in the network interface and a new API that provides information not
obtainable from the standard Java APIs as well as the corresponding
requirements.

The RFP is now publicly available and we invite you to send us your comments and questions before we finalize it and start working on the RFC. The RFP can be found at: RFP 154