Monday, July 14, 2014

The Big Ball of Mud

I am a tad frustrated about the whole µService discussion. How carefully designed concepts and terminology gets trampled over by a crowd that doesn't bother to look around in their rush to next year's fashion. When Neil Bartlett posted a link to an interesting blog that discusses exactly this subject I was therefore pleasantly surprised. It is about how our industry tends to swing from hype to hype and does not even try to learn any lessons from the down-swing. That is, if those lessons requires more than a minimal attention span. Ok, Simon Brown  does not appear to be an OSGi aficionado but he hits the nail on the head that your monolithic big ball of mud is not going to be a tender agile application overnight by adopting micro-web-services. Or, how a new fashion is always more attractive than the grunt work of making the existing stuff right. Hey, maybe they did find the silver bullet this time! #not

We actually know quite well, and for a long time, how to build large complex systems: contract based component designs scale very well. It just so happens that in practice the siren song of short cuts are much more attractive than doing it right ... that is, in the short run. We show beautiful layered architectures that are in reality camouflage nets over the actual big ball of muds. Read Simon's blog and weep.

Now, although I agree with the view in this blog, I think he misses the key reason why we always end up with that big ball of mud. Unfortunately, experience shows that this is not an easy story that fits the 140 character attention span, the cause of the big ball of mud does require some work to understand. Please shut off twitter and bear with me.

In the seventies, structured programming was the then current best practice. The mantra's in those times were 'Low Coupling' and 'High Cohesion'. Then, in the early eighties, we got Objects! New! Improved! No need to bother about these old geezers with their coupling and cohesion muck anymore. (I never really got that cohesion stuff anyway.) Progress, New! We were young and innocent until time taught us that the old geezers had probably learned a lessons or two that we had missed. Embarrassingly, the anticipated reuse we had promised to our managers were much harder to achieve than we had thought in our youthful exuberance. What killed us most were the transitive dependencies that are implicit in the object programming models. It is wonderful to reuse another object, but the damned thing should then not do the same thing and depend on others, ad nauseum. Instead of picking and choosing reusable objects, systems quickly became big balls of mud, or more appropriately, a different form of spaghetti coding. We had carefully avoided the GOTO statement but the result was painfully similar, just on another level.

In the nineties I had learned my lessons about coupling, cohesion, and objects. So when I met Java in 1996 I  immediately realized what interfaces were doing, they were addressing the transitive dependency problem that had caused me so much pain! With an interface you could program against a contract, not against an implementation! This did wonders for the transitive dependency problem since you were now bound to an interface. Interfaces were much less coupled by design than implementation classes. I obviously wasn't alone and interfaces became highly popular in Java and are pervasive today because they do provide freedoms that transitive dependencies kill.

Alas, we then witnessed a demonstration of how little intra-generational learning is going on. Two things went haywire. First, interfaces raised this pesky problem of 'How on earth do we get the instance in runtime?` Factories were kind of awkward so we found the greatest ostrich solution of all software times: 'We hide it in a XML!'. Obviously, the fact your coupling Java analysis tools did not see these hidden dependency anymore did not mean they stopped hurting you over time. An implementation class reference is a transitive dependency as much as any "new" expression is in your compiled code. Worse, a class referenced in text bypasses all those type safe-guards in Java you just spent all that money on.

The other, and surprisingly even worse, mistake was that we happily copied the transitive implementation dependencies model that failed so spectacularly with objects into our modules. And surprise, we got the same ball of mud, just now on a tad larger scale. Don't get me wrong, it is not maven that downloads the Internet, it is people that download the internet. People are addicted to solutions that look easy. That is, until that proverbial stuff hits the fan but then the original designers have then usually moved on.

Hard coding a dependency on another module will inevitably create more and bigger balls of mud, we learned this with structured programming, with object oriented programming, and now we are learning it with modular programming. The structural problem is identical in all these cases, so why should the result differ? (Though I find that a surprisingly large group of people disconnect now, 'What on earth has an interface to do with modules? This guy is an idiot ...'. Then again, their attention span probably already had made me loose them to that interesting tweet that just came in.)

Obviously, when we started designing OSGi in 1998 we did not have the clear view of today. However, the crew that did the early design had a lot of experience and provided a very elegant, µService based, model.  It solved the 'how on earth do I get my instance of that interface' problem in a non-ostrich way, it also innovatively used Java packages as the dependency model.  The Java package plays the same role for a module as the interface does for a class.

If you find this hard to swallow then you're not alone (which does not make you right btw, just more common). At JavaOne, I actually once overheard a discussion between three (at that time) Sun/Oracle high level executives;  they were making fun of package dependencies, radiating an amazing ignorance of the underlying issues. Even with very bright people, some that I admire a lot, I've seen that it takes effort to accept this structural analogy between classes and modules. However, even at the JCP, the Java package is already playing a large organizing role in most, if not all, specifications. It acts as the mediator between the provider (e.g. Jersey) of the specification (JAX-RS in packages) and the consumer of that specification.

The key consequence of the OSGi model is that the µservice is not an implementation (a module) as many people tend to talk about it, it is a contract. Modules can provide implementations for these service contracts and they can consume objects that implement these service contracts. The modularity model is thus not: module A depends on module B. No, module A depends on service S and module B and module C can provide such a service. It is this crucial indirection that I think Simon misses in his blog that is the only known answer against drowning in the big ball of mud over time. We figured this out with Java interfaces, we now only need to understand how we can eradicate the same transitive pest with modules as well.

Yes, I know the model is not as simplistic as the module A depends on module B model that we know so well; it adds an additional indirection that is not free. The difference is that you pay that cost up-front while the long term cost of module dependencies is much, much higher but drawn out over time.  Though we like to postpone costs, we actually did learn a lot of lessons over time that made us take some up-front cost, Java as a type safe language is the prime example. So I am utterly convinced that model we learned in OSGi will prevail over time. Just hope that I do not have live through another hype that presents (parts of) these ideas as if they are something new.

Peter Kriens


Monday, June 23, 2014

Software mixed with marketing: micro-services

A few months ago I was on the phone when the speaker on the other side mentioned micro-services. Many moons ago I'd written a blog about OSGi's µservices because people were getting confused with the term service. Though this name was unique enough in 1998, the advent of Service Oriented Architectures (SOA) had imprinted people to think about heavy weight, costly, and complex things requiring stacks and other things a developer wants to avoid. That is, everything OSGi is not. I even used the Mu symbol (µ) because it was also the symbol for friction, of which µservices have so little. I therefore kind of naturally assumed the guy was talking about our wonderful in-VM µservice oriented architecture (µsoa). Actually, I smugly told him that I coined the term, trying to impress him. This however turned the discussion from straightforward discussion about technologies into a rather confusing one, that is, until I realized that they had stolen our name again!

Looking at the discussions it seems that micro-services talk like a web-service, they walk like a web-service (that is, they don't walk, they are actually kind of static), and they quack like a web-service.  Ergo?

They could not even call it tiny-service, puny-service, or small-service (getting the alliteration for free). Oh no, they had to jump the 6 magnitudes with micro! The worst thing is that it is a terrible misnomer to call a web-service a micro-service. I get the service part, a service is clear a point to access some functionality according to a contract; all service models share this core idea. However, a REST-service and a WS-* service use the Web/REST prefix to identify the communication stack. Since the OSGi µservices have zero overhead because they are actually local calls it makes sense to call them µservices. However, this new 'micro-service' is still a web-service or a REST-service, inheriting the often sizable communications stack and overhead associated with remote procedure calls. These new incarnation of web-services are only supposed to be conceptually small.

Which is of course is quite silly, since size is not the primary factor for module decomposition (of which service design is an instance). The things that mostly counts in good design is cohesion. Cohesion measures the conceptual closeness of the service's methods in relation to the overall system, a cohesive services has a well defined single responsibility. When I read about the micro-web-services I think they actually just mean cohesive web-services.

So can someone please ask them to call their bright idea cohesive web-services? Then we all know what they are talking about, we keep the vocabulary consistent, and we can keep using the term µservices because in OSGi they actually are!

Peter Kriens


Thursday, June 12, 2014

OSGi BOF Tonight @ OSGi DevCon 2014

OSGi DevCon 2014 is well underway and Tonight we have an OSGi BOF.

Its at 19.30hrs in Comedians Room in the South Tower at the New York Marriott at the Brooklyn Bridge.

Its your chance to come and meet with your peers, network and ask any questions.  There will be a number of the OSGi Alliance Board Members at the BOF including Dan Bandera, the President.

David Bosschaert will kick us off with a quick demo creating a project through Bndtools and then showing how to work with this project both inside Bndtools as well as from the command line with Maven.

Peter Kriens will also be around and keen to talk with anyone about OSGi enRoute or answer any questions.

In the true spirit of a BOF there will be plenty of opportunity for everyone to take part asking and answering questions, sharing and gathering insights and experience.

We hope you can join us at 19.30hrs this evening.


PS Don't forget its tutorial day tomorrow at OSGi DevCon - sign up to be sure of your seat.

Wednesday, June 11, 2014

Draft OSGi Enterprise R6 specification available

The OSGi Enterprise Expert Group has released a draft of the upcoming OSGi Enterprise R6 specification. You can find the following updates in the download package:

  • The new Asynchronous Services specification has been added which allows clients to invoke OSGi services in an asynchronous way.
  • A Promises API specification has been added. Promises are a very popular programming model in the Javascript world, where they make it easy to work with operations that execute in an asynchronous way. OSGi Promises follow a similar pattern. They are used by the Asynchronous Services specification, but can also be used independently.
  • A new specification on Framework management via REST has been added. This facilitates remote framework management in cloud and other contexts where REST is the protocol of choice.
  • A number of updates are made to the Remote Service Admin specification. RSA now supports remote endpoint modification notifications through new modification events. This means that if you update the service properties on a remoted services, this will no longer appear to clients as if the service was removed and then re-added. With the new modified event a previously remoted service will remain, and is updated in place.
  • Subsystems have been updated to version 1.1, a number of changes have been made to this specification including the ability to provide the Deployment Manifest separate to the subsystem archive.
  • The Repository spec has been updated to version 1.1, allowing more complex queries to be passed to the Repository.
  • A new HTTP Whiteboard specification is being worked on. This specification brings Servlet support up to Servlet 3.x and adds whiteboard-based Servlet and Filter registration.
  • Declarative Services is receiving an update which adds a new and really convenient way to work with Configuration, support for Prototype Service Factory (which is introduced in Core R6) and an introspective API.
  • The MetaType service is being extended with annotations.

You can download the draft specification from the OSGi website on this location. Note that the zip file contains the draft spec, as well as three RFCs for which no spec chapter is ready yet, these will also be included in the final specification.

Also released this week, the Core Platform R6 final specification. Get it here.

OSGi EEG co-chairs,
David Bosschaert
Tim Diekmann

Thursday, May 29, 2014

Only 2 weeks to go to the Big Apple...

June 11 to 13, New York at the New York Marriott at the Brooklyn Bridge.

We are on the final countdown to OSGi DevCon 2014 in New York.  There are just 2 weeks to go, so if you have not registered yet be sure to REGISTER NOW.

You can still save $200 by booking before June 9 and for groups of 3 or more there is a group booking discount.

We have an excellent schedule for the event this year and don't forget all attendees also get full access to the QCon New York conference as well as OSGi DevCon. 

Thanks to the OSGi DevCon sponsors Adobe and Cloudyle for supporting this event.

In total you have access to 4 keynotes and over 100 speakers, along with 2 dedicated OSGi Tutorials and an OSGi BOF. In addition there are deep technical case studies from organisations including NASA, Netflix, Twitter, Facebook, Amazon, and others.

With a 1:5 speaker to attendee ratio, over 500 of your peers in attendance, and numerous breaks and parties, the combined OSGi DevCon and QCon event offers fantastic opportunities for peer sharing and networking.

You can full details about OSGi DevCon on the conference home page including how to register, the conference location, how to book accommodation and the schedule.

We look forward to seeing you in the Big Apple on June 11.

If you have any questions you can contact the Program Committee by email.

Monday, May 12, 2014

Register For OSGi DevCon By May 25 For A Chance To Win.....

The countdown is on with just 1 month to go before OSGi DevCon 2014 in New York (11 to 13 June).

The Schedule is announced, the hotel is booking up, NOW is the time TO REGISTER if you haven't done so already.

There are some key money saving deadlines ending on Friday this week

  • Registration savings: You can save $300 if you book before May 17, that's this Friday. 
  • Hotel savings: The discounted hotel conference rate is only guaranteed for reservations made by this Friday too. 

Of course don't forget if there are 3 or more of you registering at the same time you can contact the QCon Registration Team for a group discount.

And now for the chance to WIN.....

Our friends at Manning Publications Co. have kindly donated some of their excellent OSGi Books that you have a chance of winning just by registering by May 25.  Don't worry if you have registered already you will also be entered in to the draw.

All you need to ensure is that when you make your reservation you select OSGi as the option in the last question (Are you primarily registering for:) and you will automatically be entered in the draw!

There is 1 copy of each of the following books and we will draw three names from the hat.

Good luck and get your registration sorted this week to benefit from the savings and be in with the chance of winning one of these books.

OSGi DevCon 2014 Program Committee

Friday, April 11, 2014

OSGi DevCon 2014 Schedule Announced

We are pleased to announce that the OSGi DevCon 2014 Schedule is now available.

Register before April 19 to save $400 per ticket. Yes thats just 8 days away!  Don't forget there is a group discount available if there are 3 or more of you registering at the same time.

As you will see there is plenty to keep you busy for the 3 day conference (June 11 to 13) in New York.

There are:
  • 23 Talks
  • 2 Tutorials
  • 4 keynotes
  • An Open Space session
  • A BOF

In addition attendees to OSGi DevCon 2014 also have full access to the QCon New York schedule which offers 5 tracks each day.

Plus there are plenty of social activities on offer to:
  • Exhibitor Reception
  • Conference Party
  • IBM Reception
  • QCon Reception
Your conference pass also includes breakfast (8am to 9am) and lunch each day.

Thanks to everyone who made a submission to the CFP. We really appreciate the time and effort that goes in to these.  The quantity and quality of the submissions was extremely good which gave the Selection Committee a tough time in coming up with the Schedule.

Thanks must also go to the Selection Committee for their assistance and time to produce the excellent Schedule for OSGi DevCon this year.

We hope yo can join us in New York in June.  As ever if you have any questions you can contact the Program Committee by email.

Click here for the OSGi DevCon 2014 Schedule

Blog Archive