Thursday, September 11, 2014

More OSGi events in Spain this month.....

So it looks like September is turning in to OSGi month in Spain....

*** Update - Live Stream for tonight available from 19.00hrs CEST on You Tube ***

Hot on the heels of the Peter Kriens presentation about enRoute to the Madrid JUG on Sept 10, there is an event from Barcelona JUG this Friday and Saturday (Sept 12 and 13) by Jean-Baptiste Onofré.

This is then followed by another event from Barcelona JUG on Friday Sept 26 and Saturday Sept 27 with Peter Kriens.  More details on Peter's sessions are to follow in a later post.

The presentation this coming Friday by Jean-Baptiste Onofré is called "What is OSGi and why and how I should consider it for my projects?" This will cover some of the OSGi specifications and introduce the Apache Karaf container.  Hopefully this session will be streamed live and further information will be updated here (see above) and posted on the OSGi Alliance twitter feed as it becomes available.

The next day (Saturday Sept 13) a workshop will be held that will show you how to create a complete enterprise level OSGi application by leveraging the different layers provided by Karaf.

Full details for these two events and how to register are as follows:

Fri Sept 12 - Presentation
Start Time: 19.00hrs CEST
Location: Plaça Catalunya, Portal de l’Àngel, 08002 Barcelona (Map)

Sat Sept 13 - Workshop
Start Time: 10.00hrs CEST
Location: Espai Jove La Fontana, Carrer Gran de Gràcia, 190, 08012 , Barcelona (Map)


Wednesday, September 10, 2014

Live Streaming of Peter Kriens presenting enRoute Today @ Madrid JUG

Peter Kriens is presenting at Madrid JUG this evening (Sept 10) about OSGi enRoute. The presentation will be in English.

Our friends at the Madrid JUG have kindly arranged to stream the session. You can also follow the event and ask questions on twitter using the hashtag #mjugOSGi.

Its taking place at 19.00hrs CEST/ 18.00hrs BST/ 13.00 EDT / 10.00hrs PDT /  02.00hrs on Sept 11 (JST) (apologies to anyone in Asia).

You will be able to watch the stream from You Tube.  If you are in Madrid its being hosted by Liferay (thanks) but there are no spaces left so you will have to watch the live stream.

Hope you can join us for the stream if you aren't one of the lucky ones attending in person.

*** Update - the video recording below of the event will be made available in due course ***


In my last whiny post I stated that we tend to solve the symptoms and not the root problem, Jaco Joubert challenged me to define what the real problem is; fair enough.

The root problem is that we write fragile software that stumbles when there are unexpected changes in the environment. The simplest solution to this problem is to lash down the environment. If the environment does not move, then our software will not fall over. This is exactly what Docker does. And if a software system consisted of a single delivery then the cheapest solution is likely the best solution. 

However. My unease is caused by two fundamental issues: 

* First, we will write the same amount of software in the next 7 years as all the software written since the big bang. 
* Second, successful software systems tend to live a long time and evolve continuously. 

In such a gyrating environment the only thing constant is change. Change that Docker will allow us to largely ignore because we create a warm nest where this bad outside world does not intrude. Sloppy programming like hard coded path names and making other assumptions about our environment will not be punished. Initially this will work perfectly ok. However, over time your modules will become more and more tied to your unique environment because we have a hard time resisting a short-cut when they see one. These short-cuts will make the modules less and less reusable, which will probably make it easier to double the lines of code in the next 7 years but efficient is different. It will also make it harder to evolve your own environment because a lot of its details will be hardcoded in all the components. Expect another variation on the spaghetti model.

The root of software bugs is almost always a violated assumption in the code, therefore, not having these assumptions is the way to go, even though this can be very hard work. Not only will this reduce bugs, it will also make code more reusable because not-knowing something also reduces constraints between modules. A great way to structure this is to do contract based programming, something which reaches its (current) perfection in the OSGi service model. 

The argument is therefore that the virtual-image model (which Docker just makes more usable) will let sloppy programming go unpunished until the big bang.

Now, the picture is not black and white; I actually feel a bit uncomfortable with such an argument in other places and I clearly see the shorter term advantages of the container model. Maybe jet fighter pilot training provides some insight. Their simulators are running significantly faster than real life. The result is that when the pilots are deployed in a real fight it actually feels a tad boring. This is why during development I want an open world and as much dynamics as possible to capture our team's false assumptions early. However, when the code actually gets deployed to Q&A I want a locked down environment that will be bitwise identical to the image that goes to production.

Ah well, that is why I love this industry, never a dull moment.

Peter Kriens

Monday, September 8, 2014

The Unbearable Uncoolness of Java

This weekend I played with Docker  to better understand this technology and this was quite fun. Interesting how we focus our energy on solving symptoms instead of addressing the root problem. But I deviate.

What actually shocked me was the current standing of Java in this world, many devops seem to hate Java with a vengenance. After trying to create a Docker image with Just Java, I can (partially) see their point. Java sticks out like a (very) sore thumb in the Linux landscape. Virtually everything can be installed with all different package managers until you hit Java, then suddenly you need a click-trough license (except for some reason when you download for the Mac, there downloading is sufficient to accept the license). But the frustration is not limited to not being able to automate an install, it is actually not that hard to bypass, just ugly. It is the whole experience. Just compare the Python, Ruby, Go, and node.js website with the java download ordeal, which even includes a nice 404 now and then. Also Eclipse, despite it new look, feels like it lost a lot of steam in the past years.

It hurts to say, but Java looks stuffy and tired ...

We have the surreal situation that Java is by far the most advanced software development environment in our universe. We have amazing tools like Eclipse, Hotspot, based on an incredibly mature environment. We have at our disposal a humongous repository with libraries. It actually has a really good module system. Java is used to build gigantic applications of sizes that most other environments can only have nightmares off. Every day Java is involved in moving around trillions of dollars. Untold number of developers use it day in day out. There is just nothing out there that can handle the size of large systems Java can.

I know, any technology that is actually solving real problems becomes complex over time and Java is no exception. I also know that sometimes you need a clean break to get rid of the scar tissue or parts that proved unnecessary but I do not see anything on the language horizon that is providing anything fundamentally better than Java 8. If Go is as successful as Java it will become as complex as Java. Most often advantages are synthetic sugar or only work for specific problems. Overall Oracle is doing a pretty decent job keeping this technology up to date. That said, they really need to rethink their position in the computing landscape, especially in the server area. However, Java surely has a very long life ahead of it. 

That said, it would be a lot more fun if Java looked more like a sexy 20 year old instead of a stuffy octogenarian. Anybody any ideas?

Peter Kriens

Friday, August 29, 2014

Is Docker Eating Java's Lunch?

The reason I've put up with Java's deficiencies so long (ok, now with Lambda's it is finally starting to become a real language) is that I truly believe in the 'Write once, run anywhere' mantra. I've never understood why you want to write software that is not reusable and thus portable. Java achieved its portability by creating an abstract API for the interaction with the host Operating System (OS). I am duly impressed with Sun's engineers of how close they got to this goal, the Java VM is amazing piece of technology.

However, in practice, writing portable software still required effort from the developer. Despite the abstractions (alas, if only they had stayed close to POSIX), there are many subtle, and some not so subtle differences, that must be handled by the developer. Just like the differences in the configuration of an OS or the way its file system is used (i.e. hard coded path names).

The dreaded 'But it works for me!' frustration drove the idea of virtualization, where the software runs on a standard (Linux) OS that is virtualized on each developer's machine, which is then bitwise identical in test & production. The cost of this model was that it required schlepping around large images of many Gigabytes whenever the tiniest detail changed. Docker is, however, removing this disadvantage by making it very efficient to create, manage, and deploy these custom images.

So it looks like that the majority of Java applications in the future will run on Linux even if they run on MacOS or Windows, voiding the unique selling point of Java: portability. From now on, Java will be on par with every other language in the world regarding portability.

So what is left?

Java's greatest asset is that it allows you develop very large applications with sizable teams and stay sane. Though its type safety and long name feel cumbersome for small applications it becomes a necessity when the code base grows, which it inevitably will for any successful product. Type safety gives you confidence that all the different parts are actually compatible; the long names prevent conflicts. Even more important, it gives you navigability in the IDE, crucial when you maintain a large code base. This is of course exactly the area where OSGi provides it benefits: extending the type system with private Java namespaces, a time dimension (semantic versioning), dependency model, and into the runtime. There is no competitor on the horizon here.

So though Java/OSGi seems still have some unique selling points, it is the programmer in me that is a bit sad because the Docker revolution is another win for the sloppy programmer. There is something fundamentally wrong in our software industry if our software is so brittle and fragile that we can only make it run in a rigidly defined bitwise identical world, it points at a fundamental failure in the way we develop software. Sadly, we have so far been unable to learn from nature. In nature, systems survive and evolve because they are dynamic, adaptable, and resilient, often even anti-fragile. The Docker revolution will actually make our systems even sloppier since now many bad practices will go unpunished and this will have a price tag. These now unpunished bad practices will make maintenance and evolution of the code base much harder, aggravating the situation down the line.

Then again, who cares what happens in the next quarter? This Docker development model will likely lower the use of the  excuse: 'But it works for me!' in this quarter.

Peter Kriens

Follow me on Twitter @pkriens

Monday, July 14, 2014

The Big Ball of Mud

I am a tad frustrated about the whole µService discussion. How carefully designed concepts and terminology gets trampled over by a crowd that doesn't bother to look around in their rush to next year's fashion. When Neil Bartlett posted a link to an interesting blog that discusses exactly this subject I was therefore pleasantly surprised. It is about how our industry tends to swing from hype to hype and does not even try to learn any lessons from the down-swing. That is, if those lessons requires more than a minimal attention span. Ok, Simon Brown  does not appear to be an OSGi aficionado but he hits the nail on the head that your monolithic big ball of mud is not going to be a tender agile application overnight by adopting micro-web-services. Or, how a new fashion is always more attractive than the grunt work of making the existing stuff right. Hey, maybe they did find the silver bullet this time! #not

We actually know quite well, and for a long time, how to build large complex systems: contract based component designs scale very well. It just so happens that in practice the siren song of short cuts are much more attractive than doing it right ... that is, in the short run. We show beautiful layered architectures that are in reality camouflage nets over the actual big ball of muds. Read Simon's blog and weep.

Now, although I agree with the view in this blog, I think he misses the key reason why we always end up with that big ball of mud. Unfortunately, experience shows that this is not an easy story that fits the 140 character attention span, the cause of the big ball of mud does require some work to understand. Please shut off twitter and bear with me.

In the seventies, structured programming was the then current best practice. The mantra's in those times were 'Low Coupling' and 'High Cohesion'. Then, in the early eighties, we got Objects! New! Improved! No need to bother about these old geezers with their coupling and cohesion muck anymore. (I never really got that cohesion stuff anyway.) Progress, New! We were young and innocent until time taught us that the old geezers had probably learned a lessons or two that we had missed. Embarrassingly, the anticipated reuse we had promised to our managers were much harder to achieve than we had thought in our youthful exuberance. What killed us most were the transitive dependencies that are implicit in the object programming models. It is wonderful to reuse another object, but the damned thing should then not do the same thing and depend on others, ad nauseum. Instead of picking and choosing reusable objects, systems quickly became big balls of mud, or more appropriately, a different form of spaghetti coding. We had carefully avoided the GOTO statement but the result was painfully similar, just on another level.

In the nineties I had learned my lessons about coupling, cohesion, and objects. So when I met Java in 1996 I  immediately realized what interfaces were doing, they were addressing the transitive dependency problem that had caused me so much pain! With an interface you could program against a contract, not against an implementation! This did wonders for the transitive dependency problem since you were now bound to an interface. Interfaces were much less coupled by design than implementation classes. I obviously wasn't alone and interfaces became highly popular in Java and are pervasive today because they do provide freedoms that transitive dependencies kill.

Alas, we then witnessed a demonstration of how little intra-generational learning is going on. Two things went haywire. First, interfaces raised this pesky problem of 'How on earth do we get the instance in runtime?` Factories were kind of awkward so we found the greatest ostrich solution of all software times: 'We hide it in a XML!'. Obviously, the fact your coupling Java analysis tools did not see these hidden dependency anymore did not mean they stopped hurting you over time. An implementation class reference is a transitive dependency as much as any "new" expression is in your compiled code. Worse, a class referenced in text bypasses all those type safe-guards in Java you just spent all that money on.

The other, and surprisingly even worse, mistake was that we happily copied the transitive implementation dependencies model that failed so spectacularly with objects into our modules. And surprise, we got the same ball of mud, just now on a tad larger scale. Don't get me wrong, it is not maven that downloads the Internet, it is people that download the internet. People are addicted to solutions that look easy. That is, until that proverbial stuff hits the fan but then the original designers have then usually moved on.

Hard coding a dependency on another module will inevitably create more and bigger balls of mud, we learned this with structured programming, with object oriented programming, and now we are learning it with modular programming. The structural problem is identical in all these cases, so why should the result differ? (Though I find that a surprisingly large group of people disconnect now, 'What on earth has an interface to do with modules? This guy is an idiot ...'. Then again, their attention span probably already had made me loose them to that interesting tweet that just came in.)

Obviously, when we started designing OSGi in 1998 we did not have the clear view of today. However, the crew that did the early design had a lot of experience and provided a very elegant, µService based, model.  It solved the 'how on earth do I get my instance of that interface' problem in a non-ostrich way, it also innovatively used Java packages as the dependency model.  The Java package plays the same role for a module as the interface does for a class.

If you find this hard to swallow then you're not alone (which does not make you right btw, just more common). At JavaOne, I actually once overheard a discussion between three (at that time) Sun/Oracle high level executives;  they were making fun of package dependencies, radiating an amazing ignorance of the underlying issues. Even with very bright people, some that I admire a lot, I've seen that it takes effort to accept this structural analogy between classes and modules. However, even at the JCP, the Java package is already playing a large organizing role in most, if not all, specifications. It acts as the mediator between the provider (e.g. Jersey) of the specification (JAX-RS in packages) and the consumer of that specification.

The key consequence of the OSGi model is that the µservice is not an implementation (a module) as many people tend to talk about it, it is a contract. Modules can provide implementations for these service contracts and they can consume objects that implement these service contracts. The modularity model is thus not: module A depends on module B. No, module A depends on service S and module B and module C can provide such a service. It is this crucial indirection that I think Simon misses in his blog that is the only known answer against drowning in the big ball of mud over time. We figured this out with Java interfaces, we now only need to understand how we can eradicate the same transitive pest with modules as well.

Yes, I know the model is not as simplistic as the module A depends on module B model that we know so well; it adds an additional indirection that is not free. The difference is that you pay that cost up-front while the long term cost of module dependencies is much, much higher but drawn out over time.  Though we like to postpone costs, we actually did learn a lot of lessons over time that made us take some up-front cost, Java as a type safe language is the prime example. So I am utterly convinced that model we learned in OSGi will prevail over time. Just hope that I do not have live through another hype that presents (parts of) these ideas as if they are something new.

Peter Kriens


Monday, June 23, 2014

Software mixed with marketing: micro-services

A few months ago I was on the phone when the speaker on the other side mentioned micro-services. Many moons ago I'd written a blog about OSGi's µservices because people were getting confused with the term service. Though this name was unique enough in 1998, the advent of Service Oriented Architectures (SOA) had imprinted people to think about heavy weight, costly, and complex things requiring stacks and other things a developer wants to avoid. That is, everything OSGi is not. I even used the Mu symbol (µ) because it was also the symbol for friction, of which µservices have so little. I therefore kind of naturally assumed the guy was talking about our wonderful in-VM µservice oriented architecture (µsoa). Actually, I smugly told him that I coined the term, trying to impress him. This however turned the discussion from straightforward discussion about technologies into a rather confusing one, that is, until I realized that they had stolen our name again!

Looking at the discussions it seems that micro-services talk like a web-service, they walk like a web-service (that is, they don't walk, they are actually kind of static), and they quack like a web-service.  Ergo?

They could not even call it tiny-service, puny-service, or small-service (getting the alliteration for free). Oh no, they had to jump the 6 magnitudes with micro! The worst thing is that it is a terrible misnomer to call a web-service a micro-service. I get the service part, a service is clear a point to access some functionality according to a contract; all service models share this core idea. However, a REST-service and a WS-* service use the Web/REST prefix to identify the communication stack. Since the OSGi µservices have zero overhead because they are actually local calls it makes sense to call them µservices. However, this new 'micro-service' is still a web-service or a REST-service, inheriting the often sizable communications stack and overhead associated with remote procedure calls. These new incarnation of web-services are only supposed to be conceptually small.

Which is of course is quite silly, since size is not the primary factor for module decomposition (of which service design is an instance). The things that mostly counts in good design is cohesion. Cohesion measures the conceptual closeness of the service's methods in relation to the overall system, a cohesive services has a well defined single responsibility. When I read about the micro-web-services I think they actually just mean cohesive web-services.

So can someone please ask them to call their bright idea cohesive web-services? Then we all know what they are talking about, we keep the vocabulary consistent, and we can keep using the term µservices because in OSGi they actually are!

Peter Kriens


Blog Archive