Monday, July 22, 2013

Accidental Complexity

I am currently writing an RFP for the increasing adoption work I will do for the OSGi Alliance. To understand the landscape, I interviewed a number of software companies. The first interview was a Spring/Maven shop, representing mainstream Java development. However, I've seen that PHP is very  popular in the lower segments of the web applications. Since I am a strong believer in the powers of Java, I always wondered why those people 'struggled' with PHP so I decided to interview a small PHP shop to find out why they used PHP.

The story they told me over lunch was quite interesting. Early 2000 they had some people go on a course at IBM to learn about Java EE. After many thousands of dollars and some weeks later they all had become utterly confused. In their eyes, Java was just way too complicated for what they wanted to do. PHP gave them a simple platform that was more than sufficient for their needs. And not only was it much easier to learn, it was also much easier to find libraries and snippets that they could use in their daily work.

When I tried to defend Java (neutrality is not my stronghold) I found that most of those really good things in Java fell on deaf ears. For example, refactoring is one of my top features in Java but they had no idea what it entailed. So I found myself desperately trying to explain refactoring in terms of a type safe search and replace but it was clear that their eyes glazed over, wondering what the big deal was (ok, the rosé might not have helped). It was clear to me that much of the accidental complexity we have in Java was not understood to have sound software engineering reasons, for them it was just painful and overly complex. PHP was so much easier to get started with. I must admit that when a friend, who is a non-technical sociology professor, built a professionally looking PHP website, including large forms and payments, I was duly impressed.

After some more Rosé my lunch partners did actually come up with problems. Problems that I think could be addressed but the problem was that they saw these 'problem' as facts of life, not something that could be addressed.

Therefore, the primary problem that we should try to address is how to cross the steep threshold that Java puts between an uninitiated developer and a web app. I think having a framework for web apps that provides a skeleton for other applications is the start point that will make a difference. However, to also attract non-Java developers we will need to minimize the accidental complexity of Java and OSGi. With DS annotations and bndtools I think OSGi's accidental complexity is ok for the powers it offers. However, after researching how to handle persistency in Java I find it hard to apply the Java EE APIs in OSGi so that they provide an out of the box experience like PHP. At the same time, I find some other libraries that are not Java EE but seem a lot easier to use. It will be interesting to find out how we will weigh the compatibility requirements against the simplicity requirements.
Peter Kriens (@pkriens)


  1. Interesting post. I like how you said, "It was clear to me that much of the accidental complexity we have in Java was not understood to have sound software engineering reasons, for them it was just painful and overly complex." It is kind of a sad truth. One only really sees the value in all of it after becoming intimately familiar with it. Yet if the barrier to entry is high, many will never even get to the point where they will see it. For me, I grew up with Java since it first came out - so I evolved with it and never experienced the complexity as it is today anew. It is interesting to think about.

  2. Bullshit. You can success with Java or PHP or anything else. The point is: large teams, large projects. PHP is a pain in the ass with big projects and large teams. You can do it, but it's not easy. Java brings all the tools and better support for that scene. Java is a lot easier with big teams, large projects over time. But with great engineering and great programmers, you can success with ANY tech. Call it Java, PHP, .NET, Python, even BASIC. Great human resources can make great things... even with shit.

    1. No doubt you are right, but kind of irrelevant isn't it? 'Great human resources' would be even more efficient with good engineering, it is not about the goal, it is about the process to get there. I think you also ignore the maintenance phase, the part where your software will reside for the majority of its life and those 'Great human resources' have long left the project.

  3. I beg to disagree. While your background story is accurate, your conclusion is not on point.

    Example is Rails. And Play. Those are complex technologies with lots of metaprogramming magic inside. Average Ruby or Java developer will have difficulties in understanding the framework.

    Complexity of the framework has little to do with adoption. It's about usability.

    It's as easy to get started with Rails as it is with Play. And it's somewhat normative: you get straightforward 5-minute or so tutorial right from their official websites.
    Neither is true with OSGi.

    Yeah OSGi is abstract, so mention a framework then. Pax Wicket comes to mind. I adore Pax Wicket, but on the "get started" easiness scale, Rails and Play wins easily.

    Starting out is just a start, the developer needs to evolve the app to more complex.. Rails ecosystem provides plugins and general coherence in the way things USUALLY work. So ask any Rails expert about how to do X and even if the answer varies, there's usually a common lining. And it usually works.

    OSGi? 99% the answer is "it depends". Database? It depends if you use JDBC/Spring/JPA/Hibernate/OpenJPA etc.

    That's the most basic thing one can get and Rails has its basics covered, while in OSGi land we're still wondering why JPA fails some operations under OSGi and works properly without.

    Any OSGi app dev will hit (lots of) CNF within his first week. This is a bizarre experience that never happens on PHP, Rails, and even to some extent Java.

    And the more complex the app gets, the more libs get used, it gets worse. Java ecosystem is huge but most of them aren't 100% OSGi friendly yet. Even those who try are struggling. Rails...well as long as you can specify a version then that plugin should work just fine.

    Note I am a proponent of OSGi. And as an insider, Java is the best tech for me and it's my main language. But since the topic is adoption, there I said it honest and front.

    Note also that barrier of adoption is not exclusive to OSGi but also to Java. So OSGi only compounds the barrier that already exist in Java, not reducing it.

    Good luck!

    1. I fully agree with Hendy. The biggest problem in Java is that for every technical part of a typical application there are many alternatives and they do not integrate well.

      If you want to create a business app there is a lot of effort to setup the technical infrastructure. The only kind of consistent framework is Java EE. It offers a coherent model from UI over services to persistence.

      In OSGi it is even worse. We have some nice service frameworks like blueprint and declarative services but the rest is largely missing. How would I do the UI, how persistence.

      Blueprint at least has namespaces that allow integration with aries jpa, cxf and camel. It is too inconvenient with the xml though.

      DS on the other hand does not provide integration with other frameworks at all. So it lacks the completeness to build a full app. Bndtools is nice but a far as I know it lacks integration with maven. As maven is used in most companies this is a big issue.

      The only candidate to do complete applications in OSGi I currently see would be Java EE adapted for OSGi and with good maven integration. The are efforts to achieve this already like Karaf EE or pax cdi but they are far from production ready.

      So to sum it up we need a consistent model for building business apps including build integration.


    2. Hendy you are dead on. As a developer with experience in many different languages and frameworks it has completely confounded me that the only framework I've taken more than a week to feel comfortable in are OSGI implementations. I still don't feel comfortable in them and have considered migrating a very large project I'm the lead on to a non-osgi framework.

      I like java as a language. It enjoys wide support and has very good documentation overall. OSGI and it's lack of drop in support for existing libraries has made a terrible fumble in my eyes. The key to wide adoption and admiration is a low barrier to entry.