When SpringSource announced their Application Platform and Bundle Repository I was very excited. I had been informed something was coming and picked up rumors from the grapevine.The bundle repository was in one word: stunning. I had been looking into providing such a service but had decided that I could not afford it, despite the fact that I think that many companies would really benefit from such a service. This bundle repository must have been a terrible job to do. Kudos!The Spring Source Application Platform was, however, a bit of a shock. Looking through the documentation I found lots of headers that felt like OSGi but that I did not recognize: Import-Library, Import-Bundle, Application, etc. It looked like SpringSource had “improved” OSGi extensively. This was a bit of a shock, especially because they looked so much like standard OSGi headers.
I learned over the past 10 years that just adding a number of headers that solve a particular problem usually backfires. Something that sounds really good, does not have to be that good at all when you inspect it from a different perspective. The problem with standards is, is that it is very easy to add something but impossible to remove. You really have to get right as much as possible the first time; it is therefore better to err on the side of leaving things out. Due to the embedded origin of OSGi we have always been extremely careful to minimize the number of feature. Each additional feature require a lot of eyes and discussions and a very thorough analysis before it makes into the specification. Especially today, with a mature specification the feature interaction can be quite painful.
Do not get me wrong, as a practitioner I do know the importance of getting things to work and standards, by definition, are not always sufficient. I also know that a standardization process feels sometimes excruciatingly slow when you live in the trenches, and I love experiments.
However, Eclipse has the same problem, but they have always been a good citizen in this respect. They always added proprietary headers in their own name space, learned from their experiences, and then proposed a new standard. In all cases so far, the specification process was able to make the concept more general and fit better in the overall OSGi model. Having a separate name space was also a convenience for their customers. A specification process will undoubtedly change the semantics, requiring awkward constructs to distinguish between the SpringSource defined meanings and the OSGi defined meanings. I do realize that OSGi does not own the name space in the manifest, and I do regret heavily that we did no better scope our header names, however, this did not seem overly important 10 years ago.
Why would the standardized version be better? Because more eyes looked at it, from different perspectives (which sometimes can be quite bad for one's ego). One man's convenience is another man's complexity. This is why so far we have always focused on providing the primitives and not convenience functions because convenience is often in the eye of the beholder and can be provided during the development process in the IDE or run-time libraries.
The biggest danger of a standard like OSGi today is feature creep. The leanness of OSGi is one of its main attractions for many newcomers. It really took real hard work and untold man hours to make it this lean, believe me. Maybe the SpringSource headers are a significant improvement over the existing specification, maybe they turn out to be superfluous. Time will tell. I guess it is a sign of becoming more popular that companies try to stretch the specification in their world view. However, I do hope that most developers realize that restricting oneself to the standard may sometimes be a little bit less convenient, but it does provide one with a more options and no vendor lock-in down the line.
So sadly, my feelings are mixed. I love what they have done with the bundle repository, and also many parts of the S2AP look very impressive, Neil Bartlett's blog provides an overview that I can very much sympathize with. I know several Spring Sourcerers and they are very clever and capable people. SpringSource undeniably is meaning a lot for the OSGi organization and the adoption of the specifications. However, multiplying bundle headers feels more like coming from an apprentice than from the experts I know them to be.
Peter Kriens
P.S. On a more positive note. We are reaching the first deadline for registration of the OSGi Community event in Berlin, June 10-11. I am part of the program committee and I can ensure you that it is worth coming. We actually had to split the second day into two tracks because we had so many good sessions submitted. Looking at the number of submissions it will look like we will sell out the places we have this year. This is the premier OSGi event in Europe, we are really expecting all the key OSGi people to be there. Take advantage of the early registration fee and register today!
I had the same misgivings on reading about the Import-Library and Import-Bundle, but on further reading I had thought these were only syntactic sugar, as they expand to Import-Package under the covers.
ReplyDeleteI've also read that the bundles available from the SpringSource repository will only use standard OSGi headers; not the new Import-* headers. This still leaves me a bit confused on how the new Import-* headers are used and where - playing with the Spring app platform will, hopefully, answer those questions.
I did not comment the semantics of the headers. It is great that they are experimenting and this will provide invaluable feedback. Even if it is only syntactic sugar, these headers look like they are part of the OSGi family and that concerns me. All OSGi headers have been analyzed and vetted to death, these are very interesting experiments and will likely influence the spec in the long run. They should have made this more clear by naming them S2AP-Import-Library or so.
ReplyDeleteKind regards,
Peter Kriens
Peter,
ReplyDeleteI actually think we are in violent agreement about many of the points you raise in your post. There seem to be three basic points of confusion/contention about the new manifest headers supported by the SpringSource Application Platform being raised here and elsewhere, namely:
* The fact the we introduced new headers at all
* The fact that Import-Library and Import-Bundle sound similar to existing headers
* The fact that we didn't attempt to standardize these headers before including them in our beta (not a criticism made in your post, but heard elsewhere)
Let me address each of these points in turn, responding to some of your comments as we go.
* The fact that we have introduced new headers (at all)
Peter wrote:
>The Spring Source Application Platform was, however, a bit of a shock. Looking through the documentation I found lots of headers that felt like OSGi but that I did not recognize: Import-Library, Import-Bundle, Application, etc.
The "lots" of headers that you refer to for MANIFEST.MF basically boil down to Import-Library and Import-Bundle (there is no "Application" manifest header).
>It looked like SpringSource had “improved” OSGi extensively.
The important thing to understand about these headers is that they add no new underlying semantics to an OSGi Service Platform, and are essentially just syntactic sugar for a set of Import-Package statements. Any bundle that uses these headers could be deployed on any OSGi R4 compliant Service Platform following a simple mechanistic expansion of the headers into their Import-Package equivalent (more on that later, we'll be shipping the tools to perform exactly this translation). The SpringSource Application Platform uses the OSGi RI (Equinox) unmodified at its core.
>I learned over the past 10 years that just adding a number of headers that solve a particular problem usually backfires. Something that sounds really good, does not have to be that good at all when you inspect it from a different perspective. The problem with standards is, is that it is very easy to add something but impossible to remove. You really have to get right as much as possible the first time; it is therefore better to err on the side of leaving things out.
I completely agree. The key point is, *we haven't changed the standard*! At SpringSource we firmly believe in standardizing solutions that have been proven to work in the field, as opposed to solutions that have not yet been battle hardened. What we have done is added syntactic sugar particular to the SpringSource Application Platform, that if it proves to be successful in the marketplace we would then put forward through the OSGi RFP/RFC process for potential standardization. I agree it would be a mistake to add these headers to the specification now. The specification should err on the side of leaving things out, and only include new features that have proven their worth in the field.
>This is why so far we have always focused on providing the primitives and not convenience functions because convenience is often in the eye of the beholder and can be provided during the development process in the IDE or run-time libraries.
The SpringSource Application Platform is a demonstration of this split you describe working in practice. The OSGi R4 specification defines the primitive (in this case Import-Package) and the SpringSource Application Platform provides two convenience functions on top of that (Import-Package and Import-Bundle). These are supported during the development process (in the Eclipse tools that accompany the Application Platform) and in the runtime libraries that translate the convenience headers into their standard equivalents before deployment into a standard OSGi Service Platform. So once again we find ourselves in agreement.
* The fact that Import-Library and Import-Bundle sound similar to existing headers
>>This was a bit of a shock, especially because they looked so much like standard OSGi headers.
The headers are of course designed to feel like a natural fit on top of the OSGi standard. From a usability point of view, I argue that this is a good thing! I've heard two arguments made as to why this "usability" is a bad idea:
1) Users may be confused about when they are using standard headers and when they are using non-standard headers, and
2) A future OSGi specification process may want to use exactly the same names, and that would cause confusion.
Taking these points in turn...
1) Users may be confused about when they are using standard headers and when they are using non-standard headers.
The fear here is clearly one of "accidental lock-in" to the SpringSource Application Platform by inadvertent use of non-standard headers. The fact is that since the headers introduce no new underlying semantics, any bundle using these headers can be trivially converted back to a bundle that will deploy on any vanilla OSGi Service Platform: there is no intention on our part to create lock-in with these headers. We will be shipping the tools (both command-line, and as part of our Eclipse-based tool suite) to convert any bundle using these new headers into one that uses only Import-Package instead. So anyone developing bundles for the SpringSource Application Platform and that uses these headers has a trivial path to port them to any OSGi Service Platform implementation. Moreover, we like Neal Bartlett's suggestion of a warning marker in Eclipse whenever one of these headers is used, and will be building it into our Eclipse tools as a "strict OSGi compliance" mode that can be enabled if desired. (So you don't need to write the plugin Neal, we'll be doing it for you ;) ). The SpringSource Application Platform is just as happy with Import-Package as it is with Import-Library or Bundle.
2) A future OSGi specification process may want to use exactly the same names, and that would cause confusion.
>A specification process will undoubtedly change the semantics, requiring awkward constructs to distinguish between the SpringSource defined meanings and the OSGi defined meanings. I do realize that OSGi does not own the namespace in the manifest, and I do regret heavily that we did no better scope our header names, however, this did not seem overly important 10 years ago.
First a general point. The underlying issue here is really that Java itself has no conventions for qualifying manifest headers in the same way that for example the package namespace is segregated. As you point out, OSGi does not own the manifest namespace and is an equal partner with everyone else in defining headers for use in a manifest and associating semantics with them.
Let's examine the hypothetical "worst case" : the OSGi Alliance decides it wants to introduce standardized headers with exactly the same names as the ones used by SpringSource, but with semantic differences. These headers would clearly not be recognized by an R4.1 based OSGi Service Platform. The "awkward constructs" to distinguish between the SpringSource meanings and the standard meanings therefore already exist and aren't really that awkward at all: it's the existing Bundle-ManifestVersion header. At Bundle-ManifestVersion 3.0 (for sake of argument) these headers would clearly have whatever semantics the OSGi specification defined for them, and it would be a problem for the SpringSource Application Platform itself (not the OSGi specification) to deal with the migration of existing bundles written at Bundle-ManifestVersion 2.0 to bundles at Bundle-ManifestVersion 3.0.
I guess it's fortunate for SpringSource therefore that the OSGi Alliance exhibits such a deep belief that new manifest headers should be qualified. Clearly it would be a hypocritical position for the Alliance to further propagate its "mistake" of 10 years ago by introducing new headers without (e.g.) an "OSGi" qualifier. (And if the Alliance doesn't qualify any new headers, on what grounds can it legitimately expect anyone else to do so??). Therefore the chance of a clash (which in any case would be easily resolved by Bundle-ManifestVersion) appears to be negligible.
I would actually fully support an RFP proposing that in R5 all OSGi manifest headers get an "OSGi" qualifier, and the existing unqualified headers be deprecated for removal in some future version of the specification. This is I believe the right solution to the perceived problem as the OSGi Alliance *can* legitimately claim some ownership of manifest headers starting with the OSGi qualifier.
* The fact that we didn't attempt to standardize these headers before including them in our beta
It sounds like both you and I are in strong agreement that premature inclusion of new headers in the standard before they have been proven in the field would be a mistake. See my post on the spring-osgi mailing list for a fuller discussion of this point.
The next OSGi CPEG and EEG meetings will be held at the SpringSource offices in a couple of weeks time. I'm looking forward to continuing this debate in that forum. The SpringSource Application Platform is still in beta, and now that it is in the public domain I'm looking forward to the EG feedback on what we have done so far. Making changes to the model we are putting forward is definitely still within scope, but so far I'm not persuaded by the arguments that we need to do so in this particular case.
-- The Apprentice ;)
Steve wrote:
ReplyDelete>I had the same misgivings on reading about the Import-Library and Import-Bundle, but on further reading I had thought these were only syntactic sugar, as they expand to Import-Package under the covers.
That's absolutely correct. They are syntactic sugar and using the tools we will provide any bundle that uses these headers can be easily converted into one that uses only Import-Package with no change in semantics.
>I've also read that the bundles available from the SpringSource repository will only use standard OSGi headers; not the new Import-* headers. This still leaves me a bit confused on how the new Import-* headers are used and where - playing with the Spring app platform will, hopefully, answer those questions.
This is a very good question, and one that deserves a longer answer in a dedicated blog posting of its own (which Glyn Normington has posted on the SpringSource blog). The short answer is that Import-Bundle and Import-Library are intended for use when referring to existing enterprise libraries that a bundle depends on (and not for use for importing e.g. user-defined application bundles). We do this for two reasons: (a) it more naturally expresses the user intention (the statement "my application depends on Spring Framework 2.5.4" is at a higher level of abstraction than 50 or 60 import package statements), and crucially (b) the set of Import-Package statements needed for many enterprise libraries that use load-time weaving, runtime subclass generation and other enterprise tricks is non-obvious to the user (it's not just the set of packages they explicitly depend on in their own code at runtime) and would require a tedious trial-and-error process to get right - and even then it would depend on internal implementation details of the library in question and may change with future versions making the construct fragile.
Adrian,
ReplyDeleteAlthough you are right in principle about the Import-* naming, SpringSource should act as a well-behaved OSGi citizen, as Eclipse has done in the past, and chosen a prefix.
I certainly don't believe that S2 has not made a conscious decision that a non-prefixed header serves the corporation better, i.e. accidental lock-ins, since(as you point out) you have chosen the risk of dealing with a very possible problem in the future.
I would urge S2 to make this change (prefix) prior to 1.0-final, to keep its status as one of the most outstanding contributors to OSGi technology. It is not too late.
Other than the above, I am generally positive to this new product. The choice of GPL/dual-licensing is a sign that S2 is now looking into making some money out of all their market-share, which I can easily sympathize with. Whether it will work is a different story, but I think the big winner here is the OSGi community at large.
ReplyDeleteGood Luck with S2AP (just add the prefix... ;o) )
@Adrian and @Niclas
ReplyDeleteI too, would urge S2 to add a prefix on their non-standard headers. I read Adrians response, I can see the points in there, and I *still* would like clearly spoken in the header name, that it is S2-specific.
It is not as if it is a new concept, to do this when "enhancing" a spec. It is just plain playing nice. Something S2 has a reputation for (playing nice), and I urge you to keep that reputation.
In short: Add the damned prefix to the non-standard headers! :-)