Release 6 of OSGi Compendium, OSGi Enterprise and OSGi Residential specifications available

I am pleased to announce that the OSGi Expert Groups and the OSGi membership has approved Release 6 of the OSGi Compendium, OSGi Enterprise and OSGi Residential specifications. They are now available for download from OSGi at http://www.osgi.org/Specifications/HomePage. You can download the specification PDFs, the API JAR files, and also the javadoc.

For you maven users, I also released the API JAR files to Maven Central and Jcenter. The maven coordinates are org.osgi:osgi.cmpn:6.0.0, org.osgi:osgi.enterprise:6.0.0, and org.osgi:osgi.residential:6.0.0. (I just uploaded these, so it make take a bit of time for them to be visible.)

Now time for some vacation... :-)

Avoiding Class Loader Muck with the Weaver. Wonkish

We're quite busy to finalize OSGi enRoute (the 1.0 date is September 1) and one of the significant parts of OSGi enRoute is bnd(tools). A shameful secret of bnd is that it knows a lot about OSGi but its plugin system is actually not based on OSGi. Fifteen years ago when bnd started OSGi was deemed too heavy to run as a command line tool. Ah, the crimes we commit in the name of optimizations.

The bnd plugin system regularly makes me want to pull out my (by now grey) hair;  it confirms my believe that class loaders originated from Dante's ninth circle. Things that work correct without effort in an OSGi environment are yucky in a raw class loader environment. They work, but the solutions are surprisingly fragile without writing another OSGi.

That said, bndlib is used in several OSGi frameworks; it is a real bundle. As can be expected, OSGi is pretty obnoxious when it comes to class loader muck. (Google for OSGi and class loader problems.) The worst part is of course that OSGi is actually right in being obnoxious.

Over time bnd got expanded with several repository types, a resolver, and other plugins. In bndtools (the Eclipse version) we actually install these plugins as bundles but due to reason I will not explain here, they were not always used. So we needed to clean up this mess.

Initially I created a mucky class loader solution that would do the evil thing of searching a class name in all bundles, a.k.a. buddy loading. Evil because you are in lala land, the whole OSGi engine to make sure proper classes are aligned is utterly ignored. You also create wiring dependencies that are not visible to the OSGi framework and this can cause other nastiness. Yuck.

After making my hands dirty and getting it to work I discussed the solution with BJ Hargrave. He, however, came up with a very interesting solution: the Weaving Hook.

The Weaving Hook is normally used to do byte code weaving. I am not that crazy about weaving because it is hard to make it work in an open and shared environment like OSGi. However, in this case it was actually quite easy and useful. Let me elucidate you.

When you change the byte codes of the classes, you need to add imports for any new dependencies. However, nothing in the Weaving specification forces you to weave, you can also use it to just add imports any bundle you want.

So this is what we ended up doing. In bndtools (or any OSGi environment) we register a Weaving Hook service. The hook tracks bundles that can provide plugins (They opt-in with an attribute on an exported package) and then adds these packages to the bndlib bundle as new imports.

This solution worked the first time and provided us with all the OSGi goodies. For the OSGi Framework it looks like the bndlib bundle now imports the plugin bundles. No more class loader yuck!

Peter Kriens

Say Hello to the IoT Expert Group

IoT (the Internet of Things) is a hot topic in technology today, and those of you who follow OSGi will have noticed a number of IoT related events over the past year. 

The first IoT demo/hackathon took place at the OSGi Community Event in late 2014, and served as an excellent demonstration of what can already be achieved with OSGi specifications. More recently in 2015 the OSGi Alliance hosted a pair of public IoT requirements gathering meetings in Berlin and San Jose. Some of the requirements gathered at these events are already addressed by new specifications in the OSGi R6 release (Final drafts are available at http://www.osgi.org/Specifications/Drafts), however others are not completely addressed by any existing OSGi specification.

The interest generated by these meetings (representatives from 35 companies attended) has made clear that the Alliance's work with IoT has significant support, and so the OSGi Board has approved the creation of an IoT Expert Group within the Alliance specification process. The Expert Group charter is available at http://www.osgi.org/IOTEG/HomePage and, in summary, it states that the Expert Group will work to provide solutions for IoT use cases within the scope of an OSGi framework.

As interim chair of the IoT Expert Group it is my job to bootstrap the expert group membership, and to begin the process of turning IoT requirements into OSGi specifications. As part of this process the OSGi Alliance IoT Expert group would like to invite you all to a public IoT day at the next Expert Group meeting in Turin, Italy on Thursday 17^th September 2015 between 0930 and 1730. This will be a free public event, open to both OSGi members and non-members, but does require attendees to pre-register by contacting s.schwarze@prosyst.com.

Please join us if you can. We'll be discussing how you can get involved with the IoT Expert Group, whether that is providing requirements, writing the specifications or developing a reference implementation. We'll also spend much of the day gathering requirements, discussing their relative priorities, and seeing which OSGi specifications may already address them.

If you can't make it to Turin (or even if you can) then anyone can raise requirements using the OSGi bugzilla at https://osgi.org/bugzilla/ - Make sure you use the IoT RFP (RFP 174) as a reference for your “Document Feedback”. You can also follow the RFP (and any new IoT RFPs) at https://github.com/osgi/design. OSGi members can also join the IoT Expert Group mailing list to begin following the Expert Group's work.

I look forward to seeing you in Turin,

Tim Ward (Paremus)

IoT Expert Group Interim Chair

P.S.

I just wanted to add that while IoT is “the latest hot topic” it is nothing very new for OSGi. OSGi was originally created to support home automation devices, residential gateways and embedded systems. As a result of this OSGi already has most of the features needed for an IoT platform – no wonder the IoT demo was so successful! If you want proof of how long OSGi has been working in this area then take a look at this video...

New RFPs under discussion at OSGi

While the Enterprise R6 release is currently in the process of being released, new topics are being worked on in the OSGi Alliance. These topics are currently discussed in RFP documents, which hold the requirements. The plan is to finalize these RFPs by the end of the summer, then we can start on the RFCs, the technical design that will ultimately be the basis for future spec texts.

Below you can find a brief 'elevator pitch' of the currently active RFPs, as discussed in the Expert Groups. For details click on the links that will bring you to the actual text. All documents are available publicly in GitHub here: https://github.com/osgi/design/tree/master/rfps.

The following RFPs are discussed in the context of the Core Platform and Enterprise Expert Groups:

• RFP 163 Log Service Update - This RFP is about modernizing the OSGi Log Service and making is easier to use. 
• RFP 164 Authentication and RFP 165 Authorization - Are about providing APIs to integrate various security providers and mechanisms into OSGi for authentication and authorization.
• RFP 166 Scheduling - Provide a cron-like persistent scheduling API.
• RFP 167 Manifest Annotations - Adds annotations to generate OSGi Manifest Headers. In particular this will make generating Provide-Capability and Require-Capability headers easier.
• RFP 168 Configurer - This RFP is about providing a standard way to store OSGi Configuration Admin configuration inside a bundle.
• RFP 169 Object Conversion - The converter that converts anything into everything. This can be especially useful when working with configuration data or service properties.
• RFP 170 Managed JPA - Adds Managed JPA to the OSGi JPA integration.
• RFP 171 Web Resources - This RFP talks about a standard way to provide resource for the web. In particular it's about providing JavaScript libraries and CSS files such as Angular, Bootstrap, D3 etc from an OSGi bundle.
• RFP 172 Protocols - Create a REST service using OSGi annotations and conventions.
• RFP 173 JAX-RS - Integration with JAX-RS.

The following RFP relates to the Internet of Things:

• RFP 174 Initial IoT Requirements - Requirements for the upcoming IoT Expert Group.

Have any thoughts or feedback on these requirements documents? Anyone can post feedback as described at https://github.com/osgi/design#feedback.

Want more direct involvement? If you are interested in participating in shaping these specifications, or writing an entirely new RFP, then become a member. Details on joining can be found here: http://www.osgi.org/Join

OSGi Enterprise R6 specifications proposed final draft

The OSGi Enterprise R6 specs are now available as proposed final drafts: http://www.osgi.org/Specifications/Drafts
OSGi members are currently voting on these and once that is done the same text will be released as 'final' specification. Let's dive in to see what's new!

New specifications

• Promises - Promises are a popular asynchronous programming paradigm regularly used in the JavaScript world. The Promises specification brings a similar asynchronous programming model to Java. It allows you to define a chain of operations that are executed asynchronously, every next step being started once the previous one has completed. For example, let's say you have some code that needs to prepare a download, then find the appropriate mirror and then start downloading, all time consuming operations. This can nicely be modeled asynchronously using promises:
    Promise<InputStream> masterURL =
      prepareDownload().then(p -> getMirror(p.getValue()).
      then(p -> p.getValue().openStream());
  Promises integrate very well with Java 8 lambdas and support monadic programming. They can be used both inside as well as outside of an OSGi Framework.
• Asynchronous Services - This specification can turn any regular OSGi service into an asynchronous service. It leverages the Promises API as described above and supports turning ordinary services into async ones. It also provides support for developing new services that are specifically implemented to be asynchronous. 
• REST Management Service - This specification adds remote framework management via REST over HTTP, either using XML or JSON. This is particularly useful in cloud settings where traditional Java remote management via JMX does not work very well. The REST management service can either be invoked directly, or via the Java and JavaScript clients that are defined in the specification. The JavaScript client allows you to use the REST Management service directly from your JavaScript code in the browser. 
• HTTP Whiteboard Service - OSGi has provided a simple mechanism to register servlets for a long time via the HTTP Service specification. This specification, although widely used, has aged a little. The new HTTP Whiteboard specification supports Servlets, Filters, static resources and Http/Servlet listeners based off the Servlet 3.1 specification. It leverages the OSGi Whiteboard pattern (www.osgi.org/wiki/uploads/Links/whiteboard.pdf) to register them, which effectively means that if you register a Servlet or a Filter as a service, it will be made available by the whiteboard runtime. This specification also includes a wide range of DTOs to allow runtime introspection. 

Updated specifications

• Declarative Services - This specification has undergone extensive changes bringing several exciting enhancements: new annotations for field injection, a strong-typed mechanism based on annotations for providing configuration and DTOs for introspection. Declarative Services now also support Prototype Service Factories which were introduced in the Core R6 release. Numerous other small improvements were also made to DS. 
• Remote Service Admin - This specification has had minor updates which allow remote endpoints to be updated. In the past the only way to do this was to remove/re-add the endpoint but in practise this caused unnecessary volatility in the system, especially for discovery providers. With the update facility this volatility is now avoided.
• Repository - The OSGi Repository service now supports composite queries that span multiple namespaces. With this you can ask the repository questions like 'Is there a bundle that exports javax.mail and has the ASL license?' As this information is expressed in different capability namespaces, it's an example of a composite query. A requirement builder utility has also been added to by this specification to make the developer's life easier. 
• Subsystems - This specification has been updated to allow providing a Deployment Manifest at deploy time. No longer does it need to be embedded in the subsystem archive. Deployment manifests freeze the dependencies that a subsystem has to particular versions; for example the versions validated during a QA phase. With this release these manifests can be provided separately, without the need to modify the subsystem archive (.esa file). The subsystems specification also received a number of other updates. 

A number of smaller changes were made to other specifications. Look for the Changes sections at the back of each chapter for details.

Capability Namespaces

The Enterprise R5 specification provided a number of capability namespaces for general use: osgi.extender, osgi.service and osgi.contract. The Enterprise R6 specification adds the osgi.implementation capability. This capability is useful to ensure that a given technology is present. For example, if you want to ensure that the HTTP Whiteboard implementation is present in a given framework, just add a requirement of the osgi.implementation capability:

  Require-Capability: osgi.implementation;
  filter:="(&(osgi.implementation=osgi.http)(version:Version>=1)(!(version:Version>=2)))"

Note that if the implementation is an extender, then osgi.extender is used instead of osgi.implementation.
Specifications that have been updated in this round have these standard capabilities added, where this made sense.
In addition, Portable Java Contracts have been defined using the osgi.contract namespace to achieve portable bundles for JSR-based java packages that don't follow semantic versioning. This allows for portable consumer bundles of these packages. See here: http://blog.osgi.org/2014/09/portable-java-contracts-for-javax.html and http://www.osgi.org/Specifications/ReferenceContract.

Enjoy, take a look at all this good stuff. The PFD spec is available here: http://www.osgi.org/Specifications/Drafts. A number of (near) finished implementation are available, you can find them listed on the OSGi Specification Implementation wikipedia page: http://en.wikipedia.org/wiki/OSGi_Specification_Implementations

David Bosschaert
Raymond Augé

OSGi Enterprise Expert Group co-chairs

IoT Requirements Meeting Berlin & San Jose

Last week Thursday we had an OSGi IoT Requirements day in Berlin and next week Thursday (June 4) we will have another workshop in San Jose.

Now I always like to go to Berlin (I worked there part-time for Deutsche Telekom for about 3 years) but this time was even more special. The reason was the excitement that made me fondly remember the early days of the OSGi. The more than 30 people that signed up for the requirements day represented the industry very well and it was good to see some faces that had disappeared over time but that are now interested again.

A certain 'déja vu' was definitely present. Then again, the differences with those early days are huge. Today we have a mature proven specification, many implementations, books, and so much more. We also now face an industry where Java is no longer seen as overweight except for the smallest of devices. However, the field today is also more crowded. An impressive number of organizations have sprung up around IoT or are taken advantage of the energy in the industry. Over the past few weeks I had moved back to the future a bit by extending OSGi enRoute with some IoT support at the request of the Alliance's board. (Watch this blog if you want to use that dusty Raspberry Pi that has been lying on your shelf for the past 3 years.) I confess, I find that writing software that actually moves the world more fun than being locked up in cyberspace. And best of all, in this world they the dynamic nature of OSGi is a prerequisite.

Anyway. The workshop turned out to be a very interesting day with some really good results. We gathered quite a few topics and their requirements and had some interesting discussions. After the next workshop in San Jose (contact us on iot@osgi.org if you still want to participate, there are a few seats left) we will disseminate the results through a mailing list that we will open up after San Jose.  I will also use this blog to report about progress.

Let's see what San Jose brings!

Peter Kriens

Yet Another Maven Bundle Plugin!

It must have been 8 years ago that Richard S. Hall visited me here in the south of France. During these days we hacked together a rudimentary plugin for Maven so that Richard could use bnd's functionality in his maven build. This was where we laid the foundation for the "funky" XML that described the bundle. As this was our first maven plugin, we had no clue what we were doing but it seemed to work for Richard. It is now 19 releases later. Surprisingly, the Apache Felix bundle plugin is one of the most popular downloads outside the standard Sonatype maven plugins. 

That said, the plugin was showing some strains that were not easy to correct in an existing plugin for backward compatibility reasons. Therefore, last week Neil Bartlett published a new variation on this plugin in the bndtools project.  This plugin fits better in the maven model, instead of taking over the JAR process it now only outputs the contents of the JAR in the classes folder. This fitted better when you had later phases that further processed the classes directory. Neil Bartlett, the author of the plugin describes the process in detail in his blog. Since we receive a lot of pressure to be more maven like, it is not unlikely that we will add more features to this plugin and bndtools to make them interoperate more closely.

Inspired by this new maven plugin, BJ Hargrave, added similar support to the Gradle plugin, making it easier for native Gradle builds to use the standard bnd plugin. Similar support was already available in ant.

The new plugins will be part of the bndtools build so that we can release the plugins synchronized with the new features in bnd so that we can publish them timely on Maven Central.

This does not mean that the older plugin will be disbanded any time soon. Since it is so popular and has some behaviors that are different than the new plugin we will continue provide support for the Apache Felix group to keep publishing this plugin.

Peter Kriens