OSGi R7 Highlights: Java 9 Support

With the Java 9 release, the Java Platform Module System (JPMS) is a major feature that impacts many areas of the Java Platform.  The JPMS is used to modularize the class libraries provided by the Java Platform.  In addition, it has been proposed that the JPMS can be used by developers to modularize their own applications, although the OSGi Alliance cautions against its use here.

Meanwhile the OSGi Core specification has provided a stable platform for developing modular Java applications for well over 17 years.  The OSGi Core R7 specification continues this tradition by providing updates to the OSGi Core specification which are fully compatible with previous versions of the OSGi Core specification.  OSGi Core R7 Framework implementations will be able to run on top of Java 9 without requiring any changes to existing OSGi application bundles (assuming no internal packages from the Java platform are being used).

The OSGi Core R7 specification has a couple of new features that build upon some features new to Java 9.  This includes support for multi-release JARs and runtime discovery of packages provided by the JPMS modules loaded by the platform.  First I will tell you about the multi-release JAR support.

Multi-Release JAR Support

Java 9 introduces a new type of JAR called a multi-release JAR.  A multi-release JAR file allows for a single JAR to support multiple major versions of the Java platform. For example, a multi-release JAR file can depend on both the Java 8 and Java 9 major platform releases, where some class depend on APIs in Java 8 and other class depend on APIs in Java 9.

The purpose of multi-release JARs is to support alternative implementation of select classes to deal with changes in the visible APIs of the Java platform.  That is, it is not meant as a means to supply new function or new API on different Java platform versions. As a best practice new APIs should be released in an updated version of an existing bundle or released as part of a new bundle.

The OSGi Core R7 specification adds support for multi-release JAR files. An OSGi bundle file can be a multi-release JAR by adding the following manifest header to the bundle manifest:

 Multi-Release: true

Any JAR included on the bundle class path can also be independently declared as a multi-release JAR by including the same manifest header:

  Multi-Release: true

in its own JAR manifest. For example, an OSGi bundle can include third-party multi-release JARs on its own bundle classpath. In this case the third-party multi-release JARs will be loaded as multi-release JARs by the OSGi Framework even if the bundle JAR itself is not a multi-release JAR.

When a JAR on the bundle class path is a multi-release JAR, then the Framework must search the JAR's versioned directories when attempting to locate a class or resource in the JAR. For more information on how classes and resources are loaded from multi-release JARs see the multi-release JAR documentation.

Different implementations of a Java class for different versions of the Java platform can affect the requirements of the bundle.  For example, what packages need to be imported from the running Java platform. The R7 Framework supports supplemental manifest files that can be used to specify alternative values for the Import-Package and Require-Capability manifest headers for different versions of the Java platform.

When a bundle file is a multi-release JAR, then the Framework must also look for a supplemental manifest file, OSGI-INF/MANIFEST.MF, in the versioned directories. For example:

  META-INF/versions/9/OSGI-INF/MANIFEST.MF

The Framework finds the highest versioned manifest available in the multi-release JAR which can be applied to the running Java platform and uses the Import-Package and Require-Capability manifest headers as replacements for the values specified in the bundle's manifest.

The purpose of the supplemental manifest is to specify requirements on capabilities provided by different versions of the Java platform which are needed by the implementation classes for that Java platform version. As a best practice, the supplemental manifests should not contain additional requirements on capabilities which are not supplied by the Java platform for the Java version associated with the supplemental manifest.

For more information about multi-release JAR support you can refer to the following sections of the OSGi Core R7 specification:

1. Bundle File Multi-release JAR
2. Bundle class path Multi-release Container

Runtime Discovery of Java Platform Packages

Now that the Java platform is modularized in Java 9, the runtime can be configured to load only the modules which are required by the application.  This allows for a smaller custom runtime that is tailored to the needs of a specific application.  The set of java.* packages provide by the running Java platform is no longer constant for a specific version of the Java platform.

Bundles typically depend on different packages by using requirements specified with the Import-Package header, but as of the OSGi Core R4 specification, bundles have been prohibited from using Import-Package to specify requirements on java.* packages.  Instead bundles have used the osgi.ee namespace (or the deprecated Bundle-RequiredExecutionEnvironment header) to specify a dependency on specific versions of the Java platform. It has been assumed that the set of java.* packages available for a specific version of the Java platform is constant and therefore there is no need to specify additional requirements on specific java.* packages.

With Java 9 this is no longer a valid assumption.  With OSGi Core R7, the osgi.ee namespace should only be used to specify the minimal level of the Java platform required by the Java class bytecode level included in the bundle.  Dependencies on specific java.* packages should to be specified using the Import-Package header.

The OSGi Core R7 specification now allows bundles to use the Import-Package header to import java.* packages.  The OSGi Framework implementation is required to discover the all available java.* packages in the running Java platform and automatically provide the java.* package as a separate system packages (see the system packages configuration property).  The system bundle is the only bundle that is allowed to provide java.* packages. It is an error for normal bundles to attempt to export java.* packages using the Export-Package header.

When importing java.* packages only the package name should be specified, no other matching attributes are needed.  For example:

  Import-Package: java.sql

Note that bundle class loaders in R7 continue to have full visibility to all available java.* packages available in the running Java platform.  The class and resource loads for the java.* packages will continue to be boot delegated by the framework implementation even when the bundle does not specify any Import-Package header. The java.* package requirements are only necessary to prevent a bundle from resolving if the necessary java.* packages are not available at runtime.  For example, if a bundle must only resolve if the java.sql module is loaded.

For more information about java* package support you can refer to the following sections of the OSGi Core R7 specification:

1. Execution Environment
2. Parent Delegation 

Bundles Supporting R6 and R7 Frameworks

The OSGi Core R6 specification prohibits bundles from importing java.* packages and will throw a BundleException if such a bundle is attempted to be installed. Bundles that must work on OSGi R6 and earlier Frameworks but also want to run on R7 Frameworks have two options:

1. Do not import the java.* packages.  The bundle will continue to work on an R7 framework because java.* packages are still boot delegated by the bundle class loaders. This implies the bundle may resolve when running on Java 9 even when all the java.* packages required by the bundle are not available.
2. Package the bundle as a multi-release JAR to allow alternative values for Import-Package to be specified.

A bundle that must still be able to be installed on an OSGi R6 or earlier Framework must not specify java.* packages in their main bundle manifest Import-Package header. A supplemental manifest must also be included to be able to specify java.* package imports.

This supplemental manifest must contain the original Import-Package header from the main bundle manifest as well as any java.* package required by the bundle. When running on an OSGi R6 Framework, this supplemental manifest will be ignored and therefore the bundle will install successfully as the bundle did not specify an import for the java.* packages in the main manifest.

If a bundle will always require OSGi R7 or greater Framework, there is no need to use multi-release JARs for this purpose even when the bundle must support Java 8 or earlier. This is because OSGi R7 frameworks must provide the java.* package capabilities on all Java platform versions the framework implementation supports. As long as the java.* package is available on the running Java platform, the bundle with an import for the java.* package must resolve.

I advise using option 1 here because it is by far the most simple approach.  If you really have a strong need to use this new R7 feature then I recommend you do not try to produce a single bundle version that supports both R6 and R7 Frameworks.  Instead simply release a new version of your bundle that only supports OSGi R7 or greater Frameworks.

In Summary

The OSGi Core R7 release continues the tradition by providing a stable modular platform with updates to the OSGi Core specification which are fully compatible with previous versions of the OSGi Core specification.  OSGi bundle developers can continue to develop advanced modular applications and run their applications on Java 9 without having to commit to migrating to the green initial release of JPMS.

The OSGi Core R7 specification does add new functionality in support of new features available in Java 9, such as the support for multi-release JARs and importing of java.* packages.  The OSGi specifications will continue to evolve in order to provide the necessary tools for developing advanced modular applications.

OSGi R7 Highlights: Proposed Final Draft Now Available

I am pleased to announce that the OSGi Alliance has published the Proposed Final Drafts of the OSGi Core R7 and Compendium R7 specifications. We expect that the final versions of these specifications will be published in April 2018 after OSGi Alliance member approval.

The R7 release builds upon the long history of the OSGi Alliance’s leadership in Java modularity and reflects a significant amount of effort from the technical members of the OSGi Alliance expert groups over the last 2 years. Thanks go to all of the members who have contributed to this release.

R7 represents many significant new features and capabilities and provides an open standards-based approach for a number of modern valuable and simple-to-use technologies important to Java developers.

This blog post is the start of a series of blog posts from the technical experts at the OSGi Alliance to share some of the key highlights of R7. The blog posts in this series will come out over the coming weeks and cover the following topics:

• Java 9 Support – Multi-release JAR support and runtime discovery of the packages provided by the JPMS modules loaded by the platform.
• Declarative Services – Constructor injection and component property types.
• JAX-RS – A whiteboard model for building JAX-RS microservices.
• Converter – A package for object type conversion.
• Cluster Information – Support for using OSGi frameworks in clustered environments.
• Transaction Control – An OSGi model for transaction life cycle management.
• Http Whiteboard – Updates to the Http Whiteboard model.
• Push Streams and Promises – The Promises packages is updated with new methods and an improved implementation and the new Push Streams package provides a stream programming model for asynchronously arriving events.
• Configurator and Configuration Admin – Configuration Admin is updated to support the new Configurator specification for delivering configuration data in bundles.
• LogService – A new logging API is added which supports logging levels and dynamic logging administration and a new Push Stream-based means of receiving log entries is also added.
• Bundle Annotations – Annotations that allow the developer to inform tooling on how to build bundles.
• CDI – Context and Dependency Injection support for OSGi developers.

Stay tuned and I hope you find the technical information in the blog post series useful to you as developers!

Join Us for OSGi in Ghent - Weds, Sept 20, 2017

The OSGi Alliance is holding its next technical Expert Group meetings in Ghent later this month.

On Weds, Sept 20, 2017 you can join us for an evening of OSGi. Organised in collaboration with BeJUG.

Registration is required as places are limited so be sure to sign up on the BeJUG meetup event page.

ACA-IT Solutions are kindly hosting the event and providing sandwiches while everyone assembles from 18.30 hrs.  And then the evening kicks off with a welcome from 19.00hrs.

The agenda for the rest of the evening is:

• 19.10 hrs - What's coming in the OSGi R7 Specification (David Bosschaert & Carsten Ziegeler)
• 19.50 hrs - OSGi Push Streams, scalable event processing - pushing the limits (Tim Ward)
• 20.30 hrs - OSGi; from amateur to invited contributor (Tim Verbelen)

There will also be opportunity to network and ask questions with your peers and OSGi Expert Group members.

The event is being held at ACA-IT Solutions Ghent office (Voorhavenlaan, 31/008, Ghent). ACA have been actively using OSGi for many years and support the community by committing to open source OSGi projects. Thanks also to BeJUG for their support in promoting the event.

We hope you can join us. Don't forget to REGISTER in advance.

New OSGi R7 spec drafts published

In the OSGi Enterprise and Core Platform Expert Groups we are working hard on the upcoming R7 specifications, which are planned to be released later this year. Recently the OSGi Alliance published the current drafts of some these specs.

In this blog post you can read a very brief summary of what is new, for all the further details look in the drafts.
In the Enterprise draft specification - new and changed are:

• Remote services (ch 100) - has been enhanced to include new standard intents that require minimum remote service capabilities, asynchronous processing or a private remote connection.
• Declarative Services (ch 112) - this spec has received a substantial update, including delayed object reclamation, component property type enhancements and other updates.
• JPA Service (ch 127) - has been updated to support JPA 2.1, supports standard JPA 2.1 configuration properties with the EntityManagerFactoryBuilder and now specifies a number of standard Capabilities to integrate better with the Resolver.
• new Transaction Control Service (ch 147) - this new specification provides an improved mechanism to perform work in a transaction scope. It provides a higher level of abstraction than the existing JTA integration specification, and makes it really easy to write transactional code, especially when using Java 8 lambdas.
• new Configurator Specification (ch 150) - the configurator makes it possible to store configuration data for Configuration Admin driven entities in a bundle. The configuration is stored in a spec-defined format inside the bundle and can be used to configure any number of bundles, components, services or other entities via the Configuration Admin Service.
• Promises specification (ch 705) - this spec is now updated with a number of improvements including timeouts, easier exception handling and callbacks.
• new Push Streams (ch 706) -  This new spec provides a programming model similar to Java 8 pull-based streams, but then using a push model. This can be very useful for data such as events that arrives asynchronously. The data can be mapped, buffered, splitted, filtered or otherwise processed before it gets pushed to the receiver. 
• new Converter (ch 707) - This spec defines a portable converter that can convert nearly everything to nearly anything. Especially useful then converting untyped data such as a Map to a typed API such as an interface, DTO or annotation. Converters can also be customized to support user-defined types or to create deviations from standard behavior.

The Core Specification has also been updated. Besides a number of small changes and improvements - see the Changes sections in each chapter for details - the Core spec now defines new Annotations in the org.osgi.annotation.bundle package that can be used to define bundle headers from inside the Java code of your bundle. For example, these can be used to declare requirements and capabilities that your bundle could have or to declare which packages should be exported.

The R7 March 2017 drafts for the Core and Enterprise specs can now be downloaded from here: https://www.osgi.org/developer/specifications/drafts

Finally, not yet in the draft, but coming up as well are:

• Cluster Information Specification (previously known as Cloud Ecosystems), from RFC 183.
• CDI integration, you can read more about this in RFC 193.
• JAXRS integration, see RFC 217.
• HTTP Whiteboard Extensions, RFC 223.
• Resolver Service updates: RFC 224.
• Configuration Admin updates, RFC 227.

OSGi API Snapshots are Live!

Sometime back, OSGi began to make all design document (RFC) drafts available in real time from the OSGi Design GitHub repository. In furtherance of providing real time access to the latest design work at OSGi, OSGi made a change to the OSGi build process. Previously, the various OSGi specification's API code had only been published by the Alliance once a specification release has been declared final. Since mid February, however, the OSGi build has been publishing API snapshots to the public https://oss.sonatype.org/content/repositories/osgi/ group repository! Many thanks to BJ Hargrave for making this happen.

So why should you care?

Broadly speaking, the people who care about the pre-release OSGi APIs fall into two sets:

1. The people who want to try out the new draft APIs in their applications.
2. The people who want to implement the new draft APIs.

At first glance, it looks like the new release policy only helps the second group - after all it saves them from some issues

• How do I keep up to date with the OSGi RFCs?
• Is there a way to avoid duplicating the OSGi draft API source code?

But actually there are two important ways that publishing snapshots directly helps users:

1. By making it easier for implementors to keep up to date with RFCs, users get faster access to new implementations of the draft API.
2. For implementation code published by the OSGi Alliance, you get access to the draft code right now!

A good example of this would be OSGi Push Streams. You may have already watched the talk I gave in Portland about them, and been wondering when you could try them out. The answer is right now, and the same is true for the new 1.1 version of OSGi Promises!

Helping implementors

Open Source implementations of OSGi specifications are already moving to use the new draft API snapshots from OSGi (for example the Aries JAX-RS whiteboard). In most cases the API can be simply referenced from the OSGi group repository and no further action is needed.

One thing worthy of note is that the latest OSGi APIs are all compiled using Java 8. For some implementations, this is a problem as they wish to include the API inside their bundle but also to be usable on older versions of Java. In these cases, implementors may wish to try using a tool such as the Retrolambda project, so that they can still benefit from the published snapshots.

We hope that you enjoy the chance to play with all the cool new specifications!

OSGi R7 early spec drafts available

Recently OSGi published the first early draft for the OSGi R7. While not all specs aimed for R7 are present yet, a few new ones have appeared that can be checked out. The new ones are:

• Chapter 147 - Transaction Control Service. This specification provides an improved mechanism to perform work in a transaction scope. It provides a higher level of abstraction than the extising JTA integration specification, which makes it really easy to write transactional code, especially when using Java 8 lambdas.
• Chapter 148 - Converter. Convert anything into everything, and back. A universal object converter which can be used to convert simple values between datatypes, but it can also be used to convert complex structures such as maps or DTOs to interfaces or annotations. Previously functionality such as this was already supported in DS, which allows you to access a component's configuration map via an annotation. Now, this can be used anywhere you like...
• Chapter 706 - Push Streams. These provide a programming model similar to Java 8 pull-based streams, but then using a push model. Useful for data such as events that arrives asynchronously. The data can be mapped, buffered, splitted, filtered or otherwise processed before it gets sent to the receiver. 

You can find the R7 early drafts at the OSGi website here: https://www.osgi.org/developer/specifications/drafts/

Opensource implementations are already starting to appear, check https://en.wikipedia.org/wiki/OSGi_Specification_Implementations for where to obtain OSGi specification implementations.

As always, these are early drafts. Things will definitely change in some of the details. If you want to learn more about these and other upcoming OSGi specifications, come and visit the OSGi Community Event at EclipseCon Europe in Ludwigsburg https://www.osgi.org/2016-osgi-community-event.

New OSGi Early Access Drafts available

Earlier this week a new OSGi Early Access Draft was made available. A lot of new work is currently underway in both the Enterprise Expert Group (EEG) as well as the Core Platform Expert Group (CPEG) and in this document you can see where things currently are. Below you can find a short summary of each of the RFCs to get an idea what they are about.

RFC 180 - Portable Java Contracts. This RFC is about how you import technologies that originated in the JCP and specifically how these are versioned. A previous version of this RFC was included in the last EA draft and you can find more information on what RFC 180 is about in this posting:
http://osgithoughts.blogspot.com/2012/10/new-osgi-eeg-early-access-draft.html

RFC 182 - Defines a REST Management Interface for OSGi Frameworks. A number of remote management specifications already exist for OSGi, but especially in the context of Cloud Computing a REST-based management API is highly desirable. This is defined in RFC 182.

RFC 183 - Cloud Ecosystems. This RFC defines how a number of OSGi frameworks can work together in a Cloud Computing context, or other environment where multiple frameworks collaborate together. It describes how OSGi frameworks and their associated node-related meta-data can be discovered within the Ecosystem. It also covers how these frameworks can communicate across nodes to provide a fluid environment where a logical application is spread across multiple nodes and its components and services can easily be moved from one VM to another to deal with changes in machine load, available resources and other constraints.

RFC 184 - Blueprint 1.1. This RFC defines a collection of updates to the Blueprint specification that include the ability to switch off damping, change the behavior of the grace period and other changes.

RFC 185 - Data Transfer Objects. OSGi has always provided mechanisms for remotely managing its technology, but as the number of specs and the number of management technologies both grew maintaining coverage across all specs for all management technologies became a challenge. Data Transfer Objects provide a means for an OSGi specification to provide information suitable for a management agent in a technology-neutral style. The idea is that every OSGi spec defines its DTOs and that this can then be easily, possibly mechanically, mapped to most management technologies, so that they are kept up-to-date with a lot less effort than what was previously required. RFC 182 builds on top of these DTOs, other management specs (such as JMX) will most likely be updated in the future to take advantage of them too.

RFC 187 - Complex Repository Requirements. The Enterprise R5 specification defines the OSGi Repository spec. This was really the first Bundle Repository specification, although the Apache Felix project did have an OBR project before. The OSGi Repository is a simple yet powerful spec, which defines an API that allows one to find resources based on capabilities. RFC 187 adds a small enhancement to this specification which makes the querying mechanism even richer. It allows the combination of multiple capability namespaces into a single query so that you can express things like: I need a bundle that exports package org.foo and has license X. Or I need a bundle that provides the Declarative Services capability and has passed our company's internal QA process.

RFC 188 - Native Namespace. This specification adds a capability namespace for the generic OSGi Capabilities and Requirements model to represent the Bundle-NativeCode header. While other requirements in the Bundle Manifest, such as Import-Package or Fragment-Host can already be described using generic requirement namespaces, the Bundle-NativeCode did not have such a mapping yet. This RFC addresses this issue.

RFC 189 - HTTP Service Updates. The HTTP Service specification has been around for a long time and has seen many projects using it. However, it was long due an update to modernize it in relation to the Servlet Specification and the way people generally use OSGi Services. This RFC brings the HTTP Service spec up-to-date. It makes it possible to use Servlet Filters, adds the ability to use the Whiteboard pattern and provides additional configuration mechanisms.

RFC 191 - Weaving Hook Enhancements. This RFC enhances the Weaving Hooks allowing consumers to find out in what stage the weaving of a certain class is. In particular it helps the Subsystems specification to react appropriately to any imports that were added dynamically during the weaving phase.

RFC 193 - CDI Integration. This RFC focuses on how CDI beans can be deployed in OSGi and how these interact with the OSGi Service Registry. It brings annotation-based interaction with OSGi Services based on the popular CDI annotations. As with any OSGi component technology that uses the Service Registry, it will allow you to mix-and-match them. So a CDI bean can be injected by a service created in a Blueprint component. Similarly a simple Declarative Services client could obtain a service that was created in a CDI bean, and so on...

RFC 195 - Service Scopes. Up until now the OSGi Service Registry has supported two models. Either a registered service instance is shared across all consuming bundles or each distinct client bundle gets a separate instance. Recent work in the context of CDI and EJB integration (an OSGi/EJB RFC is being worked on but didn't make this draft) indicates that additional models are desirable. For example where a new service instance is returned every time the service is obtained, to fit stateful session models. This RFC describes additions to the OSGi Service Registry that support these models.

You can find the full documents in the EA draft at the following link: http://www.osgi.org/Download/File?url=/download/osgi-early-draft-2013-03.pdf

If you are interested in becoming involved in any of these RFCs, join our efforts: http://www.osgi.org/Join. If you already work for a member company, contact the EEG or CPEG chairs for more information on getting involved see http://www.osgi.org/EEG or http://www.osgi.org/CPEG.

Also note that some of these topics will be discussed at EclipseCon/OSGi DevCon Boston March 25-28. For more information see here: http://blog.osgi.org/2013/03/boston-calling.html

Core Release 5 and Enterprise Release 5 specifications

The OSGi Alliance has just published the recently approved Core Release 5 and Enterprise Release 5 specifications and made them available for download.

Some highlights from the new specifications include:

OSGi Core Release 5

• New Resource API for modeling generic capabilities and requirements.
• New Version Range class.

OSGi Enterprise Release 5

• New Repository Service Specification provides declarative access to artifact repositories based on the generic capabilities and requirements model. Where traditional repositories have typically provided artifacts based on their name, version and group, the OSGi Repository can provide artifacts based on capabilities, such as packages exported, services provided, extender functionality provided or custom-defined capabilities.
• New Resolver Service Specification. Based on the generic capabilities and requirements model, a management agent can use the Resolver service to compute the set of necessary resources needed to satisfy the given set of requirements. The Resolver is designed to work with the Repository Service, if available.
• New Subsystems Service Specification provides the ability to group multiple bundles into a single manageable entity, allows for complete isolation as well as various sharing models of code, services, and resources through a management agent. The Subsystem Service Specification defines an archive format to package multiple bundles, the Enterprise Subsystem Archive (.esa).
• New Service Loader Mediator Specification addresses common problems of bundles that rely on the java.util.ServiceLoader API to load custom Service Provider Implementations. It describes how to use the service registry for lookup of Service Providers as well as a solution for existing code to continue functioning using Service Loader API in a OSGi environment.
• New Common Namespaces Specification for use with the generic OSGi capabilities and requirements model.
• The Extender Namespace allows a bundle that requires an extender, such as Declarative Services or Blueprint, to express this dependency.
• The Contract Namespace provides a shorthand for many Import-­‐Package statements for technologies which span multiple packages.
• The Service Namespace allows a bundle to express that it provides or consumes a certain service.
• Updated JMX Management Model Specification.
• Object names now contain framework name and UUID, which allow multiple frameworks to be represented side-­by-­side.
• Updated the JMX API to reflect the latest Core API, specifically the bundle wiring API.
• Many improvements as requested by users, often focused on limiting the amount of data communicated via JMX APIs.
• Updated Configuration Admin Specification.
• Added targeted PIDs, which can be useful when configuring multiple versions of the same bundle through Configuration Admin.
• Added persistent change count to make it easier to detect changes.
• Added Synchronous Configuration Listener.

The specification PDFs, companion code jars and javadoc are now all available for download from the OSGi website.