OSGi Core Release 8 is now final and published

The specifications, reference implementations, and compliance tests for OSGi Core Release 8 have been approved as final by the members of the OSGi Alliance as one of the final actions of the OSGi Alliance as we transition the mission to the OSGi Working Group at the Eclipse Foundation.

You can find the online version of the Core Release 8 specification on the OSGi Specification Project Documentation website. A PDF version is also available for download. Javadoc is also available and the artifacts are now available on Maven Central.

As part of the mission transfer to the OSGi Working Group, the work will of course be done in open source projects. So I have recently submitted project proposals at Eclipse to form the OSGi Specification Project and the OSGi Technology Project. The OSGi Specification Project will be responsible for OSGi specification development and evolution and will take direction from the OSGi Working Group as a specification project under the Eclipse Foundation Specification Process. The OSGi Technology Project will host OSGi technology-related (non-specification) projects like the exciting osgi-test project we recently presented at EclipseCon 2020.

One of the immediate results of moving the mission to Eclipse is that we have now made the OSGi git repositories public. Check out the OSGi GitHub organization and, in particular, the OSGi Specification Project Build repository.

Finally, I would like to encourage everyone to consider joining the OSGi Working Group and get involved in continuing to evolve the OSGi specifications. Your company can join and individuals can also join as well. Work on Compendium Release 8 was well underway in the OSGi Alliance when we started the mission transfer to the OSGi Working Group at the Eclipse Foundation and we will get right back at it once we get the OSGi Working Group and OSGi Specification Project bootstrapped at Eclipse. I hope to see you there!

Announcement of Transition to Eclipse Foundation

It has been over twenty-one years since the OSGi Alliance was formed - sponsored by several major corporations such as Ericsson, IBM, Oracle, and Sun Microsystems - to create open specifications, reference implementations, and compliance tests to establish an architecture and programming model that was modular, dynamic, and service-oriented.  These standards were initially targeted at “Internet gateways”, what we would call today an Edge Computing device, but over the years would address a plethora of computing environments.

The world has changed a lot since 1999.  Oracle acquired Sun Microsystems, Ericsson is a much different company, and IBM and Oracle are no longer the biggest names in tech.    The world of software specifications development has also seen vast changes, being heavily impacted and influenced today by open source projects.

In 1999 open source projects were just starting to become recognized by major Information Technology providers.  The Apache Software Foundation was just getting started, the Linux operating system was beginning to be supported by computer hardware manufacturers, and it would be another 5 years before the Eclipse Foundation was established.

Now open source projects have become the primary source for open technology for software developers.  Even the OSGi Alliance depends on open source projects for the vast majority of its reference implementations.   A “code first” approach has taken hold for open standards too.  We can see examples of this at the Eclipse Foundation in their Jakarta EE platform project and at OASIS in their OASIS Open Projects.

Another thing has changed too, the OSGi Alliance no longer has the critical mass necessary to continue as a stand-alone organization.   

When this became evident to the OSGi Board of Directors, they began to examine and plan how best to deal with the future.  After a great deal of work examining every reasonable alternative it was decided the best course of action was to transfer the assets of the OSGi Alliance to the Eclipse Foundation, with the expectation that future specification development will continue there, and dissolve the OSGi Alliance.  The OSGi Board of Directors has reached an agreement with the Eclipse Foundation to establish an Eclipse OSGi Working Group which will be the home for the continued evolution of the OSGi specifications.  Current OSGi Alliance members and others that wish to participate will need to be members of the Eclipse Foundation itself, and also members of the Eclipse OSGi Working Group.   The Eclipse Foundation intends to waive the membership fees for the OSGi Working Group for the rest of 2020. 

To quote Mike Milinkovich, the Executive Director of the Eclipse Foundation, “The Eclipse Foundation is pleased to become the home for the future evolution of the OSGi's specifications.  Our working group model and the Eclipse Foundation Specification Process both provide a natural framework for the important work that continues under the OSGi banner, and our communities are both very comfortable with each other already. We thank the OSGi Alliance’s Board of Directors for their trust, and we look forward to working with OSGi Alliance’s members and developers as we continue down this path.”

I certainly hope that everyone will join the efforts at the Eclipse Foundation to continue the work that started 21 years ago.  

It has been a pleasure to serve you all these years and I look forward to continuing to work with you in the Eclipse OSGi Working Group.

Best wishes for the future,

Dan Bandera, President of the OSGi Alliance

------------------------------------------------------------------------------------------------

 

FAQ

Q: Why the Eclipse Foundation? 

A: The OSGi Alliance and the Eclipse Foundation share many of the same members, and we have had a long, synergistic relationship between our organizations.  

• The Eclipse Equinox project has hosted the OSGi Framework Reference Implementation for many years and many releases.  
• The Eclipse Foundation, like the OSGi Alliance, is a non-profit, vendor-neutral organization which simplifies the transfer of OSGi Alliance assets and mission to them.
• We have held joint EclipseCon / OSGi Community Event developer conferences for the past 8 years.    
• The Eclipse Foundation is home for many Java technology related open source projects like Jakarta EE, OpenJ9, OMR,  MicroProfile, Transformer, Vert.x and many more.
• The Eclipse Foundation has developed a robust specification process over the last two years which can produce specifications allowing for Intellectual Property grants and benefits very similar to those of OSGi Specifications.
• The Eclipse Foundation continues to thrive and grow which provides a safe home for the planned Eclipse OSGi Working Group.

Q: Where will we find the old OSGi specifications in the future ?

A: Currently published OSGi specifications are available at https://docs.osgi.org/. The planned Eclipse OSGi Working Group will decide if a different location is appropriate.

Q: For existing published OSGi Specifications what will be users/implementors rights be after the OSGi Alliance dissolves?

A: There will be no change in rights. The existing published OSGi Specifications will remain available under their existing license.

Q: What about reference implementations and compliance tests?

A: Almost all existing OSGi Reference Implementations are in open source projects at the Eclipse Foundation or the Apache Software Foundation (Eclipse Equinox, Apache Aries, Apache Felix, etc.) and should continue to be available directly from those projects.  Those that are not will most likely be made available in a GitHub repository sometime in the future. The planned Eclipse OSGi Working Group will make this decision.

Q: How will the Eclipse specification process differ from the OSGi specification process?

A: BIG Question! There will be changes. Please see the following resources (in the order listed):

• https://www.eclipse.org/community/eclipse_newsletter/2019/january/specification_process.php
• https://waynebeaton.wordpress.com/2019/03/08/eclipse-foundation-specification-process-step-by-step/ 
• https://www.eclipse.org/projects/efsp/ 

Q: Will new specifications from the Eclipse OSGi Working Group have a different license ?

A: It is expected that specifications released from the Eclipse OSGi Working Group will use the Eclipse Foundation Specification License. See https://www.eclipse.org/legal/efsl.php.

Q: How will this transition affect the development of Bnd/Bndtools?

A: This transition won't affect Bnd/Bndtools which is a separate open source project.

Q: What are the next steps? What should be done to participate in the Eclipse OSGi Working Group?

A: To participate in future OSGi specification development you will first need to be a member of the Eclipse Foundation itself. Subsequent to that, you will also need to be a member of the Eclipse OSGi Working Group.

Q: How can we join the Eclipse OSGi Working Group?

A: At this time, the Eclipse OSGi Working Group has yet to be formed. So right now, you can join the Eclipse Foundation itself and we will communicate when the Eclipse OSGi Working Group is ready to be joined.

Q: Who do I contact if I have further questions?

A: For OSGi Alliance questions, you can contact the OSGi Executive Director John Ehrig. For Eclipse Foundation questions, you can visit https://www.eclipse.org/membership/, or contact Eclipse Foundation VP Member Services Paul White.

OSGi Core R8 Proposed Final and Compendium R8 new Draft

Recently, the OSGi Alliance has published 2 new specification drafts with a lot of new content.

The Core R8 Proposed Final Draft is now available. New in Core:

• Condition Service Specification - the Condition Service generalizes what is sometimes referred to as the Ready Service: a potentially complex set of states that, when combined, indicate that the system has reached a certain point, such as being ready for use.
• Connect Specification - OSGi Connect specifies how OSGi works in environments where some of the class loading is provided by a non-OSGi entity, for example Java Modules, or Spring, or just a plain old classpath. Read more about Connect in this blog post.

Additionally there are a number of smaller changes in the Core spec, you can find them by looking at the Changes section at the end of each chapter.

The first Compendium R8 Draft has also been published. New chapters are:

• Service Layer API for oneM2M™ - This chapter specifies an integration with the oneM2M IoT platform.
• Condition Factory Specification - Adds a configurable layer to the Condition Service specified in Core R8.
• Typed Event Service Specification - A new eventing specification that supports strong data typing for events.

Some chapters have moved into Compendium that were only available in earlier Residential/IoT or Enterprise specifications. So while they appear for the first time in Compendium, these are not actually new specs. Also the new chapter 158 Log Stream Provider spec was previously part of chapter 101 Log Service. The Log Service chapter has been moved to the Core spec in R8.

While the Core R8 spec is technically completed, being a Proposed Final Draft, the Compendium is only in its first draft, so the content of the Compendium R8 spec can still change significantly. 

These drafts are available in HTML and PDF formats here: https://docs.osgi.org/specification/#release-8

OSGi Core R8 Specification Draft Available - Get Connected

The OSGi Core Expert Group has released a draft of the upcoming OSGi Core Release 8 specification. The draft includes two new specifications for the Core Framework as well as a number of smaller improvements.

• OSGi Connect - Provides a mechanism to connect bundles in the Framework with content managed outside of the Framework itself
• OSGi Condition Service - Provide a mechanism to signal that a condition is satisfied at runtime

For this post, I will focus on the OSGi Connect specification included in chapter 60 of the draft specification. Karl Pauls previously blogged about OSGi Connect last year and now I would like to describe the progress made since that post for the draft specification.

A Bit of History

Over the last 20 years, the OSGi Framework has provided a solid foundation to develop modular software.  This foundation is built upon the various layers provided by the Core OSGi Framework itself.

At the bottom of the foundation is the module layer.  This layer provides the rules for how modules (or bundles) can share or hide the packages they include in their own bundle JAR.  The Framework provides the class loader implementation that enforces the rules of the module layer.  This includes a rich capabilities and requirements model and a resolver that wires up the requirements to the available capabilities in the Framework.  Once a bundle is installed and resolved it is then able to participate in the layers that are built on top of the module layer.

The lifecycle layer provides control over the activation of the bundles resolved in the Framework.  Activation provides an entry point for each bundle to allow them to interact with the Framework and other bundles installed in the Framework.  A bundle may include a bundle activator which allows them to execute code when they are activated.  The activation concept also allows for more powerful runtimes to be built on top that can use introspection of the bundle capabilities and provide a mechanism for enabling the capabilities of the bundle at runtime.  This concept is called the extender pattern.  Two good examples of the extender pattern are the OSGi Declarative Services specification and the CDI Integration specification.  These two examples provide powerful dependency injection runtimes for developers.

The service layer provides a dynamic, concise and consistent programming model for developers, simplifying the development and deployment of services by de-coupling the service's specification from its implementations. This model allows developers to bind to services only using their interface specifications. The selection of a specific implementation, optimized for a specific need or from a specific vendor, can thus be deferred to runtime. The service layer model provides a common layer which allows for things like declarative service components and CDI components have dependencies on services available in the service registry.  This is powerful because it allows the components from declarative services and CDI to interact with each other through a shared layer without needing to know the details of each other’s runtimes.

The combination of the activation model with the service model in OSGi provides for an unparalleled platform for developing loosely coupled components that are highly configurable at runtime. In order to use this powerful tool, developers must be able to live within the confines of the OSGi Framework module layer. In some scenarios, it is challenging to live within the OSGi module layer as it is defined today. The following are a few examples:

• Modules built into a jlink image. The jlink tool was introduced in Java 9 along with the Java Platform Module System (JPMS) and provides a way to “right size” the JVM along with a set of application modules to provide an image that only includes what is required by an application’s modules. At runtime Java modules are typically loaded by class loaders provided by the JVM.  In the case of a jlink image all the modules are loaded by a single class loader.  This limits the ability to load bundle content out of the modules included in a jlink image.  Similar limitations exist with modules provided on the module path.
• Applications natively compiled using something like Graal Substrate.  With native-image compilation not only is it not possible to have a custom class loader, the classes at runtime are not really loaded at all.  Instead they simply exist with the execution of the native-image. This limits the ability to load bundles out of content compiled into the native-image.
• Bundles included on the class path or more generally loaded by some form of the URLClassLoader. The concept of the Java class path has been around since the early days of Java and many of today’s popular frameworks take advantage of the class path environment.  For example, Spring Boot loader mimics the class path when a Spring Boot application is packaged as a uber JAR (a JAR with many embedded JARs).  It has been challenging to integrate OSGi technologies into such environments.
• Other environments that compile Java into alternative deployment artifacts, such as Android, limit the ability of the framework to control the actual installation and deployment of new bundles.

The Idea to Connect

It would be helpful to allow content that lives outside of the module layer to participate in the lifecycle and service layers of OSGi. The new OSGi Connect specification introduces a mechanism that allows content managed outside of the Framework control to be represented (or connected) with a bundle installed in the Framework.  Because the content is managed outside of the Framework itself it may not follow all the rules of the OSGi module layer. For example, multiple bundles connected to outside content may be loaded by the same class loader. The class loader loading the content may not provide the same isolation as the Framework managed class loaders. With Connect now that content has the ability to participate in the activation model as well as the service layer of the framework.

The Connect specification introduces a concept of a module connector. The module connector is called by the framework when the framework needs to access content of the bundle. The content of the bundle includes the following:

• A list of entries contained in the bundle and access to read from them.  For example, a declarative service component XML file, the bundle manifest, or any other resources included in the bundle.
• An optional map of bundle manifest headers. Typically bundle manifest headers are specified in the bundle’s META-INF/MANIFEST.MF entry, but for connect content the headers may come from an alternate source.
• An optional class loader for the bundle. If a class loader is provided then the framework must use it for the bundle and must not create a framework managed class loader.

In order to install connect bundles, the Framework must be created with a module connector instance (defined by the interface org.osgi.framework.connect.ModuleConnector). The module connector hooks into the initialization of the framework as well as the activation, allowing it to interact with the Framework lifecycle and service layers. More importantly the module connector hooks into the installation of bundles in the Framework.  When a bundle is installed into the Framework a location string and an optional input stream to the bundle content is provided by the installer.  When no input stream content is provided the Framework must determine what the content of the bundle is.  Typically the Framework implementation will attempt to convert the location string into a URL and load the bundle content from there.

When a module connector is used the Framework must first ask the module connector if it can provide content for the bundle location.  If the module connector can provide content then it supplies a connect module to the framework (defined by the interface org.osgi.framework.connect.ConnectModule). A connect module is then associated with the connect bundle installed in the framework.  For each revision of the connect bundle the connect module provides connect content to the Framework (defined by the interface org.osgi.framework.connect.ConnectContent). A connect content provides access to read entries out of the content, provides an optional class loader for the content and may provide the bundle manifest headers for the content.

To create a new Framework that uses a module connector a new framework factory has been introduced with the interface org.osgi.framework.connect.ConnectFrameworkFactory.  A Framework implementation that supports the Connect specification provides a ConnectFrameworkFactory just like the Framework provides the org.osgi.framework.launch.FrameworkFactory.  A launcher that is looking for a factory to create a Framework instance can use the Java ServiceLoader to load a ConnectFrameworkFactory implementation.  This factory can then be used to create a new Framework instance that uses a module connector instance.

Seeing it in Action

While developing the Connect specification, Karl Pauls (Apache Felix project lead) and myself (Tom Watson - Eclipse Equinox project lead) have been busy implementing the Connect specification in our respective OSGi Framework implementations. I have also been working on a project called Atomos that I used as a proof of concept to test out the ability of the Connect specification to connect bundles to various content from different environments, such as: a jlink image, Graal substrate, Spring Boot uber JAR and an Android Dexified JAR (still a work in progress).

With the Connect specification entering into the draft phase, Karl and I thought it would be a good idea to have my Atomos project contributed to the Apache Felix project.  This has been done and is now available in the GitHub repository https://github.com/apache/felix-atomos

Among other things, the Atomos project provides a runtime with a module connector and a launcher that can be used to easily launch a framework and a set of bundles contained on the class path or module path.  For example, to launch a framework with the Apache Felix Gogo console you could have the following directory containing the necessary JARs:

bundles/

bundles/atomos.osgi.framework-0.0.1-SNAPSHOT.jar

bundles/jline-3.14.0.jar

bundles/org.apache.felix.atomos.runtime-0.0.1-SNAPSHOT.jar

bundles/org.apache.felix.gogo.command-1.1.0.jar

bundles/org.apache.felix.gogo.jline-1.1.0.jar

bundles/org.apache.felix.gogo.runtime-1.1.2.jar

bundles/org.eclipse.osgi-3.16.0.tjwatson_osgiConnect11.jar

The org.apache.felix.atomos.runtime JAR is the Atomos runtime snapshot built from the felix-atomos GitHub repo.  It contains the Atomos module connector implementation and the AtomosLauncher class which discovers the framework implementation and launches it with the Atomos module connector. The org.eclipse.osgi JAR is a snapshot of the Equinox framework that implements the connect specification. The atomos.osgi.framework JAR is a Java module that acts as a facade for the Framework implementation so that the Atomos runtime module can require it instead of requiring directly the org.eclipse.osgi module.  This allows Atomos to work with other Framework implementations, such as Felix.  The atomos.osgi.framework JAR is only required if you are launching from the module path.

To launch Atomos with the Gogo console using the class path, the following Java command can be used:

java -cp "bundles/*" org.apache.felix.atomos.launch.AtomosLauncher

To launch using the module path instead the following can be used:

java -p bundles -m org.apache.felix.atomos.runtime

Both will give you a gogo console with the bundles, Framework implementation, and Atomos all loaded from the same class loader.  The Atomos runtime does support Java 8 when using the class path, but class path mode can also be used with Java 11. If you are using Java 11 or higher you will notice that a number of other bundles are installed from the modules included in the JVM.  Atomos discovers all the modules that got loaded and will map each of them as an OSGi connect bundle.  This includes not only the modules from the specified module path, but also the modules from the boot layer of the JVM.  It includes things like the java.base module.  For example, if you run the Gogo command “lb” you will see things like the following:

g! lb | grep java.base

   38|Active     |    1|java.base (11.0.6)|11.0.6

Atomos will read the module descriptors and generate an equivalent OSGi bundle manifest for it so that it can be represented as a connected bundle in the Framework.

At this point, you have a fully functional OSGi Framework instance.  You can even install other bundles dynamically.  Any additional bundles that you install will not be connect bundles because they will not have been included on the original class path or module path at launch.  That means they will have the typical Framework managed class loader and have all the expected behaviors of living in the OSGi module layer.

The Atomos project also has a number of examples that can be looked at in https://github.com/apache/felix-atomos/tree/master/atomos.examples. This includes a jlink, Spring loader and a few Graal Substrate examples.  The Spring Loader and substrate examples all include a version of the Apache Felix WebConsole and the substrate examples also include the Felix SCR (Declarative Services implementation) and a set of test bundles that have declarative service components.  On my laptop the substrate examples can be launched in an impressive 40 milliseconds.

We are also working on a Maven plugin in Atomos to make the configuration of a Graal substrate compilation easier and automatic.  Also in the works is the ability to produce something that can easily be used on Android.

I am excited about the upcoming OSGi Core R8 specification release with the Connect specification.  I think this will help enable the use of OSGi technologies in environments that were not previously easy to do and in some cases not possible.  Now go download the draft specification and read more details about the new Connect specification.

OSGi Alliance IoT Vision

The OSGi Alliance started more than 20 years ago with the mission to develop specifications for connected homes and buildings – something we now consider part of the Internet of Things. Over the last couple of months, we have worked on an update of our IoT vision that describes challenges and requirements, and how OSGi addresses these in the chaotic IoT standards landscape. The challenges and requirements are not necessarily all new, but the ever-growing IoT landscape now requires more than ever standards to ensure that IoT solutions are economically and technically sustainable, maintainable, agile and evolvable over many decades. The OSGi Alliance with its specification for standardized software modularity and life-cycle management, connectivity of IoT devices, device management and software provisioning have been industry proven for many years with millions of deployments. The IoT Expert Group will continue to work on new specifications such as interoperability with oneM2M and support for real-time environments.

Please review the OSGi Alliance IoT vision and give us your feedback. Should you feel that we are on the right track, please join us to help make sustainable IoT solutions a reality.

To Embed or Not To Embed Your API Packages

For many years the OSGi Alliance, like other Java Standards organisations, has been creating APIs as part of its specifications. Multiple communities have then been involved in providing implementations of those APIs and specifications (see https://en.wikipedia.org/wiki/OSGi_Specification_Implementations).

In Java, not just OSGi, to use a specification you need both the API and the code which provides the API implementation. As a convenience many OSGi bundles implementing an API also provide the API package(s) from the bundle via ‘Substitutable Exports’ (see https://docs.osgi.org/whitepaper/semantic-versioning/050-exporter-policy.html). This means that the API package is both exported as well as imported by the implementation bundle. Because the OSGi framework controls the classloading on the package level, when multiple bundles export the API package(s), the OSGi framework can decide which exporting bundle ultimately provides the API package(s) to each bundle importing the API, that is, to each API consumer. This is not possible in “vanilla” Java, where the first API package(s) found on the classpath will be used, potentially causing problems if multiple versions of the API are present.

The use of substitutable exports had historical advantages because it made the job of a management agent easier when trying to find a set of functional bundles to install into an OSGi framework. By providing the API packages that they implement, implementations make themselves easier to deploy. The management agent (or a human) does not also have to find a bundle providing the API to make an implementation resolve. As the OSGi specifications continued to evolve the process to  search for dependencies was replaced by a solution that uses the standard Resolver and Repository services. Due to the limited metadata available, the use of substitutable exports also had advantages here. If a client bundle imports an API package and that API package is provided by another bundle which also provides an implementation of the API then a resolver will automatically pull in the implementation bundle along with the API when resolving the set of bundles to install.  If the API is provided by a separate bundle from the implementation then the resolver needed some proprietary way to discover the requirement for the implementation, and then to discover the bundles that provide the implementation.  In modern OSGi the “resolution problem” can now be completely solved by using the generic capability and requirements functionality provided by the OSGi core specification.  Not only can bundles declare requirements on the API using the Import-Package header, but they can also declare requirements on an implementation using the Require-Capability header.  Similarly bundles can declare that they provide an implementation using the Provide-Capability header.

In the past there was another benefit to substitutably exporting your API, getting API package(s) for individual OSGi specifications used to be tricky. The main Jar file that had them available was typically the osgi-cmpn.jar which contained all the API packages for all the Compendium specs and this Jar was certainly not designed or intended to be used at runtime.  Fortunately this last problem is now resolved since OSGi Release 6 as individual specification API jars are now available in Maven Central for each specification and these jars can be used at runtime.

So while the approach for embedding API packages has served the OSGi community well for many years, it requires a runtime supporting type visibility encapsulation, such as an OSGi Framework, that is in control of the class loading to select the appropriate package provider at runtime. Recently, work has started at the OSGi Alliance to support operating in environments where the class loader may not be provided by an OSGi Framework, or where there is no class loader at all. See RFC 243 OSGi Connect and RFC 245 Resource Encoding For Java Modules. This could be when running with the Java Platform Module System (JPMS), running as part of another framework such as Spring Boot, or even in a AOT-compiled environment such as SubstrateVM. Many existing OSGi bundles can work in these environments, providing the dynamic OSGi service model and allowing users to continue using popular OSGi technologies such as Config Admin, Declarative Services, the Converter and many others. However the embedding of API packages defined in specifications becomes a problem if there are multiple bundles that contain and provide the packages, since in these contexts there is no OSGi classloader to select the proper packages and hide, through type visibility encapsulation, the unused copies of the packages.

Going Forward

Going forward, to support all environments for your bundles, you should consider not embedding API packages that are defined by OSGi specifications. The better approach is to declare them as dependencies using Import-Package manifest headers. Most build tools will do this automatically for you. At runtime, simply use the API runtime bundles provided by OSGi. If you do embed the API packages in an implementation bundle then be sure to only embed the API packages you directly implement.  In most all cases you should not embed the APIs that you use but do not implement.  For example, if you use the OSGi Log Service you should avoid embedding the org.osgi.service.log package unless your bundle is actually providing the implementation of the Log Service.