Monday, February 25, 2013

New OSGi Tool Chain Documentation

People sometimes say that OSGi is hard to use because tooling support is lacking. Well, there are actually quite a lot of OSGi-related tools out there, but many of them address a specific area. They often work together with other tools to address a larger context. A modular approach to tooling, if you like... One of the issues with this approach was however that it could be hard to find out how these tools are working together to address a real life scenario. An example scenario would be how to set up an entire development environment that includes a headless build, and IDE with debugging facilities and a testing environment. Documentation describing how to do all this using a combination of existing tools was not easily available.

Over the past months a number of people have worked together on creating a set of documentation pages that describe various OSGi toolchains: how a number of tools can be combined to create an end-to-end solution. For example, it describes how Bndtools can be used with Ant and Eclipse to create a comprehensive development setup. But there are more options. Not everyone uses Bndtools, other people might use Tycho with Eclipse and Maven. This is also documented. As is how to use Maven with Bnd, Eclipse and Pax-Exam. 
Additionally there is a page describing how to set up a Scala build environment for OSGi using SBT.

All the details of how to get going are described on a new set of pages focusing on documenting OSGi tool chains on the OSGi wiki. You can find them here: http://wiki.osgi.org/wiki/ToolChains

The pages that are there now are just a starting point. They should get you going with a number of OSGi tooling technologies. As most of the pages use a similar example it is also possible to compare the various approaches. However, there is always room for improvement. As this is an open environment feel free to contribute. If you are using a different toolset, document it! Or if you have anything to improve the descriptions add it in!

http://wiki.osgi.org/wiki/ToolChains

Tuesday, February 19, 2013

java.util.ServiceLoader in OSGi

When migrating existing projects to OSGi, generally the biggest issue is the modularization of the code. Non-modular projects might have built up dependencies over time that you weren't aware of, with the risk of creating a spaghetti-like dependency chain in the end. Migrating to OSGi modularity gets rid of that spaghetti. However there are sometimes other challenges, one of which can be getting java.util.ServiceLoader to work inside OSGi.

ServiceLoader was introduced in Java 6 to generalize the various forms of 'FactoryFinder' patterns that were in existence throughout the JRE. It provides a basic plug-in model where implementations of a class (subclasses) or an interface can advertise themselves via a file that has the name of that interface (or class) in the META-INF/services directory of a Jar file. When looking up implementations via ServiceLoader.load(someClass.class), ServiceLoader scans all the Jars visible to the Thread Context Classloader for the associated file in META-INF/services and then uses a zero-arg constructor to instantiate the classes found.

There are a number of issues with this for a modular environment. First of all, ServiceLoader is non-modular by design in that it expects visibility of all Jars in the system, as they can all potentially contain an implementation. Additionally, depending on the Thread Context Classloader is also a questionable design decision as that can only really work well in situations where there is only 1 classloader (e.g. the system classpath) or in situations where all threads are managed by a container. In OSGi this is clearly not the case, as OSGi bundles are free to create threads and an OSGi system typically contains many classloaders.
Note that ServiceLoader also has a load(Class, ClassLoader) API which does not suffer from the issues described above.
Still we want to be able to make existing libraries that rely on ServiceLoader work in OSGi. The OSGi Enterprise R5 specs contain the ServiceLoader Mediator specification which addresses this issue.

ServiceLoader provider
Let's first look at the provider side. An obvious thing to do with bundles that contain META-INF/services resources is to register them as OSGi Services so that OSGi-aware consumers can simply use them from the OSGi Service Registry. So you'll get that with an implementation of this spec. For example take the following provider bundle where the org.acme.MySpiImpl class implements the foo.bar.MySPI interface. In this bundle,

the META-INF/services/foo.bar.MySPI file contains:
  org.acme.MySpiImpl

META-INF/MANIFEST.MF contains:
  Require-Capability: osgi.extender;
    filter:="(osgi.extender=osgi.serviceloader.registrar)"
  Provide-Capability: osgi.serviceloader;
    osgi.serviceloader=foo.bar.MySPI

The ServiceLoader mediator is controlled through the OSGi Manifest via generic capabilities and requirements. This mechanism was introduced in the OSGi 4.3 core specification. Basically what the two lines in the manifest state is that a Service Loader registrar extender is required and that the bundle provides the osgi.serviceloader capability for foo.bar.MySPI.

Once the provider bundle has been extended by the ServiceLoader mediator, you can find its corresponding service present in the OSGi Service Registry:

  g! inspect capability service 8
  org.acme.provider.bundle [8] provides:
  -------------------------------------------
  service; foo.bar.MySPI with properties:
    service.id = 17
    serviceloader.mediator = 6

you can identify services that are registered this way by the fact that the serviceloader.mediator property is present.

The OSGi Service Registry provides a much richer service environment than the functionality provided by java.util.ServiceLoader as besides the ability to plug-in service implementations it supports a dynamic lifecycle (services can come and go) and it has its rich service lookup query mechanism - plus it is designed to work in a modular environment.

If you can change the client-side code that uses the provider to consume it from the OSGi Service Registry, you have all you need at this point.

ServiceLoader consumer
However, in some cases you may not have access to the client-side code that is using ServiceLoader.load(). Maybe you're using an existing library that was written in another project. Maybe you're using a commercial library that you can't modify. For these cases the Service Loader Mediator specification also describes a way to get existing ServiceLoader.load() consumer code to work. While implementations can implement this in any way they like, the Reference Implementation in Apache Aries (SPI Fly) makes this work by keeping track of the providers and then, using byte code instrumentation, set the Thread Context ClassLoader to provide visibility of the appropriate provider jars for the duration of the ServiceLoader.load() call in the consumer bundle. Here's an example:

The consumer may contain code similar to:
  ServiceLoader<MySPI> ldr = ServiceLoader.load(MySPI.class);
  for (MySPI spiObject : ldr) {
    spiObject.doit(); // invoke the SPI object
  }

In the META-INF/MANIFEST.MF, the consumer bundle has:
  Require-Capability: osgi.extender;
      filter:="(osgi.extender=osgi.serviceloader.processor)",
    osgi.serviceloader;
      filter:="(osgi.serviceloader=foo.bar.MySPI)";
      cardinality:=multiple

This may look like a mouthful, but it's basically two generic OSGi requirements stated in a single Require-Capability header. The consumer requires the osgi.serviceloader.processor extender. This extender is responsible for ensuring that the ServiceLoader.load() call has the Thread Context Classloader set. The consumer also requires the osgi.serviceloader capability for foo.bar.MySPI. This capability is provided by the provider bundle and effectively instructs the extender what to set the TCCL to.

As with the provider, the consumer needs to opt in to the process by specifying the requirement headers in the OSGi manifest, however you don't need to change the actual client side code. If you don't want to modify the original Jar file at all you can wrap it as-is in an OSGi bundle, and provide the additional Manifest header in the wrapped bundle. Or you can simply add the additional manifest header to the existing Jar, whichever you prefer.

The Service Loader Mediator specification helps with migrating existing libraries that use java.util.ServiceLoader to OSGi and is available since June 2012 as part of the Enterprise R5 release. The Reference Implementation at Apache Aries is now released at version 1.0. For the consumer side, Aries comes in 2 variants: one that uses static weaving: you run a command-line tool on the consumer jar which then produces a new jar that works in OSGi. The other variant uses dynamic weaving, which means that you can install your consumer bundle as-is and have it processed at run-time in the OSGi Framework. More details on the RI can be found here: http://aries.apache.org/modules/spi-fly.html

As with any OSGi Specification, you can always check this Wikipedia page for additional implementations: http://en.wikipedia.org/wiki/OSGi_Specification_Implementations

Thursday, February 14, 2013

New Server for OSGi

The OSGi Alliance had been using a dedicated server for various functions for many years. But it was getting to be too old to continue to be used. It was running RHEL 4 and I could not find a Java 7 binary for that and I was not interested in trying to compile OpenJDK from source!

So this past week, I migrated and upgraded the various services (osgi.org and osgiusers.org websites, bugzilla systems, hudson build system, mailman mail lists, git and subversion repositories, ldap user database) from the old server to a shiny new server. I even able to install Java 7.

Most people probably did not notice the migration as the websites were only down for about 10 minutes while the ip addresses were switched between servers.

So hopefully, the OSGi community can enjoy our new server for a number of years to come.

Tuesday, February 12, 2013

OSGi DevCon 2013 Registration Deadline & Workshop Announced

OSGi DevCon 2013 in Boston, MA is inching closer. If you haven't registered yet, then be sure to do so by close on Thursday this week (14 Feb) as thats the deadline for the Advanced discounted passes giving you a saving of $500.

Plus if you are from an OSGi Member company then be sure to use the OSGI coupon code when you register for an addition $100 off your pass. You can find further details on how to register on the OSGi DevCon 2013 home page.

OSGi & JavaScript Workshop Announced

We are pleased to announce an OSGi and JavaScript Workshop at OSGi DevCon 2013. This will be taking place from 13.30hrs to 15.30hrs on Thursday March 28. The workshop is free to attend and open to all (both conference and non-conference delegates), however places are strictly limited so be sure to REGISTER soon.

The workshop will be hosted by Simon Kaegi, from IBM, who has been working in this area for some time.  The workshop will include a short presentation looking at how OSGi concepts might map to JavaScript and will be followed by open discussion. The aim of the session will be to gather requirements and foster discussion around producing an OSGi Alliance RFP.

Further details about the workshop and how to register can be found here.  As ever if you have any questions you can reach the Program Committee by email.

Looking forward to seeing you in Boston next month.

OSGi DevCon 2013 Program Committee
BJ Hargrave & Mike Francis

Monday, February 11, 2013

RFP 153 Resource Management now publicly available

Resource Management is essential in embedded systems where you have to deal with keeping the system healthy within constraint resources. It can also be highly valuable in cloud environments where you need to decide when to dynamically scale up, down or fail-over, or to ensure that you are running within agreed SLAs. Resources to be monitored and managed may include CPU, memory, disk space, network bandwidth, service response times and more.

The OSGi Alliance is currently working on the definition of use cases and requirements for resource management in OSGi environments. The RFP is now publicly available and we invite you to send us your comments and questions before we finalize it and start working on the RFC. The RFP can be found at: RFP 153