Tuesday, February 19, 2013

java.util.ServiceLoader in OSGi

When migrating existing projects to OSGi, generally the biggest issue is the modularization of the code. Non-modular projects might have built up dependencies over time that you weren't aware of, with the risk of creating a spaghetti-like dependency chain in the end. Migrating to OSGi modularity gets rid of that spaghetti. However there are sometimes other challenges, one of which can be getting java.util.ServiceLoader to work inside OSGi.

ServiceLoader was introduced in Java 6 to generalize the various forms of 'FactoryFinder' patterns that were in existence throughout the JRE. It provides a basic plug-in model where implementations of a class (subclasses) or an interface can advertise themselves via a file that has the name of that interface (or class) in the META-INF/services directory of a Jar file. When looking up implementations via ServiceLoader.load(someClass.class), ServiceLoader scans all the Jars visible to the Thread Context Classloader for the associated file in META-INF/services and then uses a zero-arg constructor to instantiate the classes found.

There are a number of issues with this for a modular environment. First of all, ServiceLoader is non-modular by design in that it expects visibility of all Jars in the system, as they can all potentially contain an implementation. Additionally, depending on the Thread Context Classloader is also a questionable design decision as that can only really work well in situations where there is only 1 classloader (e.g. the system classpath) or in situations where all threads are managed by a container. In OSGi this is clearly not the case, as OSGi bundles are free to create threads and an OSGi system typically contains many classloaders.
Note that ServiceLoader also has a load(Class, ClassLoader) API which does not suffer from the issues described above.
Still we want to be able to make existing libraries that rely on ServiceLoader work in OSGi. The OSGi Enterprise R5 specs contain the ServiceLoader Mediator specification which addresses this issue.

ServiceLoader provider
Let's first look at the provider side. An obvious thing to do with bundles that contain META-INF/services resources is to register them as OSGi Services so that OSGi-aware consumers can simply use them from the OSGi Service Registry. So you'll get that with an implementation of this spec. For example take the following provider bundle where the org.acme.MySpiImpl class implements the foo.bar.MySPI interface. In this bundle,

the META-INF/services/foo.bar.MySPI file contains:
  org.acme.MySpiImpl

META-INF/MANIFEST.MF contains:
  Require-Capability: osgi.extender;
    filter:="(osgi.extender=osgi.serviceloader.registrar)"
  Provide-Capability: osgi.serviceloader;
    osgi.serviceloader=foo.bar.MySPI

The ServiceLoader mediator is controlled through the OSGi Manifest via generic capabilities and requirements. This mechanism was introduced in the OSGi 4.3 core specification. Basically what the two lines in the manifest state is that a Service Loader registrar extender is required and that the bundle provides the osgi.serviceloader capability for foo.bar.MySPI.

Once the provider bundle has been extended by the ServiceLoader mediator, you can find its corresponding service present in the OSGi Service Registry:

  g! inspect capability service 8
  org.acme.provider.bundle [8] provides:
  -------------------------------------------
  service; foo.bar.MySPI with properties:
    service.id = 17
    serviceloader.mediator = 6

you can identify services that are registered this way by the fact that the serviceloader.mediator property is present.

The OSGi Service Registry provides a much richer service environment than the functionality provided by java.util.ServiceLoader as besides the ability to plug-in service implementations it supports a dynamic lifecycle (services can come and go) and it has its rich service lookup query mechanism - plus it is designed to work in a modular environment.

If you can change the client-side code that uses the provider to consume it from the OSGi Service Registry, you have all you need at this point.

ServiceLoader consumer
However, in some cases you may not have access to the client-side code that is using ServiceLoader.load(). Maybe you're using an existing library that was written in another project. Maybe you're using a commercial library that you can't modify. For these cases the Service Loader Mediator specification also describes a way to get existing ServiceLoader.load() consumer code to work. While implementations can implement this in any way they like, the Reference Implementation in Apache Aries (SPI Fly) makes this work by keeping track of the providers and then, using byte code instrumentation, set the Thread Context ClassLoader to provide visibility of the appropriate provider jars for the duration of the ServiceLoader.load() call in the consumer bundle. Here's an example:

The consumer may contain code similar to:
  ServiceLoader<MySPI> ldr = ServiceLoader.load(MySPI.class);
  for (MySPI spiObject : ldr) {
    spiObject.doit(); // invoke the SPI object
  }

In the META-INF/MANIFEST.MF, the consumer bundle has:
  Require-Capability: osgi.extender;
      filter:="(osgi.extender=osgi.serviceloader.processor)",
    osgi.serviceloader;
      filter:="(osgi.serviceloader=foo.bar.MySPI)";
      cardinality:=multiple

This may look like a mouthful, but it's basically two generic OSGi requirements stated in a single Require-Capability header. The consumer requires the osgi.serviceloader.processor extender. This extender is responsible for ensuring that the ServiceLoader.load() call has the Thread Context Classloader set. The consumer also requires the osgi.serviceloader capability for foo.bar.MySPI. This capability is provided by the provider bundle and effectively instructs the extender what to set the TCCL to.

As with the provider, the consumer needs to opt in to the process by specifying the requirement headers in the OSGi manifest, however you don't need to change the actual client side code. If you don't want to modify the original Jar file at all you can wrap it as-is in an OSGi bundle, and provide the additional Manifest header in the wrapped bundle. Or you can simply add the additional manifest header to the existing Jar, whichever you prefer.

The Service Loader Mediator specification helps with migrating existing libraries that use java.util.ServiceLoader to OSGi and is available since June 2012 as part of the Enterprise R5 release. The Reference Implementation at Apache Aries is now released at version 1.0. For the consumer side, Aries comes in 2 variants: one that uses static weaving: you run a command-line tool on the consumer jar which then produces a new jar that works in OSGi. The other variant uses dynamic weaving, which means that you can install your consumer bundle as-is and have it processed at run-time in the OSGi Framework. More details on the RI can be found here: http://aries.apache.org/modules/spi-fly.html

As with any OSGi Specification, you can always check this Wikipedia page for additional implementations: http://en.wikipedia.org/wiki/OSGi_Specification_Implementations

Thursday, February 14, 2013

New Server for OSGi

The OSGi Alliance had been using a dedicated server for various functions for many years. But it was getting to be too old to continue to be used. It was running RHEL 4 and I could not find a Java 7 binary for that and I was not interested in trying to compile OpenJDK from source!

So this past week, I migrated and upgraded the various services (osgi.org and osgiusers.org websites, bugzilla systems, hudson build system, mailman mail lists, git and subversion repositories, ldap user database) from the old server to a shiny new server. I even able to install Java 7.

Most people probably did not notice the migration as the websites were only down for about 10 minutes while the ip addresses were switched between servers.

So hopefully, the OSGi community can enjoy our new server for a number of years to come.

Tuesday, February 12, 2013

OSGi DevCon 2013 Registration Deadline & Workshop Announced

OSGi DevCon 2013 in Boston, MA is inching closer. If you haven't registered yet, then be sure to do so by close on Thursday this week (14 Feb) as thats the deadline for the Advanced discounted passes giving you a saving of $500.

Plus if you are from an OSGi Member company then be sure to use the OSGI coupon code when you register for an addition $100 off your pass. You can find further details on how to register on the OSGi DevCon 2013 home page.

OSGi & JavaScript Workshop Announced

We are pleased to announce an OSGi and JavaScript Workshop at OSGi DevCon 2013. This will be taking place from 13.30hrs to 15.30hrs on Thursday March 28. The workshop is free to attend and open to all (both conference and non-conference delegates), however places are strictly limited so be sure to REGISTER soon.

The workshop will be hosted by Simon Kaegi, from IBM, who has been working in this area for some time.  The workshop will include a short presentation looking at how OSGi concepts might map to JavaScript and will be followed by open discussion. The aim of the session will be to gather requirements and foster discussion around producing an OSGi Alliance RFP.

Further details about the workshop and how to register can be found here.  As ever if you have any questions you can reach the Program Committee by email.

Looking forward to seeing you in Boston next month.

OSGi DevCon 2013 Program Committee
BJ Hargrave & Mike Francis

Monday, February 11, 2013

RFP 153 Resource Management now publicly available

Resource Management is essential in embedded systems where you have to deal with keeping the system healthy within constraint resources. It can also be highly valuable in cloud environments where you need to decide when to dynamically scale up, down or fail-over, or to ensure that you are running within agreed SLAs. Resources to be monitored and managed may include CPU, memory, disk space, network bandwidth, service response times and more.

The OSGi Alliance is currently working on the definition of use cases and requirements for resource management in OSGi environments. The RFP is now publicly available and we invite you to send us your comments and questions before we finalize it and start working on the RFC. The RFP can be found at: RFP 153

Monday, January 21, 2013

Get help adding OSGi metadata to your Jars!

More and more Java projects are adding OSGi metadata to their Jar files. The benefits are clear, if you add OSGi metadata your Jar can also be used in the OSGi modular environment and be an OSGi Bundle. In many cases an ordinary Java Jar file can be an OSGi Bundle at the same time, if the appropriate OSGi metadata is added to its manifest.

So what does OSGi metadata look like? OSGi metadata is simply a number of extra headers in the Jar's META-INF/MANIFEST.MF file. While quite a large number of OSGi headers exist, the following would typically be the ones that you'd find in a library Jar file that provides OSGi metadata.
Bundle-ManifestVersion: 2
Bundle-SymbolicName: org.acme.mybundle
Bundle-Version: 1.4.1
Export-Package: org.acme.mycomponent;version="1.1.6";uses:="org.acme.tools"
Import-Package: org.acme.tools;version="[1.1, 2)"

Adding OSGi metadata to your project's Jar files can increase its user base, as OSGi-based projects can now also use it. However for some projects this might be a bit of a challenge, as not everyone is entirely familiar with the OSGi headers and especially properly versioning your bundles and packages requires some thought.

From today there is a new facility available to get help around OSGi metadata. If you are thinking of adding OSGi headers to your project but are unsure about the metadata, you can ask the OSGi community for help.

How to get help?
The OSGi bugzilla system now has a new 'product' called Metadata Advice. Anyone who is looking for help around OSGi metadata for their project can create a bug with their question at http://www.osgi.org/bugzilla/enter_bug.cgi

Metadata Advice is open to everyone who needs some help. There is no need to be an OSGi Alliance member. The nice thing about using Bugzilla for this is that each bug effectively gets its own associated mailing list.

This is a community-based system, help is provided on a voluteer basis. So if you are knowledgeable about OSGi, feel free to sign up for receiving help requests. To do this go to the Bugzilla 'Email Preferences' and watch a user called metadata.advice-inbox@www.osgi.org
When watching this user you will get notified when new requests for metadata help come in.

So - when in doubt about your OSGi metadata, create that bug and share your question with the community!
Already know all about OSGi? Then sign up to watch metadata.advice-inbox@www.osgi.org and help others with their metadata...

Happy New Year!

Friday, December 28, 2012

Register by Dec 31 to Save for OSGi DevCon 2013

The preparations for OSGi DevCon 2013 are well underway. Its taking place in less than 3 months between March 25 to 28, 2013 in Boston.


It is set to be an excellent event with a packed OSGi Program of 17 talks, 2 tutorials, a BOF and a Workshop over the 4 days. All attendees also get access to all of the EclipseCon 2013 sessions.

With a 40% reduction in registration fees compared with last year there has never been a better time to feast on the huge SmörgÃ¥sbord of content for learning new things and brushing up your skills. Not to forget the opportunity to network with your peers and many of the OSGi experts from around the world.

There are only a few days left to secure the best price for attending as the Early Bird price of $800 expires on New Years Eve.  And if you are an OSGi member you can benefit from an additional $100 saving.

So why delay? Sign up today and secure you place to join us for one of the key OSGi events of next year.

If you have any questions you can reach the Program Committee by email.

Happy Holidays.

OSGi DevCon 2013 Program Committee
BJ Hargrave & Mike Francis

Monday, December 10, 2012

The Growing OSGi Ecosystem

Let’s step back for a second to look at OSGi, the big picture. We’re all so busy facing both everyday and strategic challenges that we don’t always see how many we are, how much OSGi is being deployed and, as a result, how much the OSGi ecosystem is growing.

Industry conferences are an excellent illustration of ecosystem growth and OSGi value.

We just wrapped the OSGi Community Event 2012 last quarter.  The buzz around OSGi was very visible. The event was co-located with EclipseCon Europe and some talks drew 80 attendees, a healthy snapshot of the OSGi ecosystem.  The OSGi keynote by John Duimovich from IBM showed just how well-known OSGi technology is and how it is deployed in other communities. It was exciting to see business deployments, such as the QIVICON solution from Deutsche Telekom or Cisco’s product solution, not only show the benefits of OSGi technology, but also attract additional market players to become solution partners and, accordingly, grow the ecosystem. Also, the technical presentations received positive feedback and covered a range of markets, from enterprise, cloud, and embedded topics to native OSGi. An engaged audience and an inspiring BoF showcased the high interest in the various topics and the increasing interest in OSGi-based solutions.

Excitement about OSGi is expected at the OSGi Community Event, but it also builds at other events and shows the breadth of the OSGi ecosystem as it presents the value of OSGi. For instance, there were 14 sessions with OSGi technology at Java One 2012. Importantly, they weren’t all “OSGi sessions,” but OSGi is a part of so many efforts that it works its way into diverse talks.

There is an increasing understanding and recognition that OSGi is the right technology when it comes to reducing complexity with modularity, whether it’s for large-scale distributed systems or small, embedded applications.

In the Smart Home market, which touches both large-scale utilities as well as embedded applications, OSGi adoption is gaining speed. OSGi is deployed in a variety of Smart Home devices and portal-based solutions because it provides a dynamic programming model for all applications and the capability to integrate and enhance multiple devices in a networked environment at runtime. There is an entire ecosystem building around Smart Home and Smart Energy solutions – and OSGi is integral to more and more of these solutions. You can expect that several operators and their partners will commercially launch OSGi based solutions in 2013.

Key standardization organizations are joining forces at their industry members’ request to further speed up the process for such end-to-end solutions. An OSGi workshop in October launched coordinated efforts regarding the device abstraction layer and follow-up meetings and action plans have followed.  It’s moving fast because the market wants a standardized device abstraction layer solution as soon as possible – and OSGi is considered to be a key piece of the puzzle. 

OSGi is also an industry standard for enterprise application server providers, and is embraced by open source projects within the Apache and Eclipse communities. Enterprise adopters don’t necessarily promote their use of OSGi yet, but it could become a notable competitive differentiator, as it has in the Smart Home market.

This momentum across industries promotes and builds the case for both OSGi adoption and Alliance membership. Open source as well as commercial projects propel the development of tooling and fuel steady development of the OSGi ecosystem itself, including a broad variety of industry players from enterprise software companies, operators and utility providers to software providers, manufacturers of Customer Premise Equipment (CPE), white goods, SoC vendors, Independent Software Vendors (including portal and application vendors), automotive manufacturers, and telematics providers. The cross-industry OSGi ecosystem allows companies to discover new partners and to share the workload while enhancing their product portfolio and service offerings –even in the aftermarket. That’s why it’s beneficial to become part of it – a self-fulfilling prophecy.

Inspiring times -- when you take a step back to notice. Of course, there is always more work to do and more ideas of where we can go, but we know the cycle between specification work and adoption is shrinking while the ecosystem grows. And that is to all of our benefit.

Susan Schwarze
OSGi VP Marketing