OSGi R7 Highlights: Cluster Information Specification

In my experience, one of the most powerful parts of the OSGi specification are the Remote Services and Remote Service Admin, which enable you to develop complex, distributed applications in a modular way. However, in order to deploy such a distributed application, it is key to find out on which devices bundles can be deployed. In previous releases, OSGi specifications for remote management are already available for certain domains (i.e. telecommunications with TR-069 Connector Service) or protocols (i.e., REST Management Service).

With OSGi R7, we now introduce the Cluster Information specification, a protocol-agnostic specification to discover, list and inspect the available devices in a distributed compute environment, and to provision these devices with OSGi bundles.

A Cluster of Nodes

At the heart of the cluster information specification, is the NodeStatus service, which is used to indicate the presence of a node within the cluster. This could be any entity, such as a virtual machine, a docker container, a physical machine, a database, your Raspberry Pi, etc. Each node has a globally unique identifier, a name of the cluster it belongs to, and optionally a number of other properties such as the endpoint at which the node can be accessed, the physical location where this node is located or some application specific tags. These properties are available as service properties on the Node Status service. For a full list of properties, take a look at the specification.

A special case of node is, of course, an OSGi framework, in which case we use the FrameworkNodeStatus as presence service, which extends NodeStatus. In addition to the Node Status service properties, this one also provides some OSGi-specific properties, such as the framework version, Java version, etc. Also, when an OSGi framework is part of a cluster, this means it gets access to remote services of any other OSGi framework in that cluster. Ensuring the discovery, visibility and access of remote services within the cluster is the responsibility of the Remote Service Admin.

As we all know service properties are quite cumbersome to parse and process. Therefore, we also provide DTO types NodeStatusDTO and FrameworkNodeStatusDTO to have type-safe access to these properties. Converting service properties to these DTO types can be easily done using the Converter specification. For example, the following component lists each node in the cluster with name mycluster and prints out its id.

@Component
public class MyClusterManagement {
    private static final Converter CONVERTER
                        = Converters.standardConverter();

    @Reference(cardinality=MULTIPLE, 
               policy=DYNAMIC,
               target="(osgi.clusterinfo.cluster=mycluster)")
    void addNode(NodeStatus ns, Map props) {
        // Convert properties to the DTO for type safe access
        NodeStatusDTO dto = CONVERTER.convert(props)
                                     .to(NodeStatusDTO.class);
        // Print out the node id
        System.out.println("Node added: "+dto.id);
    }
}

For more info on using the Converter, take a look at David Boschaert's blog post.

Provisioning Nodes

The Framework Node Status service also provides a way to interact with the OSGi framework by also extending the FrameworkManager interface. This provides a similar interface as the REST Management service, but accessible as an OSGi Remote service.

Now we can extend the previous example and write a provisioner that deploys some location specific bundles.

@Component
public class FrameworkProvisioner {
    private static final Converter CONVERTER
                        = Converters.standardConverter();

    @Reference(cardinality=MULTIPLE,
               policy=DYNAMIC)
    void addFramework(FrameworkNodeStatus fns, Map props) {
        // Convert properties to the DTO for type safe access
        NodeStatusDTO dto = CONVERTER.convert(props)
                                     .to(NodeStatusDTO.class);

        // Check the ISO 3166-1 alpha 3 country code
        if ("DEU".equals(dto.country)) {
            // If this framework runs in Germany
            // install a special bundle into it
            try {
                fns.installBundle("Germany specific bundle");
            } catch (Exception e) {
                // log
            }
        }
    }
}

Metrics

Besides announcing static properties about a node by means of the service properties, it is also possible for a node to give access to dynamically changing properties, which we call metrics. To that end, the NodeStatus service provides the getMetrics() method, which returns a map with the current metric values. Which metrics are actually provided depends on the type of node and is implementation specific. For example, a NodeStatus of a VM could return usage metrics of that VM given by your cloud provider.

Again, an implementation can also provide DTO types to convert this map to a type-safe object. For example, the Eclipse Concierge reference implementation provides a JMXMetricsDTO that exposes JMX metrics of the JVM.

// From service registry
NodeStatus ns = ...;
// Obtain all metrics for this node
Map metrics = ns.getMetrics();

// Convert the metrics map to a DTO for type-safe access
JMXMetricsDTO dto = Converters.standardConverter()
                              .convert(metrics)
                              .to(JMXMetricsDTO.class);

// Use metrics
System.out.println("System Load Average: " 
                   + dto.systemLoadAverage);

Case Study: OSGi Community Event Trains Demo

Those of you who attended the OSGi Community Event the last couple of years will most likely have seen the trains demo. This demo uses a number of different OSGi open source projects, and commercial products from OSGi Alliance members to manage and control the operation of LEGO^® trains.

All the trains, signals, track switches, train control buttons, notification screens etc. are controlled by different Raspberry Pi devices, adding up to a total of 8 different devices running an OSGi framework that have to be provisioned with different bundles depending on their role in the demo. This makes it a very tedious setup to get all devices up and running correctly.

For this use case, we can use the osgi.clusterinfo.tags property of the Node Status service to associate application specific tags to each OSGi framework node, indicating the roles it has to fulfill (i.e., "switchcontroller","traincontroller","signalcontroller", ...). We can then use these tags in a provisioning component similar to the example above, installing the correct bundles depending on the roles.

These tags can be used in a very versatile way, and in fact, any bundle in the OSGi framework can add a custom tag to the Node Status service representing this framework. To do so, any bundle can register a service with the org.osgi.service.clusterinfo.tags property set, which will automatically be added to the Node Status service tags.

The Cluster Information specification can also be used to manage non-OSGi entities. For example, in the demo, we also used an MQTT broker to connect with various external sensors. Again, this resulted in configuring on each of the devices the correct URI for connecting to the broker. By using a Node Status service that announces the presence of the MQTT broker, we have a single point of configuration in the cluster, and any component can depend on the presence of the broker by adding the following reference.

@Reference(target="(osgi.clusterinfo.tags=mqttbroker)")
void setBroker(NodeStatus ns, Map props) {
   // Convert properties to the DTO for type safe access
   NodeStatusDTO dto = CONVERTER.convert(props)
                                .to(NodeStatusDTO.class);
   // Now connect to the broker given the endpoint
   connect(dto.endpoint);
}

Notice again how we can use a custom tag to indicate this is an MQTT broker.

Want to know more?

This blog post gave a quick overview on the Cluster Information specification and some potential use cases. For more detailed information, see the spec chapter. If you want to play around with some actual code, the reference implementation is available in the Eclipse Concierge repository, with an actual release to follow soon.

---

Want to find out more about OSGi R7?

This is one post in a series of 12 focused on OSGi R7 Highlights.  Previous posts are:

1. Proposed Final Draft Now Available
2. Java 9 Support
3. Declarative Services
4. The JAX-RS Whiteboard
5. The Converter

Be sure to follow us on Twitter or LinkedIn or subscribe to our Blog feed to find out when it's available.

OSGi R7 Highlights: The Converter

One of the entirely new specifications in the OSGi R7 set of specs is the Converter specification. The converter can convert between many different Java types and can also be a highly convenient tool to give untyped data a proper API.

Let's dive in with some simple converter examples:

// Obtain the converter
Converter c = Converters.standardConverter();

// Simple scalar conversions
int i = c.convert("123").to(int.class);
UUID id = c.convert("067e6162-3b6f-4ae2-a171-2470b63dff00")
           .to(UUID.class);
BigDecimal bd = c.convert(12345).to(BigDecimal.class);

Collections, arrays and maps can also be converted:

List<String> ls = Arrays.asList("978", "142", "-99");

// Convert a List of Strings to a Set of Integers, 
// note the use of TypeReference to capture the generics
Set<Integer> is = c.convert(ls)
                   .to(new TypeReference<Set<Integer>>(){});

// Convert between map-like structures
Dictionary<String, String> d = ...; // some dictionary
Map m = c.convert(d).to(Map.class);

// Even though properties contain String keys and values, you 
// can't assign them to a Map<String, String>, the converter 
// can make this easy...
Properties p = ...; // a properties object
Map<String, String> n = c.convert(p)
           .to(new TypeReference<Map<String,String>>(){});

The above examples show that many different types of conversions can be made with the converter, using the same convert(object).to(type) API.

Note that the converter is obtained by calling Converters.standardConverter() so by default it's not registered in the service registry. This makes the converter usable in both OSGi Framework as well as in plain Java environments.

Backing Objects

An interesting feature of the converter is that it can convert between maps and Java types such as interfaces, annotations and java beans. Together we call these map-like types. In certain cases the conversion can provide a live view over the backing object. 

For example, let's say you have an interface MyAPI:

interface MyAPI {
    long timeout();
    long timeout(long defval);
}

Now you have a map that contains a timeout key with some value, if you want to expose that via the MyAPI interface the converter can do this:

Map<String, Long> m = new HashMap<>();
m.put("timeout", 999L);
MyAPI o = c.convert(m).to(MyAPI.class);
System.out.println(o.timeout()); // prints 999
m.put("timeout", 1000L);         // update the underlying map
System.out.println(o.timeout()); // prints 1000

The other way around is also possible. For example, if you have a JavaBean that you need to view as a Map, then the converter can do this. In the following example, we see that in order for the converter to handle a source object as a JavaBean you need to specify the sourceAsBean() modifier. Additionally, live views for maps are not enabled by default and are enabled with the view() modifier.

MyBean bean = new MyBean();
bean.setLength(120);
bean.setWidth(75);
Map<String, String> bm = c.convert(bean).sourceAsBean().view()
           .to(new TypeReference<Map<String,String>>(){});
System.out.println("Bean map:" + bm);
bean.setWidth(125)                    // update the bean
System.out.println("Bean map:" + bm); // map reflects the update

In addition to the map-like types mentioned above, the converter can handle DTOs and when an object exposes a method called getProperties(), it can take this as the source map as well.

Defaults

Converting a map or properties object to a Java interface can be a really useful way to give some untyped configuration a well-defined API. Often you'd want to be able to specify some defaults for the configuration values that you're using, in case they are not specified. When converting to interfaces or annotations defaults can be specified.

When converting to an interface, the interface may define a one-argument method to specify the default, as is done with the MyAPI interface above. To specify the default pass it into the method:

Map<String, Long> m = new HashMap<>(); // empty map
MyAPI o = c.convert(m).to(MyAPI.class);

// default to 750 if no timeout is in the map
System.out.println(o.timeout(750L));

Handling defaults via an annotation is even a little neater, as the Java annotation syntax allows for the specification of defaults.     

@interface MyAnn {
    long timeout() default 100;
}

MyAnn a = c.convert(m2).to(MyAnn.class);
System.out.println(a.timeout()); // use default 100
m2.put("timeout", 500L);
System.out.println(a.timeout()); // now get the actual value 500

Customizing Converters

While the converter can convert between many types (see the spec chapter for the actual list of supported types) it doesn't understand every single Java type ever created. Let's say we have two custom types Foo and Bar. There is a conversion between the two, but the converter doesn't know about it. We can create a customized converter that, on top of its existing conversions, knows about our special conversion as well!

Let's say our custom classes look a bit like this. The Foo class can be represented as a string which you can obtain via its read() method. The Bar class can hold a byte array.

public class Foo {
    // ...
    public String read() {
        return val;
    }
}

public static class Bar {
    byte[] buf;

    public static Bar create(byte[] b) {
        Bar bar = new Bar();
        bar.buf = b;
        return bar;
    }
}

To create a converter that, on top of the existing conversions, knows about Foo and Bar, we use a converter builder and register a rule with that to do the conversion. In the example below, the Rule object declares the source and the target of the rule. The conversion is provided as a lambda:

Foo f = ...; // Some Foo object
Converter custom = c.newConverterBuilder()
    .rule(new Rule<Foo, Bar>(
        foo -> Bar.create(foo.read().getBytes())){})
    .build();

// Now we can convert Foo to Bar!
Bar b = custom.convert(f).to(Bar.class);

Now the converter can convert our custom objects. An additional benefit is that the converter is an immutable object. You can customize an already customized converter by creating a new converter based on the previously created custom one.

Because the converter is immutable it's ideal for sharing, so once you've customized the perfect converter for your application, a good way to share it is by registering it in the OSGi Service Registry, and because the default converter isn't available in the service registry, your application components can simply look up the Converter service to get the one you registered for application wide usage.

Getting Started

This blog post doesn't cover all the details of the converter specification. See the spec chapter for more info on those. Like to get started straight away? At the time of this writing the specs are still under vote and not yet finally released. However, you can obtain a snapshot converter implementation from the OSGi Maven Repo: https://oss.sonatype.org/content/repositories/osgi/org/osgi/org.osgi.util.converter/ or you can build one yourself from the codebase in the Apache Felix project: https://svn.apache.org/repos/asf/felix/trunk/converter/converter.

---

Want to find out more about OSGi R7?

This is one post in a series of 12 focused on OSGi R7 Highlights.  Previous posts are:

1. Proposed Final Draft Now Available
2. Java 9 Support
3. Declarative Services
4. The JAX-RS Whiteboard

Be sure to follow us on Twitter or LinkedIn or subscribe to our Blog feed to find out when it's available.

An evening of OSGi with BGJUG on April 17 Sofia, Bulgaria

The BGJUG and Software AG are kindly hosting an Evening of OSGi on Tuesday April 17 from 7pm.

The OSGi Alliance is in Sofia conducting its Expert Group face to face meetings and Board meeting so this evening event will provide an excellent opportunity to meet with the local Java community while we are in town.

There are two talks lined up from the OSGi Alliance:

• the first is an 'Introduction to OSGi' by BJ Hargrave (OSGi Alliance CTO) and 
• the second is a look at 'How OSGi Alliance specifications have influenced the IoT market' by Pavlin Dobrev & Kai Hackbath (Bosch). 

In addition Todor Boev from Software AG will be presenting 'OSGi and JPMS in practice' which will discuss Software AG's experiences with OSGi and Jigsaw/JPMS now that Java 9 is released.

The event is being held at Sofia Tech Park / Incubator building, Tsarigradsko shoes 111B, 1784 Sofia [MAP].

Please be sure to register in advance to secure your place.

Thanks to BGJUG and Software AG for their assistance in arranging this event and of course to the speakers for their support.

We hope you can join us.