Tuesday, September 17, 2019

Type Safe Eventing - Teaching an Old Spec New Tricks

The rapid growth in the number of connected devices in homes, factories, cities, and everywhere else embracing the IoT revolution has driven a huge increase in the volume and rate of data that systems are expected to handle. Properly managed, the use of "events" can provide a flexible, scalable way to deal with these large data volumes.

Doesn't OSGi Already Support Eventing?

Yes! Event-based applications have long been supported by OSGi using the Event Admin Specification. This provides a broker-based mechanism for distributing event data to whiteboard recipients, either synchronously or asynchronously.

So if we already have an Eventing Specification, why do we need another?

While the current Event Admin does offer a solution for Event Based applications, there are some ways in which it is a little limited:
  • The Event Admin API is based around Maps with opaque String keys and Object values
  • There is no way to monitor the flow of events through Event Admin
  • There is no way to determine whether an event was actually received by anyone
These limitations can make it challenging to use Event Admin, and also lead to a lot of boilerplate code. For example, if you've ever used Event Admin you'll know that you need to be very defensive when trying to consume an event - you may expect the value for the "price" key to be a Double, but it's easy for someone to accidentally supply a Float or a String!

What's being proposed?

The primary enhancement being proposed is to allow the event data to sent in a type-safe way, but what does this actually mean? Well at the moment all the data that goes into an event is stored as a set of String keys which map to Object values, and the only way for an event consumer to know what keys and associated value types to expect is based on the Event Topic. This means that my event consumer is relying on an implied contract where the topic defines the schema of the event data.

The really yucky part of this isn't that the schema is implied, it's that it isn't able to be validated and enforced very easily. If I write an event consumer then it has to rummage around inside the map (hopefully using the correct keys) and assume that the associated values will match the types I expect.

So far Type Safe Eventing is only an RFP, so there isn't any implementation to discuss, however, you can imagine how much nicer it would be to use a type-safe data object like an OSGi DTO. Rather than receiving an opaque map an Event Handler can receive a data object which matches the implied contract of the topic and formalizes the key names and value types. Instead of checking your code for typos in the keys the Java compiler will guarantee that the key exists and that the value is of the right type!

Further proposed enhancements include:
  • Monitoring the flow of events sent via the broker - at the moment the only way to determine what events are flowing is to add your own Event Handler, however, it would be much simpler (and less invasive) if the broker could provide a "monitor" view of the data flowing.
  • Handlers of last resort - the current Event Admin gives no feedback if an event is not handled by anyone. This can lead to problems, especially when debugging, as the data flow seems to "stop" without any obvious reason. Adding the equivalent of a "Dead Letter Queue" would help in this situation, and it could enable other more advanced use cases.

What's Next?

This RFP initially came about as a result of requirements gathered in the development of the BRAIN-IoT Horizon 2020 project, an event-based IoT Platform. The RFP is nearly complete, but it's not too late to supply more requirements if you have any. Soon this specification will turn into an RFC and the implementation work can begin. You can take a look at it at Github:

No comments:

Post a Comment