OSGi Blog
Tuesday, February 20, 2007
Friday, February 2, 2007
Tidbits
I wanted to blog about the OSGi Enterprise Expert Group but Eric Newcomer (IONA), one from the chairs beat me. I do not think I can improve his report, so please check it out. I fully agree with Eric that it was an exciting and energetic day with a lot of hope for the future (and work). If you want to participate in this work, you will have to become an OSGi member, but believe me, it is definitely going to be worth it!
I just finished working on the final details OSGi DevCon/EclipseCon . It really is becoming an amazing conference. We had many more submissions than we could select, and almost all submissions were of very high quality. You still get an early bird discount for the next two weeks, so sign up quickly. I sure hope to meet many of my blog readers.
Ok, one thing technical. Jon Bostrom showed me the N800, which is an improved version of the N770. This is an OSGi dream come true machine. For some reason, it does not include a GSM chip, but that could make the N800 an interesting companion with the E70 that I have, which contains also an OSGi framework. The devices could communicate over wifi or bluetooth to exchange data and code. Jon promised me an N800 once the channels are filled, once I get one I will report about my experiences. So many interesting areas, so little time ...
Peter Kriens
I just finished working on the final details OSGi DevCon/EclipseCon . It really is becoming an amazing conference. We had many more submissions than we could select, and almost all submissions were of very high quality. You still get an early bird discount for the next two weeks, so sign up quickly. I sure hope to meet many of my blog readers.
Ok, one thing technical. Jon Bostrom showed me the N800, which is an improved version of the N770. This is an OSGi dream come true machine. For some reason, it does not include a GSM chip, but that could make the N800 an interesting companion with the E70 that I have, which contains also an OSGi framework. The devices could communicate over wifi or bluetooth to exchange data and code. Jon promised me an N800 once the channels are filled, once I get one I will report about my experiences. So many interesting areas, so little time ...
Peter Kriens
Tuesday, January 23, 2007
Can Someone Tell Steve Jobs About OSGi?
Steve Jobs in an interview with Jon Markoff: “Java’s not worth building in. Nobody uses Java anymore. It’s this big heavyweight ball and chain.”. Mr. Jobs is known to use the hyperbole, but I am afraid he has a point. I remember having a fiery discussion with Mr. Smalltalk, Ralph Johnson, in ’97 about how nice Java was. One of my key arguments was that Java was so lean and mean in comparison to Smalltalk. He shook his wise head and told me to wait and see. Oh boy, how right he was. Since then Java’s library growth has consistently beaten the growth of even the best hedge fund. Time to reflect and refactor are words one can not easily associate with the Java world. The JCP has given us a hodgepodge of inconsistent APIs split between worlds that seem to diverge farther every day.
Yes, I know. Four billion phones or more run Java, generating billions of dollars of revenue. However, everybody in family has a Java phone and non uses Java applications. When you look at what people run on their telephone then it is mostly games. Where are the killer apps? These apps would appear in the market when the operators and manufacturers would allow third parties to access all the resources. Of course, JSR 232 provides a very interesting architecture for interesting applications that will enable these types of applications, but phones are only just now appearing in the market.
Sadly, Java largely failed on the desktop. How many pleasant applications do you use that are written in Java? Eclipse is still the exception. Java seems promise so much for the embedded market but it is still a hard to sell Java in a consumer device due to the extra cost of the VM and increase in resource consumption. Java only shines in the server market, where resources are plenty and inefficiency is a minor inconvenience.
Why is the situation so bleak? The “Write Once, Run Anywhere” mantra (WORA) of Java is highly attractive. The software industry spends countless billions every year porting applications from one environment to the other. Application developers limit themselves to fewer markets because the cost to develop for additional platforms is just not worth it. There have been significant problems in the past with the WORA (write once, debug anywhere israther tired joked repeated ad nauseum by Microsoft) the joke), but today it works amazingly well across platforms, note again the Eclipse success story.
Personally, I think the biggest problem is that the Java architecture is not suited to what it desires to be: a ubiquitous computing environment. A key aspect of a ubiquitous computing environment is that it should be lean and mean at the core. Few want to carry the dead weight of someone else’s requirements. The key mistake Java/JCP made was splitting Java up in different domains: JavaCard, J2ME, JSE, and JEE. Then in different configurations: CLDC and CDC. Then further refined with profiles like Foundation, Personal, Basic, etc. The intentions were good; the result is a disastrous fragmentation.
Lean and mean means you need to be able to mix and match. One of the most interesting Java projects at this moment is therefore Apache Harmony. Harmony is an open source implementation of the Java SE VM and class library. Instead of taking a short-cut, cloning the Sun VM, they use a novel architecture of modularizing the Java class library. With Apache Harmony, you can start really small and add what is needed. Such an architecture permits the use of the same basic environment across industry and platforms. Developers no longer target a specific pre-defined profile but only express their requirements on the platform. When the application gets deployed and certain packages are missing, the platform can be dynamically extend the platform by adding the missing packages.
Modularization creates a market for solutions. It is no longer required to come up with a catch-all API in the JCP, which is too often designed under political pressure. For example, J2EE would never have been a success in a free market. It received its life because it was the “ordained” API for enterprise applications by virtue of its JCP origin. It was not until an external party like Interface21 showed with Spring that Enterprise applications can actually be lean and mean as well.
I think that Java is too big and too important to our industry to be shepherded by a single company like Sun. By trying to shield the environment from fragmenting they sadly have achieved the opposite. By exerting too much control they have enabled some pretty bad APIs. Why not set firm ground rules, that is a small VM and rules for modularization, then let different parties compete with solutions. The market will quickly pick the best practices and create de-facto as well as de-jura standards.
Time to reflect. What do Eclipse, Apache Harmony, and Spring, actually have in common? Hmm, maybe Steve Jobs should invite an OSGi evangelist to give a presentation?
Peter Kriens
P.S. You did register for EclipseCon I hope?
Yes, I know. Four billion phones or more run Java, generating billions of dollars of revenue. However, everybody in family has a Java phone and non uses Java applications. When you look at what people run on their telephone then it is mostly games. Where are the killer apps? These apps would appear in the market when the operators and manufacturers would allow third parties to access all the resources. Of course, JSR 232 provides a very interesting architecture for interesting applications that will enable these types of applications, but phones are only just now appearing in the market.
Sadly, Java largely failed on the desktop. How many pleasant applications do you use that are written in Java? Eclipse is still the exception. Java seems promise so much for the embedded market but it is still a hard to sell Java in a consumer device due to the extra cost of the VM and increase in resource consumption. Java only shines in the server market, where resources are plenty and inefficiency is a minor inconvenience.
Why is the situation so bleak? The “Write Once, Run Anywhere” mantra (WORA) of Java is highly attractive. The software industry spends countless billions every year porting applications from one environment to the other. Application developers limit themselves to fewer markets because the cost to develop for additional platforms is just not worth it. There have been significant problems in the past with the WORA (write once, debug anywhere israther tired joked repeated ad nauseum by Microsoft) the joke), but today it works amazingly well across platforms, note again the Eclipse success story.
Personally, I think the biggest problem is that the Java architecture is not suited to what it desires to be: a ubiquitous computing environment. A key aspect of a ubiquitous computing environment is that it should be lean and mean at the core. Few want to carry the dead weight of someone else’s requirements. The key mistake Java/JCP made was splitting Java up in different domains: JavaCard, J2ME, JSE, and JEE. Then in different configurations: CLDC and CDC. Then further refined with profiles like Foundation, Personal, Basic, etc. The intentions were good; the result is a disastrous fragmentation.
Lean and mean means you need to be able to mix and match. One of the most interesting Java projects at this moment is therefore Apache Harmony. Harmony is an open source implementation of the Java SE VM and class library. Instead of taking a short-cut, cloning the Sun VM, they use a novel architecture of modularizing the Java class library. With Apache Harmony, you can start really small and add what is needed. Such an architecture permits the use of the same basic environment across industry and platforms. Developers no longer target a specific pre-defined profile but only express their requirements on the platform. When the application gets deployed and certain packages are missing, the platform can be dynamically extend the platform by adding the missing packages.
Modularization creates a market for solutions. It is no longer required to come up with a catch-all API in the JCP, which is too often designed under political pressure. For example, J2EE would never have been a success in a free market. It received its life because it was the “ordained” API for enterprise applications by virtue of its JCP origin. It was not until an external party like Interface21 showed with Spring that Enterprise applications can actually be lean and mean as well.
I think that Java is too big and too important to our industry to be shepherded by a single company like Sun. By trying to shield the environment from fragmenting they sadly have achieved the opposite. By exerting too much control they have enabled some pretty bad APIs. Why not set firm ground rules, that is a small VM and rules for modularization, then let different parties compete with solutions. The market will quickly pick the best practices and create de-facto as well as de-jura standards.
Time to reflect. What do Eclipse, Apache Harmony, and Spring, actually have in common? Hmm, maybe Steve Jobs should invite an OSGi evangelist to give a presentation?
Peter Kriens
P.S. You did register for EclipseCon I hope?
Tuesday, January 16, 2007
OSGi Vehicle Workshop in Detroit
Last Thursday (11/01/2006) we met for an automotive workshop at the Delphi headquarters in Troy with participants from Intel, Wind River Systems, Delphi, Renesas, Vetronix, Booz Allen, Ford, GM, Aonix, Siemens VDO, ProSyst, and Telcordia. This workshop was initiated by Kai Hackbarth (ProSyst). Kai is working with a number of companies in the VII project and noticed that there is a lot of interest in OSGi from the participants in this project.
The VII project has a very ambitious goal: it wants to reduce the number of deaths on the American roads and decrease congestion (this is similar to the CVIS project in Europe). To achieve this goal, the American Department of Transport (DOT) wants to build an extensive communication infrastructure that supports vehicle to vehicle communication as well as vehicle to centers communications based on the DSRC communications service. Vehicles will get an on board unit and the roads will get a roadside unit that manages the communication of nearby vehicles. This infrastructure can be used in many different ways. Some examples:
The number of scenarios that can be supported is of course only limited by your imagination. It is therefore that OSGi plays an important role in this project. The life cycle of a car is around 10 years while the life cycle of these types of services is significantly smaller. It is therefore obvious that some form of software downloading and management is required. This is of course the archetypical application area of the OSGi specifications. Maybe the most interesting aspect is that the on board equipment (OBE) can be used for other, commercial, applications as well. This is not a primary goal, but it enables a lot of possibilities!
We had modeled the meeting after the successful Enterprise meeting in August 2006. This meeting resulted in the OSGi Enterprise Expert Group, which will meet in Dublin at the end of this month. The format is simple; some of the participants present their ideas in a 5 minute presentation. This kicks off a discussion where we gather all the different areas that could be relevant for OSGi standardization. This list is then prioritized and further discussed.
We found the following areas of highest interest:
Resource management and hard real time issues
The traditional embedded concerns of resource management and hard real time issues came far ahead of all other items. There exists an OSGi RFC addressing some of these concerns but the focus on server side and desktop applications have moved these concerns to the background. It seems clear that this should be a number one concern for the vehicle group. Interestingly, Aonix presented a very interesting idea. They had create a VM where you could run hard real time applications written in a subset and annotated Java. The code would run in a special memory space that did not require garbage collection. This concept meshes obviously very nicely with the OSGi model of deployment. Special metadata could be used to indicate that specific classes should run on the hard real-time VM.
Vehicle Interface
A vehicle interface provides access to the vehicle configuration and status. There has been work in GST, AMI-C, and the OSGi VEG in this area. Both GST and OSGi use the OSGi Device Management based on OMA DM as the foundation. The complex aspect of the vehicle interface is to model the car. How many chairs, how many axis has each chair, how many windows, where are the seats located. Such an ontology requires vetting by a lot of knowledgeable parties.
HMI
Human-Machine Interface. We have been talking about the HMI since day one but we have never been able to standardize a single HMI. There are developments that seem to make the development of a generic HMI more feasible. Normally, the car manufacturers regard the look and feel of the vehicle as their prerogative but it is becoming clearer that the largest number of applications requires some form of a HMI. Therefore, an OSGi standard should address some way that applications can interact with a user while at the same time allowing as much as possible freedom to the car manufacturer.
Non-Java code integration
The demand to expand from Java to native code becomes larger and larger. In the embedded world it is rarely that there are green-field applications. Embedded systems must work with legacy code that is not easy ported to a Java environment. From the participants it was clear that we need a story how to integrate non-java environments with OSGi. As I discussed elsewhere, I strongly believe that our service registry is a very good mechanism to let different subsystems communicate, regardless of the language that is spoken.
Other
There were several other areas discussed, like: Integration with AutoSAR, handling of safety information, vehicle specifics around provisioning and management, a comprehensive driver model, diagnostics, provisioning of data, navigation model, security, a lightweight form of OSGi, privacy issues, reprogramming ECUs (other processors in the car), distribution, and some none technical areas.
Conclusion
A great deal of time was spent on discussing the OSGi vision of universal middleware and how to get large car companies to buy into the technology. Though the basic OSGi technology has proven itself in a large number of applications, it is not well known in higher echelons. Marketing the technology is hard because there is no product that needs to be sold, what we are selling is a promise. To fulfill this promise we need the participation of the key players in this market: the car manufacturers. Without strong backing from the car manufacturers, few suppliers will invest in pushing a technology like OSGi. We discussed how to get support from the car manufacturers (several were around the table) but that is a difficult issue due the state of the current car industry and lack of knowledge about OSGi in the higher echelons. I guess we have to continue as we do today and hope that the combined results of this work will convince the car manufacturer’s management to invest in our technology.
However, it was clear that OSGi plays an important role in the thinking of many participants of the VII project, just like the European GST and the CVIS project. For these architectures the only option is the OSGi service platform, or otherwise limit the standards to protocols only. Though protocols only can reduce the cost of the OBE, it adds cost in other places because the diversity is increased on the management level. It is therefore not unlikely that in the long run the OSGi service platform becomes a standard component in the car.
Overall, this was very good meeting where both technical and non-technical issues were openly discussed with representatives of the key player; there was a lot of energy in the room. We also intend to follow up this meeting with an OSGi Vehicle Expert Group meeting, further detailing the requirements. It looks like the OSGi technology will happen in our industry, now we only have the small detail of making the vehicle specific specifications left.
Peter Kriens
The VII project has a very ambitious goal: it wants to reduce the number of deaths on the American roads and decrease congestion (this is similar to the CVIS project in Europe). To achieve this goal, the American Department of Transport (DOT) wants to build an extensive communication infrastructure that supports vehicle to vehicle communication as well as vehicle to centers communications based on the DSRC communications service. Vehicles will get an on board unit and the roads will get a roadside unit that manages the communication of nearby vehicles. This infrastructure can be used in many different ways. Some examples:
- Vehicles will transmit roadside information to central management centers which will be digested and dispatched as traffic information.
- When a vehicle breaks, it can send out this information to other vehicles
- Stop lights can send their state to oncoming vehicles
The number of scenarios that can be supported is of course only limited by your imagination. It is therefore that OSGi plays an important role in this project. The life cycle of a car is around 10 years while the life cycle of these types of services is significantly smaller. It is therefore obvious that some form of software downloading and management is required. This is of course the archetypical application area of the OSGi specifications. Maybe the most interesting aspect is that the on board equipment (OBE) can be used for other, commercial, applications as well. This is not a primary goal, but it enables a lot of possibilities!
We had modeled the meeting after the successful Enterprise meeting in August 2006. This meeting resulted in the OSGi Enterprise Expert Group, which will meet in Dublin at the end of this month. The format is simple; some of the participants present their ideas in a 5 minute presentation. This kicks off a discussion where we gather all the different areas that could be relevant for OSGi standardization. This list is then prioritized and further discussed.
We found the following areas of highest interest:
Resource management and hard real time issues
The traditional embedded concerns of resource management and hard real time issues came far ahead of all other items. There exists an OSGi RFC addressing some of these concerns but the focus on server side and desktop applications have moved these concerns to the background. It seems clear that this should be a number one concern for the vehicle group. Interestingly, Aonix presented a very interesting idea. They had create a VM where you could run hard real time applications written in a subset and annotated Java. The code would run in a special memory space that did not require garbage collection. This concept meshes obviously very nicely with the OSGi model of deployment. Special metadata could be used to indicate that specific classes should run on the hard real-time VM.
Vehicle Interface
A vehicle interface provides access to the vehicle configuration and status. There has been work in GST, AMI-C, and the OSGi VEG in this area. Both GST and OSGi use the OSGi Device Management based on OMA DM as the foundation. The complex aspect of the vehicle interface is to model the car. How many chairs, how many axis has each chair, how many windows, where are the seats located. Such an ontology requires vetting by a lot of knowledgeable parties.
HMI
Human-Machine Interface. We have been talking about the HMI since day one but we have never been able to standardize a single HMI. There are developments that seem to make the development of a generic HMI more feasible. Normally, the car manufacturers regard the look and feel of the vehicle as their prerogative but it is becoming clearer that the largest number of applications requires some form of a HMI. Therefore, an OSGi standard should address some way that applications can interact with a user while at the same time allowing as much as possible freedom to the car manufacturer.
Non-Java code integration
The demand to expand from Java to native code becomes larger and larger. In the embedded world it is rarely that there are green-field applications. Embedded systems must work with legacy code that is not easy ported to a Java environment. From the participants it was clear that we need a story how to integrate non-java environments with OSGi. As I discussed elsewhere, I strongly believe that our service registry is a very good mechanism to let different subsystems communicate, regardless of the language that is spoken.
Other
There were several other areas discussed, like: Integration with AutoSAR, handling of safety information, vehicle specifics around provisioning and management, a comprehensive driver model, diagnostics, provisioning of data, navigation model, security, a lightweight form of OSGi, privacy issues, reprogramming ECUs (other processors in the car), distribution, and some none technical areas.
Conclusion
A great deal of time was spent on discussing the OSGi vision of universal middleware and how to get large car companies to buy into the technology. Though the basic OSGi technology has proven itself in a large number of applications, it is not well known in higher echelons. Marketing the technology is hard because there is no product that needs to be sold, what we are selling is a promise. To fulfill this promise we need the participation of the key players in this market: the car manufacturers. Without strong backing from the car manufacturers, few suppliers will invest in pushing a technology like OSGi. We discussed how to get support from the car manufacturers (several were around the table) but that is a difficult issue due the state of the current car industry and lack of knowledge about OSGi in the higher echelons. I guess we have to continue as we do today and hope that the combined results of this work will convince the car manufacturer’s management to invest in our technology.
However, it was clear that OSGi plays an important role in the thinking of many participants of the VII project, just like the European GST and the CVIS project. For these architectures the only option is the OSGi service platform, or otherwise limit the standards to protocols only. Though protocols only can reduce the cost of the OBE, it adds cost in other places because the diversity is increased on the management level. It is therefore not unlikely that in the long run the OSGi service platform becomes a standard component in the car.
Overall, this was very good meeting where both technical and non-technical issues were openly discussed with representatives of the key player; there was a lot of energy in the room. We also intend to follow up this meeting with an OSGi Vehicle Expert Group meeting, further detailing the requirements. It looks like the OSGi technology will happen in our industry, now we only have the small detail of making the vehicle specific specifications left.
Peter Kriens
Tuesday, January 9, 2007
Spring and OSGi: Jumping Beans
This week there was an interesting discussion going on in the Spring-OSGi mailing list. In this mailing list, a number of people discuss their fantastic work marrying Spring with OSGi. This project has created a lot of interest; Even JBoss seems to have been inspired by this work and is moving to OSGi, putting the majority of application servers on firm OSGi footing!
Anyway, the discussion was about the dynamics of services. Hal Hildebrand (Oracle) started a discussion about the cardinality of services. This opened a floodgate of mails that I tried to ignore due to after-vacation overload. However, Costin Leau (Interface21) asked me to jump in so I was forced to analyze the thread. I got myself a cup of strong coffee and wrestled through the mail flood.
The issue was revolving around the cardinality. The cardinality indicates how many services of a given type you require and how many of them you are interested in. The OSGi-Spring specification treats OSGi services similar to beans. This allows you to inject beans into other beans, following the concept of inversion of control and dependency injection. For unary services like an OSGi Log Service this works fairly well. However, the devil is usually in the details, so let us take a closer look.
The key differences between beans and OSGi services are:
Most Java programmers will recognize this model. Many developers use the Class.forName to find a class and then use Class.newInstance to create an instance for extendibility. Virtually any (non-OSGi) Java application uses this model to extend or configure itself this way. Notice that there is a strong 1:1 relation between the Class and the instance in this model.
In contrast, OSGi services are more like jumping beans; they have a life of their own. When beans start jumping all over the place (spring, as they say in Dutch!) then most programmers will be shocked to the bones. Fortunately, once they grasp the power of a dynamic model they usually fall in love it. However, this model implies that the IoC is not limited to configuration and activation, it means that the IoC container must handle life cycle issues during the active life of the application.
Obviously, many Spring applications will not be able to handle these dynamics, or even want to handle these dynamics. The Spring-OSGi group therefore chose to hide these dynamics behind a proxy. Instead of directly injecting the service object, a proxy to this object is injected into the configured bean. This gives us a choice to control the behavior later when the underlying service arrives or goes away. However, it also gives us a choice of optionality and multiplicity. The proxy mechanism of Spring gives us full control, the tricky thing is to define what to do with this control.
These are exactly the issue we spent a lot of time discussing during the development of declarative services, inspired by Richard Hall’s service binder and designed by BJ Hargrave (IBM). Declarative services do not use proxies but instead bind and unbind services to the application using injection. After lots of deep discussions we came up with the following policies for binding and unbinding services to the application.
Mandatory/Optional
A mandatory service means that the service must be present before the application is activated and the application will be deactivated when there is no such service available anymore. An optional service does not influence the life cycle of the application. The declarative runtime can unbind a service and bind another service at will when the service is optional. Normal Spring beans are similar to mandatory services.
Mandatory is normally indicated in the cardinality as 1.., and optionality as 0.. .
Unary/Multiple
The application can be interested in a single service (unary) or it can track a set of services. The set may require some explanation. The dynamic service registry enables some very interesting software patterns. For example, one bundle manages a Bluetooth hardware interface. The whiteboard pattern shows that the simplest solution for this bundle is to register a service per discovered device, and unregister this service when the device is no longer reachable. Other bundles can then just track these services from the registry and use when required. This pattern nicely decouples the lifecycle of the bundles that participate, making clients easier to develop.
The idea that you can add and remove beans dynamically from a configured bean is new to Spring but it follows naturally from the OSGi model.
This cardinality is indicated with ..1 for unary and ..n for multiple.
Static/Dynamic
Once an application is activated because its dependencies are met, the lifecycle state of a referred service can change. The traditional choice is to deactivate the application and restart it when the conditions are right again; this is called the static policy. Some people are repelled by this idea but in an OSGi system this is quite natural, and well under the control of the operator. An update can cause a ripple effect of many bundles restarting but well designed systems should be robust enough to handle this. Hmm, let me rephrase, if your system can’t handle this model, it will fail in the end anyway.
In certain cases, it would do no harm to replace a service on the fly. If an application uses the Log Service, it is likely not interested which Log Service it uses. If this service is unregistered, it could easily be unbound and then bound again with a new instance. Most stateless services can be replaced this way. Binding and unbinding during the active life of the application is called the dynamic model.
Usage
Let us now take a look at the possible combinations and see what they mean, and where they are applicable.
Spring
The Spring-OSGi spec can copy the concepts from the declarative services because they have exactly the same issues. The discussion in the mail group got confused because the interaction between the policies for proxying (static/dynamic) were confused with optionality.
Spring adds the concept of proxies. It is my advice that the rules are as close as possible to the declarative services model which means that in most cases the application must participate in the binding and unbinding. It is tempting to automatically replace the service reference in a proxy but most of the time services are stateful and then it is easy to confuse them. Associating proxying with dynamic does not make sense because normally the application must do some action during binding. Maybe Spring needs an extra policy to indicate that replacing the service in the proxy is allowed or not, for example statefull and stateless. Then again, I am starting to have serious doubts how useful proxy rebinds are in reality.
Anyway, this year will be an interesting year for declarative services. It is clear from this issue that declarative services did a good job in this area but the possibilities of Spring will likely encompass all the important features of declarative and then add more. I am not sure if it is a good idea to have declarative services compete with Spring. My personal preference is that the Spring-OSGi specification becomes an official OSGi R5 spec … Let me know if you have ideas how we should handle this.
Peter Kriens
Anyway, the discussion was about the dynamics of services. Hal Hildebrand (Oracle) started a discussion about the cardinality of services. This opened a floodgate of mails that I tried to ignore due to after-vacation overload. However, Costin Leau (Interface21) asked me to jump in so I was forced to analyze the thread. I got myself a cup of strong coffee and wrestled through the mail flood.
The issue was revolving around the cardinality. The cardinality indicates how many services of a given type you require and how many of them you are interested in. The OSGi-Spring specification treats OSGi services similar to beans. This allows you to inject beans into other beans, following the concept of inversion of control and dependency injection. For unary services like an OSGi Log Service this works fairly well. However, the devil is usually in the details, so let us take a closer look.
The key differences between beans and OSGi services are:
- Beans are passive, beans are non-existent until someone creates them with Class.newInstance(). The whole idea of Inversion of Control (IoC) is that this creation is done by an outsider and not the POJO itself.
- You rarely require two beans of the same class. Why would you use two different database objects or log objects?
Most Java programmers will recognize this model. Many developers use the Class.forName to find a class and then use Class.newInstance to create an instance for extendibility. Virtually any (non-OSGi) Java application uses this model to extend or configure itself this way. Notice that there is a strong 1:1 relation between the Class and the instance in this model.
In contrast, OSGi services are more like jumping beans; they have a life of their own. When beans start jumping all over the place (spring, as they say in Dutch!) then most programmers will be shocked to the bones. Fortunately, once they grasp the power of a dynamic model they usually fall in love it. However, this model implies that the IoC is not limited to configuration and activation, it means that the IoC container must handle life cycle issues during the active life of the application.
Obviously, many Spring applications will not be able to handle these dynamics, or even want to handle these dynamics. The Spring-OSGi group therefore chose to hide these dynamics behind a proxy. Instead of directly injecting the service object, a proxy to this object is injected into the configured bean. This gives us a choice to control the behavior later when the underlying service arrives or goes away. However, it also gives us a choice of optionality and multiplicity. The proxy mechanism of Spring gives us full control, the tricky thing is to define what to do with this control.
These are exactly the issue we spent a lot of time discussing during the development of declarative services, inspired by Richard Hall’s service binder and designed by BJ Hargrave (IBM). Declarative services do not use proxies but instead bind and unbind services to the application using injection. After lots of deep discussions we came up with the following policies for binding and unbinding services to the application.
Mandatory/Optional
A mandatory service means that the service must be present before the application is activated and the application will be deactivated when there is no such service available anymore. An optional service does not influence the life cycle of the application. The declarative runtime can unbind a service and bind another service at will when the service is optional. Normal Spring beans are similar to mandatory services.
Mandatory is normally indicated in the cardinality as 1.., and optionality as 0.. .
Unary/Multiple
The application can be interested in a single service (unary) or it can track a set of services. The set may require some explanation. The dynamic service registry enables some very interesting software patterns. For example, one bundle manages a Bluetooth hardware interface. The whiteboard pattern shows that the simplest solution for this bundle is to register a service per discovered device, and unregister this service when the device is no longer reachable. Other bundles can then just track these services from the registry and use when required. This pattern nicely decouples the lifecycle of the bundles that participate, making clients easier to develop.
The idea that you can add and remove beans dynamically from a configured bean is new to Spring but it follows naturally from the OSGi model.
This cardinality is indicated with ..1 for unary and ..n for multiple.
Static/Dynamic
Once an application is activated because its dependencies are met, the lifecycle state of a referred service can change. The traditional choice is to deactivate the application and restart it when the conditions are right again; this is called the static policy. Some people are repelled by this idea but in an OSGi system this is quite natural, and well under the control of the operator. An update can cause a ripple effect of many bundles restarting but well designed systems should be robust enough to handle this. Hmm, let me rephrase, if your system can’t handle this model, it will fail in the end anyway.
In certain cases, it would do no harm to replace a service on the fly. If an application uses the Log Service, it is likely not interested which Log Service it uses. If this service is unregistered, it could easily be unbound and then bound again with a new instance. Most stateless services can be replaced this way. Binding and unbinding during the active life of the application is called the dynamic model.
Usage
Let us now take a look at the possible combinations and see what they mean, and where they are applicable.
- static, 1..1 – The simplest case. The service must be present before the application is activated and the application is deactivated when the service disappears. The world looks perfectly simple and static for the application. The typical use case for this policy is the Http Service. Many applications should only be started when the Http Service is present and when it goes away the application has no purpose anymore. A once chosen Http Service can not just be replaced because the application must register the servlet with the Http Service; the Http Service is not stateless with respect to the application.
- static, 0..1 – The common case when the service is not really required to operate the application and the programmer does not want to bother when the service shows up later. The application is therefore unconditionally activated, regardless if the service is present. However, if a service is bound to the application before activation, the application is deactivated when that specific service goes away, even if another becomes available. This cardinality could for example be used for the Configuration Admin service because it is likely to be there or not.
- static, 1..n – This case is not very useful, tracking multiple services require a dynamic policy to make sense.
- static, 0..n – See previous.
- dynamic, 1..1 – The service must be present before the application is activated, however, the service can be unbound and bound again if a replacement can be found. This mode is useful when you have a stateless service but you require its presence to operate. For example, if the application needs to send messages then it can not function without a message service. However, it does not really care which message service is used.
- dynamic, 0..1 – An optional service that can dynamically be replaced. The application is unconditionally activated and never deactivated on behalf of this service. This policy is typically used for a Log Service. Logging is nice, but not generating a fire alarm because you can not log it is bound to make some unhappy customers.
- dynamic, 1..n – At least one service is needed, but many are handled. The application is only activated when at least one service is present and is deactivated when no more services can be found. This can be useful if you perform an expensive function that is only relevant when there is at least one consumer. For example, assume you run a GPS device in a phone so battery power is at premium. If there are no positioning listeners, it is no use to run the GPS chip thereby reducing power.
- dynamic, 0..n – The typical tracking case. The application is unconditionally activated and never deactivated on behalf of this service. Services are bound and unbound as they come and go. For example, if you are tracking a set of Bluetooth devices in the service registry then this is your policy to use.
Spring
The Spring-OSGi spec can copy the concepts from the declarative services because they have exactly the same issues. The discussion in the mail group got confused because the interaction between the policies for proxying (static/dynamic) were confused with optionality.
Spring adds the concept of proxies. It is my advice that the rules are as close as possible to the declarative services model which means that in most cases the application must participate in the binding and unbinding. It is tempting to automatically replace the service reference in a proxy but most of the time services are stateful and then it is easy to confuse them. Associating proxying with dynamic does not make sense because normally the application must do some action during binding. Maybe Spring needs an extra policy to indicate that replacing the service in the proxy is allowed or not, for example statefull and stateless. Then again, I am starting to have serious doubts how useful proxy rebinds are in reality.
Anyway, this year will be an interesting year for declarative services. It is clear from this issue that declarative services did a good job in this area but the possibilities of Spring will likely encompass all the important features of declarative and then add more. I am not sure if it is a good idea to have declarative services compete with Spring. My personal preference is that the Spring-OSGi specification becomes an official OSGi R5 spec … Let me know if you have ideas how we should handle this.
Peter Kriens
Friday, December 22, 2006
OSGi Alliance at the end of 2006
The end of the year is nearing and that is a time to reflect. What did we achieve this year and where will we be heading to? Bear with me.
Evangelism
For me, this year was special because for the first time in my life I spent much of my time in non-technical work. The OSGi Alliance board had appointed me their official evangelist. That was a bit getting used to; suddenly I had to visit conferences, present OSGi technology and just talk about it instead of doing it. The hardest part was learning how to consistently use the word “OSGi” as an adjective for some trademark reason.
This evangelism thing is a bit hectic: EclipseCon in Santa Clara, ApacheCon in San Diego, ApacheCon Europe Dublin, Oredev in Malmö, Javapolis in Antwerp, CCNC in Las Vegas, OOPSLA in Portland, and BCC in Birmingham. Busy, but most of the conferences were very gratifying because I met so many people that are enthusiastic about the OSGi technology. It never ceases to amaze me that there are so many people that can enthusiastically tell me that they had been using OSGi for many, many years; most of the times from companies that I never heard of. But I love those stories and they make my work very rewarding. Next year it looks like there will be a similar schedule so do not hesitate to talk to me when you see me in my “Ask me about OSGi” shirt..
Mobile Specifications
In 2006 we had to finalize the Mobile specifications, which had some political hurdles. The project had run as a joint project between the JCP (JSR 232) and the OSGi Alliance. One of the key aspects has been the device management; a specification that uses the same concepts as the OMA DM protocol. JSR 246 had developed a similar specification for MIDP that was originally based on the work done in the OSGi but unfortunately deviated significantly. Fortunately, the spec leads Jon Bostrom (Nokia) and Jens Petzold (BenQ) realized that two similar but different specifications in the JCP would not be such a good idea. It was therefore great that the companies worked together and aligned the APIs.
There is a always a lot of work in finalizing specifications. Not only must the text be edited, reviewed, and reviewed, and reviewed … it is also necessary to ensure that the reference implementations and test suites are correct and nicely packaged. The mobile specification was truly an international effort, which did not make it easier. We had teams in Brazil, US, Hungary, Finland, and Bulgaria all working together to finalize the required artifacts. Great work and thanks all for the cooperation and patience!
After the mobile specification was finalized, I received one of the first prototypes from Nokia, an E70, that offered an OSGi framework. I remember racing to the local UPS to get it when it had arrived in Montpellier last June. The first versions needed some work but if everything works out I should get a new version today! I hope to write about many demos with this phone in the next year.
Even after working with the mobile specification for a long time, I still believe that the vision of an OSGi platform in a mobile phone opens a huge market of both niche and mass market applications.
Automotive
From the outside there was not much news from the vehicle/automotive front. Part of the reason is that many members have been too actively involved in the EU sponsored Global System for Telematics (GST) project to spent much time inside the OSGi Alliance. However, a number of companies have remained active in the OSGi vehicle expert group and are working on some very interesting projects. One good decision of this group was to align the vehicle management specification with the mobile device management specification, including diagnostics. The Mobile OMA DM based device management architecture was slightly extended in cooperation with the mobile expert group and then included wholesale. Based on this specification, the vehicle group then developed an abstraction of the vehicle data. This abstraction allows applications to use vehicle data without having to concern themselves with actual models and car types. For example, an application can close all the doors on any compliant car or find out if it is raining.
However, the most important and interesting work has been the navigation model, driven by Olivier Pavé and Rob van den Berg (both Siemens VDO). This navigation model is my personal favorite. By providing a standardized API to the on-board navigation system the OSGi specifications will enable a very large number of interesting applications. A standard navigation system allows third parties to combine their unique data with position and guidance information. These applications can leverage a highly complex navigation system for their own goals, enabling applications that today are not feasible.
For example, the Guide Michelin could offer a bundle that helps you select a restaurant, make the reservations, and guide you to the selected restaurant. Or it could even notify you for a restaurant that is worth a detour. They even could push the envelope and suggest a trip to a restaurant that is worth the trip.
I hope that we will be able to finalize the vehicle work next year. I also have good hopes that some of the results of the GST project will be merged with the OSGi vehicle specifications. Next month (Jan 11) we will have a vehicle workshop in Detroit, looking at the guest list I am sure this will be a very interesting meeting that will put the OSGi vehicle group back on track.
Enterprise
The new kid on the block is the OSGi Enterprise Group! The OSGi Alliance is gaining a momentous amount of steam in the enterprise/server world lately. Though the core technology is very useful as is, there are clearly many requirements that could improve the core and extend the set of available standardized services. Service distribution seems to be at the top of the list, but there are many more areas, as became visible in the workshop. The challenge will be to keep the core lean and mean so that its applicability for the other markets does not get lost. However, I have full confidence that we will succeed there; there are too many people involved that distinctly dislike bloat and the success of POJO’s is a clear sign that our industry is becoming acutely aware of the threats of coupling.
BJ Hargrave (IBM) and I arranged a very successful Enterprise Workshop in September in the San Francisco bay area. This workshop has resulted in an active group of members that are willing to run an OSGi EG. The board approved a new Enterprise Expert Group last month, appointing Tim Diekmann (Siemens) and Eric Newcomer (IONA) as co-chairs.
Spring-OSGi
Before 2006, OSGi technology was put on the map for most people by the Eclipse adoption of the OSGi service platform. This year, the visibility increased significantly because Interface 21, the company behind Spring, decided to support OSGi frameworks. That definitely was a good idea looking at the market response. Adrian Colyer (Interface21) took the lead but the work was shared by Hal Hildebrand (Oracle) and Andy Piper (BEA). The most interesting part for in this process was the reaction of the participants of this group to the OSGi technology. Initially the approach was to “support” OSGi frameworks. During the project, you saw the participants get more enthusiastic about the combination. It would surprise me if OSGi did not become a premier deployment platform for Spring. And vice versa. Spring has some very interesting techniques that overlap the Declarative Services from OSGi. I think it is definitely worth the effort to investigate how these two can work together. I hope Interface 21 will become an OSGi member soon so we can explore the possibilities of specifying the Spring-OSGi collaboration in a future specification.
JSR 277 and JSR 294 module systems
The dramatic highlight of this year is obviously JSR 277 and its brethren JSR 294. JSR 277 attempts to define a module system for the future Java 7 that has a significant overlap with the functionality of the OSGi core framework, JSR 294 is doing the same but restricts itself to changes to the VM and language.
I desperately tried to be on the JSR’s expert group but I was repeatedly denied, despite efforts from people like Doug Lea. Despite this rejection, I tried to evaluate the first draft from JSR 277 without too much bias in my blog. This blog was very well read by the industry. In general the reaction was that Sun should align with other standard groups if they have achieved significant results in the market and not try to reinvent the wheel. Last week I ran into Stanley Ho and Alex Buckley (JSR 277 and JSR 294 spec leads from Sun) at Javapolis an Antwerp. We had a good and pleasant discussion and I hope this will bear some fruit. However, these JSRs will be an ongoing saga so I am curious what next year will bring us.
JSR 291 OSGi Service Platform for J2SE
IBM filed JSR 291 with Glyn Normington (IBM) as lead this year. Though the OSGi specifications were already part of the JCP in JSR 232 (Mobile) it was felt that the visibility of a JME specification by the JCP SE/EE Executive Committee was too low for comfort. JSR 291 was setup as an open expert group that would provide requirements to the OSGi Alliance. All design work is done inside the OSGi Alliance by its members. Interestingly, this process was setup because of intellectual property rights but the separating the requirements gathering process from the API design worked really well in JSR 232 and now in JSR 291. Maybe this separation of concerns should be tried more often in the JCP?
Anyway, JSR 291 is moving along and will produce a specification next year. This will come out as OSGi Release 4.1, likely in April.
Apache Felix
Last year I was asked to support an incubation project at Apache: Felix. This project was based on Oscar, the first and popular open source implementation of the OSGi specifications. It was really nice to see how Richard S. Hall rallied a large group of people with enormous energy around this project. They have been working in incubation status during this year and are now ready to graduate to a top level project. I hope, and expect, that this graduation will push many more Apache projects to first enable their artifacts to run on an OSGi platform and later commit themselves to Felix. Many of Apache’s software projects scream for an underlying OSGi platform that can unify the myriad of plugin systems and class loader abuse present in Apache and other open source code.
OSGi Popularity
The best part for me this year was the tremendous increase in outside awareness of the OSGi Alliance. I remember sadly staring at 8770 Google hits for OSGi in 2005, today we have between 1.5 and 2.5 million hits. I do not really know what the number means, and it seems to depend on where you ask it, but the trend is to my liking. Same for Google alerts. In the beginning of this year I received about one Google Alert per week on the OSGi keyword (My week was good when I received two). Today I receive one every day with 5-10 links. Also the statistics of this blog look like we are on the root of the archetypical hockey stick curve. And best, the last few weeks we received a flurry of requests for membership as well as introductions to speak on conferences. If this trend continues, next year will be very busy, but I will love it!
Future
So much more has happened this year but I am already impressed that certain readers got this far. Let us close with a short outlook for next year.
I expect that 2007 will be the year of the Enterprise. Much of the OSGi Alliance’s effort will go into creating a specification for server side/enterprise applications. I think that the key parties in this market will join the work so we can have a lean mean industry wide platform for server side applications.
I als see many signs that the home automation market is making a revival. This market is where we came from and it would do me great pleasure if we could increase our activities in this market. For the mobile market I expect several vendors to provide OSGi capability. This will create a flurry of development work. However, most of this work will be in specialized applications. The popularity of the platform will then likely become visible in 2008/2009. For 2007, mobile will offer interesting opportunities that can pay off handsomely.
OSGi DevCon
At last, I’d like to put in a plug for the OSGi developers conference which is held jointly with EclipseCon 2007 in Santa Clara 5-8 March. The OSGi track will be focused on server side, embedded, desktop, mobile, and other areas where OSGi platforms are used. If you have an interest in this area, please join us and meet all those other people that are involved in creating an eco-system for universal middleware.
Ok, this blog is way too long. Let me finish by wishing all the readers a fantastic 2007 and I hope to meet many of you next year on a workshop, member meeting, a conference, or online.
Peter Kriens
Evangelism
For me, this year was special because for the first time in my life I spent much of my time in non-technical work. The OSGi Alliance board had appointed me their official evangelist. That was a bit getting used to; suddenly I had to visit conferences, present OSGi technology and just talk about it instead of doing it. The hardest part was learning how to consistently use the word “OSGi” as an adjective for some trademark reason.
This evangelism thing is a bit hectic: EclipseCon in Santa Clara, ApacheCon in San Diego, ApacheCon Europe Dublin, Oredev in Malmö, Javapolis in Antwerp, CCNC in Las Vegas, OOPSLA in Portland, and BCC in Birmingham. Busy, but most of the conferences were very gratifying because I met so many people that are enthusiastic about the OSGi technology. It never ceases to amaze me that there are so many people that can enthusiastically tell me that they had been using OSGi for many, many years; most of the times from companies that I never heard of. But I love those stories and they make my work very rewarding. Next year it looks like there will be a similar schedule so do not hesitate to talk to me when you see me in my “Ask me about OSGi” shirt..
Mobile Specifications
In 2006 we had to finalize the Mobile specifications, which had some political hurdles. The project had run as a joint project between the JCP (JSR 232) and the OSGi Alliance. One of the key aspects has been the device management; a specification that uses the same concepts as the OMA DM protocol. JSR 246 had developed a similar specification for MIDP that was originally based on the work done in the OSGi but unfortunately deviated significantly. Fortunately, the spec leads Jon Bostrom (Nokia) and Jens Petzold (BenQ) realized that two similar but different specifications in the JCP would not be such a good idea. It was therefore great that the companies worked together and aligned the APIs.
There is a always a lot of work in finalizing specifications. Not only must the text be edited, reviewed, and reviewed, and reviewed … it is also necessary to ensure that the reference implementations and test suites are correct and nicely packaged. The mobile specification was truly an international effort, which did not make it easier. We had teams in Brazil, US, Hungary, Finland, and Bulgaria all working together to finalize the required artifacts. Great work and thanks all for the cooperation and patience!
After the mobile specification was finalized, I received one of the first prototypes from Nokia, an E70, that offered an OSGi framework. I remember racing to the local UPS to get it when it had arrived in Montpellier last June. The first versions needed some work but if everything works out I should get a new version today! I hope to write about many demos with this phone in the next year.
Even after working with the mobile specification for a long time, I still believe that the vision of an OSGi platform in a mobile phone opens a huge market of both niche and mass market applications.
Automotive
From the outside there was not much news from the vehicle/automotive front. Part of the reason is that many members have been too actively involved in the EU sponsored Global System for Telematics (GST) project to spent much time inside the OSGi Alliance. However, a number of companies have remained active in the OSGi vehicle expert group and are working on some very interesting projects. One good decision of this group was to align the vehicle management specification with the mobile device management specification, including diagnostics. The Mobile OMA DM based device management architecture was slightly extended in cooperation with the mobile expert group and then included wholesale. Based on this specification, the vehicle group then developed an abstraction of the vehicle data. This abstraction allows applications to use vehicle data without having to concern themselves with actual models and car types. For example, an application can close all the doors on any compliant car or find out if it is raining.
However, the most important and interesting work has been the navigation model, driven by Olivier Pavé and Rob van den Berg (both Siemens VDO). This navigation model is my personal favorite. By providing a standardized API to the on-board navigation system the OSGi specifications will enable a very large number of interesting applications. A standard navigation system allows third parties to combine their unique data with position and guidance information. These applications can leverage a highly complex navigation system for their own goals, enabling applications that today are not feasible.
For example, the Guide Michelin could offer a bundle that helps you select a restaurant, make the reservations, and guide you to the selected restaurant. Or it could even notify you for a restaurant that is worth a detour. They even could push the envelope and suggest a trip to a restaurant that is worth the trip.
I hope that we will be able to finalize the vehicle work next year. I also have good hopes that some of the results of the GST project will be merged with the OSGi vehicle specifications. Next month (Jan 11) we will have a vehicle workshop in Detroit, looking at the guest list I am sure this will be a very interesting meeting that will put the OSGi vehicle group back on track.
Enterprise
The new kid on the block is the OSGi Enterprise Group! The OSGi Alliance is gaining a momentous amount of steam in the enterprise/server world lately. Though the core technology is very useful as is, there are clearly many requirements that could improve the core and extend the set of available standardized services. Service distribution seems to be at the top of the list, but there are many more areas, as became visible in the workshop. The challenge will be to keep the core lean and mean so that its applicability for the other markets does not get lost. However, I have full confidence that we will succeed there; there are too many people involved that distinctly dislike bloat and the success of POJO’s is a clear sign that our industry is becoming acutely aware of the threats of coupling.
BJ Hargrave (IBM) and I arranged a very successful Enterprise Workshop in September in the San Francisco bay area. This workshop has resulted in an active group of members that are willing to run an OSGi EG. The board approved a new Enterprise Expert Group last month, appointing Tim Diekmann (Siemens) and Eric Newcomer (IONA) as co-chairs.
Spring-OSGi
Before 2006, OSGi technology was put on the map for most people by the Eclipse adoption of the OSGi service platform. This year, the visibility increased significantly because Interface 21, the company behind Spring, decided to support OSGi frameworks. That definitely was a good idea looking at the market response. Adrian Colyer (Interface21) took the lead but the work was shared by Hal Hildebrand (Oracle) and Andy Piper (BEA). The most interesting part for in this process was the reaction of the participants of this group to the OSGi technology. Initially the approach was to “support” OSGi frameworks. During the project, you saw the participants get more enthusiastic about the combination. It would surprise me if OSGi did not become a premier deployment platform for Spring. And vice versa. Spring has some very interesting techniques that overlap the Declarative Services from OSGi. I think it is definitely worth the effort to investigate how these two can work together. I hope Interface 21 will become an OSGi member soon so we can explore the possibilities of specifying the Spring-OSGi collaboration in a future specification.
JSR 277 and JSR 294 module systems
The dramatic highlight of this year is obviously JSR 277 and its brethren JSR 294. JSR 277 attempts to define a module system for the future Java 7 that has a significant overlap with the functionality of the OSGi core framework, JSR 294 is doing the same but restricts itself to changes to the VM and language.
I desperately tried to be on the JSR’s expert group but I was repeatedly denied, despite efforts from people like Doug Lea. Despite this rejection, I tried to evaluate the first draft from JSR 277 without too much bias in my blog. This blog was very well read by the industry. In general the reaction was that Sun should align with other standard groups if they have achieved significant results in the market and not try to reinvent the wheel. Last week I ran into Stanley Ho and Alex Buckley (JSR 277 and JSR 294 spec leads from Sun) at Javapolis an Antwerp. We had a good and pleasant discussion and I hope this will bear some fruit. However, these JSRs will be an ongoing saga so I am curious what next year will bring us.
JSR 291 OSGi Service Platform for J2SE
IBM filed JSR 291 with Glyn Normington (IBM) as lead this year. Though the OSGi specifications were already part of the JCP in JSR 232 (Mobile) it was felt that the visibility of a JME specification by the JCP SE/EE Executive Committee was too low for comfort. JSR 291 was setup as an open expert group that would provide requirements to the OSGi Alliance. All design work is done inside the OSGi Alliance by its members. Interestingly, this process was setup because of intellectual property rights but the separating the requirements gathering process from the API design worked really well in JSR 232 and now in JSR 291. Maybe this separation of concerns should be tried more often in the JCP?
Anyway, JSR 291 is moving along and will produce a specification next year. This will come out as OSGi Release 4.1, likely in April.
Apache Felix
Last year I was asked to support an incubation project at Apache: Felix. This project was based on Oscar, the first and popular open source implementation of the OSGi specifications. It was really nice to see how Richard S. Hall rallied a large group of people with enormous energy around this project. They have been working in incubation status during this year and are now ready to graduate to a top level project. I hope, and expect, that this graduation will push many more Apache projects to first enable their artifacts to run on an OSGi platform and later commit themselves to Felix. Many of Apache’s software projects scream for an underlying OSGi platform that can unify the myriad of plugin systems and class loader abuse present in Apache and other open source code.
OSGi Popularity
The best part for me this year was the tremendous increase in outside awareness of the OSGi Alliance. I remember sadly staring at 8770 Google hits for OSGi in 2005, today we have between 1.5 and 2.5 million hits. I do not really know what the number means, and it seems to depend on where you ask it, but the trend is to my liking. Same for Google alerts. In the beginning of this year I received about one Google Alert per week on the OSGi keyword (My week was good when I received two). Today I receive one every day with 5-10 links. Also the statistics of this blog look like we are on the root of the archetypical hockey stick curve. And best, the last few weeks we received a flurry of requests for membership as well as introductions to speak on conferences. If this trend continues, next year will be very busy, but I will love it!
Future
So much more has happened this year but I am already impressed that certain readers got this far. Let us close with a short outlook for next year.
I expect that 2007 will be the year of the Enterprise. Much of the OSGi Alliance’s effort will go into creating a specification for server side/enterprise applications. I think that the key parties in this market will join the work so we can have a lean mean industry wide platform for server side applications.
I als see many signs that the home automation market is making a revival. This market is where we came from and it would do me great pleasure if we could increase our activities in this market. For the mobile market I expect several vendors to provide OSGi capability. This will create a flurry of development work. However, most of this work will be in specialized applications. The popularity of the platform will then likely become visible in 2008/2009. For 2007, mobile will offer interesting opportunities that can pay off handsomely.
OSGi DevCon
At last, I’d like to put in a plug for the OSGi developers conference which is held jointly with EclipseCon 2007 in Santa Clara 5-8 March. The OSGi track will be focused on server side, embedded, desktop, mobile, and other areas where OSGi platforms are used. If you have an interest in this area, please join us and meet all those other people that are involved in creating an eco-system for universal middleware.
Ok, this blog is way too long. Let me finish by wishing all the readers a fantastic 2007 and I hope to meet many of you next year on a workshop, member meeting, a conference, or online.
Peter Kriens
Monday, December 11, 2006
PHP on an OSGi Framework?
Last week (2006/12/08) the OSGi Alliance had a webinar for their members. Last month, Susan Schwarze from marketing had asked me very sweetly to provide a demonstration for this webinar. After some soul searching I had proposed to make a demonstration using the OSGi Framework in a multi language setting. The scenario of the demo was simple: show how to start a framework, get some basic bundles installed, install some applications from the OSGi Bundle Repository (OBR) to show it all works, and then as grande finale, install a PHP interpreter and port a PHP Wiki to an OSGi bundle in real time. And this all must happen in 20 minutes.
The most interesting part of the demo was obviously the PHP part. I always thought that having a PHP interpreter on an OSGi framework would be really cool (However, hard this is to explain to my kids and wife)! With an OSGi based PHP interpreter you can deploy PHP applications as OSGi bundles. This is attractive to companies that today use PHP on remote servers as well as device vendors that want to design their web sites in PHP.
The demo was not the first time I had come up with PHP. I had searched the web for a PHP interpreter written in Java already several times but I had never found one until I discussed this demo with BJ Hargrave. He mentioned that there was an open source project/company doing a Java based PHP interpreter: Quercus. The interpreter is a part in the bigger Resin webserver. With my usual optimism, I though: “How hard can it be to port this interpreter to an OSGi bundle?”
Well, hard.
It started all very hopeful because there was a class Quercus Servlet and the OSGi HttpServer supports servlets very well. Unfortunately, the Quercus Servlet was assuming it ran inside the Resin server, making it impossible to use with another web server. Writing my own servlet was luckily only a few hours. Unfortunately, I could not resist simplifying the Quercus Servlet which added a few unnecessary hours to undo these simplifications when I understood why the complications were there in the first place.
The next problem I ran into was the file system. Normally PHP assumes that all the sources are in the file system and in our case the files had to come from a bundle. Quercus/Resin supported a virtual file system but it took some time to find out how you could provide a new source for files. It turned out you had to write your own Path sub-class and then set the root path to an instance of your type. All other files are then relative to that root. Reading was very easy to program, I just mapped all the read primitives to the resource access in the bundle. However, writing was a lot harder. At first, I decided to allow the bundle that carries the PHP files to specify its URI and a set of writable directories. That is, I architected the following headers:
PHP-Path: /mywiki
PHP-Data config, data, wiki.d
I got this to work but ran into the problem that some PHP programs actually re-write their own source files. Having special writable directories forced me to modify several PHP programs, obviously not a desirable architecture. Then it suddenly hit me (under the shower!) that I could just do the rather standard technique of copy-on-write. So I got rid of the PHP-Data header altogether. A read would now always first look on the file system (actually the bundle’s data directory) and then in the bundle’s resources. A bit tricky was the handling of directories; they should be automatically created when you wrote a file that also resided in the jar. After this (awfully late) insight, it became quite easy and I could use several PHP programs from the net without any modifications. This worked so well that I could show off this model with a real functioning Wiki that was wrapped in a bundle directly from the Internet!
However, I wanted one more splash and I found this splash in the Quercus modules. The Quercus interpreter was designed to be very modular; all its commands were written in separate modules which were loaded at startup. This model was too tantalizing to not use it to show off the dynamic capabilities of the OSGi Framework. Would it not be impressive if you could install a new Quercus Module on the fly as a bundle (Again, an excitement my family rarely shares)? So I decided to create a Service Tracker that listened to Quercus Module services. Those service objects were then added/removed as a module to the Quercus interpreter. I had to write a new remove function inside the interpreter because, like so many programs, it had no concept of modules that could go away.
Quercus Modules could now be written as OSGi bundles. This turned out very simple to do because bnd (a program/plugin that creates bundles from a recipe) supports declarative services as well. I could just create a small bnd file that builds the bundle and indicates what class should be registered under the QuercusModule interface. Like:
Private-Package: com.caucho.quercus.lib.zip
Import-Package: *
Service-Component: com.caucho.quercus.lib.zip.ZipModule;
provide:=com.caucho.quercus.module.QuercusModule
I had decided early on that I did not want to do the demo live. I guess I am becoming chicken at my old age; the chance that something fails is just a bit too high. I therefore used Instant Demo. This application can record everything that happens in a window. You can edit this recording and provide sound as well. Instant Demo then converts the total set of files to a flash file so you can play it in any browser. I must say, using such a program is a humbling experience because every little error requires a restart. It was painful to have to repeat the same actions over and over again just to get it right.
Anyway, after a lot of hard work I finally succeeded in making four short movies about this demo. If you are interested, download this presentation and play it. Let me know if you think we should have more of such demos. Should we also do this for JRuby and run Ruby-On-Rails applications on OSGi Frameworks? Hmm, sounds cool!
Peter Kriens
P.S. The Quercus interpreter is not available because the code is not industrialized. It really worked for a demo (nothing was faked) but it requires more work to harden it for real use. However, I will contact the resin people to see if they like to modularize their webserver. Their web server screams for OSGi technology!
The most interesting part of the demo was obviously the PHP part. I always thought that having a PHP interpreter on an OSGi framework would be really cool (However, hard this is to explain to my kids and wife)! With an OSGi based PHP interpreter you can deploy PHP applications as OSGi bundles. This is attractive to companies that today use PHP on remote servers as well as device vendors that want to design their web sites in PHP.
The demo was not the first time I had come up with PHP. I had searched the web for a PHP interpreter written in Java already several times but I had never found one until I discussed this demo with BJ Hargrave. He mentioned that there was an open source project/company doing a Java based PHP interpreter: Quercus. The interpreter is a part in the bigger Resin webserver. With my usual optimism, I though: “How hard can it be to port this interpreter to an OSGi bundle?”
Well, hard.
It started all very hopeful because there was a class Quercus Servlet and the OSGi HttpServer supports servlets very well. Unfortunately, the Quercus Servlet was assuming it ran inside the Resin server, making it impossible to use with another web server. Writing my own servlet was luckily only a few hours. Unfortunately, I could not resist simplifying the Quercus Servlet which added a few unnecessary hours to undo these simplifications when I understood why the complications were there in the first place.
The next problem I ran into was the file system. Normally PHP assumes that all the sources are in the file system and in our case the files had to come from a bundle. Quercus/Resin supported a virtual file system but it took some time to find out how you could provide a new source for files. It turned out you had to write your own Path sub-class and then set the root path to an instance of your type. All other files are then relative to that root. Reading was very easy to program, I just mapped all the read primitives to the resource access in the bundle. However, writing was a lot harder. At first, I decided to allow the bundle that carries the PHP files to specify its URI and a set of writable directories. That is, I architected the following headers:
PHP-Path: /mywiki
PHP-Data config, data, wiki.d
I got this to work but ran into the problem that some PHP programs actually re-write their own source files. Having special writable directories forced me to modify several PHP programs, obviously not a desirable architecture. Then it suddenly hit me (under the shower!) that I could just do the rather standard technique of copy-on-write. So I got rid of the PHP-Data header altogether. A read would now always first look on the file system (actually the bundle’s data directory) and then in the bundle’s resources. A bit tricky was the handling of directories; they should be automatically created when you wrote a file that also resided in the jar. After this (awfully late) insight, it became quite easy and I could use several PHP programs from the net without any modifications. This worked so well that I could show off this model with a real functioning Wiki that was wrapped in a bundle directly from the Internet!
However, I wanted one more splash and I found this splash in the Quercus modules. The Quercus interpreter was designed to be very modular; all its commands were written in separate modules which were loaded at startup. This model was too tantalizing to not use it to show off the dynamic capabilities of the OSGi Framework. Would it not be impressive if you could install a new Quercus Module on the fly as a bundle (Again, an excitement my family rarely shares)? So I decided to create a Service Tracker that listened to Quercus Module services. Those service objects were then added/removed as a module to the Quercus interpreter. I had to write a new remove function inside the interpreter because, like so many programs, it had no concept of modules that could go away.
Quercus Modules could now be written as OSGi bundles. This turned out very simple to do because bnd (a program/plugin that creates bundles from a recipe) supports declarative services as well. I could just create a small bnd file that builds the bundle and indicates what class should be registered under the QuercusModule interface. Like:
Private-Package: com.caucho.quercus.lib.zip
Import-Package: *
Service-Component: com.caucho.quercus.lib.zip.ZipModule;
provide:=com.caucho.quercus.module.QuercusModule
I had decided early on that I did not want to do the demo live. I guess I am becoming chicken at my old age; the chance that something fails is just a bit too high. I therefore used Instant Demo. This application can record everything that happens in a window. You can edit this recording and provide sound as well. Instant Demo then converts the total set of files to a flash file so you can play it in any browser. I must say, using such a program is a humbling experience because every little error requires a restart. It was painful to have to repeat the same actions over and over again just to get it right.
Anyway, after a lot of hard work I finally succeeded in making four short movies about this demo. If you are interested, download this presentation and play it. Let me know if you think we should have more of such demos. Should we also do this for JRuby and run Ruby-On-Rails applications on OSGi Frameworks? Hmm, sounds cool!
Peter Kriens
P.S. The Quercus interpreter is not available because the code is not industrialized. It really worked for a demo (nothing was faked) but it requires more work to harden it for real use. However, I will contact the resin people to see if they like to modularize their webserver. Their web server screams for OSGi technology!
Subscribe to:
Posts (Atom)