This blog is terribly late due to a security issue at our provider that interfered with a spam filter on Gmail :-( Excuses.
The OSGi BOFs that we had organized were a complete success. The Thursday BOF attracted over a hundred people; this is incredibly successful considering that this BOF was at 8.30 PM while the “JavaOne After Dark Bash” was going on at the same time. The BOF started a bit scary because the organization had not setup the tables I had requested. Fortunately, we could steal some tables from the public space. We started with a presentation by BJ Hargrave. He was regularly interrupted with questions. Often he did not get a chance to answer the questions because someone in the audience did. This created a very interactive feeling, which worked well. After 20 minutes we asked the people to go to the demonstrations or get an OSGi T-Shirt. Though the T-Shirts won the popularity contest in the beginning, the demonstrations then won hands on. We had presentations from ProSyst, Siemens, Paremus, and Luminis. While all this was going on we presented a slide show of 240 slides (!) that consisted of presentations that I had received about OSGi related projects (Thanks all submitters!). In the end we had to be forcefully removed from the room for the next BOF.
The BOF on Wednesday, Component Programming was also quite successful. We attracted around 50 people, which at 10.30 PM and a million parties going on is amazing. I presented the introduction to the OSGi. Thomas Watson gave a presentation how Eclipse can be used to program components and Jeff McAffer talked about the Slug. We got lots of good questions and the feedback was very positive.
We also had lots of questions about the OSGi during the conference. I had made some nice shirts with “Ask me about OSGi” on the back and this generated the desired response. I also had printed stickers with this slogan and hand wrestled everybody that knew anything about the OSGi specifications to wear them. It was a bit awkward that there were actually a surprising number of people that had never heard of “OSGi”. Well, I guess there still is a lot of evangelizing to do.
Very positive are the reactions about OSGi and the enterprise. I have met many people that are extremely excited about OSGi technology being used with Spring and other enterprise technologies. I think the coming year will be very exciting with all the developments in this area. The OSGi Alliance really must organize a workshop on OSGi in the Enterprise soon.
From an OSGi Alliance point of view JavaOne was a great success!
The conference itself was, from my point of view, rather disappointing. It is big and it is monopolized by Sun; it was not easy to find a technical presentation by a non-Sun employee. I actually think I got a bit of a rash from too much sun exposure …
Peter Kriens
OSGi Blog
Monday, May 22, 2006
Wednesday, May 10, 2006
Why Installers are Evil
About 7 years ago the OSGi Alliance forced me to switch from a Macintosh to a PC. Ok, they did not wrestle my arm but I needed access to the latest (and not so greatest) version of Java to do my work properly. With regret, I parted with my dear Macintosh and dived into the wonderful new world of PCs. Now, as a Mac owner you are bound to be a bit naïve about computing, you actually think it is not such a big deal. Well, you find out how wrong you are when you migrate to the PC.
The first time I stepped into the PC mess was when I wanted to clean up my disk. Excited about all the free downloadable goodies (and baddies) I had collected quite a set. I had installed these applications in different places on my hard disk and and decided to group them nicely in a folder. As you know, and I do know now, that is a distinctly stupid idea. It is useful, but still stupid because Windows does not want you to do that and in a fight between you and Windows, you find out why many call it “Win”.
Well, maybe that is too much credit; Windows has actually no clue that you moved the application file; the next time it needs the application it will blindly look in the old place and fail miserably with a cryptic message. After having too many of those cryptic message, I decided to reformat and reinstall Windows. I learned my lesson and after that occasion I never cleaned up anymore (My wife sometimes has a problem with this behavior, but fortunately she is significantly more benign than Windows).
So what is this rant doing in an OSGi related blog? Well, there is a lesson that we learned early in the OSGi: installers are evil. This is easy to say, but I will try to make it clear to you. Let us analyze why it became so painful to move these files around on a PC and not on a Mac. On the Mac, an application is a single file (on System X, it is actually a directory). This file is a simple database that consists of the code resources and other resources like configuration files, icons, html pages, properties, translations, etc. Anything that the program needs is stored in this single file. This file is introspective; this means that other programs can inspect the contents and react according to this content. We call these progams/utilities that react to this content extensions. Extensions are extremely useful.
Application files have a special type in the Mac, and this type is recognized by the disk operating system. Whenever you copy or move an application file the OS will track this in an application database. The OS actually looks in the application file's database and finds the information about the files it can open and other useful aspects. Ergo, you can move the applications around as much as you want; the OS just finds them when you need them. It has this genuine advantage that if you trash the application file, you really cleaned up all aspects of that application.
So why does Windows fail so miserably when you move an application to another directory? Well, Windows does not track the applications because it has installers! An installer is an ingenious piece of software that most PC users think is a necessity of nature. It is usually zipped and then compressed after which it is squeezed, requires a lot of thinking, asks you questions you have no clue how to answer or could not care less (fortunately there are defaults), shows lines that you cannot read because they are replaced too quick (which at least gives you the cozy feeling something is happening), splatters files all over your hard disk (sometimes in quite unexpected places), overwrites important DLLs, then spends minutes updating the registry, after which it asks you to read a README file that is incomprehensible until you need it because the application balks at you; unfortunately at that time you have no clue where to find it anymore. And if this all went well, you can run the application. The fact that this is all overkill is demonstrated by Eclipse that has no installer.
You would expect that for security reasons Windows would have a registry of these installed applications, and a way to forcefully uninstall an application. Well, you thought wrong. There is something of a voluntary registry that the installers can fill in if they feel like it. However, the information is not guaranteed by Windows. Worse, to uninstall an application Windows will ask the original installer to remove all the files, remove the registry changes, and generally clean up. The uninstaller sometimes fails because a file is missing or something else is wrong. At such a moment you have an orphan application that likely remains as a bit-squatter until you have to reformat your hard disk. Lots of these orphans can bring that time significantly closer.
When we designed the OSGi framework we were sure that the Windows Installer model was not where we wanted to go; the Mac model looked a lot more attractive. Fortunately, the JAR file model of Java is an even more powerful format than the Mac resource file. The ZIP file format is flexible enough to allow all kinds of application extension scenarios based on resource name and directory. However, due to the security risk attached to running applications we decided to strictly control all the aspects of the installation, update, and uninstallation; all these are primitive in the OSGi, there is no untrusted code involved. OSGi applications (bundles) can normally not move files to odd places and modify all kinds of settings. OSGi applications can declare information in their JAR files and other applications or utilities can inspect this information and act upon it. This is a clear example of Inversion of Control or IoC: “Don't call us, we'll call you!” For example, an application using Declarative Services contains an XML file with the proper setup. When the application is installed the Declarative Service implementation will find this file, the application does not have to register.
This is an incredibly robust model that prevents a lot of abuse and errors. As a simple example, assume an application fails all the time and you want to uninstall it. The fact that the application fails makes it very likely that the uninstaller will also fail. Leaving remnants in lots of places on your system. An OSGi application can always be uninstalled by the Framework. The uninstallation will notify any applications that performed some function on behalf of the uninstalled application. These other applications are in a good state and can therefore cleanup correctly, regardless of the state of the culprit.
The application model of the OSGi has learned from its predecessors and provides excellent characteristics. Most importantly: An overview what you have really installed and a guarantee that you can always throw anything away.No code can run without the Framework being aware of it.
Every time when new groups enter the OSGi Alliance we have the Installer discussion again. Every time people ask me how it can be wrong when Windows does it that way? 100 million users can't be wrong, can they? I leave that up to you to figure it out.
Peter Kriens
The first time I stepped into the PC mess was when I wanted to clean up my disk. Excited about all the free downloadable goodies (and baddies) I had collected quite a set. I had installed these applications in different places on my hard disk and and decided to group them nicely in a folder. As you know, and I do know now, that is a distinctly stupid idea. It is useful, but still stupid because Windows does not want you to do that and in a fight between you and Windows, you find out why many call it “Win”.
Well, maybe that is too much credit; Windows has actually no clue that you moved the application file; the next time it needs the application it will blindly look in the old place and fail miserably with a cryptic message. After having too many of those cryptic message, I decided to reformat and reinstall Windows. I learned my lesson and after that occasion I never cleaned up anymore (My wife sometimes has a problem with this behavior, but fortunately she is significantly more benign than Windows).
So what is this rant doing in an OSGi related blog? Well, there is a lesson that we learned early in the OSGi: installers are evil. This is easy to say, but I will try to make it clear to you. Let us analyze why it became so painful to move these files around on a PC and not on a Mac. On the Mac, an application is a single file (on System X, it is actually a directory). This file is a simple database that consists of the code resources and other resources like configuration files, icons, html pages, properties, translations, etc. Anything that the program needs is stored in this single file. This file is introspective; this means that other programs can inspect the contents and react according to this content. We call these progams/utilities that react to this content extensions. Extensions are extremely useful.
Application files have a special type in the Mac, and this type is recognized by the disk operating system. Whenever you copy or move an application file the OS will track this in an application database. The OS actually looks in the application file's database and finds the information about the files it can open and other useful aspects. Ergo, you can move the applications around as much as you want; the OS just finds them when you need them. It has this genuine advantage that if you trash the application file, you really cleaned up all aspects of that application.
So why does Windows fail so miserably when you move an application to another directory? Well, Windows does not track the applications because it has installers! An installer is an ingenious piece of software that most PC users think is a necessity of nature. It is usually zipped and then compressed after which it is squeezed, requires a lot of thinking, asks you questions you have no clue how to answer or could not care less (fortunately there are defaults), shows lines that you cannot read because they are replaced too quick (which at least gives you the cozy feeling something is happening), splatters files all over your hard disk (sometimes in quite unexpected places), overwrites important DLLs, then spends minutes updating the registry, after which it asks you to read a README file that is incomprehensible until you need it because the application balks at you; unfortunately at that time you have no clue where to find it anymore. And if this all went well, you can run the application. The fact that this is all overkill is demonstrated by Eclipse that has no installer.
You would expect that for security reasons Windows would have a registry of these installed applications, and a way to forcefully uninstall an application. Well, you thought wrong. There is something of a voluntary registry that the installers can fill in if they feel like it. However, the information is not guaranteed by Windows. Worse, to uninstall an application Windows will ask the original installer to remove all the files, remove the registry changes, and generally clean up. The uninstaller sometimes fails because a file is missing or something else is wrong. At such a moment you have an orphan application that likely remains as a bit-squatter until you have to reformat your hard disk. Lots of these orphans can bring that time significantly closer.
When we designed the OSGi framework we were sure that the Windows Installer model was not where we wanted to go; the Mac model looked a lot more attractive. Fortunately, the JAR file model of Java is an even more powerful format than the Mac resource file. The ZIP file format is flexible enough to allow all kinds of application extension scenarios based on resource name and directory. However, due to the security risk attached to running applications we decided to strictly control all the aspects of the installation, update, and uninstallation; all these are primitive in the OSGi, there is no untrusted code involved. OSGi applications (bundles) can normally not move files to odd places and modify all kinds of settings. OSGi applications can declare information in their JAR files and other applications or utilities can inspect this information and act upon it. This is a clear example of Inversion of Control or IoC: “Don't call us, we'll call you!” For example, an application using Declarative Services contains an XML file with the proper setup. When the application is installed the Declarative Service implementation will find this file, the application does not have to register.
This is an incredibly robust model that prevents a lot of abuse and errors. As a simple example, assume an application fails all the time and you want to uninstall it. The fact that the application fails makes it very likely that the uninstaller will also fail. Leaving remnants in lots of places on your system. An OSGi application can always be uninstalled by the Framework. The uninstallation will notify any applications that performed some function on behalf of the uninstalled application. These other applications are in a good state and can therefore cleanup correctly, regardless of the state of the culprit.
The application model of the OSGi has learned from its predecessors and provides excellent characteristics. Most importantly: An overview what you have really installed and a guarantee that you can always throw anything away.No code can run without the Framework being aware of it.
Every time when new groups enter the OSGi Alliance we have the Installer discussion again. Every time people ask me how it can be wrong when Windows does it that way? 100 million users can't be wrong, can they? I leave that up to you to figure it out.
Peter Kriens
Friday, May 5, 2006
The OSGi Needs You!
More than 20 years ago Brad Cox promoted his Software ICs. In those 20 years, Object Oriented technologies have solved many problems and during this time frame it has become a tremendous success, Java could not have existed without OO. However, so far we did not succeed in creating the component market that Brad Cox once envisioned. The key reason that it never worked out is that software is the wrong name for what we are producing; brittleware would have been a better name. Despite the improvements in tools and knowledge, component oriented systems turned out to be very difficult to engineer. What are the reasons that this attractive idea is so hard to achieve?
The following is my list of requirements that must be satisfied for a successful component model:
This is clearly not a list that a single company can develop as a product. It is also clearly more than just the Java environment, though Java provides a big chunk. A cross industry component model requires the cooperation of large number of companies to create the required eco system. This is exactly what the OSGi Alliance has done. Over the past 7 years they developed a component framework that satisfies all these technical requirements, and more.
We seem to have achieved our technical goal as is proven by the large number of applications building upon OSGi technology. However, to achieve our long term vision of “components roaming freely”, we need more, much more. To achieve adoption in the mainstream software shops we need to cross a chasm; a chasm that any new technology must cross. This is a difficult passage because most people resist change.
To resist the change, people currently use the following arguments against OSGi based solutions:
Hardware cost? We have been telling people that the cost issue will go away and today I am glad to tell you, it really did go away. OK, kidding, but with flash going for 1 cent per MByte the hardware cost issues is really on its way of becoming moot. There is of course the extra processor capacity required and the VM can be a significant cost but more and more vendors are providing good alternatives to SUN’s VMs for reasonable cost. The OSGi component model simplifies the use of alternative VMs so much that the Apache Harmony VM (free!) has chosen OSGi at its foundation.
Testing is another problem. Mainstream developers, project managers, and general managers are skeptical that anything dynamic in their software configuration can really work. Most seasoned developers have lived through nightmarish situations with late and buggy software. To manage their processes, they have installed safeguards and measures to ensure that the product that goes out the door is actually working in at least one context. Often these safeguards require a frozen system when it is tested; something that is very much at odds with a component model where the components each have a different life cycle.
This fear of change in itself is at odds with where the mainstream of computing is heading. Devices are networked and must survive in a continuously changing world. As an industry we must evolve to a model where software is less brittle and more robust. Freezing the world during testing, as so many do, just means that many, many errors will be found by the users in the real world where time tends not to be frozen. Facing the problems up front is the better solution.
Unproven? Type in OSGi in Google and browse … Should the US National Guard order a system to detect bio terror attacks on an unproven technology? Would Eclipse knowingly put unproven technology under the hood knowing that their IDE will be downloaded millions of times? Don’t think so, the OSGi specifications have already clearly proven themselves over and over, in many different situations.
I am convinced that the OSGi Alliance provides a crucial function to achieve this ubiquitous computing model we have trying to achieve for so many years. We have the technology, we have the enthusiastic early adopters, and we have the ears of many in the mainstream markets, among them some of the largest software companies in the world like IBM, Siemens, and Nokia. However, to continue, we also need your help as well. To achieve the credibility needed to cross the chasm, the OSGi Alliance needs many more mainstream companies to use and help promote the OSGi specifications. You can do this for the pay off, which can be big: a cross industry component market will be gigantic. You can do this because a unified component market will simplify your development and reduce its cost. You can do it because you share our vision and want to help it come true. Or you can do this because you believe this is the right way to go. Whatever your motives, the OSGi Alliance needs your support as a member, as an adopter associate, or if you cannot afford the fee, as an enthusiastic user.
Hope to see you on the next OSGi member meeting!
Peter Kriens
The following is my list of requirements that must be satisfied for a successful component model:
- Portability – The model must be supported on virtually any device, or critical market size will not be achieved.
- Deployment & Management – Today’s devices are all networked. This will require remote management and deployment of components.
- Loosely coupled – Objects never made it as components because they were too tightly coupled to their environment. Components should be allowed to mix and match.
- Security – A component model for networked devices requires security to prevent the system against viruses, malicious code, and erroneous code.
- Dynamic – Component updates should not cause the system to shut down completely. Change is an integral part of today’s computing world.
- Non Intrusive – The programming model should not overly restrict the programmer or create unnecessary intrusions in plain old java code.
This is clearly not a list that a single company can develop as a product. It is also clearly more than just the Java environment, though Java provides a big chunk. A cross industry component model requires the cooperation of large number of companies to create the required eco system. This is exactly what the OSGi Alliance has done. Over the past 7 years they developed a component framework that satisfies all these technical requirements, and more.
We seem to have achieved our technical goal as is proven by the large number of applications building upon OSGi technology. However, to achieve our long term vision of “components roaming freely”, we need more, much more. To achieve adoption in the mainstream software shops we need to cross a chasm; a chasm that any new technology must cross. This is a difficult passage because most people resist change.
To resist the change, people currently use the following arguments against OSGi based solutions:
- Too expensive or too slow
- Hard to test
- Unproven
Hardware cost? We have been telling people that the cost issue will go away and today I am glad to tell you, it really did go away. OK, kidding, but with flash going for 1 cent per MByte the hardware cost issues is really on its way of becoming moot. There is of course the extra processor capacity required and the VM can be a significant cost but more and more vendors are providing good alternatives to SUN’s VMs for reasonable cost. The OSGi component model simplifies the use of alternative VMs so much that the Apache Harmony VM (free!) has chosen OSGi at its foundation.
Testing is another problem. Mainstream developers, project managers, and general managers are skeptical that anything dynamic in their software configuration can really work. Most seasoned developers have lived through nightmarish situations with late and buggy software. To manage their processes, they have installed safeguards and measures to ensure that the product that goes out the door is actually working in at least one context. Often these safeguards require a frozen system when it is tested; something that is very much at odds with a component model where the components each have a different life cycle.
This fear of change in itself is at odds with where the mainstream of computing is heading. Devices are networked and must survive in a continuously changing world. As an industry we must evolve to a model where software is less brittle and more robust. Freezing the world during testing, as so many do, just means that many, many errors will be found by the users in the real world where time tends not to be frozen. Facing the problems up front is the better solution.
Unproven? Type in OSGi in Google and browse … Should the US National Guard order a system to detect bio terror attacks on an unproven technology? Would Eclipse knowingly put unproven technology under the hood knowing that their IDE will be downloaded millions of times? Don’t think so, the OSGi specifications have already clearly proven themselves over and over, in many different situations.
I am convinced that the OSGi Alliance provides a crucial function to achieve this ubiquitous computing model we have trying to achieve for so many years. We have the technology, we have the enthusiastic early adopters, and we have the ears of many in the mainstream markets, among them some of the largest software companies in the world like IBM, Siemens, and Nokia. However, to continue, we also need your help as well. To achieve the credibility needed to cross the chasm, the OSGi Alliance needs many more mainstream companies to use and help promote the OSGi specifications. You can do this for the pay off, which can be big: a cross industry component market will be gigantic. You can do this because a unified component market will simplify your development and reduce its cost. You can do it because you share our vision and want to help it come true. Or you can do this because you believe this is the right way to go. Whatever your motives, the OSGi Alliance needs your support as a member, as an adopter associate, or if you cannot afford the fee, as an enthusiastic user.
Hope to see you on the next OSGi member meeting!
Peter Kriens
Subscribe to:
Posts (Atom)