A few weeks ago I spotted an article that stated OSGi as "the next big thing after Spring". Actually, there are a whole bunch of articles, blogs etc... out there in the wild - all praising OSGi as one of the next "big things" that will reach many sectors in the Java space. I don't doubt this and I do admit that OSGi solves many problems and supports top notch architectural models such as high modularization and micro services. However, on the other side - after using OSGi for a couple of years and see how it performs (in the meaning of development performance) in different areas I really have doubts about OSGi.
Here are some things I love and hate about OSGi:
The concept of creating a system out of even hundred of typically pretty small bundles is very good. The really good thing about bundles in OSGi is that they have defined borders - not only in the sense of dependencies but also in the sense of runtime. Each bundle can serve as a micro application, having it's own lifecycle, having it's own citizens and each bundle can carefully decide which objects to expose to the outside world. Carefully adopted, this leads to a system with loose coupling and a high probability for reuse. However, at the same time the lifecycle model of OSGi makes life complicated. Actually, tracking services and managing all the aspects of what to do when services come and go is nasty (I've seen systems that had tried to manage these things with pretty large state machines in their bundle activators ... this is boiler plate that no one wants to write!) - Luckily, there's Spring DM out there that helps us manage these things. To be honest - I'd never even think about starting an OSGi project without using Spring DM. Altough Spring DM greatly reduced the complexity of bundle setup, it's still complicated. I still need to start the OSGi runtime and install and start the bundles - and I have to take care that all other bundles that I need are installed and started too ...
I personally think that we - as developers - have to enforce constraints on ourselves! We have to automatically check the defined constraints and let the build process fail if we e.g. accessed some classes we were not meant to access. OSGi's versioning concept (as a side effect ...) makes a first step of bringing the enforcement of architectural constraints to the developer's daily life by definining export and import packages ... and introduces a completely new dimension of problems! OSGi resolves versioning issues at runtime by giving each bundle it's own class loader and let that class loader see exactly the classes in the versions it depends on. Sounds neat ? Yes it is - and you will curse it! This leads to a really new dimension of problems, because it really changes the way your environment works. Your code passes all your unit tests, but once you start it in an OSGi runtime it crashes; Libraries crash because the enhance their classes at runtime; Singletons designed as static objects get created more than once; The story continues ... You typically end up in a situation where you constantly tweak your modules build specifications (that creates your manifest) and spread absolutely counter-intuitive dependencies across your whole system. Spring DM also addressed this topic - by adding some directives to your services and tweaking around with the class loaders ... and you still need to tweak and spread dependencies. I'm shivering at the thought that I need this workaround to even get started ...
Especially that class loading hell leads to a new dimension of problems. You will quickly notice that you cannot go on without a strong and automated integration testing suite that incorporates OSGi (another thing that Spring DM supports).
Now - what's my conclusion about all that stuff. First of all - before considering OSGi I would really check if I have the ultimate requirement to exchange bundles at runtime without shutting down the system - and even in this case I would really revisit these requirements and see if I can limit them to a corner where I could use other techniques for dynamically loading modules at runtime. There are alternatives out there fostering the architectural principles (e.g. you could use an IOC container, use independent container instances to enforce module independence, do constraint checking at build time, etc...) - most of these things are much closer to the KISS principle, avoid all the additional boiler plate code and build configuration - and therefore allow you to stay more agile.
I almost forgot the title :o) There's another technology out there with pretty the same characteristics (in terms of things being complicated): EJB. Spring is propably the most popular example of what can happen when a technology is way too complicated and makes life hard. Maybe we will experience the same thing with OSGi in in the next years ? I cannot say ... however, if this is the case, then there is a certain irony in that SpringSource now heavily supports OSGi :o)
Cheers,
Peter
Subscribe to:
Post Comments (Atom)
Posts
I agree with many of your thoughts, but what do you mean by "passes all your unit tests, but once you start it in an OSGi runtime it crashes"? - As long as I understand the OSGi framework right the dependency check is done at bundle-startup. It should only be a small step into an additional check if the loading of a bundle is possible at the target system. (could be part of the continous integration process).
ReplyDeleteSo I can't see a fundamental paradigm leak. More often it’s an inadequate integration into the current development process.
Another question is if the description of the dependencies between bundles is sufficient? – I’m sure it's not but it is a first step which should be enhanced in the future.
Thanks for the first comment on my blog :o))
ReplyDeleteI do not want my objects to be bound to a runtime environment such as OSGI - therefore I unit-test my classes with the standard Java runtime environment (and hence with the default class loader). The behavior of the classloader changes dramatically once OSGi is involved - This is typically not a problem with your dependencies, however, the problems come with third party libraries (e.g. Hibernate creating new classes, JMS that serializes objects, RMI, etc...). These are exactly the things I typically mock for my unit test and that do not behave the same within an OSGi environment (mainly due to class visibility). Especially serialization and bytecode enhancement issues cannot be checked at startup but happen during runtime. To check these issue you need to add full-fledged, OSGi based integration tests. This is actually a very good thing - the more meaningful tests - the better :o) However, I experienced that it makes the build process pretty complicated. Since the integration tests are very time consuming, they are not run in the smoke tests and therefore artifacts that are initially tested green may not be green at all. Ok - that's sometimes cumbersome but it's the nature of a smoke test... what bothered me more was that the integration tests were very (!) vulnerable to changes within other artifacts (bundles). You introduce a new dependency in one of your bundles; coincidentally the object with the new dependency is serialized (e.g. because it is somehow sent through JMS); the chances that some of your integration tests fail are good. The interesting word here is "some". I found myself haunting for hours because the failure issues is not local to my artifact any more...
Once you have the whole thing running smoth and once it's easy to code integration tests I actually appreciate it ... but there still remain some doubts, where my stomach says that this does not agree with the KISS principle :o)
As most of the frameworks, they only work well if used with POJOs solely. Same goes for OSGi, I really like it for small independent services, which do real work.
ReplyDeleteBut when it comes to integration with other frameworks like Hibernate, JMS and others which are not OSGi aware and use dirty things like class modifications and dynamic class loading, I agree it's getting nasty.
Unfortunately, this is not an OSGi only problem. The argument is valid for nearly every Framework. So you might write as well an article, questioning: Why do different frameworks not work together?
Probably because Framework integration Framework creators like SpringSource, could not make money, otherwise.
And if this is not enough, we start with a "framework integration framework" integration framework project to keep SW developers busy. And there goes KISS, but CPU and memory needs to be used anyway ;-)
You are right.
ReplyDeleteHowever, especially when working with Java where I have to select the tools (frameworks) compromising my application (often) out of dozens of choices, it is essential to have an underlying infrastructure supporting these needs.
I am curios what the future will show us :o)
Peter,
ReplyDeleteFor me and no doubt for most OSGi converts modularity is an extremely important tool in managing complexity in applications. However, I also have some reservations about the developer-friendliness of OSGi which I have expressed on numerous occasions in the past.
Sadly, integration testing seems to be one of the overlooked issues with OSGi and in my view it is it's achilles heel. My other major worry has been the increased overhead of explicitly managing third party library dependencies.
Much has been made of the fact that OSGi has been taken up by all the vendors, and is therefore a "proven" technology. One area where it is still unproven is in its ability to win over mainstream developers.
Any technology which provides productivity barriers will be resisted, no matter what its technical merits are. Perceptions of increased productivity are easily the largest driver for developers moving from mainstream Java to Grails/Rails, etc.
It's for these reasons that I started the project Impala (http://impala.googlecode.com/). In my view it has almost all of the most valuable benefits you get from OSGi (and Spring DM) but without any productivity drain - on the contrary, developing with Impala once you're up to speed is much more productive than plain Spring.
Very good topic.
ReplyDeleteMy personal experience with OSGi is that I get interested once in a while, look at some examples and then decide it is too complicated for what it does.
What I would be very interested is to see some kind of real world requirement example where this kind of overhead is a justified choice to make (besides Eclipse). I haven't had such a situation myself yet.
I think you raise some very important points. There are no silver bullets and OSGi is clearly not a such a bullet either. However, it does provide some very important features that are crucial for large and very large projects. Eclipse is one example, but there is a clear reason why all major application servers went to OSGi and that is that modularity is crucial for these huge software systems. Or for example LinkedIn where over 800 bundles provide the functionality for this very cool website.
ReplyDeleteI do believe OSGi also provides value in the smaller scale because it is the first Java platform that provides a strong component model. Unfortunately, many open source software projects (the potential components) use Java in a way that is not compatible with strong modularity. Class path scanning, dynamic class loading, context class loader, and custom class loading are cool tricks but they do require big holes in the modularity. OSGi was developed for greenfield applications that worked properly in a modular, service oriented environment. In that context, it does a wonderful job with very little overhead and no need to revert to class loader hacks. But trying to run existing apps with all the hacks that many have learned to love over time just does not fly very well.
For the large scale applications OSGi works clearly very well looking at the examples. The challenge for the coming year is to make OSGi more in line with the expectations of the average J2EE programmer because we see that need. Companies like SpringSource and others are already working very hard to provide the tools and platforms to support this. But this will take time. The only excuse we have that these tools are not in place is that OSGi never was intended to end up in the enterprise space ...
Kind regards,
Peter Kriens
This comment has been removed by the author.
ReplyDeleteSorry for the previous mishap. . . needed minor edit
ReplyDeleteNice article
To add to Peter's "there are no silver bullets" comment, to address some of the issues you describe, we need "silver bundles" and developing these bundles should not be part of application development cycle.
OSGi does provides one with practically everything you need to solve most of the problem you have described (from adapter hooks and various startup options to writing plumbing bundles), but as you pointed out several times and rightfully so, it comes at the cost. . . at the cost of developing such "silver bundles", hooks and other components which essentially is nothing more then higher level OSGi based Application platforms and frameworks. . . to address service dynamics, dynamic class loading, load time weaving etc. . .
The capabilities of todays application servers and platforms are way to limited to the requirements of today's enterprise and there is not much they can do outside of creating their own proprietary modular system. OSGi gives vendors a new and standard model to develop next-gen server product, components as well as application frameworks on top of OSGi so the application developer instead of suffering will benefit from the new development and thinking model that OSGi presents
You were right Spring-DM has paved the way to simplify OSGi when it comes to application development (similar to what Spring has done to simplify J2EE) and is currently trying to standardize component model around OSGi as part of RFC-124. Spring also built dmServer - an OSGi based Enterprise Java Platform which some major pains the developer would have to go through while building an application using raw OSGi. And although I am a S2 employee who honestly believes in this product, I'll admit, there are other vendors who are entering the same competing OSGi market with some pretty cool innovations :-)
In any event, I don't think anyone in the right state of mind would deny OSGi
Oleg
@pkriens: Thanks for your comment! It kind of makes me proud seeing you
ReplyDeletereading my post :o)
You are right! Modularity is the key issue - and I absolutely agree with you
that the notion of (pretty small) bundle-scoped micro services is something that
enterprise developers have longed for!
The world of the enterprise Java developer is a world of almost extreme
diversity - When I start up a greenfield enterprise Java application, I have to
decide from a huge variety of components (the world of .NET is much easier in
regard to diversity ...). Most of us decide on an open-source stack - and there
you hit the point - no "silver bundles" (I really like that term :o) out there.
I am a big fan of OSGi modularity and I've used OSGi from large scale embedded
up to enterprise application - At the current point of time I would be very
careful upon making a pro OSGi-decision for an enterprise application. However,
I would use the "low-hanging-fruits" of the OSGi mindset:
- Use bundle-scoped micro-services (e.g. by just structuring my software into
bundles and exporting/importing to/from a service registry)
- Give developers what makes life easier for them (e.g. classpath scanning,
automatic startup of bundles if they are on the classpath etc.)
- BUT: Heavily constrain the developer at build-time (like OSGi does - but
e.g. through the use of Classycle or similiar tools)
- BUT: Loosely constrain the developer at runtime (bundles can only get objects
they are intended to get, e.g. through service lookups)
- AND: put all these things into relation with existing best practices (e.g.
Spring DM like IOC, AOP, etc...)
I think that this would strengthen the communities modularity awareness, bring
benefits to development performance and potentially allows a budge to OSGi
once "silver bundles" are available. AND: it's KISS!
I've already started with a proof of concept based on Spring-Core. I will post
an article in the next few days putting down some of the major concepts. I've
seen that Phil Zoio seems to have done something similiar - Maybe we can merge
our sources to help paving the way for OSGi into enterprise application
development!
I think the integration problems you mention about classloading and dynamic instrumentation that other non osgi-aware framework do (e.g Hibernate) is not due to OSGi but the very Java ClassLoader architecture.
ReplyDeleteClassLoaders mix two responsabilities:
1) Load the class (and have a chance to instrument it at load-time)
2) Define the visibility on other classes loaded in the JVM (e.g. via ClassLoader hierarchies.
Thus, this lack of separation of concerns leads to conflicts between frameworks/libraries even if they need to use only one of these concerns.
OSGi implementation have no choice, they must tweak with the class loaders to enforce visibility rules.
Ideally, the concepts of class loading and visibility should be orthogonal. Java should maybe evolve in this direction, by integrating the bundle concept into the JVM for dealing with visibility rules (in a way like .NET assemblies) and let the classloader only deal with class loading.
This comment has been removed by a blog administrator.
ReplyDeleteThis comment has been removed by a blog administrator.
ReplyDeleteInteresting blog, i usally be aware all about all different kind of sofware. i am online all the time, and this action allow me to see a site costa rica homes for sale and i like it too much. beyond all doubt without my computer i never would have seen this site too.
ReplyDeleteHey, nice site you have here! Keep up the excellent work!
ReplyDeletesoftware engineering services
Nice start guys...I went through the website and I found that you made decent point here. Keep up the topic that everyone can choose one of the best. Thanks. Feel free to know more about online software engineering
ReplyDelete