Beyond The Yellow Bubble

It would be fair so say over the past few weeks that I have been caught up in all the hype about XPages.  So much so that I had forgotten about the thing I was most looking forward to when doing Notes 6 development.  For me the single most important recent enhancement to Notes was not XPages or even the new LotusScript editor (in 8.5.1) but the ability to design Notes applications using a Services Oriented Architecture.  To me this one new capability ensure Notes future more than anything.  Let me explain why.....

What Is SOA?
There are many definitions out there on SOA.  To me SOA extends the concept of Object Oriented Programming to an entirely new level.  With OOP we define applications as objects (classes) comprising properties (data) and operations (methods).  Now step back and look at all the applications in your organization.  Not as a disparate set of applications but rather as a series of components capable of interacting with each other without regard of the application platform in which they were developed.

Why Is SOA so Important?
Suddenly the application platform is no longer as important as it once was.  As long as that platform supports Web Services (and other higher levels of SOA such as property brokers etc.) it really doesn't matter platform what they are written in.  Notes applications no longer need to be second class citizens within an IT department.  By creating Web Services Notes applications can be accessed by non-Notes applications in the same way as any other SOA application.  Likewise Notes applications can also access data from these other applications directly without the need for LEI or other data synching processes.  Moving towards an SOA has the potential to solidify a place for Notes as a strategic application development environment.  For the great many small-medium sized applications Notes will often offer a clear advantage over alternatives due to its RAD environment.  In fact, the need for large applications can be replaced by building a a larger number of smaller components that are intergrated via SOA.

And Xpages?
Xpages can consume Web services so they are not excluded from this and this may even represent a better way to integrate XPage and non-Xpage Notes applications.  Even within a single Notes applications, web services represent a way you can invoke LotusScript code from inside an Xpage.

SOA will not happen overnight.  Some organizations are much more advanced than others.  With Notes 8.x SOA is relatively simple to implement .  So the opportunity is there for Notes developers to lead the way   Web service providers were added in Notes 7.  Web Service Consumers and Composite applications were added in Notes 8.0.  My suggestion would be to become familiar with these design elements and play around with them.  I will be developing SOA components as part of the .Dominio Framework 2.0 and you will probably see as many blogs from me about SOA  as XPages in the coming months.  My goal is to transform my own Notes applications into components and then start integrating them, in ways that were never possible before.

I have often asked myself similar questions?  At one of my previous clients an audit was done on Notes applications and it was found that there were 70,000 Notes databases!  This led everyone to start asking - where the **** did all these applications come from?  During my 16+ years involvement with Lotus Notes I have observed the following key sources for Notes applications:-

1. Non-IT Solutions: My biggest beef with IT departments has been that we often make it harder, rather than easier,  for people to get applications developed/enhanced.  The response to this is typically universal - finding a way to do it themselves. (that's how I started doing Notes)  So when a business process becomes too large or complex to be handled manually or via MS Office, Notes often provides a way for an enterprising business professional to automate that process.  This is based upon the speed with which a Notes application can evolve, the relative simplicity of its programming model, and the absolute ease in which Notes applications can be deployed (including security and replication). These are usually the most common form of Notes applications (in sheer numbers).  Many of these applications I never get to see as a Notes developer.  Those that I do are usually because there is a need for a change that cannot be implemented by the inventor.   Often because they are no longer in that role. These applications are highly effective but often not the prettiest, most efficient, or best engineered solutions. The dark forces that often line up against Notes use these examples to paint a picture of Notes as an inferior application development platform without truly understanding the unique nature and characteristics of such applications.  I am sure Java programs would look/behave just as bad if non-programmers could ever figure out how to use Java!
2. RAD:  One of the key reasons I get asked as a Notes developer to build a new Notes application is the speed with which the solution can be built and significantly lower cost structure this entails.  This is perhaps best demonstrated by one IT project which was intending to replace a series of existing Notes applications with a new Java/Oracle application.  As is often the case, design alternatives were requested and a Notes alternative outlined.  The proposed Java approach would cost over $1 million.  The Notes alternative was costed at less than $250K.  A greater focus on cost reduction helped sway that decision to the obvious solution - build it in Notes.
3. Building A Brand: Once a particular business area has one or more professionally built Notes applications the customer is more inclined to specifically request future needs be met by Notes-based solutions or they direct their requests to the Notes development team that have been so responsive in the past.  The only time these requests seem to be intercepted is when the project is big enough or strategic enough to gain senior management attention or the organizational structure is one in which a direct superior of the Notes development group does not (yet) bleed yellow.  Many Domino Web applications seem to get developed because of this predisposition to use Notes and/or a Notes development team.
4. Simply Too Expensive To Replace:  I have never yet had the pleasure of working in an IT department in which Notes was considered strategic. (That would be a dream contract).   But despite the efforts of many non-believers, Notes applications continue to flourish because of the above.  Too often it is simply too expensive to replace existing Notes applications with a "strategic" alternative.  e.g. I once was involved with the support of a Notes application that started as (1), became a (2) before evolving to a (3).  The database had grown to over 1 million records presenting a few challenges on the performance side.  When I was first asked to do some performance tuning for this application the brief I was given was  to give the application legs for a few more months as it was being replaced.  Four years later I left that contract without seeing that application replaced.  The project to replace the application had become a huge money pit.  As the business needs constantly changed we were able to quickly enhance the Notes application while the alternative (C#/SQL Server) simply couldn't keep up with the changes.  It was too big and was never completed.

There are other reasons but I believe the above account for the majority of Notes applications that get built.  What you do not see there is Notes applications being chosen as part of a deliberate long term strategy or because Notes had specific features and capabilities that alternatives did not.  They were chosen largely because they were quick, low-cost, and very effective.  To me this is what makes Notes Application development yellow and differentiates it from most other products.  It is this aspect of Notes I am hoping we don't loose in the efforts by big blue to align its products.  Just as we are starting to realize that our long term future on this planet will require us reverse some of our non-green habits there is a risk that by becoming too blue and less yellow we may also cause an imbalance in the Notes eco-system.

In Part 1 and Part 2 I presented the opportunity to add functionality to applications by generating a collection of data (dataset) at run-time  and presenting this data within a Notes application as a datagrid.  Using this technique it becomes possible to do such things as embed multiple category views, display the top 5 x by category, or generate dynamic columns based upon logic built into a form.  The components to implement this in the .Domino Framework are based upon Notes 6.0 (Form, View, Folder, LS Class).

In July I will be starting work on .Domino Framewok 2.0.  This new version of the framework will be targeted specifically at Notes 8.5.1.  (I previously blogged about bypassing 8.5.0 and whilst a year has now passsed my views havent changed about this).  One of the first things I want to do with 2.0 is extend the concept of datasets/datagrids even further....

In .Domino Framework 2.0 the process for building a dataset will be converted to a Web service.  The Web service will then return a Notes URL that correspons to either an existing view/folder that can be used to view this dataset or a folder that has been dynamically generated by the Web service.  This takes the task of (potentially) building a document collection away from the Notes client and places it on the Domino server.  What it also means is that I now have a simple way to build a dataset in one Notes database and pass it to another Notes application where it can potentially be embedded in a frame, form, page, or XPage.  This may not always be the most efficient way to achieve the end result, but in implementing an SOA approach, it eliminates the need to customize views in application A to meet the needs of Application B.