Service Oriented Architecture
Is it Ready for Prime Time?

The short answer to the question in the title of this piece is, “Yes and no; it depends on what you mean by prime time.”

When I first heard the buzzword Service Oriented Architecture (SOA), it sounded like a pretty good idea. After all, we in IT have been talking obsessively for years about better service for our users, while not doing a very good job of providing it. Developing an architecture (a blueprint) for providing good service to users seemed like a natural next step. So I decided to look into it.

As the hype began to build I found myself asking the same questions over and over again without getting any clear answers. “What is SOA?” “Who is getting the service, the end user or another computer program?” It was déjà vu, back to the beginnings of client/server architecture, trying to find out exactly what a client is and what a server is. We now have a pretty good idea about these things and how they interact. But it was a struggle to reach this point.

Initially I naively assumed that the services in question were oriented toward the end users. It took me several months and a couple of dozen interviews to learn that the services in question were oriented toward IT applications. In a service oriented architecture, when a program – typically an application – requests a service, say a customer name and address, it requests this service from an independent program that provides this service not only to the application in question but also to other applications requiring the same kind of information. SOA has no direct contact with a user. Rather, its goal is to enable and expedite the work of application programs.

Recently a client (a human, not a workstation) asked me attend a conference on SOA and give him my opinion of its maturity and suitability for major rollout. I attended an IDC conference and then did some further research. Here is a summary of what I learned. (Incidentally, the presentations are on IDC’s web site. Go to http://www.idc.com/. E-mail idc_support@idc.com to get a password. The event name is IDC Service-Oriented Architecture Forum Midwest.)

The core idea of SOA is to design applications around the use of various services, each dedicated to a specific task and each usable and reusable by multiple applications.

The goals of SOA as recited by several authors and speakers are the same as the goals we have been hearing about for many other architectures and technologies: lower costs of new applications; quicker deployment of new systems; quicker and less expensive updates; lower total costs of ownership. All of these can be achieved by having widely useful reusable services.

If you have been around IT for a while, this probably sounds familiar to you. We have been talking about reusable code for decades. COBOL has stored procedures; FORTRAN has callable subroutines. object-oriented programming (OOP) has – what else – objects. In every case, the purpose was to save money and deployment time by having pre-coded subprograms that already had been used in other applications and could be used in still other applications in the future.

We had a little success with reuse in COBOL, and a modest amount in FORTRAN. There were a few successes in OOP. The most promising user was a large financial services company which was so pleased with its use of OOP that it spun off a company to go into the business of selling the objects it had created. This company was not a resounding success.

Note that in every case cited above, the code in question was in fact reusable. It simply wasn’t reused very much. Why? The promise was there. The engineering was done. The potential benefits were large. Before we jump into SOA with both feet, we should figure out why previous attempts to reach the goal of reuse failed.

One difference between the world of SOA and the prior world of reusable code is the presence of the Internet as a host for services. It is in many ways the ideal host. It uses industry standard interfaces; it transfers the work and responsibility for operation and maintenance of the service from the user to an independent vendor. And its use prevents the application programmers and architects from meddling with the service to customize it for a specific use.

The current level of discourse about SOA in the trade press and on the seminar circuit suggests that SOA is far from a mature technology. One article lists the Five Things You Must Do to succeed with SOA. Another lists the Seven Keys to SOA Success, Unfortunately, some of the Five contradict some of the Seven. One suggests that the beginnings of SOA be small and low key; another says that SOA must be a long term enterprise-wide activity,

There is some consensus that a successful SOA program will require changes in the IT organization, changes in IT governance including SOA steering committees and SOA standards committees, enhanced cooperation across business units (to assure that the services developed are in fact reusable and reused), and increased understanding of business needs by the IT community. None of this is new, and none of it is unique to SOA. Look at any of the surveys of CIO concerns over the last couple of decades and you will find the same issues. We haven’t resolved them before. Why should we think that the new perspective of SOA will enable us to resolve them now?

The hype surrounding SOA has called forth the usual identity theft associated with new software; other things are renamed SOA to ride the marketing wave. Be sure you are getting the real thing. Lack of common definitions at all levels, including the definition of “ service” itself causes confusion and mistakes.

As of now, all of the tools are not in place. Any real world application of SOA will require some custom code. There will be legacy systems which cannot be effectively served by SOA. Further, you will need a new set of tools to manage both the services and the applications that rely on them.

Finally, there remain significant design issues that are unlikely to be resolved in any definitive way. One is the question of granularity. How much detail should, say, a name-and-address service provide? Zip code (5 digit or 9)? E-mail address? Identification of spouse and children? Making these decisions is an art, not a science, and is likely to remain so.

Please do not interpret my comments above as a condemnation of SOA. Quite the contrary. I think it is a positive development, particularly the use of Web based services. The attention SOA is getting is reemphasizing the importance of the perennial IT issues of better understanding of the business by IT, better communications between IT and the business, and better governance of IT activities shared between IT and business executives.

SOA is mostly old wine in new bottles, but this is not bad. It is offering us another chance to get at core IT issues that have bedeviled us for years. And, finally, the re-use idea may actually work. If it does, it will constitute a major advance in the way we do things, and provide us with systems that are less expensive, more agile, and more focused on business needs.

Is SOA ready for prime time? I recently read about a new kind of cable TV channel. Each cnannel serves a specific individual resort community such as Aspen, Vail, or Stowe. These channels are focused on visitors rather than residents. They report snow conditions, weather, and information about local events. Prime time for this small, select audience is not 7 PM to 10 PM as it is for network television; it is 8AM to 9 AM and 5 PM to 7 PM.

The answer to our topic question is this: SOA is ready for prime time for a small and select group of organizations: those that can bear the risks and live within the constraints noted above, those that have advanced technical skills available for this work, and those willing to make the changes in business processes (IT and other) that a successful SOA implementation requires.

SOA is too unstable and too iffy at this point in its history to be a wise choice for an organization-wide architecture of a large company. But it may be a good choice as a target architecture “to be” in five to ten years.

I have left until last the most important question about SOA. Will your designers and implementers actually use the reusable services designed and built for other applications? The passive aggressive opposition by IT professionals to reusable code is what doomed most previous efforts. Can your organization overcome this mind set?

I am reminded of the old riddle about statisticians. Why do statisticians distrust all data? They divide data into two classes: data collected by others and data they themselves have collected. They distrust data collected by others because they do not know how it was collected; they distrust data collected by themselves because the know exactly how it was collected.

Make you own translation to the world of system designers and programmers.

<><><><><>

The Very Worst Identity Theft

“Who steals my purse steals trash; ‘tis something, nothing; ‘Twas mine, ’tis his, and has been slave to thousands. But he that filches from me my good name robs me of that which not enriches him but makes me poor indeed.” Shakespeare, Othello, Act III, Scene 3.

The worst kind of identity theft is the theft of reputation. The purpose of most identity theft is to enable the thief to steal money or merchandise that is the equivalent of money. This is important to the victims but it is small in the larger scheme of things. The smooth functioning of our complex society depends on trust and trusted relationships. When this trust is compromised, bad things often happen.

There are many examples in the marketing arena. False trademarks on fashion goods “steal” the identity of the owner to enable the counterfeiter to sell his goods for more that they are worth.

A slight variant of this is common in the IT world. Marketers change the names of their products to associate them with some new fad. A recent example is in the field of knowledge management (KM). When this became a hot topic a couple of years ago, some vendors of document management systems suddenly discovered that these systems were knowledge management systems – which of course they were not – and proceeded to market them under that banner. Who suffered? Some naïve customers were taken in by the propaganda, but more important, the discipline of knowledge management is degraded by these distortions and false promises. In the end we all suffer from this debasement of what will become an important part of our business activities.

The very worst identity theft of our time is the rampant theft of the identity of Science. This theft will be much more damaging in the long run than monetary fraud. The identity of Science is regularly and shamelessly stolen by social activists, politicians, marketers., and others with some axe to grind and weak justification for their positions. They pretend that their ideas are supported by the kinds of objective observations and analysis that are the core of the scientific enterprise. The example of current interest is the debate over global warming. Much of this debate depends on pseudo-science at best, and cynical distortions and misrepresentations at worst.

It is useful to set the debate about global warming and the related apocalyptic predictions into the larger context of models of the world, of which the global warming model is merely the latest.

Models of the World – Thomas Malthus

In 1798, Thomas Malthus created one of the earliest models of society in his paper “An Essay on the Principle of Population, as it Affects the Future Improvement of Society”. [You can find it at htttp://www.ac.wwu.edu/~stephan/malthus/malthus.0.html. Read the first chapter or two to get a sense of the argument and the data with which he supports it.] He says:

I think I may fairly make two postulata.
First, That food is necessary to the existence of man.
Secondly, That the passion between the sexes is necessary and will remain nearly in its present state.

He argues that the rate of increase in food production will be arithmetic – it will grow by a constant amount each year as limited amounts of new land are turned to agriculture and as improvements are made in agricultural practices and tools.

He goes on to assert that the passion between the sexes will cause a geometric growth rate in population unless checked by famine, disease, war, or natural catastrophe.

Based on these arguments, he reaches the inevitable conclusion: population will outrun food supplies until some limiting event, such as one of the catastrophes listed above, occurs. He further suggests that there will be cyclical oscillations between a condition of too much food and not enough food, and that the brunt of the hardship will fall on the urban working (lower) classes.

Malthus’ work was (and still is) enormously influential, so much so that to this day the term ‘Malthusian’ refers to the idea that human population and welfare are limited by available land and other resources, and that we are nearing the limits.

Malthus’ model was not a model in the sense that we use the term today. It is verbal, not mathematical. (Babbage did not begin work on his Difference Engine until 1833.) It relied on the personal observations of Malthus and a few others. But, given the postulates and the data, the analysis and conclusions are faultless.

One of the beauties of Malthus’ paper is the clarity of thought and exposition that he provides. He starts with the purpose of the model – which is to study whether it is possible to improve Society. He describes his assumptions and methods clearly, and argues forcefully for his conclusion, which is that Society cannot be improved very much.

Malthus’ predictions proved unfounded. What did he miss? He missed the possibilities of innovation, invention, and huge amounts of new land available for agriculture. What he saw as the arithmetic rate of increase in agricultural productivity has turned out to be geometric, with production rising at a much faster rate than population growth.

Let’s not fault him for these omissions. Hindsight is about 20/20 after a couple of hundred years. He did a first rate job based on the science of his time.

Models of the World – The Club of Rome

174 years later, The Club of Rome revisited the issue. The Club is a global think tank, a non-profit, non governmental organization (NGO). Its basic premise is precisely the opposite of Malthus’ conclusion. In the words of its Web site, “it brings together scientists, economists, businessmen, international high civil servants, heads of state and former heads of state from all five continents who are convinced that the future of humankind is not determined once and for all and that each human being can contribute to the improvement of our societies.”

But its most prominent work, the report “The Limits to Growth” (1972), reaches precisely Malthus’ conclusion, although by a different route. The report is based on the outputs of a large scale computer-based model of the world and its economy. It includes representations of natural resources – renewable and non-renewable – human resources and behavior, and it incorporates elaborate feedback mechanisms that describe the interactions among these factors.

It also includes Malthus’ two assumptions. The first assumption, in slightly more general terms is that all resources – not merely agricultural land – are finite in quantity and even with increased productivity and capital investment, we will still run out of one or another in a rather short time, 100 years or so. Malthus’ second assumption, “That the passion between the sexes is necessary and will remain nearly in its present state” is included intact, although expressed in much less charming terms: population will grow geometrically, unless restrained by natural catastrophe or human planning.

The Malthusian assumptions inevitably led the Club of Rome to the Malthusian conclusion, restated in “The Limits to Growth”: population growth and economic growth are limited and the task of man is to guide the world to live within these limitations. The main point where the Club of Rome diverges from Malthus is that Malthus thinks that no long range improvement is possible, whereas the Club thinks that the elite can guide us to a constrained and limited future. Market forces are seen as inadequate to the task.

We are still within the time frame of the Club of Rome’s predictions, so it is difficult to prove that they are wrong. But the trends are clear. In many parts of the developed world, population is decreasing rather than increasing. The green revolution, advanced plant genetics, and improved methods of cultivation have turned the world from a place of chronic food shortages to a place of chronic food oversupply. (There are plenty of food supply problems, but they are almost all in the area of distribution and political constraints, not in the area of production.)

What about shortages of other resources, copper, iron, coal, petroleum? While these materials are essential to life as we know and want it, the proportion of our wealth and productivity used in the production of physical goods decreases year by year (12% of GDP in the USA last year), as services and intellectual property become more important in our lives. There is no shortage of mental resources, and there never will be until every brain cell of every human being is fully occupied. Even this may not turn out to be a limit. See Sharon Begley’s recent book “Train Your Mind, Train Your Brain”.

Models of the World – Global Warming

The latest incarnation of the Malthusian disaster scenario is the specter of global warming. The models are too complex to analyze here. You can find them described on many web sites; I just Googled “global warming” and got 70,400,000 hits.

The important point is this: the people who predict that global warming will happen claim Science as the basis for their conclusions, and as the justification for the social and political changes they propose ‘to save the earth.’ This claim is dubious at best. Let me count the ways.

1. It is widely asserted that the consensus of scientific opinion is that global warming is occurring and will continue. This is simply false. The assertion is often based on a report several years ago by the National Science Foundation that collected the significant papers on the topic and published them in one volume to facilitate discussion of the issue. But the introduction/summary of the report (which is all that most people read) commented favorably on the papers which supported the global warming hypothesis, and completely ignored all the papers in that volume that took contrary views.

And anyway, since when is scientific truth decided by a vote? Check with Galileo.

2. Many of the advocates of the warming-is-inevitable position are attempting to stifle debate on the issue, contending that it is “settled”. A couple of weeks ago Senators Jay Rockefeller and Susan Collins sent an open letter to the CEO of Exxon/Mobil demanding that it stop supporting groups that don’t believe in global warming. A couple of weeks later, Exxon/Mobil bowed to the threat of federal punishment, and changed its position.

Putting aside the First Amendment to the Constitution, this is not the way science is conducted. Anyone who has a strong scientific position welcomes debate because history has shown that scientific truth can only be discovered through the filter of controversy.
3. The whole global warming argument is based on the results of a number of different large-scale computer models of various aspects of climate – atmospheric trends, oceanographic studies, sunspot activity, and others. These in turn are supported by other related data: the fossil record, studies of volcanic activity, and so forth.

There are two separate but related problems here. One has to do with the reliability of large scale computer models in general. These models are hard to build and even harder to validate. In the end, the only reliable validation is by comparing the results of the models with what happens in the real world. This validation has not been achieved with the global warming models.The second problem is related to the data required by the models. All of the data about the future is derived essentially from extrapolations of prior trends, modified according to the preferences of the modeler. Any errors in extrapolation for year 1 and magnified in year 2 and more in year 3. By year 50 or 100, most of the data has lost all relation to reality. Meteorologists have been developing computer models of the weather for three or four decades, and they have reached the point where their predictions are pretty good for three or four days in advance. I am not denigrating their work. It constitutes a major scientific advance. But what does this imply about models that purport to look 50 years into the future?

Why Worry?

My purpose in writing this diatribe is not to defend one or another position on global warming. It is to defend Science from the cynics and opportunists who attempt to hide their social and political agendas behind a shield of Science and who if left unchallenged, will undermine the Scientific enterprise which is at the core of our prosperity and our culture.

<><><><><>