SOA Myth and Mystery

Update of original essay from May 14, 2007.

Service-orientation is a good, but old, idea and the benefits it heralds are great indeed. Like all bandwagons that have come before, it’s attracted more than its fair share of misinformation, consultant-speak, and myth. Search for the one true definition on Google and you’ll find plenty, walk into any corporate strategy meeting and they’ll be working themselves up into a frenzy over it, ask a consultant for advice on it and they’ll look surprised that you aren’t already doing it. SOA is the must-have accessory for software success.

I’ve now “done” SOA for four different organisations. In the early days there was a lot more debate about its relevance and usefulness, yet now it seems everybody is an expert. But where are the success stories? How come businesses aren’t talking about it more? As one recent post asked:

If SOA is about the business, why hasn’t it ever been mentioned in Fortune magazine?

Here’s what I hope is a useful summary of the state of the art. My best advice, from bitter experience, is this: take your time to get it right, your success measures will be specific to you, and set by your customers, not vendors or consultants. Be open to the idea that you may not even need it, and don’t lose focus on the many other spinning plates that make up a ‘good architecture’.

A Quick SOA History

The momentum behind what’s now called the Service Oriented Architecture (SOA) arose from frustrations with Enterprise Application Integration (EAI), although the basic idea of SOA has been around much longer. EAI, which become popular in the late 1990s, centred on creating ‘adaptors’ to bridge the gaps between a central message bus and the various connected enterprise applications and data stores. The EAI promise of loose coupling (an important architectural concept that means changes in one area do not require changes elsewhere) was somewhat watered down by the complexity of code required to manage individual adaptors, and the proliferation of adaptors across the IT landscape.

SOA takes a different approach, suggesting that rather than pass data back and forth across a central integration zone through adaptors, each application (or applications) should provide ‘services’ to the rest of the world. These services should be made available to other applications in a way that loosely-couples the consumer and provider of the service.

The industry excitement about SOA is driven by the fact that services are intended to be business-aligned, so not only should it address some of the integration issues uncovered by large-scale EAI projects, it should also bring the business and IT closer together. That’s important because the more parity there is between the business and software operating models, the less likely it is that small business changes will require expensive and time consuming changes to the supporting software.

No specific software is required to deploy an SOA, it’s a design concept not a product, but certainly the software that underpins it must support the creation of services, a repository to manage them, a registry for other applications to find them, and a mechanism for service consumers and providers to exchange business information.

But critically, overarching all of this must be a governance process that ensures business services really are business services, as it’s very easy to create services that are anything but (and non-business services are not useful in an SOA context, as described in the next section). So SOA is not a silver bullet, and all the other aspects of good architecture must be in place to make it work. If these architecture basics are not in place, then they must be implemented and institutionalised first, otherwise any SOA initiatives will fail.

SOA in Practice

SOA is defined by providers, services, and consumers. A consumer looks for, and discovers, the service it is looking for in a registry. The registry holds enough information to tell the consumer how to find the service and also how that service will operate (a service contract). Armed with this information, the consumer can then efficiently exchange information with the provider without knowing any more details about which applications, data bases, business rules, workflow, etc are involved in providing that service.

If the service granularity (the amount of work the service will do, and the consequent information it requires to do it) is set correctly then a service will become useful over and over to multiple applications and solutions. When there are lots of useful services available, new functionality can be provided simply by calling a sequence of services, with very little new code written. This is the ultimate promise of SOA and is called the Composite Application.

Despite a good story though, achieving anything like an SOA is very difficult indeed. Most SOA programmes fail, very few composite applications exist, and some companies that have tried it have only made their confused EAI pictures more confusing.

Some SOA Myths

• You can buy an SOA

  You can buy tools to automate parts of the service lifecycle, but having them does not give you an SOA.

• You need to map services to business processes

  Not true. There’s a well known bad-practice called percolating processes that refutes this. Services and processes are not the same thing. A service will often encapsulate many process elements, and business processes can be too fragile to base services on.

• SOA needs application vendor support

  SOA is an approach that explicitly recognises that environments are multi-vendor and a mixture of new and legacy systems. The abstraction of functionality into services hides this complexity from other applications. Therefore vendors should ensure that their integration options are flexible and friendly to service-orientation; they do not need to go the final step and make them SOA-enabled, as the definition of a service will be different for each of their customers.

• SOA needs Web Services

  Although the de facto standard for deploying SOA is Web Services (services that comply with the raft of XML and communication standards known as WS-) they are not mandatory for deployment. The complexities of the web services WS- standards (well over 100, at various stages of acceptance), their performance and management overhead can overwhelm small scale developments. That’s not to say Web Services are bad, but just using them is actually called a JBOWS (just a bunch of web services) architecture, not an SOA.

• SOA doesn’t need an Enterprise Data Model

  The payload passed between services needs to be based on some kind of standard, otherwise each service will represent its business entities (customers, orders, etc) in different ways. This does makes development harder for applications exposing services that don’t have the required data available locally, but in the long run is worth the effort.

• SOA doesn’t need discovery

  Services that are only available from well-known locations (IP addresses etc) couple the consumers of those services to the provider. Discovery provides a look-up mechanism, so that when service providers move location (or when multiple providers are available) consumers can easily, but dynamically, find them.

• SOA can be deployed globally

  Services represent the local dialect of the business. If this is extensible to a wider stage then SOA can be deployed on a global basis, but due to legal, regulatory, compliance, custom, cultural, and market conditions this is frequently not possible. Because of the technology and governance pitfalls, SOA is not a good vehicle around which to build global architectures in organisations with multiple trading locations.

• SOA reuse comes naturally

  Reuse of software has always been easy to achieve on a small scale (utility functions for mathematical operations, string handling etc) but notoriously difficult at the enterprise level. SOA is no different and does not solve this issue. Forced reuse can be counterproductive and it’s important not to set out to update services for every project (this creates maintenance/compatibility problems). Therefore it’s best to create reusable services only when they are being reused, if something isn’t in demand for reuse, it’s probably not a service, and therefore shouldn’t be governed as such.

• An ESB is a good route to an SOA

  An Enterprise Service Bus (ESB) is a marketing term usually indicating a Web Service enabled integration platform that supports multiple mechanisms of inter-application communication such as plain asynchronous (one-way) messaging and request-reply operations. ESBs can improve ability to meet some SOA-like operational requirements (monitoring, support, etc) but they are not the same as SOA, mostly because the S in the ESB is often vendor spin.

Some SOA Golden Rules

• Less is more

  SOA isn’t more successful the more services you have. In fact, the more you have the less likely it is you actually have an SOA. If it isn’t widely reusable and representative of an understandable business activity, it’s probably just an expensive interface. And remember that you always have to be tightly coupled to something. In SOA, that’s often the data model. When your next project requires a change to the existing service data model, you will be tempted to just release another service, leading to proliferation. One answer to this is to only pass around versioned business documents, but you’ll still have to parse and route them. The fewer services you have, the easier it will be to manage - hence the need to be relatively coarse-grained.

• When is a service reusable?

  When it’s already being reused. This is a hard one to get across, but in my experience, the candidates for SOA-style services actually suggest themselves, you can’t design them. After you’ve modified an interface a couple of times to facilitate reuse, the ‘service’ starts to present itself. Then you can make it discoverable and add it to your registry and repository etc.

  I’ve seen, at first hand, people sitting around a table, desperately trying to work out how they can impress their sponsors by describing clear RPC calls as services. There’s no need. It’s far more productive to build with reuse in mind, and then make the commonly used (business) calls into services later. And far more Agile too.

• You may not even need it

  Seriously. Think before you even embark on an SOA project. Why SOA? Why not just try to meet the business requirements whilst engineering as much as possible into a domain-specific suite of Lego-style functionality. Following the point above, if you have services they will find you, you don’t need to find them, and if your business isn’t very service-orientated then you just may never have any. Insurance, for example, is a business that stills thinks in terms of business documents passing down a workflow chain. Perhaps some of the actions within those workflows are indeed SOA services, but to start with SOA is to miss a great opportunity to concentrate on defining those documents and enabling the workflow to be loosely coupled.

• The composite application is mostly myth

  Maybe I’ll be proven wrong in years to come, but in six years of doing this I have yet to see a real composite application magically appear from well-defined previously-deployed services. I’ve seen it in demos, and been impressed, but I’ve yet to see a real business service that’s remained static long enough to be cobbled into a properly useful composite application. It’s just too much like the promise of component reuse from the late nineties that never came to fruition, and as Robert L. Glass said - reuse in the small has always been easy, reuse in the large much more difficult.

Like so much in software, we are hampered by the term more than the idea. SOA seems to have made it much further up the management line than other fads and, as a consequence, is going to be much harder to do sensibly. We’re only just getting to grips with all the bad decisions made around EAI in the early twenty-first century, so the more common sense that can be brought to bear on this right now the better.