Law and Order

In the technology justice system, the people are represented by two separate yet equally important groups: the architects who investigate standards and the developers who commit the offences. These are their stories.

<doink doink>

INT. COURTROOM - MORNING

The case before us has troubled me deeply over recent years. It’s a question that periodically pops up in various online forums and one which often invites heated debate - probably why it never seems to achieve any sense of resolution.

My serious and honest question is this:

Has the experiment failed? Is it time to say goodbye to Enterprise Architecture?

As someone who’s spent a significant proportion of their time ministering to large corporations on the subject of Enterprise Architecture, it’s perhaps a strange question for me to ask. Strange, but appropriate - being neither a patronised and pissed-off developer, nor a disenchanted and cheated CEO, I am in the rare position of being able to answer it with a certain disinterest. And as a sometime Enterprise Architect myself this may represent the best chance of a fair trial.

In April of this year Evan Weaver, Infrastructure Tech Lead at Twitter, put together a nice list of papers that describe fundamental concepts in distributed systems. It’s a solid list that I’d advise everyone to take the time to read because as you come to understand the ideas presented (and assuming you follow up some of the cited references within those papers for more detail/opposing views etc), you also come to know a lot about Enterprise Architecture.

I came across his post after I noticed a spike in my traffic due to the fact he was kind enough to quote my “Death of Architecture” article in the prelude. The Death of Architecture was written to point out that an architecture plan (whether you call it a Target Architecture, a Reference Architecture, or a Blueprint for the Future) is just that - a plan. It’s not real until it’s real and therefore, however ‘good’ it is, its competitive, scalable, performant, money-earning potential is locked away until that time. It wasn’t meant particularly to be anti Enterprise Architecture because clearly without even the plan nobody is going to get very far. Smart people pulling in different directions pretty much cancel each other out.

The month before, in March, the British Computer Society magazine IT NOW published an article entitled “Whither Enterprise Architecture?” that was much more anti EA, questioning some of its aims:

In the same way that a large organisation is staffed by diverse human beings, why shouldn’t the IT systems display a degree of diversity?

to its very existence:

How practical it is for one EA team to think, work and operate at an enterprise level and to make decisions that are beneficial to the business without working on specific business problems?

Oh dear.

All rise. This court is in session.

The Case for the Defence

Let’s begin by defining what we are talking about. I have three definitions for EA: My first is that it’s primarily about dealing with systemic/non-functional requirements. And why would you need a special team to do this? Well, as I have said many times, projects are strange tactical affairs and they rarely, if ever, allow room for considerations beyond the immediate needs of getting it done and getting it out. Pressures are such that gratifying the business and spending less are more important than getting overly concerned with additional requirements that are hard to quantify.

So the Enterprise Architecture team are there to act as a kind of business conscience, making sure that as well as delivering the things the business wants (functionality) they also avoid as many of the things the business don’t want as is reasonable (lack of scale, security, ability to extend, maintain, etc). And speaking of business conscience, my second definition includes some account of strategic aims - because systemic requirements like maintainability, extensibility, and monitoring will impact the company’s capacity to deliver the next project and the one after that (i.e. the requirements that come from strategic demands). My third definition is simply that an EA team exists to cover off all the things that are hard to keep track of when working in a business-as-usual or project environment - which means all the things needed to support the first two definitions: an architectural goal of some sort, a governance process to achieve it, technology standards to support it, and so on.

If you have other definitions then there’s no need to be swayed by mine. You see it doesn’t really matter how you define EA, because we can reduce all of them to one simple factor - money. Grubby and unseemly, I know, but it is ultimately the lifeblood and the source of stability and future growth.

Businesses want to spend as little as possible building software so they push for aggressive time-scales, the use of fewer people, faster responses, sharper contracts, value for money. The less you spend on doing it the more you will make from it. The less time you take the less money you will spend and the sooner you will make those returns. In the short term.

In the long term, business also want to spend less maintaining the software that they built, use fewer people, fewer servers, cheaper hosting, getting better information, changing their mind more easily. Even seemingly altruistic aims like great customer service, happy staff, great working conditions are indirect ways of saying more customers, bigger spenders, smarter people, lower staff turnover.

Ultimately it’s all about the money. Enterprise Architecture exists to manage the risk that’s naturally present when you want to build things for less and maintain and change them for less.

EA is critical because if you concentrate too much on the former the latter becomes impossible. A clear sign of a lack of architectural governance is rising costs, both in hosting and maintaining the IT landscape and in disproportionately high costs to modify it.

The reason EA is difficult, mostly unrewarding and rarely appreciated, is because it sits right in the fault line that represents a business conflict of interest. Projects and business sponsors hate it because EA can get in the way of making progress and saving money, and eventually the wider business resents it because the consequences of overriding EA recommendations (which always involves spending more money) is long term cost increases. For an EA team it’s a lose-lose situation.

Here’s a true story.

I once worked for a company that launched itself on the principal that it would sell its services based on one very simple, easy to understand tariff. In a market that, at the time, comprised of competing complex pricing structures it was a stroke of genius. Customers loved it and IT loved it because one tariff makes for a simpler billing architecture. Ads ran, brochures were printed, CPUs processed and customers flocked to the one place where they would always know precisely where they stood.

Competitors were worried and eventually they started to cut their prices. Some customers found that, based on their needs, there were now cheaper options, so they went back. As prices were tweaked and cut further, more customers left.

When you think about it all this makes perfect sense. Customers like simple prices, but we’re also all different, so what’s best value for one will be different for another. Also where the best deal is for one individual will change over time.

My company’s response was rational, smart, and simple: add new pricing structures so that customers that might otherwise leave would have a choice, thus making a minor change to the business plan that would guarantee improved competitiveness.

Ah .. but .. the whole enterprise architecture had been built to support one tariff. More tariffs were possible, sure, but not easy. And not quick. And, oh yes, not cheap.

Up until this very moment, business and IT had been in full alignment - a phrase I particularly dislike because, as this story illustrates, the more you align the less capability for manoeuvre you provide.

The business response ranged from derision to rage. Whose fault was it? The business for not making it clear that they might one day want to bill differently? The developers for coding around a one-tariff structure? Or the EA team for not managing that risk?

In truth I have no answer to the question (except to be wary of blind alignment..). I joined after the designs had been cast so had to deal with the situation as it was, not as I wished it could be. But that experience highlights a key decision that must be made before an EA team can work effectively. It’s decision about power and influence and how much of it EA should wield. Where is the line between merely advising the business on where they should take more time, or spend more money, and forcing projects to do so because they don’t have the technical wherewithal to understand the consequences?

Too little influence and EA is pretty much a redundant force - there’s little point in blaming a team you never supported when they gave you good advice.

Too much influence and .. well, the Defence rests.

The Case for the Prosecution

The ladies and gentlemen of the jury should disregard the entire defence statement. Life is complex enough without having to listen to the sob story of an overpaid PowerPoint jockey who spends half his time professing to be there to save the business from itself and the other half showing us how to effect this so-called ‘rescue’ with solutions that are completely out of touch with the realities of reasonable delivery.

For there is only one thing of any merit in business and that is delivery.

Voltaire famously said:

The best is the enemy of the good.

Which perfectly sums up why Enterprise Architecture is an irrelevance - if we concern ourselves with all the wonderful things that might be, we risk not getting on with the job in hand, and worse, convincing ourselves that nothing good can be achieved until such time as everything is homogeneous and perfect.

The one immoveable object in any organisation is how things really are right at this very moment. Everything must, and does, follow from this, no matter how high the expectations of the business. The reality of any IT landscape, anywhere, is that you would not choose it as a place to start. But then doesn’t that apply to most things in life?

EA is an expensive license to dream. Everything that it professes to provide can either be achieved by other (better) means or be shown for what it is - an idealistic fallacy that ultimately uses up valuable resources that could be better used making real progress.

But let’s start with something we agree on. Systemic requirements are important. Nobody wants to put in a system that works well only for the first three months, or one that can’t be changed easily. What Enterprise Architects neglect to take account of is that the pain of changing systems is felt as much by development teams as by the business. Anyone who has had to spend days trying to understand how crufty code works just so that they can change a few lines knows this. Poorly structured systems are an argument for better developers or better techniques or better tooling, not Enterprise Architecture.

And are Enterprise Architects necessary to act as a business conscience? If we can agree that good developers prefer to write code that withstands the test of time (because they are professional and because they may be the one having to make changes later) why can they not also engage with the business in the dialogue over time vs. quality? Who better to explain the ramifications of a more open design than the person that’s going to write it?

The argument about business conscience is ultimately rather patronising. It assumes that both the business and the developers are too stupid to realise that knocking something up quickly will cause pain later. The choice about whether that pain is worth it lies with the business and when the pain manifests itself it simply requires another project to address it. One of the fundamental principles of Agile is frequent communication so that there are no surprises. The argument for a business conscience is really the argument for better communication techniques and project rhythms. Again, not an argument for Enterprise Architecture.

What about standards? It all sounds plausible when it’s written up on some nicely laid out slides, but do many of the arguments for standards actually stack up? Take development languages. Of course it would be great if everyone coded in one language, but they don’t. One language will always be better at solving one business problem than another. For any standard to make sense, the cost of maintaining two or more languages have to far outweigh the cost savings of being able to develop neater, cleaner, more maintainable code in the (non-standard) language that solves the business problem best.

Sometimes it might, sometimes it won’t. But that’s an argument for better domain understanding, not Enterprise Architecture - and before the Defence suggests that domain understanding is EA territory we would like to refer to the IT NOW quote above which plainly states that you get better domain understanding by solving specific business problems.

And talking of money, when is “best” ever needed? Enterprise teams and their processes and standards cost a lot of money to maintain. If EA aims for the best there really ought to be many cases where the best is the minimum standard required. Most companies don’t need to scale like eBay. They don’t need to manage product databases like Amazon or vast information stores like Google. They need good, not perfect.

And finally, if none of the above is convincing enough. Here’s our final thought on Enterprise Architecture: it hasn’t worked. It’s been around for years. Smart people have tried to make it work. Huge sums of money have been spent on seeking its elusive goals. And yet rather than follow their great words with tangible change, all EA ever seems to do is re-brand and reinvent itself, though acronym after impenetrable acronym. It was EAI, then SOA, then ESB, then CEP, BPM. WTF? Isn’t it time to call out Enterprise Architecture for what it is:

A fun pastime for people who don’t like to get their hands dirty that’s only useful for padding out conference slides.

The Prosecution rests.

The Judgement

That I can write both sides of this argument with comparative ease illustrates just how conflicted the matter is. But let’s stick with the incontrovertible facts:

• Businesses are squandering their IT potential

  I’m not sure why the majority of companies fail to tap into the goodwill, talent and capabilities of their IT people, but that this happens is beyond question. Perhaps the ubiquity of information technology and the ease with which a layman can create a sophisticated spreadsheet or a shell script has created a sense that it’s not a very complex discipline and therefore not a profession in the true sense. Professions are treated differently. The moment I step into my doctor’s surgery he is in charge. I may have self-diagnosed my symptoms and understood all the treatment options on Wikipedia, but I know I am not the expert and so his pronouncements are treated with some reverence.

• Commercial sensibilities and projects are naturally dissonant

  There is a genuine and potentially harmful conflict in businesses when it comes to having things sooner versus having them better, so it does make good sense to be aware of it and to contain the risks it creates. Nobody thinks it odd that finance keeps a close eye on marketing budgets (or tries to), to make sure they are compatible with sales forecasts. The fact that a technical team exists to keep a close eye on projects, to make sure they are compatible with strategic needs, should not really surprise anyone.

However..

• As a supporting service to projects, Enterprise Architecture has failed miserably

  Instead of addressing its failures EA has regularly sought to blame them on flaws in the last philosophy whilst promoting the new one. But the game is up. We can all see that the emperor isn’t wearing any clothes. It’s time for him to get dressed, not to look for another emperor.

• As a vehicle for change, Enterprise Architecture has failed miserably

  Despite fortunes being spent on promoting EA concepts most organisations are still powered by out-of-date, brittle and hard-to-change software.

• As an enforcer of standards, Enterprise Architecture has failed miserably

  Firstly EA rarely allows for standards to be questioned and changed by projects with incompatible needs. Secondly, commercial figures to maintain standards (vs. the efficiencies projects gain by using the best tool for the job) are hardly ever debated. Thirdly, companies that have done well seem to do so without much in the way of enterprise standards (case in point: Amazon and the notion of the autonomous “two-pizza-team”).

Therefore, with more than a touch of sadness, this court finds Enterprise Architecture guilty. The experiment has failed and so it is time to say goodbye.

The Sentence

Extremists would fire all enterprise architects and have them write code on government projects for five years to make up for all the bollocks they’ve talked and all the time they’ve wasted. Whilst this court acknowledges the strength of feeling in this matter there is perhaps a more suitable punishment that better gets to the heart of the issue.

EA hasn’t failed because it’s not needed. Remember those papers that Evan Weaver posted that deal with the problems of distributed systems? Those problems aren’t going to go away. They were the reason EA got invented.

EA has failed because it has lost sight of what it’s there for - to provide practical support to developers so that what they produce works in a distributed environment over the long term - and it has lost this mostly because of the people that practice it: consultants who’ve never delivered anything, never experienced an operational environment for any length of time and high-level ‘strategists’ who prefer to live in an un-measurable and unaccountable world of conference-speak spin.

The solution is simple. The brand of EA has been tainted; its name misappropriated by charlatans and snake oil salesmen. EA has to die and be replaced with something much smaller and more targeted, with a name that’s prosaic and unsexy. Enterprise sounds too egotistical, Architecture too highfalutin. Something like Project Support perhaps. No consultant worth their two-thousand a day would want to be seen doing something as mundane as project support.

And this new team must work on real, right-now, problems that need long-term answers. Just like real developers. They must make things that run on code, and they must be accountable for those things. And their worth should be measured by their customers - projects that save time by having an immediate answer to integration, CEOs that can spin the company on a dime without breaking the bank.

There might even be enough time and good will left to do it too. I could be wrong about all of this. I just hate to see opportunities wasted. To paraphrase Law and Order’s great Jerry Orbach:

I specifically asked for him to be put on suicide watch. Apparently here .. that means that they watch you commit suicide.”

References

1. Martin Fowler wrote a fairly famous article entitled Who Needs An Architect which nearly got quoted here, but I’m not sure what he ends up referring to is what many companies these days call ‘architecture’ work. That is to say, whilst I don’t disagree with what he says about Architectus Oryzus (a wry Latin pun on his colleague Dave Rice’s name) what he actually describes - an architect who codes with the team, ups the game of the developers, pro-actively manages technology issues - is what I would call a Tech Lead, a Dev Lead, or if you insist on inserting architect somewhere, a Solution Architect. I’ve never had a problem with that role, and in my experience neither does anyone else. This piece is specifically about the team that exists outside individual solutions and focuses on something called the enterprise.

2. I knew I was on to something when, as I put the final touches to this, I came across a post on InfoQ about the Gartner EA summit promoting ‘Emergent Architecture’. The ridicule it received in the comments says it all.