The Lean Architect

What line of work are you in? Waste management consultant.

9 minute read


the lean architect
Any man mentions he’s a Certified
ScrumMaster spends a night in the box

Anyone who’s soaked their head in agile techniques for a while can’t help but become a little obsessed by the concept of “lean”. Behind all the fine words they’re essentially the same thing - a quest to discover ways of delivering with the minimum amount of waste: stand-ups instead of meetings, cards instead of documentation, conversations instead of process, value instead of activities.

We’ve come a long way in the years since the manifesto - these days I find the problem is not so much about getting management to buy into the least-waste concept, it’s getting newly-minted practitioners not to focus on stand-ups, cards, etc as ‘the new process’. Old way bad. New way good. Long live the process.

That’s why some study of lean is both inevitable and worthwhile - it subjugates process to its role in optimising the effort-to-value ratio. Stand-ups and conversations are the best alternative to waterfall-style meetings we have right now. By putting them in their rightful place we may uncover other techniques that are better, who knows? More likely, we’ll be able to identify the stand-ups and conversations that shouldn’t happen at all. That’s always the target, because the least waste there can be is, of course, zero.

The other problem, that’s unfortunately all too common, is a lack of software engineering focus on internal code quality, as described by Martin Fowler in Flaccid Scrum. Practices like test/behaviour-driven development and pairing seem to the uninitiated like waste (“Two coders? One coding, one shooting the breeze? Testing before the code even exists? I’m not paying for crazy shit like that!”) and yet they save a ton of time and money (waste) later in the project, because they remove uncertainty. Broad behavioural test coverage means we can refactor or change direction in very safe ways as we discover all those surprising things about where we’re headed, whereas pairing blows a little spring-clean fresh air through all that test and execution code in ways that mostly results in better software than either of the pair could have achieved alone.

This notion of adopting a practice ‘upfront’ in order to minimise waste later is to my mind exactly what being an architect is all about. You do a little work to form a basis for some decisions to be made and then continue iteratively until there’s no value to the work being done, at which point you stop. If there’s no value from the outset then you don’t do it. Some projects (shock horror) just don’t need architects.

The value, while the work is being done, comes in the form of reduced uncertainty which saves developer time. In an enterprise this can sometimes involve a lot of work, more than is necessary or healthy, but I’ve always taken the view that if it saves developers time later and keeps the bureaucrats happy then that’s a kind of value in itself. You have to decide which battles to fight. Delivering a project successfully can often provide enough credibility to take on a bit more of that environmental waste next time.

If that seems depressing then think of it this way - life is full of wasteful activities that in the end actually reduce waste. Where people typically go wrong is in either how they see the activity (as a focus in itself, rather than the waste it’s intended to reduce) or the value of the outputs from the activity.

Dwight Eisenhower famously said:

Plans are worthless, but planning is everything.

We know plans are worthless because “no plan survives contact with the enemy” (often attributed to Eisenhower, though it seems Helmuth von Moltke may be more appropriate) but I don’t think collectively we appreciate that planning is actually useful even if it’s also wasteful at the same time. This duality can be extended to architecture.

My principle is:

Your architecture is worthless, but the thinking you did to come up with it is priceless, even if your conclusions were wrong

The more freedom I have had to do that supposedly wasteful thinking, the more developer time I have saved. Not, I hasten to add, by having a blueprint for developers to follow - blueprints are contemptible nonsense - but simply by ensuring that developers don’t lack valuable context, don’t go to long meetings and don’t get stuck with technology choices that make no sense.

In fact, so critical is this kind of work that I put together a conference talk on it and called it Responsible Innovation. The thrust of the talk is simple. We are led to believe that start-ups are the best places to innovate, that innovation is impossible in enterprises because of their bulk and their attitudes to risk. Enterprises therefore regularly create ‘special teams’ to manage this innovation. They call them start-ups, innovation labs, or futures teams. These special teams usually work very well to begin with because they are left alone by all the cruddy nonsense that usually holds enterprise IT back (process, standards, governance, architects, waste) but ultimately they fail because the products they produce still have to be marketed, managed, sold, warehoused, despatched, secured, scaled by the enterprise.

The irony is they fail because they don’t communicate enough, they don’t do enough up front work. You don’t need a special team to innovate, in a special building, isolated from the rest of the company. You do need to give them the freedom to work as they see fit of course but success doesn’t come from how the team is treated, it comes from how the team treats the enterprise. As that smart team’s product starts to emerge it’s got to be woven back into the (wasteful, bureaucratic, annoying) fabric of the company that birthed it. Yes, it’s hard and no it doesn’t feel cool and hip and agile. But like I said, deliver one thing and you get the credibility to take on more next time.

You have to ask yourself what risks could actually cause you to fail and drive your project risk-first. With internal innovation projects the risk is not process or the lack of urgency that infects large companies. Painful, tedious, soul-destroying they may be, but they won’t kill your initiative off on their own. In big companies the risk is not communicating well to the channels that deliver into, and support, production systems, in start-ups it’s not having customers.

Talking of start-ups. One of the hottest books around right now is The Lean Startup by Eric Ries. It’s a great book, but if you don’t want to buy it then you can get a good feel for what it’s about by watching his Google Talk on YouTube. He mentions Steve Blank in the presentation, who in turn mentions Alexander Osterwalder the creator of the Business Model Canvas, in one of his talks.

All three speak in various terms of something that strongly resonates with me when it comes to thinking about what it means to be an effective, least waste, architect.

A start-up is a temporary organisation, designed to search

The job of a start-up is not to build a product, so much as find a product. Once a product that’s viable (makes money, clinches a buy out, values an IPO) is found it’s not really a start-up any more, it’s a company. That’s why innovation centre’s in big corporates fail - they don’t stop once the search is over and instead fool themselves into thinking they can make a better job of being a company than the company that funded them. Big companies might suck at many things but it’s a mistake to dismiss them.

Ries, Blank and Osterwalder focus on startups, but aren’t software projects of all kinds just temporary structures designed to search? With a few rare exceptions I think they are. Searching is about asking questions and trying to get data to support your answers. It’s also about being open to the many places those answers might take you, all of which are usually not where you envisioned you would go. Planning and architecting are the same - searching for answers via questions, hypotheses, data and answers. Plans and architecture diagrams quickly become worthless because the source of the questions (the market, shareholders, customers) don’t really ever rest.

Eric Ries said something else (in his talk above) that also gave me pause for thought.

We can build anything we can imagine .. the dominant question of our time is not “Can it be built?” but “Should it be built?”

Not too long ago for startups and big companies there were two problems: could the product be built in a way that was economically viable and then if that were true, would anyone want it? Now, with so many amazing web sites giving away so many support services for free, cloud based servers, social networks, and an embarrassingly rich assortment of great open source tools the answer to the first question is yes. The one question we are stuck with is if we built all the things we can imagine would anyone care?

So where does that leave the route to riches then? If we can build any software product to the point at which it will make money (or not) our only issues are commercial (pricing, marketing), logistical (distribution) and attitudinal. The first two are eminently solvable given demand. I’m not sure about attitude. I think it can be taught to some people, but for most others you either have it or you don’t. It’s why most architects create waste rather than reduce it, either because they haven’t been taught how to work any other way or because they are subject to the Shirky Principle whereby their vested interest is to preserve the problem rather than remove it.

Certainly with nothing more than the right attitude (waste-reducing, searching, questioning, data gathering) it would seem we can achieve a great deal. Lean, like agile and common sense, is really a state of mind that takes practice and learning to get into. If it were possible to measure that state of mind then hiring teams would be easy - simply pick the skills you need and make sure everyone is imbued with the lean spirit and a sense of cooperation. Startups could be funded not on product ideas but on a brain scan in the certain knowledge that the assembled team would seek out the product once pointed in literally any direction.

It won’t happen of course. It’s an interesting thought experiment but unlikely to ever be reality and the reason is that learning is hard. I’ve read a lot of books, a lot of blogs, and watched hours of conference talks. They may each have served to make me marginally better at what I do but they pale into insignificance when compared with the hours of writing software, making mistakes, facepalming, sitting in painful meetings, biting my tongue, learning diplomacy, presenting an idea, getting shot down, writing dull documents, arguing over minutiae, trying again, watching software go live, releasing updates, getting drunk at the release party. These wasteful things, combined with the list of random wasteful activities my best co-workers could also talk about are what makes us good at making software.