Dancing with the Reuse Fairy

Not long after graduating from college I went on my first skiing holiday with a large group of work colleagues. I managed to get a couple of lessons on a dry slope beforehand and turned up at the resort just about able to put my boots on, do a fairly hopeless snow plough, and carry my skis without thwacking anyone in the head every time I turned around. We were all new to the sport and so block-booked lessons for every morning of our ten-day stay, but after two days of tuition I cancelled mine and left the others to hit the slopes instead. The reaction ranged from surprise that I could have got so good so quickly, to a suspicion that I must be too much of a know-it-all to be taught by an instructor. The reality was something else.

Skiing to me was a lot like trying to learn a new programming language. I need to understand the basics and get a sense of how much there is to know to be proficient. Then I need to do it. A lot. In becoming proficient via practising it, I know I’ll get a sense of how much there is to learn to become an expert, and I can decide whether I have the talent, time and/or commitment at that point. Another way of saying this is that once I know all the things I should be able to do, but can’t, I need to practice, not find out more things I can’t do.

With skiing, once I got the idea that there was a right way to hold your knees, keep your bodyweight on the downhill ski, do a parallel turn, I knew what to do, but not how to do it. Teaching me more sophisticated concepts was not going to help. I needed to practice until I felt capable of taking the next step (or not, should I decide that skiing wasn’t for me). With languages I need to get comfortable with the basic structures, conditionals, etc before I can think too much about the specialisms available. What made me a poor ski companion fifteen years ago, makes me a better learner today. And where I say better learner, my wife would say anti-social perfectionist, but it works for me.

Wanting to master the basics, to the point at which they become intuitive and fluent, may not be conducive to group holidays like ski trips, but it is a critical behavioural practice in software engineering. If you really get interface design, you are a long way to doing SOA better than the consultants who peddle it, if you really get delivery management you are a long way to running an agile project better than those who see it as a way to condone cowboy coding, and if you really get software development you can even learn to dance with the reuse fairy.

We’ve all met the fairy. She’s been around since the late nineteen sixties. Periodically she reappears to convince a new generation of architects and consultants, who didn’t spend enough time becoming fluent in the basics, that software reuse is not only possible but will save your company. She’s the one who drew that diagram. You know. We’ve all seen it. The diagram that shows a project uncovering a new piece of business functionality, passing it through development and test, wrapping it in component clothes and depositing it as a cog in the library. Then there’s the librarian, looking after the cogs, all with the same diameter/tooth ratios (because they all work together out of the box) and the queue of future development teams lining up to mix and match cogs to amaze and impress the business.

Who could fail to be seduced by the idea? After a few months of running the library you arrive at work one day to find the Chief Executive waiting for you. He hoists you up onto his shoulders and carries you towards reception. There, forming an honour-guard, are the Sales Team, holding up the keys to their new Porsches and Jaguars. A raucous cheer goes up and they clap you into the building, tears of joy trickling from their eyes and smearing the fake tan. The CEO lowers you gently to the ground and grabs your arm, saying “Well done, your component library has been just the right kind of shake up, son”. Did he call you … son? And he says it again, “… shake up, son”, “… shake up, son”. Except it’s not shake he’s saying. You look closer at his face. He’s saying wake up. “Wake up, … hon”. Back to reality. It’s your wife nudging you awake. It was all a dream. Time to get up and get ready for another day at “Gristle and Flint” (motto: “Death or Victory. Probably death.”)

But seriously. Why, when you think about it, shouldn’t the reuse fairy be right? How much of the world’s software, in terms of pure functionality is really that new? Not very much at all I would suggest. Languages and frameworks come and go, but not that frequently. Not so fast that we couldn’t be reusing masses of lines of lines of code in the meantime, then migrating it and reusing it again.

Robert Glass, author of the great Facts and Fallacies of Software Engineering, said:

Reuse-in-the-large (components) remains a mostly unsolved problem,
even though everyone agrees it is important and desirable.

and suggests that, although we can happily build components intended for reuse

once those reusable modules are built, they have to do something that truly
matches a great variety of needs in a great variety of programs … it is simply a
problem too hard to be solved, one rooted in software diversity

thus consigning any notions of valuable reuse to our dreamtime hours.

In fact he’s mostly talking about worldwide reuse, as in internationally available component libraries that make software development ridiculously easy. At that scale it’s hard to disagree with him, but let’s look at reuse within one company and park that thought for a moment - we’ll come back to whether one component can in fact meet the needs of a “great variety” of programs or not at the end. For now let’s just agree that reuse might be a desirable thing if we could achieve it (otherwise the reuse fairy would have been shot out of the sky in the seventies) and that technically it is possible to write some code in one place and run it in many other places later.

I say reuse might be a desirable thing because sometimes it clearly isn’t. Where we might look to innovate, we would not necessarily want reuse, for example. A new business model or application will have some basic areas that don’t add much value to the overall proposition and some that are key to achieving it. Generally, unless we’re using an old thing in a new way, reuse is the opposite of innovation. Approaches like Software Product Lines (SPL) try to accommodate this by defining a core (slow changing, reusable) managed framework upon which variations (market driven, bespoke) can be built.

In practice I find SPL a bit of a heavyweight approach to a simple, but good, idea. Certainly categorising where you might get the best reuse, and identifying different sorts of reuse, is a useful exercise. Any scheme will work as long as the boundaries are clear, but I’ve chosen four categories:

Infrastructure

Reuse at the infrastructure level is both comparatively trivial and already evident. You wouldn’t write a relational database from scratch and if you already have components for security, logging, monitoring, etc, then they are (I’d hope) designed for reuse by other components. The internet is a highly visible of infrastructure-type components being reused to great affect.

And this is what I mean about totally mastering the basics before attempting the subtle, more sophisticated aspects of any practice. If you work for a company that has a built (or bought) a security component that authenticates user account against a centralised data store (LDAP, Active Directory, etc) then reuse of this, everywhere, has got to be a basic practice. Enhancements to this component could support parameter driven options like use of secureid, biometric characteristics, automatic account deactivation, suspicious login activity and so on. There are variations in needs around infrastructure components, but there’s not much of an argument for meeting these via bespoke localised development over reusable components. When you find yourself identifying such a component, you have in fact also identified an aspect in the AOP sense.

Service

Service components are in theory great candidates for reuse. Typically stateless, usually pooled, and ideally discoverable (not directly invoked) by their callers, they provide a business mapped façade to the workings that go on behind them (access to an infrastructure component for example). The reason reuse so often remains theoretical is because they have to be business mapped, not project/solution mapped, and doing this is very difficult, as we’ll see in a minute.

Session

Session components, if they are simple, like the kind you might find handling a shopping cart, can be certainly be reused. The reason they often aren’t is because of this simplicity - the effort to write one from scratch rarely shows up on the development radar. Whether that be for reasons of not-invented-here or laziness in searching for one in the first place isn’t relevant. What is clear is that reuse at the session level is either a nice-to-have (when we move beyond mastering the basics) or impractical given that session demands are so project/solution specific.

Domain

Now this is the gold mine the reuse fairy was talking about. All companies are powered by business entities, like people, orders, trades, customers, time, money and business concepts that apply to these entities like suspended, received, overdue, confirmed, approved etc. If the behaviour and management of these things could be encapsulated into components, reusable across projects, and glued together with other components that know about business rules and workflow etc, then ninety percent of new projects would be delivered by doing nothing. And nothing is faster, or cheaper, to deliver than nothing.

Holy bananafish. No wonder we get seduced by the reuse fairy. Imagine ninety percent of all project work taking only about as long as it takes to download a black box of code and cobble together some control logic.

And yet nearly every project in every major corporation still rewrites similar code over and over again, deploys their output onto new kit bought especially for them, and falls into the abstraction inversion anti-pattern where they have to construct almost identical code to that inside pre-existing applications because it’s not been exposed in a reusable way.

We do this not because we are stupid, or masochistic. We do this because domain reuse requires a style of interaction with the business that is very rare indeed (I am tempted to say non-existent, because I have never seen it, though I concede because it’s not impossible, someone somewhere will not recognise what I am about say). To get reuse IT must have what I call a thematic view of the business. That is, rather than projects and bits of ad-hoc development work, IT must be able to tap into the themes of the business vision and strategy, as I described in a previous article. This is rarely possible because the following happens instead:

The business is busy. Very busy. All the time. There’s always too much to do. The tension is heightened by a feeling that competitors are doing better. So to make next year the year that it all “gets fixed” they overload their idea buckets. They decide they want at least ten times as much change as can be delivered. They don’t want no stinking architecture. They hate IT. It’s the IT systems that causes them to be so busy, what with all those manual workarounds and extra staff to manage. Even if this change could be afforded, it would be too complex to handle in one year. And anyway there’s not the budget or the IT staff to do it. So the business panics and does what they do every year - they take the easiest route available and create a few projects to fix operational issues, add new functionality to the same old creaking applications, upgrade some stuff.

There’s no perceivable vision or strategy in this type of work, no themes to pick up on. So no components and no reuse.

It’s plainly not IT’s fault that it’s not being done. The business is to blame, not IT (though I have to say, if IT focused more on the basics, and actually doing things, rather than indulging its own vanity with change programmes and quality initiatives, this state of affairs might be better than it is). The business just doesn’t see software as an asset to be reused, they see it like a plaster for a wounded leg to keep them mobile, rather than a motorbike that would propel them forward.

We’ve ended up at a rather depressing place. If the business can’t ask for what it needs in the right way, then we can’t deliver what they would get most benefit from. Or can we? Here’s where I think there’s a possible solution. You can’t change the business attitude overnight but that doesn’t mean you can’t start yourself on the road toward making that dream of reuse possible.

First there’s that infrastructure reuse. You need to document all those bits of code you have that perform these tasks, chose the best written one and make it part of governance that projects use it. No arguments.

Then, as a precursor to any enterprise reuse notions, you need to engender the skills to achieve it at the development level. Sort of like learning to ski well before attempting the v-style technique for a ski-jump (and you thought that intro was random..). There’s always scope for more development and in-project reuse. Which we also call cutting down on refactoring time, by designing code to adapt to new demands without needing to be rewritten. Simple techniques like Dependency Injection, Command Patterns and basic OO thinking like encapsulation and polymorphism are all that’s needed here. As projects become more sophisticated at making code reusable, the enterprise case becomes more compelling.

Personally, I have doubts that significant reuse will ever be achieved inside corporate environments until it’s enforced by the development tools and languages themselves. I consider myself a prolific reuser of software, but there are times when I think I can do a better job starting with a blank screen. I guess that’s human nature. But that’s not to say we couldn’t be doing a lot more than we are, just as long as we don’t sell the full reuse dream when the business is unlikely to assist in making it real.

I’m mucking about with a lot of functional programming (FP) ideas at the moment and grappling with lambda functions, closures, currying, monads and all manner of terms that sound like kid’s TV shows. It’s been such a struggle to extract the practical usefulness from a lot of mathematics and general ideas around code with no-side-effects, that I’ve already written another article on what these look like to real people, and more importantly why they should matter to real people.

One theme that is constantly implied (but strangely not made a major selling point) is that FP is massively powerful in supporting reuse. We can visualise a “component” as a collection of interfaces that take parameters, do something useful with them, and optionally return something useful to the caller. Historically, in imperative (non-FP) languages, they aren’t much more than that. Which is why Robert Glass rightly points out that it’s fiendishly difficult to make those interfaces, and parameters, and return values useful to as-yet-unknown callers. But what if some of those parameters weren’t values, but code, making the component more finely controllable? What if it were possible to remove all the non-domain chaff from a component and have changeable domain rules injected in? It would become less like a piece of rigid take-it-or-leave-it code and more like a programmable structure.

We have to master the basics first, but if in doing so we can address enough operational issues to change the businesses perception of IT from sticking plaster to motorbike, there are plenty of technologies around, appropriately applied, to also make them believe in fairies.

Notes

• Eagle eyes will have spotted the subtitle is stolen from the 1967 film “Cool Hand Luke”. This is the line spoken by the Captain after Luke has failed to deliver on his promise of reusable components for the third time. The line is parodied back by Luke himself in the closing scene, set in the enterprise planning workshops to determine the following year’s IT project portfolio, just before the Captain shoots him dead. Or at least that’s how I remember it.