Seeing the Spoon

Do not try and bend the spoon, that’s impossible

This article is something of a preface for all other articles and, as with other posts, will be updated from time to time.

The intention for the site is to create a kind of holistic view of what can be done about the current malaise in software development. That there is a problem is in no doubt. If you work somewhere that experiences a multitude of difficulties in making your business happy by giving them what they want within reasonable parameters, then I hope it’s not too depressing to know the grass is not greener elsewhere - problems may different, but essentially the end result is the same.

I’m not so conceited as to believe I alone have all the answers, but I do believe that answers to each aspect of what can go wrong do exist. From requirements to support and all the management above, there are companies that do bits pretty well.

The Spoon metaphor from The Matrix is just my way of saying that if tech people can be a bit less introverted, insular, and well .. technical about what they do and see their business as it really is (a feat that many in the business are not able to achieve) then the malaise can be cured. Hard stuff will still be hard and complex stuff will still take time but at least it’ll all be transparent with no late surprises.

Life will never be perfect, of course. And who would want it to be? Daily intellectual battles with random events that conspire to defeat our best intentions are why we come to work every day.

But I find it profoundly disturbing that technology teams in many places have become the whipping-boy of the business, beating themselves up and spiralling down into a culture that almost expects to fail before they’ve begun. That’s not healthy and, without getting too touchy feely about it, people deserve better.

Some of the answers necessarily lie in the technology itself. Too many IT organisations just follow the herd and deploy software packages because a slick salesman did a good pitch, or say things like “we’re a Java shop” or “we only use Microsoft”, or worse “we do X because it’s what everybody else does”. Which can’t be right because on average everybody else is pretty rubbish.

Some of the answers lie in thinking less about technology and more how you can pragmatically add capability rather than blindly try to meet requirements or pursue imaginary alignment with the business strategy.

But most answers lie in the people and the culture because you will never get a team staffed entirely of blindly-cooperating mind-reading geniuses.

Here are a few of the themes you’ll find reappearing throughout the site:

• Software Vitality

  Software is an integral driving force in most businesses these days. Just like other revenue-enabling functions, such as marketing or sales. It’s more fundamental than HR or finance because, unlike these noble pursuits, it can be a differentiator where it underpins the customer experience. An article on The Register recently suggested that “around three-quarters of jobs in the Western world trade on an optimistic rhetoric of technological innovation” - so many are saying it even if possibly not following through with the requisite grass roots support.

  Anything that artificially separates the kind of software that can make you successful, from the managers of the business can negatively affect the bottom line. That wedge could be outsourcing, or it could simply be cultural.

  It’s easy to dismiss companies like Amazon or Google as tech companies. They’re not. Amazon is about anything but tech, and Google is an information company. What they have in common though is they look at software for what it can do to make them better, not for how it can meet predetermined requirements. That difference is important.

  Tech people are expensive. They are hired because they understand things that the business doesn’t. When you pay an expert to do something there’s an implied mandate for them to take an active role in applying their expertise. Not to sit back and wait to be led.

• Methodologies Suck

  All methodologies suck. They really do. A collection of good practices is fine, but like a richly endowed tool chest, you’ll always have to pick the ones that are appropriate to each occasion, be prepared to modify others, and learn completely new ones.

  Of course you need a repeatable process, so that everyone knows ‘how things get done around here’, but you also have to be prepared to adapt to changes and think on your feet.

• Nothing is New

  Except for some very specific techniques related to modern development, nearly every idea that makes a significant contribution to creating great software was defined before 1975.

  To our detriment, many of those ideas have been forgotten or ignored.

• The Blind Plan

  Hardly anybody understands what a plan really is.

  When the business needs X and you have Y developers of Z capability, then the earliest date you can deliver X is actually already predetermined.

  No amount of wishful-thinking will change that. A good plan will tell you this date.

  Then you can try and adjust what X, Y and Z are (and any number of other parameters).

  Most plans aren’t good though. They assume inaccurate values for X, Y and Z. Then it starts to look like you are behind plan just because you aren’t doing what it says on the date it says you should be doing it.

• Not Science. Not Engineering. Not Art.

  There’s no good analogy for the process of creating software that satisfies the science vs. engineering vs. art mantras. For the most part it’s bugger all like engineering, and bugger all like science, although a little bit of an engineering and a scientific brain helps.

  It’s more like the traditional craftsman role (for years I used the cabinet maker analogy) except that what you make is ultimately an intangible set of instructions for a machine to follow that can then be mass produced.

  Incidentally, the term ‘software engineering’ is actually a wilful misinterpretation. It’s a lot like some creative arts, particularly poetry, except that you can’t see or study the body of work that’s come before you, because 99% of it is legally and commercially secret.

• Languages should be Idiomatic

  Paul Graham in Hackers and Painters was right when he said all languages are progressing towards Lisp.

  I’ve always hated the idea of knocking any one computational expression medium in favour of another, but if you agree that a language is just that - a mechanism for expressing your thoughts on how an information processing solution should manifest itself - then the most efficient, elegant and precise medium must be the best.

  I’m shit at Lisp, having only really scraped the surface, but the more you understand it the more everything else starts to feel limiting. It’s not that Lisp per se is some development nirvana just that it best embodies the characteristics to build your own domain-specific language.

• Designs should be Idiomatic

  Architects (and software managers for that matter) that don’t understand coding are almost completely to blame for the present gulf between tech and the business. Being an architect (or manager) means you have to take responsibility for the link between the two.

  Coding in a way that’s friendly to the future is realy hard. A business that’s unsure of what it wants next week makes it harder.

  The answer isn’t in more buzzwords and techno-babble, it’s in culture and relationships.

• Architecture should be Idiomatic

  All mainstream architecture approaches are too narrow and, like programming languages, are evolving to something that is behaviour-driven.

  A business isn’t about services or adaptors or events. They are just the things we use to represent behaviours. If you can see the real behaviours, you can create an architecture that looks like the real business.

• All ideas should have a believable story

  One of the solutions to the cultural and communication divide between business and tech is to understand narratology.

  Work out, succinctly, what it is you need to achieve, and tell the story. If the story doesn’t work then, just like with a bad movie, nobody will be interested.

  If you can’t make a good story, the idea itself is probably wrong.

• It’s always difficult

  Doing software well is hard (see Donald Knuth’s many presentations on this).

  Even if you do everything right there is no guarantee of success. If anything goes awry in any of the lower layers of the abstraction model, you can come unstuck, and things will go awry at the most unexpected of times.

Notes

• The drawing is titled “trainees working in a Field Kitchen” and was released by the Imperial Warr Museum on their Non Commercial License