A Hidden Root Cause Problem with Software Products

“The worst programs are the ones where the programmers doing the original work don’t lay a solid foundation, and then they’re not involved in the program in the future”. – Bill Gates



Have you ever looked back at something you wrote or created years later and wished you could do it all over again?  While you may be proud of what you originally did, most of the times you can think of many things you would do differently.  Most software architects and developers can relate to this.

While the desire to rework things you’ve already done exists in most facets of business (or even life in general), the issue is magnified with software.  There are several reasons for this:

  • First, a significant amount of time lapses from creation to completion. Beyond initial completion of the software, the problem gets worse over its lifespan.
  • Typically, the first things written are the foundational items.
    • There is usually pressure to “see something” working.
    • Technology choices, architecture patterns, coding standards, and the exact way that certain things will operate are all decided in the first few months.
    • You may or may not have the right people making these decisions. The wrong people will create problems that will be with you forever. The right people
      are sometimes too in touch with the latest and greatest technologies and philosophies and also are probably their own biggest critics in the end.
    • Decisions are sometimes also too much of a function of the times. With the rapid change in technology, over any 5-10 year period, this could mean big differences between how it was done and how it should be done “today.”
  • Business Needs Change
    • It’s not practical to consider all future potential business needs up front, but in most cases throughout the life of a software product, the requirements will change
    • Many times, the changes of requirements are big enough that require sweeping global changes to the software and it is very costly and risky.

The irony is that the compounding of this problem over the years is usually the thing that ends the life of a software product, but there’s rarely a time to justify going back and changing it.  Having managed software products and divisions at both small and large companies, the one thing that is a common is that there is more work in the backlog than can ever get done with the budgets and schedules.

If I had a nickel for every time a developer came to me and said that they wanted to make some global change to the system because it was the right thing to do, I’d have a lot of nickels.  95 times out of 100, their ideas for changes would make the product better.  Unfortunately because of the circumstances, 90 out of those 95 times I couldn’t justify doing what they wanted to do.  The costs of making those global changes are enormous.  The risks to customers, sales, and internal schedules are even worse.  The further along the product is in its lifecycle, the higher the cost and risk.  In the rare event that we could make a case for something, the cost and time had a big impact and it ended up being so disruptive to the product that only future customers could benefit from it (i.e. the already impractical notion of an upgrade became impossible).

The argument that most good architects and developers make is “that’s why we need to get it right the first time” – hard to argue with that.  Unfortunately, that sometimes leads to the old “analysis paralysis” and it’s still unlikely that you won’t be looking at it a few years later with many things you wish were different.

I’ve had the good fortune (and pretty rare distinction) of being part of creating several new software products from scratch.  While there are things that you learn on each one that you apply to the next, you still fall into the trap of continuously wanting to rewrite portions of it.

In my latest endeavor, I’ve tried to tackle this problem head-on.  We’ve architected things in such a way where every piece of code is designed to be replaceable (and backward compatible) in a very cost/time efficient way (largely through built-in automation).  We have a small team whose single focus is to constantly look at the code, design patterns, and technology choices and rapidly make those sweeping changes.  In fact, many of those ubiquitous changes to the actual code can be made through external configuration.  This approach provides a pretty practical solution to this problem that seems to have no real solution.

I’ll write more detail about our solution for this later, but wanted to get something out there to see if others have closely observed this same pattern/problem and/or if others have found solutions that don’t require a lot of compromise or a lot of money.

Tom Famularo

FAST

One Response to “A Hidden Root Cause Problem with Software Products”

  1. Jeff Simpson says:

    I agree with this. I would add that the older the system, the more the magnified the problem and ramifications are. Looking at the Life Insurance and Annuity software space in the U.S., the code for three leading systems is 10-30 years old and all shared an early high growth success rate and have dealt with significant acquisitions. Companies like Apple and Google have solved half of the equation by throwing money at the problem, but most companies cannot afford to do that, nor do they have the talent levels.