I was reading Daniel C. Dennet’s excellent book “Darwin’s Dangerous Idea”, when I encountered the following excerpt:

I think the case is strong that not only do ‘over’-simplified models often actually explain just what needs explaining, but no more complicated model could do the job. When what provokes our curiosity are the large patterns in phenomena, we need an explanation at the right level.

Before reading Dennet’s statement, I had just finished a bit of text about software architecture. I immediately linked them together; software design is, no, should be such an “over”-simplified model. Precisely doing one thing: explaining the large patterns in a software system.

The source code of a system, in case of software development, is the complicated model. As developers, we all know that source code does not always explain a system that well. When we are coding, we are actually in the trenches. Which is where we usually only think in low-level terms. Variables. Statements. Type-conversions. Compiler-errors. Maybe classes or packages, at best. At that level, it is hard to overview the complete system. As Dennet puts it: “[T]he Key is often discovered when climbing out of the trenches and going for the panoramic view.”

A problem I have seen more than once: developers who are afraid to get out of the trenches. It is the fear of simplification that keeps those developers down there. Besides that, they cannot resist the urge to keep thinking in details. Unimportant details, on the wrong level. Dennet continues:

If you want to know why traffic jams tend too happen at a certain hour every day, you will still be baffled after you have painstakingly reconstructed the steering, braking, and accelerating processes of the thousands of drivers whose various trajectories have summed to create those traffic jams.

My point here is that the panoramic view, the “over”-simplified model in terms of software development is a system’s design, its architecture. This model should be able to help you understand a system. Without the need to reason about the gritty details of implementation. With all the details of the problem you are solving. However, the panoramic view itself must be kept simple and minimalistic too; we are not Designing the Space Shuttle. That’s the whole other side of the spectrum were you can go wrong: too much design.

Concluding, to really understand a complex system, you need to simplify to get the right level of problem description. You can either dig into the minute details too deeply, or end up with an “over”-complex design. In both cases the real working of the system will be concealed by the wrong details.

Dennet concludes with a calculator as an example, aptly explaining why you need the right level of problem description. Have you ever encountered a piece of software, whereby inspecting the source code you cannot believe how it works? The example shows how the wrong level of problem description miraculously changes the matter of comprehensibility:

[I]magine tracing all the electrons through a hand calculator as it multiplies two numbers together and gets the correct answer. You could be 100 percent sure you understood each of the millions causal microsteps in the process and yet still be utterly baffled about why or even how it always got the right answer to the questions you posed it. If this is not obvious, imagine that somebody made — as a sort of expensive prank — a hand calculator that usually gave the wrong answers! It would obey exactly the same physical laws as the good calculator, and would cycle through the same sort of microprocesses. You could have perfect explanations of how both calculators worked at the electronic level, and still be utterly unable to explain the intensely interesting fact that one of them got the answers right and the other got them wrong. […] of course you can’t explain all the patters that interest us at the level of physics (or chemistry, or any one low level). This is undeniably true of such mundane and unperplexing phenomena as traffic jams and pocket calculators; we should expect it to be true of biological phenomena as well.

It certainly is true for software systems.