The first time I got in touch with a software engineering process was at university. I am amazed, but very happy that that experience did not diminish my interest in software engineering processes. As you can guess, we were obliged to follow a very heavyweight process (probably one of the most extreme), that the university lend from the European Space Agency (ESA). In retrospect, it might have been one of the most educational of the practical experiences I went through in my years at the university. But while we were undergoing the project, it certainly didn’t feel that way. We tried to deliver quality software, stuck with that malicious standard that did more harm than good. In the end we spent 80% of our time on writing a pile of specification documents stretching many centimeters in height, which we ‘archived’ in a drawer somewhere, leaving us with only 20% of our time left to actually deliver a product.

I kept on thinking that there must be another way, where you do write specifications, that actually help you in the process of writing software, instead of simply following process. In the years that followed from my very first experience with a software engineering process, slowly I formed —- what I currently think is —- a reasonable statement how to address software specification. In other words, I have found the means to express, what I think are the important aspects of software specification to address in a software engineering process or project [1].

Let’s start with a little bit of history that influenced how my ideas got to their present form. As I said, I started out with the conservative ESA software engineering standard at university. I wanted to know about more lightweight standards, so I toyed a bit with the ESA lite standard which was introduced during classes as well. Still it felt it had too much overhead, especially in smaller projects (approximately 5 to 10 people, 20 to 200 KSLOC) that I was working on at the time, in which we wanted to apply the standard. What we actually did then, was stripping the ESA lite standard, leaving only the useful things for our projects, stripping everything that did not help us delivering the project. You could say we created our personal ESA ultra lite, tailored to the wishes we had based on the projects we were doing.

Finally I got to my last year where at the TU/e you have to go through a graduation or master’s project, where you write a dissertation, that is based on a (practical) research topic you work on for a minimum of 9 months. I was very lucky that, right at the same time, I got the opportunity to set up a new department for the company I worked for that time, together with a close friend, Joost, who I knew from university. We had already been working for a couple of years for that company in our spare time while we were studying computer science, so they had gained faith in our knowledge and abilities. The idea was to create a separate development group that would be responsible for researching and implementing robust generic building blocks that could help speed up the work and improve the quality of the company’s main software product. We were convinced that we could not just start this ad hoc, but that we would need some kind of process or standard, supported by a bunch of tools. Both Joost and me still needed a topic for our master’s project, and yes, as you can guess, we went lobbying.

Our best chance was the department called Software Construction, headed by Prof. Dr. Bruce Watson. We already knew him from the course of the same name (Software Construction), and we already talked to him that we were interested to graduate on a topic from the field of software development. He also thought setting up the proposed department would be an interesting project, and agreed to be our supervisor for our graduation project. We discussed our ideas with him, including our ultralite standard, and he pointed us into two directions, that heavily influenced me (up to this day, which holds for Joost as well I guess):

1. he gave me his copy of the Pragmatic Programmer, and
2. he talked about a new revolution in software engineering processes: the Agile movement.

As a result, our dissertation most importantly became research into the current available literature about software engineering processes, including setting up a process for the new department. Besides that, we investigated tool support for software engineering processes and ran a few projects in practice, with our new setup.

The process became a combination of a new subset of what we already took from the ESA lite standard, including new influences from the different Agile standards we dived into (e.g. XP, Scrum). Of course the process focused heavily on the main deliverable: the code, which makes the product, including the tests to guarantee product quality. But besides code, we invented another deliverable, which we called the Software Specification Document. In every project we included such document describing two things:

1. The functional goal of the project [3].
2. The architectural design of the project.

If you came here before, you might recognize these as two of my three pillars of abstraction. Which finally brings me to the point where I can tell what, in my opinion, are the important aspects of software specification to address in a software engineering process or project, namely: describing the software project at the functional level and at the architectural level of abstraction.

Why cover the functional level and architectural level of abstraction in a specification document for a software project? Let’s suppose we would not write any additional specification. That leaves us with the source code as only ‘specification’ of the project.

With that assumption, what functional information do we have? From source code alone, you can only guess what the program on a functional level implements. Even if you can compile the code, or have a working version of the program, you will know the basic functionality, but still it will be hard to understand what the exact intent is of every function of the program. It’s like running an application without consulting any manual, tutorial or help information. We’ve all been there; when you finally use the information at hand, you become noticeably more effective and productive with the application. Here’s where specification of the function of a program can help: if you know what the program should do at functional level, it will be easier to understand what is happening on the implementational level. In other words, you will better understand the source code if you know what it is supposed to do [2].

The same holds for architectural design. More than once, I have heard the statement that “the time it takes to really understand how a large piece of code works, is the same as the time it takes to write the code from scratch by yourself”. Personally I don’t think that it is that black and white. Reading source code can give you many hints, especially if the code is of rather high quality. However, I do believe that, in general, it is very hard to reverse engineer the intended architecture from source code alone. In this case too, I think software specification can help here: with the knowledge of the overall underlying architectural design decisions, it is easier to comprehend and thus maintain or extend the source code. That’s why I would recommend to document a software product’s architecture too (if you are interested in what I think are the important aspects of software architecture to put down, read my previous post about my five principles of software design).

To conclude, the three levels of abstraction —- function, architecture, implementation —- that I think are so important in software development can be used directly in software specification too. On top of the source code, the important information lies in functional and architectural specification.

We chose to write a Software Specification Document that was meant to end up as a hard copy, but obviously it does not have to be a paper document; you can capture the important functional and architectural information in a HTML page, or in a Wiki just as well. To be honest, currently I am a great fan of using a Wiki system for software specification.

I don’t know what came first; the idea of the software specification document or the idea of the functional and architectural levels of reasoning. I do know that for me software specification is putting down the most important information of a software project that cannot be captured in source code alone: function and architecture.