Almost two years ago, I wrote something about Conway’s law — “Any organization that designs a system […] will inevitably produce a design whose structure is a copy of the organization’s communication structure”. I concluded:

As a corollary to Conway’s Law I would say: don’t hold on to existing organization structures, if they are in the way. In a software company, it’s the product that sells. So if you want to improve the product and go forward, you will need to be flexible and willing to change: the company and its product should evolve together.

I have been contemplating recently about this statement pushed towards the other far end: how would Conway’s law apply to (mostly large) corporations that shuffle around developers, ahem, “resources” from project to project? What software architecture would you get in that case? A Big Ball of Mud. Yep, that’s what I would say. Big Ball of Mud.

I’m a big fan of having a coding style guide when working on a piece of software with a team. To be honest, I actually believe it’s an absolute must for a team to have a unified code writing style to be able to function optimally. Many software developers will agree with me, but also many will disagree, either way, it can be a pretty hot topic for a nice discussion during lunch, coffee, or maybe even while having a beer, if you’re really hardcore.

Although being very important, uniform coding style is merely a tool to ‘implement’ an underlying, more important aspect of collaborative software development, namely code ownership. That’s what I want to discuss today.

In ye olde days, I would always defend team coding style with the argument of achieving the higher goal of collective code ownership. Everybody owns all code. Everybody is allowed to change whatever part of the source base they think is necessary. Refactoring is key to keep the code base maintainable. I still believe in all of that, except for the very first thing, namely the fact that everybody should own all code. That’s just plain impossible. With a code base of say, more than 100k lines of code, you must be the smartest guy on earth to own it all. In that case, when you’re the best programmer in the world, you don’t even need a team, so you can stop reading here. However, were talking about real life here, and in real life, we cannot all be the best coders in the world. In real life, you will have a team, and you will have Bob who knows all about the network layer, Joe who is all into the GUI stuff, and then there is Collin who you need for the build and test system, and of course don’t forget about Jane, she knows a bit about everything, especially how all components are tied together.

So there you are with your utopian dream that everybody needs to own all code. Let’s take another spin at it. Is there another way to look at this issue of code ownership? Actually there is: Thanks to reading Coders at Work I’m now looking at this slightly different. Namely, we can turn it turn around: Instead of everybody owning all code we want nobody to own any code.

Nobody owning any code? So what does that mean? Well, each and every team member should be “allowed” to touch any part they want. That means that everybody has access to the whole code base, and should be able to read and understand every part. There’s where the coding style comes in to play: as a tool to facilitate team members to be able to more quickly read and understand a part of the code base that he didn’t write himself originally and is thus not yet familiar with. Another super valuable tool here is an API documentation generator. Especially together with a set of guides that prescribe how you should document your classes, methods, functions, and all other public interfaces. Next to a coding style guide, an API documentation system (such as Javadoc, Doxygen, Sphinx, etc.) is something I’d always implement in a team setting.

An important question that remains still is, if nobody owns the code, who’s responsible then? Simple: the whole team is responsible for all code. If something’s wrong anywhere, it’s the team’s responsibility to figure out who would be the best suited for solving the particular issue. Could be anyone, since everybody should be able to at least figure out what’s going on. Of course there will probably this particular person in the team that knows most about this specific part of the software where the problem occurs. Now I’m not saying he shouldn’t be the one to attack the issue, he might as well be the best person to solve it, he’ll probably be the quickest at least, I guess. However this guy could be Super Busy with some other Highly Important Things, or on vacation or whatever. With the team being responsible, the problem should be solvable by any team member.

Summarized, I’m advocating the tools to build a real team, a team that feels collectively responsible for what they are building. I believe that, in achieving this goal, the form of collective code ownership where nobody owns any part, but where you have team responsibility (but not ownership) for all code, together with tools like coding style guides and API documentation tools can be a big leap forward.

Been on an Amsterdam-San Francisco flight again, so more time to read. An excerpt from “The Design of Everyday Things”:

Would you like a pocket-size device that reminded you of each appointment and daily event? I would. I am waiting for the day when portable computers become small enough that I can keep one with me at all times. I will definitely put all my reminding burdens upon it. It has to be small. It has to be convenient to use. And it has to be relatively powerful, at least by today’s standards. It has to have a full, standard typewriter keyboard and a reasonably large display. It needs good graphics, because that makes a tremendous difference in usability, and a lot of memory —- a huge amount, actually. And it should be easy to hook up to the telephone; I need to connect it to my home and laboratory computers. Of course, it should be relatively inexpensive. What I ask for is not unreasonable. The technology I need is available today. It’s just that the full package has never been put together, partly because the cost in today’s world would be prohibitive. But it will exist in imperfect form in five e years, possibly in perfect form in ten.

Do you think what I think? I reckon that Donald A. Norman must be the owner of an iPhone. At least the quote above is a perfect description of today’s smartphones, in particular the iPhone which in my opinion is one of a kind both in design and usability. Now this book is first published in 1988, so Norman was ~10 years off, still funny to bump into, in the already quite interesting book.

I wanted to start out with some wxPython GUI testing and was trying to get dogtail installed on my Ubuntu 10.04 system. The current dogtail version that you can install with Synaptic is 0.6.1, while I want to try out 0.7.0. So I grabbed the source from dogtail’s website, unpacked it into /usr/local/src and ran:

$ cd /usr/local/src/dogtail-0.7.0$ sudo setup.py install

However, when I tried to run sniff for example, it can’t find it’s image files, because it’s looking explicitly in /usr/share instead of /usr/local/share. Ok, so let’s try again:

$ sudo setup.py install --prefix=/usr

This time, sniff starts complaining that there’s no module named dogtail.config. The problem seems to be that instead of installing into dist-packages, with a prefix other than /usr/local, the package files will be installed in site-packages. A bit of Googling tells me that Ubuntu doesn’t have /usr/lib/pythonx.y/site-packages in its path by default. Only dist-packages inside /usr/local/lib/pythonx.y and in /usr/lib/pythonx.y are added to Python’s system path by default (see this launchpad bug for more information).

Now luckily enough there’s a special argument for distutils if you want to install a “distribution” package manually, namely --install-layout=deb. Passing this argument will put the package using /usr as prefix and using dist-packages instead of site-packages, as if you were installing a deb package. So:

$ sudo setup.py install --install-layout=deb

That did the trick for me. Now back to what I was originally wanting to figure out: wxPython GUI scripting and testing with dogtail.

Edit: an anonymous user pointed out to me that no site-packages directory is in Python’s system path by default; he’s right, I revisited the issue and my post, and it is the --prefix=/usr which is the problem. Ubuntu’s distutils will put custom installed packages in /usr/local/lib/pythonx.y/dist-packages automatically, but when you use a different prefix, the default behavior is to use site-packages. That’s why you shouldn’t use --prefix=/usr, but --install-layout=deb instead.

Last month I visited Silicon Valley again. Always nice to be there. Not only for the weather and the surroundings, but also always really good to catch up and work together with my overseas colleagues face to face again. Only the flight from Amsterdam to San Francisco is not a short one, though that leaves you a lot of time for reading. So I took my chance and took with me some various writings, most notably Coders at Work, which I can definitely recommend.

Anyways, all that reading got me to pen down some random but interesting quotes I encountered:

There are two ways to design a system: “One way is to make it so simple that there are obviously no deficiencies and the other ways is to make it so complicated that there are no obvious deficiencies” —- C.A.R. Hoare.

“When documents are mostly to enable handoffs, they are evil. When they capture a record that is best not forgotten, they are valuable” —- Tom Poppendieck.

“Its easier to optimize correct code than to correct optimized code”—- Joshua Bloch.

“There’s a grand myth about requirements: If you write them down, users will get exactly what they want. That’s not true. At best, users will get exactly what was written down, which may or may not be anything like what they really want.” —- Mike Cohn

It’s interesting that only such a small number of words can express such a great deal of experience.

I already had a long standing draft to write something up about coding standards. To be more specific, coding standards about coding style. I’ve had many, no, too many (heated) discussions about coding standards and coding style, so I was — and am — reluctant to say something publicly about the topic. However, sometimes the momentum is just right, and you can’t resist.

As a developer, source code is your deliverable, it is the thing you produce, and you should try to make it look good. But what is good? How should it look? To debate this and find a compromise, you could as well host your own COP15. The results will be as disappointing. There is no such thing as the Best Looking Coding Style.

However, I do firmly believe in uniform coding style. In an independent team, all produced code style should be consistent. At all times. No exceptions. And I mean it: No Exceptions!Use iron fist if needed.

Well that’s what I believe. Do I need arguments? I think not; consistent coding style is intuitively a Good Thing. Unfortunately when talking to other developers, they want arguments. Of course — who doesn’t want arguments when somebody asks you to change habits?

So, to get back to the momentum I started with, while me and a colleague were in an email discussion with a certain module’s author about adapting his code into our team’s code base, the perfect opportunity revealed itself for me to get back to the dust-collecting coding standards post again.

My colleague wrote:

Just let me emphasize that we think style and code readability and consistency really is important, which is different from what a lot of people who write code think: if it works and the design is good, the code is good. This only holds when you’re the only one working on it, as soon as multiple people get involved it’s a very good thing to make all the code look uniformly, and keep the code readable and properly commented.

The main argument is that it helps shared ownership of the code (i.e. other people don’t feel reluctant editing your code to fix bugs, and won’t assume it is crap because it looks messy, to play the broken-window theory: it’s already crap so why should I care fixing it). Having all the code look the same also helps spotting inconsistencies, like parameters or functions that should be private but are public, debug code that is actually not debug code, comments that are misplaced and confusing etc.

It’s all a combination of minor nitpicking, but taking all of it together it definitely adds substantial value to a shared codebase.

Need I say more?

I have always been fascinated with the link between software architecture and a company’s organizational structure. For the companies I have worked for so far, I found that the company’s structure can conflict with the software architecture that would work best. For example, a software development department might be organized into separate teams, where each team is working on a separate part of the software. Usually, the software architecture clearly reflects the team structure: the software interfaces have grown to represent the borders between the parts where each team is working on. Suppose the product has evolved over the years, while the teams remained the same. At some point, the existing interfaces might become a major burden to implement new features. This seems to be a common problem, where the software design reflects the initial team structure, while the product is screaming for a new and more appropriate architecture reflecting the evolved product’s structure. As a result, team organization becomes detrimental for product progression.

This concept is known as Conway’s Law, named after Mel Conway, who published a paper called “How Do Committees Invent?”. Fred Brooks cited Conway’s paper in his classic “The Mythical Man Month”, and invented the name “Conway’s Law”. Here’s the definition from Conway’s own website (which also has the original paper in full):

Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization’s communication structure.

As Conway himself writes, “Brooks recognized that the law has important corollaries in management theory”. Here is one stated in the paper:

Because the design that occurs first is almost never the best possible, the prevailing system concept may need to change. Therefore, flexibility of organization is important to effective design.”

For example, there are many cases, where you want to improve a product and there’s no other way than to redesign its architecture. According to Conway’s Law, that means you will need to start reconsidering your development team organization: it might have to become more software (product) centered.

As a corollary to Conway’s Law I would say: don’t hold on to existing organization structures, if they are in the way. In a software company, it’s the product that sells. So if you want to improve the product and go forward, you will need to be flexible and willing to change: the company and its product should evolve together.

A very interesting contemporary (yet also for many a controversial) viewpoint about software engineering from an acknowledged software industry expert: Software Engineering: An Idea Whose Time Has Come and Gone.

Reading Jeff Atwoods latest blog entry, inspired me to kick of again [1] with a rant. I usually agree with Jeff’s statements, and this time I too do agree that scripting languages are a positive development in language evolution. They opened up many opportunities and make life a whole lot simpler with respect to mundane aspects of “real programming” like memory management, edit and compile cycles, and so on. But I cannot disagree more with the rest of what he is saying.

To summarize, he basically says that you have “real programmers” and scripters:

I remember that evening only vaguely […] Tonight, I become a real programmer. And so I began.

What happened next was the eight unhappiest hours of my computing life. Between the painfully slow compile cycles and the torturous, unforgiving dance of pointers and memory allocation, I was almost ready to give up programming altogether. C wasn’t for me, certainly. But I couldn’t shake the nagging feeling that there was something altogether wrong with this type of programming. How could C suck all the carefree joy out of my stupid little AmigaBASIC adventures? This language took what I had known as programming and contorted it beyond recognition, into something stark and cruel.

I didn’t know it then, but I sure do now. I hadn’t been programming at all. I had been scripting.

I don’t think my revulsion for C is something I need to apologize for. In fact, I think it’s the other way around. I’ve just been waiting for the rest of the world to catch up to what I always knew.

Yes, scripting languages are certainly different from the conventional compiled languages. But that does not say anything about the person writing the code: scripters are real programmers too. They are, and can be much lazier than “real programmers”. Which is a compliment: programmers should be lazy:

Only a lazy programmer will avoid writing monotonous, repetitive code. The tools and processes inspired by laziness speed up production.

Scripting langauges are such tools. But also for writing great programs in scripting languages you will need to have “real programming” knowledge.

You’ll write shit-script-code if you don’t know about data encapsulation: you won’t make use of powerful things like object oriented features of language like Perl or Python. Not only in C, but in scripting languages too you can introduce deadlocks and race-conditions when you are running your code in multiple threads or processes. You will face the same problems maintaining hundreds of thousands lines of script language code as with the same amounts of “real programming” language code.

I believe it is wrong and harmful to divide coders into two camps: the scripters and the real programmers. Whether you are programming C, C++, Python, PHP, Java, C#, Visual Basic, Javascript, Erlang, ADA, D, x86 Assembly, Tangram, Delphi, or whatever programming language of your choice: we all face the same challenges, so let’s face them together.

Near the end of A Scripter at Heart, Jeff concludes:

Every language has its place, and every programmer should choose the language that best fits their skillset and the task at hand.

I agree partly: every programmer should choose the language that best fits the task at hand. If there are multiple options that are equally well fitted for the task, then choose the one that best fits your skillset. None of these languages in your skillset? Then learn a new one, or pass on the task.