Tuesday, October 23, 2007


John C. Dvorak asks the following in his latest blog entry:
But I wonder if all open-source projects will eventually go the way of all software. Upgrade after upgrade adds more and more complexity, to the point where the software begins to get bogged down under its own weight. How many open-source projects have you seen in which the code gets leaner and meaner rather than fatter and fatter?

Good question. Software becoming more complicated and bigger is almost a universal truth. I don't see why open-source should be immune. Of course, the more features you add, the bigger your program will become. But, even software that is being maintained becomes an entangled network of goo. Brian Foote even wrote an anti-pattern about it called "Big Ball of Mud".

This is the problem that needs to be solved. I see a huge focus on creating tools that allow quick creation of code. But, I see few dealing with complexity of existing code and even dismissal of the ones that do exist. Refactoring tools are mandatory in the fight against creeping code. Tools that allow diagnosis of live running systems is a must. Code generation gets my code out the gate quicker, but without a gym, it becomes slow as molasses as the race continues down the path of new features.

I want to create systems that in their evolutionary cycle grow consistently with new features. The complexity should be in line with features. Right now, this becomes exponential and is proven by ever rising maintainance costs. Our current weapons are refactoring tools, code checkers (lint), metrics, debuggers, and let's not forget the most important: education. Actually, could that be the answer? Discipline and education just don't have the same kind of sexy ring to them, but tools can only take you so far. Still anything that can help our brains reel in the complexity is a welcome addition. Software is only going to get more complex as we go along. I think we spend so little time on reducing complexity in existing code because the feedback is not immediate. I want to get to the point where I can deliver the third version as fast I delivered the first.

It's a shame with all of the advances in our field that we have been unable to keep maintainance costs down. There are developers working on this problem. I've heard rumblings that tools are in the works. One possibly being a more macro level refactorer. I can't wait to use it. Until then, I'll keep my lint, refactorer, inspector, debugger, and metric runner close by my side. If anyone has any other tools that they like to use to keep complexity down, I would love to hear it. My brain can always use help.

I'll end with this all too true quote on Firefox in the same article as mentioned above:
Just look at Firefox, one of the poster children for open source. It loads more slowly than ever. It will be on some page within a tab, and that page will be refreshing ads or something in the background, and Firefox just hangs all its cycles there, slowing down the whole computer. Every couple of days the browser is hogging cycles, and I have to go to the Task Manager and kill it. Other people have this problem, too.

Go read the rest of the article. He makes some good points on mindless fans of technologies. Sadly, I think open-source is going to be the place where the tools to deal with complexity will be born. Until then, open-source will suffer the same as the rest of us with bloat.

1 comment:

Michael L Perry said...

Full disclosure: I sell a product based on this pattern.

That said, you asked for tools that have helped us bring complexity in line with features. That is exactly why I developed the automatic dependency tracking pattern. See the comments from your January 31 posting.

I have used this pattern on a number of projects now, and it has definitely saved my bacon. The idea is that you define the behavior of you UI not by putting your data into it (ala setters or MVP), but by having the controls ask for their data. When the data changes, they ask again. So you don't have to keep track of all the miriad ways that your UI can be affected. It works more like a spreadsheet.

The upshot is that as you add features, you don't have to go back to previous ones and look for cross-talk. The cost of the new feature is only dependent upon the complexity of that new feature, not the number of prior features that it touches. No more n-squared development.