Refactoring improves code, usually increasing the function while reducing code bulk. However, such refactoring or restructuring often forces the application to undergo a complete development cycle, including unit, acceptance, and regression testing, followed by subsequent redeployment. In a large IT or engineering production system, this can be time consuming and error prone.
Agile programming is design for change, without refactoring and rebuilding. Its objective is to design programs that are receptive to, indeed expect, change. Ideally, agile programming lets changes be applied in a simple, localized way to avoid or substantially reduce major refactorings, retesting, and system builds.
There in a nutshell is what agile is to me. The goal is not to refactor, but to try to avoid it. Let's face it, refactoring happens because understanding changes and cruft naturally builds. But, our goal should be to avoid it at all costs. Refactoring should be used for unforeseen obstacles. We shouldn't knowingly put opportunities of refactoring into our code base from the start. Let's take Dave's advice and design to accomodate change!