Before Christmas, Brian Foote made a blog entry about "The Least Understood Thing in Computing". His answer was simply "reflection". Sadly, I think he is overtly optimistic. I agree that an understanding of "reflection" is badly needed. Reflection is what makes frameworks like Hibernate and Spring possible. But, I still think they could go farther.
But, I would have answered: "encapsulation". Designs with properly encapsulated objects and layers make for great architectures. But, I've seen far too much code that exposes various aspects of the internals of the layers unneedlessly. Designs that are comprised of merely data structures and processors are complex and hard to maintain. The benefits of Good OO Design are well known and they are several good teachers. But, I keep running into procedural poo that refuses to encapsulate the internal workings of itself. Now, before anyone gets uppity, I know you can do great design in procesdural langauges as well. Information hiding works there too via modules.
I read Parnas's "Some Software Engineering Principles" article in which he talks about how easy it was for programmers to dissect programs into flows, but it was difficult to break them into modules (or objects). One of our greatest tools to aid in fighting complexity and increasing understandability is "encapsulation". Look any complex system and it will be comprised of several easy to understand sub-pieces that work well together as one cohesive. The human body is an excellent example of this. We are comprised of different systems that are comprised of even smaller systems. Nature is recursive with a keen eye for hiding information at each layer.
I think everyone in computer prgoramming knows that encapsulation is a good thing, but it's hard to achieve and is it really understood? I see designs that expose their internals all the time. It's something that is preached, but practiced in pockets. Accidental architectures arise from information exposure. Knowing what to expose and what to hide takes thought and planning in design. It just doesn't happen.
This post might seem pessimestic, but it's really a rally cry for doing better. Software projects succeed and fail for various reasons. But, I think a mixture of good technical leadership, shared vision, and good design is at the core of every success. The heart of good design is encapsulation and failure rates of 60% plus tell us we can do better. Let's start with good designs. The Pragmatic Programmers has several articles on good design and you could even pick up "Software Fundamentals" by David Parnas.