Distinguishing between testing and checking

At Agile 2013 Matt Heusser presented a history of how agile testing ideas have evolved in “Twelve Years of Agile Testing: And What Do We Do Now?” The most intellectually challenging idea I came away from Matt’s talk was the notion that testing and checking are different. I’m still trying to wrap my head around this distinction.

Disclosure: I’m not a testing insider. However, along with effective design and architecture practices, pragmatic testing is a passion of mine. I have presented talks at Agile with my colleague Joe Yoder on pragmatic test driven design and quality scenarios.

Like most, I suspect, I have a hard time teasing out a meaningful distinction between checking and testing. When I looked up definitions for testing and checking there was significant overlap. Consider these two definitions:

Testing-the means by which the presence, quality, or genuineness of anything is determined

Testing-a particular process or method for trying or assessing.

And these for checking:

Checking-to investigate or verify as to correctness.

Checking-to make an inquiry into, search through, etc.

Using the first definition for testing, I can say, “By testing I determine what my software does.” For example, a test can determine the amount of interest calculated for a late payment or the number of transactions that are processed in an hour. Using the second meaning of testing, I can say that, “I perform unit testing by following the test first cycle of classic TDD” or that, “I write my test code to verify my class’ behavior after I’ve completed a first cut implementation that compiles.” Both are particular testing processes or methods.

I can say, “I check that my software correctly behaves according to some standard or specification (first meaning).” I can also perform a check (using the second definition) by writing code that measure how many transactions can be performed within a time period.

I can check my software by performing manual procedures and observing results.

I can check my software by writing test code and creating an automated test suite.

I might want to assess how my software works without necessarily verifying its correctness. When tests (or evaluations) are compared against a standard of expected behavior they also are checks. Testing is in some sense a larger concept or category that encompasses checking.

Confused by all this word play? I hope not.

Humans (and speakers of any native language) explore the dimensions and extent of categories by observing and learning from concrete examples. One thing that distinguishes a native speaker from a non-native speaker is that she knows the difference between similar categories, and uses the appropriate concept in context. To non-native speakers the edges and boundaries of categories seem arbitrary and unfathomable (meanings aren’t found by merely reading dictionary definitions).

I’ve been reading about categories and their nuances in Douglas Hofstadter and Emmanuel Sander’s Surfaces and Essences. (Just yesterday I read about subtle difference between the phrases, “Letting the cat out of the bag” and “Spilling the beans.”)

So what’s the big deal about making a distinction between testing and checking?

Matt pointed us to Michael Bolton’s blog entry, Testing vs. Checking. Along with James Bach, Michael has nudged the testing world to distinguish between automated “checks” that verify expected behaviors versus “testing” activities that require human guided investigation and intellect and aren’t automatable.

In James Bach’s blog, Testing and Checking Refined, they makee these distinctions:

“Testing is the process of evaluating a product by learning about it through experimentation, which includes to some degree: questioning, study, modeling, observation and inference.
(A test is an instance of testing.)

Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.
(A check is an instance of checking.)”

My first reaction was to throw up my hands and shout “Enough!” My reaction was that of a non-native speaker trying to understand a foreign idiom! But then I calmed down, let go of my urge to precisely know James and Michael’s meanings, accept some ambiguity, and looked for deeper insight.

When Michael explained,

“Checking is something that we do with the motivation of confirming existing beliefs” while, “Testing is something that we do with the motivation of finding new information.”

it suddenly became more clear. We might be doing what appears to be the same activity (writing code to probe our software), but if our intentions are different, we could either be checking or testing.

Why is this important?

The first time I write test code and execute it I learn something new (I also might confirm my expectations). When I repeatedly run that test code as part of a test suite, I am checking that my software continues to work as expected. I’m not really learning anything new. Still, it can be valuable to keep performing those checks. Especially when the code base is rapidly changing.

But I only need to execute checks repeated on code that has the potential to break. If my code is stable (and unchanging), perhaps I should question the value of (and false confidence gained by) repeatedly executing the same tired old automated tests. Maybe I should write new tests to probe even more corners of my software.

And if tests frequently break (even though the software is still working), perhaps I need to readjust my checks. I’m betting I’ll find test code that verifies details that should be hidden/aren’t really essential to my software’s behavior. Writing good checks that don’t break so easily makes it easier to change my software design. And that enables me to evolve my software with greater ease.

When test code becomes stale, it is precisely because it isn’t buying any new information. It might even be holding me back.

I have a long way to go to become a fluent native testing speaker. And I wish that James and Michael could have chosen different phrases to describe these two categories of “testing” (perhaps exploration and verification?).

But they didn’t.
Fair enough.

Why Domain Modeling?

One barrier to considering rich domain model architectures is a misconception about the value or purpose of a domain model. To some, creating a domain model seems a throwback to earlier days where design and modeling were perceived to be discrete, lengthy, and mostly unproductive activities.

When object technology was young, several notable authors made a strong distinction between object-oriented analysis and object-oriented design and programming. Ostensibly, during object-oriented analysis you analyzed a task that you wanted to automate and developed an underlying conceptual (object) model of that domain. You produced a set of task descriptions and a model of objects that included representations of domain concepts and showed how these objects interacted to accomplish some work. But you couldn’t directly implement these objects. During object-oriented design you refined this analysis model to consider implementation and technology constraints. Only then, after finishing design, would you write your program. The implication was that any model you produced during analysis or design needed extensive manipulation and refinement before you could write your program.

But even in those early days, many of us blurred the lines between object-oriented analysis, design and programming. In practice, how we worked was often quite different than suggested by the popular literature of the time. We might analyze the problem, quickly sketch out some design ideas and then implement then. We might use CRC cards to model our objects (which we would then discard). There weren’t distinct gaps between analyzing the problem and designing and implementing a solution. Sometimes different people did analysis while others did design and programming; but many times a developer would do all these activities. Sometimes, we created permanent representations of some models (in addition to our code). It depended on the situation and the need.

These days, I rarely see anyone produce detailed object analysis or design models. In fact, design and modeling have gotten somewhat of a bad wrap. Good object design is deemed too difficult for “average” programmers, and there isn’t time apart from coding to think about the domain and come up with any models.

The most common object models I see are created for one of two purposes: small conceptual models constructed to gain an understanding of significant new functionality; or informal design sketches intended to provide a quick overview to newcomers or non-code reading folks who need to “know more” about the software. A lack of modeling (unless you consider code or tests to be models—I don’t) is prevalent whether or not the team is following agile practices. The most common models I do see are very detailed E-R models that are more implementation specifications than models. They don’t leave out any details making it hard find the important bits.

Understanding and describing a domain and creating any model of it in any form takes a back seat to most development activities.

But if your software is complex, rapidly changing, and strategic and you aren’t doing any domain modeling, you may be missing out on something really important. If your software is complex enough, you can greatly benefit from domain modeling and thoughtfully doing some Domain-Driven Design activities.

For example, Domain Driven Design’s strategic design is a conscientious effort to create common understanding between business visionaries, domain experts and developers. Initial high-level domain discussions lead to understanding what is central to the problem (the core domain) and the relationships between all the important parts (sub-domains) that it interacts with. Gaining such consensus helps you focus your best design efforts and structure (or restructure) your software to enable it to sustainably grow and evolve.

But it doesn’t stop there. If you buy into domain modeling, you also commit to developing a deep shared understanding of the problem domain along with your code. Your mission isn’t to just deliver working functionality, but to embed domain knowledge in your working solution. Your code will have objects that represent domain concepts. You’ll be picky about how you name classes and methods so they accurately reflect the language of the domain. You will have ongoing discussions with domain experts and jointly discuss and refine your understanding of the domain. Along the way you may sketch and revise domain models. You’ll strive to identify, preserve, strengthen and make explicit the connections between the business problem and your code. When you refactor your design as you gain deeper understanding, you won’t forget to reflect the domain in your code. Your domain model lives and evolves along with the code.

As a consequence, there isn’t that big disconnect between what you code and what the business talks about. And that can be a powerful force for even closer collaborations between developers and domain experts.

Domain-Driven Design Applied

Vaughn Vernon’s new book, Implementing Domain-Driven Design, is full of advice for those who want to apply design principles and patterns to implementing rich domain model applications.

Check out my interview with Vaughn for InformIT. We discuss why he wrote this book and some important new Domain-Driven Design patterns and their architectural impact.

There is much to learn about good design of complex software from this book. In future blog posts I plan to unpack some of the meatier domain-driven design concepts and reflect on how best to apply them to designing enterprise applications.

How far should you look ahead?

I’m a consultant. So an easy answer would be, “It depends.”

My real answer is more nuanced: only look ahead far enough to see some significant issue and a plausible way to resolve it. If you want to get better, practice looking ahead in the context of your work and its established rhythms and natural cycles.

If your organization doesn’t value planning and you want to look ahead, start by taking small steps. The reason to look ahead is to check for some pre-work or preparation that is likely to save time or money.

If you are a tech lead, try looking ahead in the context of release planning. See what architecture risks you spot (or new design requirements) as new features are scoped. Take notes on what you are concerned about. Spend some time speculating on what you should be doing architecturally to support the next release. Have a conversation with a trusted colleague about your concerns. Identify concrete follow-on activities specifically targeted at addressing your biggest concerns.

If you are an agile developer, you may not want to look further ahead than the current sprint. If you aren’t already doing so, try some iteration modeling for a couple of hours at the beginning of an iteration. Grab a handful of related story cards for a new feature and sketch out some design ideas. Draw sequence diagrams on a white board. Jot down class or component responsibilities on CRC cards. Noodle around. Give yourself the luxury of exploring with your co-workers how things might work. Don’t expect a final design, just a plausible one that will be fleshed out and revised as you implement those stories.

If your organization tends to look too far ahead, take actions to personally shift your focus away from endless planning. Winston Churchill said, “It is always wise to look ahead, but difficult to look further than you can see.” Looking too far ahead is also a big waste of time. Remedy this time suck by limiting the scope of what you plan for. Find opportunities to build and measure things.Learn to perform small experiments, make small decisions, communicate your results and move things along.

Colleagues and co-workers I’d classify as over-eager forward-looking designers typically build up comfortable infrastructure they think will be needed as a matter of course before they settle in to the programming task at hand. A good indicator that they’ve gone overboard is how others react to their work. One unhappy forward-looking architect remarked recently, “Not many people appreciate what I am trying to accomplish.”

You may not be aware that you look too far ahead. One indicator: you view most programming tasks as an opportunity to build or try out a cool new framework or create your own special DSL. (If you are part of an infrastructure team, then, fine, that is your job. Don’t embellish your creations. And make sure others can actually use your stuff.) If you are an application developer, nudge yourself towards refactoring and finding abstractions through concrete scenarios. Hold yourself back from writing speculative code.

I look to my colleague, Joe Yoder, as someone who takes a balanced approach when creating infrastructure or suggesting novel architectures. Joe likes to build flexible, adaptable systems, in particular Adaptive Object-Model architectures. He has years of experience doing this. He’s enthusiastic when opportunities come up to build these kinds of systems, but knows that this architecture style isn’t a good fit for every rapidly changing system. And he doesn’t build every part of such a system this way; only the parts that require extensive, ongoing customizations.

How far do you look ahead? Do you think you’ve found a good balance?

Agile Architecture Myth #5: Never Look Ahead

Stop it! Stop anticipating! Get coding. Get that next user story working. Don’t. Ever. Anticipate.

This seems to be a common agile mantra these days. Since we allegedly aren’t good at anticipating how code will evolve, well frack it, let’s just stop all speculation.

Instead, just do your level best to code for what you know right now. Let the design emerge. Keep it clean. Period.

But is that the best we can do?

Underlying these beliefs is the assumption that you will get burned if you look ahead because you are bad at anticipating change or the need for design flexibility. Since you never get it right, why ever bother looking ahead?

Well, who says we are bad at looking ahead? Is it really true?

The fact is we all make design mistakes, whether we look ahead or not. But that doesn’t mean looking ahead is the cause of more mistakes. (Sometimes it even prevents mistakes).

I believe that you can and should look ahead. And that most developers, given half a chance, are pretty good at incorporating past experiences and making anticipatory design choices. We might even say that an “experienced developer” is one who has the ability to use past experiences to help make good forward-looking decisions.

When you refactor code to reduce dependencies you are making an on-the-spot judgment call about what part of your code is more stable and what is more likely to change. You make these decisions based on concrete evidence. You are only speculating a little bit (things keep changing here, so I’ll try this).

Experienced designers developers look ahead at a larger scale. And the longer they’ve been at it, the more they incorporate past experiences into daily looking just a little bit ahead. You learn to anticipate where the design wrinkles will be.

If you’re less experienced, this seems like magic. How can you ever know when you’ve made an appropriate anticipatory design decision when the payoff only comes sometime in the future? Well, you don’t ever know for certain.

But you get better at looking ahead by practicing it in a safe, supportive development environment. One where you are encouraged to look around and look ahead. You get to see what more senior people do. And, if you are lucky, when you ask them why they are doing something they take the time to explain their reasoning.

Looking ahead isn’t antithetical to refactoring. In fact, it might even help you avoid some rework. Over time, we learn to make even more reasonable judgment calls based on experience, wisdom, and gut instinct. (And sometimes people think that we aren’t thinking ahead because we can fluidly make on on-the the-spot design changes).

If you never practice looking ahead, well, you might just be doomed to refactoring your way to a good design. For me, that simply doesn’t cut it. And it can simply be too painful and inefficient for large, complex projects.

Recently, a thoughtful agile coach and architect told me:

“When I start a project, I try to figure out what is the minimum architecture we need to do for that context. I start from a few key things:
What are the requirements in terms of performance, scalability, reliability, security, localization, functionality etc.?

What deployment structure best fits these requirements?

What platforms and frameworks facilitate the development of the application?

What don’t we know how to do at this point? How can we find out?”

Asking these questions helps him, “define a basic but good architecture. One that ensures the right [architectural] features, minimizes time/costs and reduces risks.” That’s just enough upfront speculation to establish a baseline architecture.

But being ever thoughtful, he also asked, “I am wondering however: Should I do more? In what circumstances?”

Under certain circumstances it may be appropriate to do even more upfront design work.

Agile heresy?

I think not.

When we tackle really big unknowns it makes sense to explore more before making costly investments. Experimentation backed up by evidence-based decision making makes our designs better. Looking ahead can save us time.

But still, too many are deceived into believing in those vividly scary look-ahead failure stories. Then they overreact, pulling back from any speculative design.

My advice: Don’t overlook the many times when you have successfully looked ahead and saved time and effort. Not looking ahead is symptomatic of inexperience.

An Architect’s Dilemna: Should I Rework or Exploit Legacy Architecture?

I recently spoke with an architect has been tuning up a legacy system that is built out of a patchwork quilt of technologies. As a consequence of its age and lack of common design approaches, the system is difficult to maintain. Error and event logs are written (in fact, many are), but they are inconsistent and scattered. It is extremely hard to collect data from and troubleshoot the system when things go wrong.

The architect has instigated many architectural improvements to this system, but one that to me was absolutely brilliant was to not insist that the system be reworked to use a single common logging mechanism. Instead, logs were redirected to a NoSQL database that could then be intelligently queried to troubleshoot problems as they arose.

Rather than dive in and “fix” legacy code to be consistent, this was a “splice and intelligently interpret” solution that had minimal impact on working code. Yet this fairly simple fix made the lives of those troubleshooting the system much easier. No longer did they have to dig through various logs by hand. They could stare and compare a stream of correlated event data.

Early in my career I was often frustrated by discrepancies in systems I worked on. I envisioned a better world where the design conventions were consistently followed. I took pride in cleaning up crufty code. And in the spirit of redesigning for that new, improved world, I’d fix any inconsistencies that were under my control.

At a large scale, my individual clean up efforts would be entirely impractical. Complex software isnâ’t the byproduct of a single mind. Often, it simply isn’t practical to rework large systems make things consistent. It is far easier to spot and fix system warts early in their life than later after myriad cowpaths have been paved and initial good design ideas have become warped and obsfucated. Making significant changes in legacy systems requires skill, tenacity, and courage. But sometimes you can avoid making significant changes if you twist the way you think about the problem.

If your infrastructure causes problems, find ways to fix it. Better yet (and here’s the twist): find ways to avoid or exploit its limitations. Solving a problem by avoiding major rework is equally as rewarding as cleaning up cruft. Even if it leaves a poor design intact. Such fixes breathe life into systems that by all measures should have been scrapped long ago. Fashioning fixes that don’t force the core of a fragile architecture to be revised is a real engineering accomplishment. In an ideal world I’d like time to clean up crufty systems and make them better. But not if I can get significant improvement with far less effort. Engineering, after all, is the art of making intelligent tradeoffs.

Agile Architecture Myths #4 Because you are agile you can change your system fast!

Agile designers embrace change. But that doesn’t mean change is always easy. Some things are harder to change than others. So it is good to know how to explain this to impatient product stakeholders, program managers, or product owners when they ask you to handle a new requirement that to them appears to be easy but isn’t.

Joe Yoder and Brian Foote, of the Big Ball of Mud fame, provide insights into ways systems can change without too much friction. They drew inspiration from Stuart Brand’s How Buildings Learn. Brand explains that buildings are made of components organized into shearing layers. He identifies six layers: the site, the structure, the skin, the services, the space plan, and physical stuff in the building.

Each shearing layer has its own value and speed of change, or pace. According to Brand, buildings are able to adapt because faster changing layers (e.g. the services layers and spaces) are purposefully designed so to not be obstructed by slower changing layers. If you design your building well, it is fairly easy to change the plumbing. Much easier than revising the foundation. And it is even easier to rearrange the furniture. Sometimes designers go to extra efforts to make a component easier to change. For example, most conference centers are designed so that sliding panels form walls that allow inside space to be quickly modified.

Brand’s ideas should’t be surprising to software developers who follow good design practices that enable us to adapt our software: keep systems modular, remove unnecessary dependencies between components, and hide implementation details behind stable interfaces.

Foote and Yoder’s advice for avoiding tangled, hard-to-change software is to, “Factor your system so that artifacts that change at similar rates are together.” They also present a chart of typical layers in a software system and their rates of change:

Frequently, we are asked to support new functionality that requires us to make changes deep in our system. We are asked to tinker with the underlying (supposedly slower changing) layers that the rest of our software relies upon. And often, we do achieve this miraculous feat of engineering because interfaces between layers were stable and performed adequately. We got away with tinkering with the foundations without serious disruption. But sometimes we aren’t so lucky. A new requirement might demand significantly more capabilities of our underlying layers. These types of changes require significant architectural rework. And no matter how matter how agile we are, major rework requires more effort.

Because we are agile, we recognize that change is inevitable. But embracing change doesn’t make it easier, just expected. I’d be interested in hearing your thoughts about Foote and Yoder’s shearing layers and ways you’ve found to ease the pain of making significant software changes.

Landing Zone Targets: Precision, Specificity, and Wiggle Room

A landing zone is a set of criteria used to monitor and characterize the “releasability” of a product. Landing zones allow you to take product features and system qualities and trade them off against each other to determine what an acceptable product has to be. Almost always these tradeoffs have architectural implications. If you’ve done something similar in the past, the criteria you should use to define your landing zone may be obvious. But for first time landing zone builders, I recommend you task someone who knows about the product to take a first cut at establishing landing zone criteria that is then reviewed and vetted by a small, informed group.

A business architect, product owner, or lead engineer might prepare a “proposed landing zone” of reasonable values for landing zone criteria that are questioned, challenged, and then reviewed by a small group. On one program I was involved with, the chief business architect made this initial cut. He was a former techno geek who knew his technical limits. More important, he had deep business knowledge, product vision, and had a keen sense about where to be precise and where there should be a lot of flexibility in the landing zone values.

Some transaction criteria were very precise. Since they were in the business of processing a lot of transactions, they knew their past and knew were they needed to improve (based on projected increases in transaction volumes). For example, that transaction throughput target for a particular business process was based on extrapolations from the existing implementation (taking into account the new architecture and system deployment capabilities). This is a purposefully obfuscated example:

Example Landing Zone Attribute
Characteristic Minimum Target Outstanding
Payment processing transactions per day 3,250,000 4,000,000 5,500,000

Some targets for explicit user tasks were very specific (one had a target of less than 4 hours with no errors, and an outstanding goal of 1 business day). On the other hand, many other landing zone criteria were only generally categorized as requiring either a patch, a new system release, or online update support. The definitions for what was a patch, a release or an online update were nailed down so that there was no ambiguity in what they meant.

For example, a patch was defined as a localized solution that took a month or less to implement and deploy. The goal was eventually to get closer to a week than a month, but they started out modestly. On the other hand, a release required coordination among several teams and an entire system redeployment. An online update was something a user could accomplish via an appropriate tool.

So, for example, the landing zone criteria for reconfiguring a workflow associated with a specific data update stream had minimal and target values of “release” and an outstanding value of “online update”.

When defining a landing zone for an agile product or program, carefully consider how precise you need to be and how many criteria are in your zone. Less precision allows for more wiggle room. Without enough constraints, however, it’s hard to know what is good enough. The more precise landing zone criteria are, the easier it is to tell whether you are on track to meet them. But if those landing zone criteria are too narrowly defined, there’s a danger of ignoring broader architecture and design concerns in order to focus only on specifically achieving targets.

We live in a world where there needs to be a balance. I’ll write more about who might be best suited to defining and redefining landing zones in another post.

Software Decision Making Under Stress

I recently blogged about my discomfort with making software design decisions at “the last responsible moment” and suggested that deciding at the “most responsible moment” might be a better approach. To me, a slight semantic shift made a difference in how I felt. Deciding at the most responsible moment made me feel less stressed and more in control of the situation.

But is this because I am basically lazy, preferring to put off decisions until they absolutely, positively must be made (and then hating that gut wrenching feeling when I finally realize that I don’t have enough time to think things through)? Or is something else going on?

I admit that the decisions we make about software development on a day in/day out aren’t always made under extreme stress; yet I thought I’d see what researchers say about decision-making under stress. As a developer I benefit from some stress, but not too much. That’s why (reasonable) deadlines and commitments are important.
But first, a disclaimer. I have not exhaustively researched the literature on this topic. I suspect there are more relevant and recent publications than what I found. But the two papers I read got me thinking. And so, I want to share them.

Giora Keinan, in a 1987 Journal of Personal and Social Psychology article, reports on a study that examined whether “deficient decision making” under stress was largely due to not systematically considering all relevant alternatives. He exposed college student test subjects to “controllable stress”, “uncontrollable stress”, or no stress, and measured how it affected their ability to solve interactive decision problems. In a nutshell being stressed didn’t affect their overall performance. However, those who were exposed to stress of any kind tended to offer solutions before they considered all available alternatives. And they did not systematically examine the alternatives.

Admittedly, the test subjects were college students doing word analogy puzzles. And the uncontrolled stress was the threat of a small random electric shock; but still, the study demonstrated that once you think you have a reasonable answer, you jump to it more quickly under stress. (Having majored in psychology and personally performed experiments on college students, I can anecdotally confirm that while college students are good test subjects, one should take care to not over-generalize any results.)

So, is stress “good” or “bad”? Is systematically considering all viable alternatives before making a decision a better strategy (or not)? Surely, we in the software world know we never have enough time to make perfectly researched decisions. And big-upfront-decision-making, without confirmation is discouraged these days. Agile developers believe that making just-in-time design decisions result in better software.

But what are the upsides or downsides to jumping to conclusions too hastily? What happens if you feel too stressed when you have to make a decision? To gain some insight into that, I turned to a summary article, Judgment and decision making under stress: an overview for emergency managers , by Kathleen M. Kowalski-Trakofler, Charles Vaught, and Ted Sharf of the National Institute of Occupational Safety and Health. These authors raised many questions about the status quo of stress research and the need for more grounded studies. However, they also drew three interesting conclusions:

1.Under extreme stress [think natural disasters, plane crashes, war and the like], successful teams tend to communicate among themselves. As the emergency intensifies, a flatter communication hierarchy develops with more (unsolicited) information coming from the field to the command centre. Under stressful, emergency situations, communication becomes streamlined and localized. Also, people take personal initiative to inform leaders about the situation.

2. Stress is affected by perception; it is the perceived experience of stress that an individual reacts to. When you perceive a situation as stressful, you start reacting as if it were stressful. What is stressful to you is what’s important. And not everyone reacts well under stress. Training helps emergency responders to not freak out in an emergency, but those of us in the software world aren’t nearly so well-trained to respond to software crises. When is the last time you had software crises training?

3. Contrary to popular opinion, judgment is not always compromised under stress. Although stress may narrow the focus of attention (the data are inconclusive), this is not necessarily a negative consequence in decision making. Some studies show that the individual adopts a simpler mode of information processing that may help in focusing on critical issues. So, we can effectively make reasonable decisions if we find and focus on the critical issues. If we miss out on a critical issue, well, things may not work out so well.

Reading these papers confirmed some suspicions I had: Stress is something we perceive. It doesn’t matter whether others share your perception or not. If you feel stressed, you are stressed. And you are more likely to make decisions without considering every alternative. That can be appropriate if your decisions are localized, you have experience, and you have a means of backing out of a decision if it turns out to be a bad one. But under extreme stress things start to break down. And then, if you haven’t had emergency training, how you respond is somewhat unpredictable.

I hope that we can start some ongoing discussions within the software community about design decisions and decision-making in general. How do you, your team or your management react to, avoid, or thrive on stress? Do you think agile practices help or hinder decision-making? If so, why? If not, why not.

Agile Architecture Myths #2 Architecture Decisions Should Be Made At the Last Responsible Moment

In Lean Software Development: An Agile Toolkit, Mary and Tom Poppendieck describe “the last responsible moment” for making decisions:

Concurrent development makes it possible to delay commitment until the last responsible moment, that is, the moment at which failing to make a decision eliminates an important alternative.

And Jeff Atwood, in a thought-provoking blog argues that “we should resist our natural tendency to prepare too far in advance” especially in software development. Rather than carry along too many unused tools and excess baggage, Jeff admonishes,

Deciding too late is dangerous, but deciding too early in the rapidly changing world of software development is arguably even more dangerous. Let the principle of Last Responsible Moment be your guide.

And yet, something about the principle of the last responsible moment has always made me feel slightly uneasy. I’ve blogged about a related topic (just barely enough design) before. And be aware that in both my personal and professional life that I am not known as someone who plans things far out in advance. As a consequence I rarely use frequent flyer miles because I don’t anticipate vacation plans far enough in advance. I am not known to get to the airport hours ahead of my flight either. My just-in-time decision-making and actions have been known to make my travelling companions a bit uneasy. They prefer a not so tight timeline.

But what about software development? Well, if I find an approach that seems worth pursuing, I’ll typically go for it. I like to knock off decisions that have architecturally relevant impacts early so I can get on to the grunt work. A lot of code follows after certain architectural choices are made and common approaches are agreed upon. Let’s make a rational decision and then vet it, not constantly revisit it, is my ideal.

Yet I know too-early architecture decisions are particularly troublesome as they may have to be undone (or if not, result in less-than-optimal architecture).

So what is it about forcing decision-making to be just-in-time at the last responsible moment that bugs me, the notorious non-planner? Well, one thing I’ve observed on complex projects is that it takes time to disseminate decisions. And decisions that initially appear to be localized (and not to impact others who are working in other areas) can and frequently do have ripple affects outside their initially perceived sphere of influence. And, sometimes, in the thick of development, it can be hard to consciously make any decisions whatsoever. How I’ve coded up something for one story may inadvertently dictate the preferred style for implementing future stories, even though it turns out to be wrongheaded. The last responsible moment mindset can at times lull me (erroneously) into thinking that I’ll always have time to change my mind if I need to. I’m ever the optimist. Yet in order to work well with others and to produce habitable software I sometimes need a little more forethought.

And so, I think I operate more effectively if I make decisions at the “most responsible moment” instead of the “last responsible moment”.

I’m not a good enough of a designer (or maybe I am too much of an optimist) to know when the last responsible moment is. Just having a last-responsible moment mindset leaves me open to making late decisions. I’m sure this is not what Mary and Tom intended at all.

So I prefer to make decisions when they have positive impacts. Making decisions early that are going to have huge implications isn’t bad or always wasteful. Just be sure they are vetted and revisited if need be. Deferring decisions until you know more is OK, too. Just don’t dawdle or keep changing your mind. And don’t just make decisions only to eliminate alternatives, but make them to keep others from being delayed or bogged down waiting for you to get your act together. Remember you are collaborating with others. Delaying decisions may put others in a bind.

In short: make decisions when the time is right. Which can be hard to figure out sometimes. That’s what makes development challenging. Decisions shouldn’t be forced or delayed, but taken up when the time is right. And to help me find the right times, I prefer the mindset of “the most responsible moment” not the “last responsible one.”