Shift Left: Testing Earlier in Development

One candidate for best experience report at XP 2015 was “High Level Test Driven Development – Shift Left” by Kristian Bjerek-Gustuen, Emil Wiik Larsen, Tor Stålhane, and Torgeir Dingsøyr. Their report describes testing strategies and tactics used on a large-scale IT development project in Norway. These authors called it “shifting left” because they wanted to move testing to as early as possible in development.

Their project was complex along several dimensions.

Coordinated work among independent teams: Several vendors were simultaneously developing and delivering systems that communicated with each other via service interfaces. Vendors did detailed design and development including unit, integration and sprint system testing of their deliverables. This work had to be integrated by the company. They performed acceptance testing after every sprint, system integration testing which ran in parallel with the vendors’ sprint system tests, as well user acceptance testing.

Significant coding and testing effort: Over 100,000 hours of testing and development went into the release.

Time pressure and project criticality: The delivery date was fixed. The system was an extension of an existing system for processing payments. It had to be delivered on time and with high quality.

The duration of the project they described was roughly 40 weeks, if I read the report correctly: 12 3-week sprints followed by 6 weeks of system testing. Faced with a time crunch, an existing group of 3 maintenance scrum teams were scaled up to 11 teams over four months. The customer who was receiving the software system didn’t have fulltime resources to dedicate to the scrum teams who were comprised of these roles: scrum master, developer, functional architect, technical architect, and QA/tester. My guess is that the customer was called upon as needed to provide clarifications, offer feedback at sprint demos, and to do all necessary testing and integration testing.

To ensure that each team had efficient and sufficient testing and quality assurance, a dedicated QA person/tester was assigned to each team. This reminds me of Stephanie Savoia’s experience report at Marchex where they embedded testers into their dev teams.

In the case of this Shift left report, the dedicated QA person (I prefer the term quality advocate) seemed very busy and vital: they ensured the quality of design documentation; decided whether the implementation was testable; prepared test data; worked with devs and the Scrum master to ensure high quality code. They also made sure test activities were performed as early as possible and that the customer provided necessary clarifications to the dev team.

Bridge, go-between, quality advocate, tester extraordinaire!

During sprint demos, in addition to demonstrating new functionality, teams also provided information on how testing was performed and any issues they had encountered in testing the implemented functionality.

There is more in the report about how they managed testing and dependencies between teams, identified and tracked high-risk modules changes and defects to aid test and development planning, and introduced exploratory testing using interdisciplinary teams. But I digress.

Back to those busy QA advocates. Not surprisingly, the experience reporters mentioned in passing that the dedicated QA advocate became one of the most central resources for the teams.

It seems that they were deeply appreciated by the whole team. That’s important. Where they ever overwhelmed by their responsibilities? Were they overworked?

Any experience report never answers all questions I have. I still find them thought provoking. Even though I’d love to sit down and talk with experience reporters and ask them more questions.

One pattern Joe Yoder and I have written about in our Shifting From QA to AQ pattern collection is called Pair With A Quality Advocate. If you purposefully pair up devs or other folks with a QA advocate, their expertise can “rub off” on less skilled/experienced testers and developers. Steadily (and sometimes stealthily), a quality mindset gets infused into the entire team. You still need quality advocates, but everyone takes on more responsibility for quality. And that’s a good thing.

Future Commitment

In early April, I spent a fun weekend at Pace University with Allen Wirfs-Brock and Mary Lynn Manns speaking with students of Pace University’s Doctor of Professional Studies in Computing program.

Mary Lynn

Mary Lynn Manns

It was a good opportunity to reconnect with Mary Lynn and to hear about new patterns in her latest book, More Fearless Change: Strategies for Making Your Ideas Happen.

Mary Lynn and Linda Rising have been working on these patterns for over a dozen years. They aren’t finished (curating long-lived patterns is an ongoing task). They continue to collect and write patterns for those who want to initiate, inspire, and sustain change in organizations they are part of or in their personal or professional life.

There isn’t one magic thing to do to institute change. Mary Lynn and Linda advise,

“You are working with humans, often in complex organizations, so results are rarely straightforward and the emergent behavior might be totally unexpected. Therefore, upfront detailed planning is rarely effective. Instead, take one small step toward your goal and see what happens. You will inevitably encounter missteps and failures along the way….uneven progress can be discouraging but may also teach you about the idea, about the organization, and, most of all, about yourself.”

Significant change often includes performing an ongoing series of experiments.

One new pattern (or strategy) in their book is Future Commitment. This one has hooked me many times.

Instead of asking a busy person for immediate help, ask them do something you need later and then wait for them to commit. Don’t worry about trying to get them to agree right away. Be patient. Keep in touch. Provide them regular information that encourages them to become more interested in your change initiative. Don’t be a pest.

Once they agree to help, solidify their commitment by recording the date and sending it in writing (along with gentle reminders so they are kept aware of their commitment). Reminders can be annoying. So include an update of what is happening and how their contribution will fit as part of a reminder. Have an alternate lined up in case that busy person can’t commit. There’s more to this strategy and the psychology behind how people commit (along with 14 other new strategies) in their book.

Like Mary Lynn and Linda, I don’t view change patterns as evil or manipulative. They are simply tools, that once aware of, you can use to engage people and get them to help you make changes.

And so here’s a future commitment if you have an urge to write about your agile experiences: We need experience reports for Agile 2016. Submit a proposal to the Agile Experience Report program. Do it soon. Within the next 30 days.

You don’t have to start writing now, but if you send in a proposal to me, it will grab my attention. Even better, if you are attending Agile 2015, we can also talk over your proposal. We ask that you write up a proposal so you get in the practice of collecting and communicating your thoughts in written form.

You could wait and submit your experience report idea via the Agile conference submission system in November. If you do, your chance of it being selected is limited. The format of the submission system makes it difficult to include details or have an ongoing conversation to sharpen your ideas. This year we had maybe 100 submissions from which we selected 20.

If you wait to submit your proposal via the conference system, it will have to really stand out from the crowd to grab our attention.

Instead, if you submit your proposal to the Agile Experience Program, it will be carefully read as soon as we receive it. If selected, we will work with you throughout the coming year. You don’t have to start writing immediately. You will get help from a shepherd as you write. As soon as you finish, your work will be published on the Agile Alliance website. You will be also invited to present your report at a future Agile conference.

I’m ready to help you write about your experience if you are ready to make a future commitment to writing. It all starts with your proposal.

Mob Programming: The Unruly Experience

This year 11 experience reports were presented at XP 2015. As co-chair of Experience Reports, along with Ken Power, one of my last tasks was to select a best experience paper. That was hard. We had several good experiences to choose from (I’ll report on more of them in future blog posts).

Ken Power and I independently read and reviewed each paper. At the conference we compared notes and agreed on 3 papers we would consider for this award. We also reviewed the authors’ presentations before making our final determination.

The best experience paper was awarded to Alexander Wilson for his report, Mob Programming – What Works, What Doesn’t. One thing that made this paper stand out, was its balanced view. Alex is clear about things you need to watch out for when and if you try mobbing—don’t be lulled into groupthink and be aware of overly dominant personalities.

Alex Wilson telling us about Mob Programming

Alex Wilson telling us about Mob Programming at XP 2015


He also gave us a close look into his company’s mob programming experience.

Mob programming is where a team of programmers swarm on a problem. They work together instead of pairing or individually writing code. One person is at the keyboard, the rest of the team helps navigate. They pay attention and guide the person at the keyboard. Team members take turns at the keyboard. When mobbing, there are more eyes on the code and more minds focused on the problem you are trying to solve.

Folks at Unruly were inspired to try mobbing after Woody Zuill talk at JavaOne in late 2014. Woody also spoke about Mob Programming at Agile 2014. You can read Woody’s experience report here.

Developers at Unruly were seasoned XPers accustomed to pair programming and test-driven development. They are responsible for the entire lifecycle of their product, from research to operational support. They are in constant touch with problems and ongoing sustainability of code they write.

They decided to form a mob to work on performance enhancements to their existing product. This involved refactoring and reworking critical code. If they made mistakes, it could result in lost data and have serious financial impacts. Mobbing for them wasn’t just a casual experiment.

After mobbing one day per week for a couple of months they found, in general, that they were pretty happy with their results. After 5 months they concluded that they don’t prefer mob programming for all user stories. They do find mobbing beneficial for complex work (where there is the potential for errors) over complicated work (where the solution is known, but is merely time consuming). They also find that tasks that are dull or repetitive were likely to cause their mob to dissolve. They only mob one day a week, unlike Woody’s team who mobs every day.

Unruly settled on a rhythm of periodic mobbing that worked for them. That’s what I like about Alex’s report. He tells us: here is what worked, here is where we tripped up, here is how we adapted our practices, and here is what we’re doing now.

Teams in learning organizations perform ongoing experiments. While they settle on a core set of practices, they also try to build upon them. They keep innovating, improving, and reflecting. And how they work continues to evolve.

What does karaoke have to do with being agile?

Last week I attended XP 2015 in Helsinki, Finland. This is my second time there…I hope to go next year, too. It is a unique blend of researchers and students along with people working in big and small companies doing all kinds and flavors of agile development from all over the world. 37 countries were represented. We had 11 experience reports (Ken Power and I were track co-chairs of experience reports. More about them in another post).

Lots of learning. We had fun, too.

One of the highlights of the conference banquet was Presentation Karaoke. 6 brave souls gave 3 minute impromptu speeches. Every 30 seconds a new, unknown slide appeared.

Llewellyn Falco’s PowerPoint karaoke at XP 2015


Here’s video of parts of a presentation by Llewellyn Falco.

Avraham Poupko won the competition by popular vote. Pardon the shaky camera…I was laughing and not holding my cell phone very steady ….

So what’s connection between PowerPoint karaoke and agile development? Each speaker ostensibly spoke on a conference theme. Mostly, they had fun. High performing agile teams, like these speakers, know how to roll with the punches. Sure there’s a theme, a plan, a backlog of work items. But sometimes requirements change and you need to adapt on the spot. Rather than throw up your hands or panic, you need to make things work. It is impossible to go with the flow if you are in a panic. Like improv, agility demands that you accept change. Unanticipated things happen. You accept them and you adjust.

Some speakers made connections between slides. Avraham spotted what he thought were mangoes on the head of someone on one slide and incorporated them throughout his talk. Giovanni summarized all the key points at the end (it is a wonder he remembered them…but again, he wasn’t in a panic). Llewellyn was master at dramatic pauses before finding something to say. Their presentations were great.

Next time you are faced with unexpected change, take some cues from these improvisers: pause, gather yourself, take stock of the situation and then figure out what you need to do. Don’t panic. You’ll get through it. And maybe with a more relaxed attitude you’ll even have some fun.

Life in the Mob

The latest report in the Agile Experiences Program has just been published. It is a story by Jason Kerney about what it is like to join a Mob Programming Team.

Jason’s report started with a conversation over lunch at Agile 2014. Jason was sitting with his manager, Woody Zuill who reported on how mob programming was born and how it works in practice. Woody introduced Jason as the newest member of their mob.

Ever on the lookout for good stories (I’m Director of the Agile Experience Report Program), I asked Jason what it was like to join the mob. And thus began a conversation which after a lot of hard work by Jason turned into this latest report.

When he first joined the team, Jason was hyper vigilant. Not wanting to let his new team down, he found it hard to take any breaks. For the first few weeks he would go home from work exhausted. By accident one day he discovered that if he stepped away from the team for awhile, he could catch up in a just a few minutes. He had stumbled on a sustainable rhythm for mob programming and went home from work energized.

Jason’s initial hyper vigilance reminds me of my first two weeks on the job as a forest service lookout. I was constantly scanning for fires through my binoculars. I got eyestrain. After two weeks, I just couldn’t keep up my constant surveillance. So I backed off to looking for fires every 15 minutes. That was plenty of attention. No fire is going to explode and burn down the entire forest in 15 minutes.

Jason also shares a keen observation into the collective way his team dynamically works. He likens how they each pay attention to different things as being like a howling wolf pack (where no two wolves sing at the same frequency). Wolves just join in and find an agreeable pitch. Mob programmers who are paying attention to their teammates find a way to contribute what’s “missing”. Jason finds mob programming powerful because:

“In coding, there are a lot of things to think about, architecture, design, the problem at hand, coding standards, testing, deployment, business impact and security to name a few. I have found that our mob treats each of these like a wolves’ howling frequency. We each take one or a few and pay attention to those. If someone else appears to be covering it, we choose something else. None of this is an active decision, it just happens.”

Don’t dismiss Mob Programming as a simple variant of an XP programming team. There’s more to it.

If you have an itch to write and share your agile experiences, please feel free to contact me at rebecca at wirfs hypen brock dot com. I know that writing is hard work. I want to hear your intriguing stories and help you tease out your wisdom. I can help you find a voice in your writing. We learn from experience (especially when we reflect on it). And it’s a wondrous gift to share your experiences with others. Thanks for sharing, Jason.

Can Nudging Help Us Make Better Choices?

Nudging, as popularized by Richard Thaler’s book Nudge, is the idea of making a by-default action the preferred choice so you don’t have to think deeply to make a good decision. However the common definition for nudge as, “a slight or gentle push or jog, especially with the elbow,” gives me reason to pause.

I don’t like being elbowed, even if it is done gently with good intentions.

Does nudging ever work?

At first glance, I was intrigued by the possibility of personally crafting my own nudges. The idea of setting up personal reminders using a service like NudgeMail, seemed promising. While potentially useful, NudgeMail reminders aren’t exactly what Richard Thaler means when he talks about nudging. A Thaler nudge is more like an unobtrusive force that moves you to make a better decision or action without feeling poked or prodded. A nudge isn’t the same as a reminder.

Successful nudges need to be unavoidable attractive shiny attractions that draw you to nearly effortlessly make good decisions or take appropriate action. They are the not prods, nags, or scoldings, either. You can’t be nudged when you are annoyed.

In today’s Oregonian there was an article about the use of electric car charging stations. The idea behind installing publicly accessible recharging stations was to “nudge” more people to buy and drive electric cars. By plunking public charging stations all over the place, the thinking went, people wouldn’t be so concerned about draining their batteries while out and about.

Public charging stations has been in failure in Oregon. Even though placed in public places, chargers are rarely used. Some appear conveniently located (like parking garages in downtown Portland). But many others are located in out of the way places (there’s one at the back end of a shopping mall near in the small town where I live. I’ve never seen a car there). Turns out they were located more as matter of political expediency than common sense or practicality.

Inconveniently, it also takes hours, not minutes, to recharge a car using them. Consequently, people find it cheaper and more convenient to plug in and recharge their cars at home overnight. They don’t need to top off their battery’s charge while working out at the gym. Often people buy electric cars with the intention of using them only for short commuter trips. They don’t need or want public charging stations.

But consider the case of Tesla. Tesla has strategically installed fast-charge station capable of delivering up to 50% battery capacity to their sporty cars in about 20 minutes (roughly 16 times faster than most public charging stations). Tesla wants its customers to feel comfortable driving their electric cars on longer trips. There’s a Tesla recharging station by the Burgerville in Centralia, Washington, halfway between Seattle and Portland. Tesla owners are more likely to travel between Portland and Seattle, a distance of 180 miles, if they can stop for a burger at the healthy, local food-sourced burger joint halfway on their journey and get their battery charged at the same time. Tesla has found the right nudge to encourage longer road trips.

The idea of painless nudges that help us “do the right thing” seems compelling. But isn’t easy to find subtle nudges for behaviors we want to change.

Merely speculating on what might be a good nudge before you implement it can be a bad idea. You need to experiment. I’m not so sure government policy-makers can or do take the time to experiment on the effectiveness of nudges before they implement them. Perhaps they should.

On a personal level, any nudge I put in place has to be cost effective, cheap to set up, and easy to tinker with. For example, buying fresh fruit (for me, anyways) is just the nudge I need to eat fruit instead of more sugary snacks. If you don’t like fresh fruit, this nudge won’t work for you.

Keeping a tab in my browser open to Duolingo, where I’m trying to learn Portuguese, is just enough of a nudge for to keep me to keep at it. I study for a few minutes every day when I need to take a break from work. And Duolingo gently sends me email reminders about keeping up my daily streak. Still relatively new to Duolingo, these reminders aren’t yet annoying. Overtime, I know they will be. I don’t like to be reminded to do what I’m already doing. I just hope they have a way to turn them off.

If motivated, I can be nudged a little. But I’ve got to be in the right mood and it has to be in an appropriate situation and context. If a nudge feels at all like a shove or a poke instead of gentle, painless, helpful guidance, I’m inclined rebel and do the opposite.

A classic example of “nudging” by policy makers is to create incentives (think tax breaks or credits). For example, giving a rebate upon buying energy efficient appliances or solar panels. That nudge has been in place for some time here in Oregon. But I haven’t bought a new energy efficient appliance simply to get a tax break. Even if the investment will be paid back in a few years in savings on my electric bill. That nudge was more likely to nudge people moving into a new home instead of someone like me.

A couple of years ago, I recycled an old refrigerator because I happened to stumble upon an extremely attractive nudge when I was searching the Internet for recycling options. I could schedule an appointment online (selecting the day and time from a list of available times) for someone to come to my house, cart away my fridge, and mail me a $50 refund check. I was in the right frame of mind for that nudge. The conveniences plus the small monetary compensation made this a perfect nudge.

But it is oh so easy to resist a nudge when I am at all forced to think even a little. I am a cranky contrarian who doesn’t like pushy nudges. Don’t tell me I have to recycle my used appliances. And don’t give me too many options to choose from, either. But give me a choice.

Nudges should be easy peasy, transparent, painless offerings of ideal options.

If you are a policy maker at your company, you might be able establish a few nudges like providing free bus passes to encourage mass transit use or putting in showers to encourage bicycling to work or exercise.

If you are an agile coach or team leader you can nudge a little, too.

Want to limit the time of a daily standup meeting? Hold it around a whiteboard and have people actually stand up. In my experience, getting comfy around a table makes meetings last longer. But watch out. I know some people who happily spend hours talking at the whiteboard. Every day. They need more than a nudge to keep stand ups brief. This leads some agile coaches to introduce protocols or standard ways of doing things that are demonstrated, and then gently enforced in order to encourage appropriate actions. How I feel about protocols and rituals is the topic for another blog post.

Nudging isn’t social engineering for social engineering’s sake, but with the intent of setting up your environment and context so you want to make the obvious better choice without feeling pressured.

Have you found effective ways to nudge yourself and change you or your team’s habits? If so, I’d like to hear from you.

Making New Behaviors Stick

My last post argued that knowing about cognitive biases and how our brain works is valuable, even if knowing isn’t enough to effectively change our behavior. Tweeter Jessica Kerr sums this up nicely, “Knowing isn’t half the battle, but without it, we don’t know the battle exists.”  Knowing doesn’t prevent  automatic behaviors and biased reactions. However, knowing gives us words we can use to reflect on our actions and feelings.

And then what? After awareness comes the really hard parts: consciously replacing undesirable, semi-automatic behaviors with more desirable ones.

Much of our daily behavior is driven by habit. Social psychologists estimate around 35-50% of our daily behaviors are driven by habit. Personally, I think that may be somewhat on the low end.

There’s a good reason for habits. If we had to consciously think through every action we took, we’d be exhausted.  When we have to consciously think about how to perform any task, it takes mental energy. And that wears us out. When we are tired we unconsciously fall back on our fast (associative) thinking to conserve mental energy and provide easy (if not always well-reasoned) answers to difficult questions.

It’s better if we can slip on our shoes in the morning and tie them without thinking.

We actively use our system 2 brain when we learn new skills or are placed into unfamiliar situations (think trying to navigate around a new neighborhood, learning customs in an unfamiliar country, learning a new problem domain or new programming language, or tracking down a tricky intermittent software bug).

When we’re not forced to work so hard, we mentally take it easy. We’re NOT lazy. But we do many things out of habit, to conserve our slow thinking, system 2 brain for when we really need it.

But what if we want to replace an old habit with a new improved one—and make that new habit automatic?

To change entrenched habits, we have to find effective replacements, insert them into our daily life…and be vigilant. This takes energy and effort.

The bad news: Old habits can still be triggered. They don’t disappear. They are simply supplanted by more recent habits (if those new habits can be successfully triggered).

One way to thwart undesirable habits is to remove their triggers from your environment. That’s why some people recommend removing unhealthy food from your household and  environment when wanting to develop healthier eating habits (easier said than done when you live with others or if there are always donuts at work on Fridays).

When it’s impractical to remove undesirable triggers, you’ve simply got to deal with them. Instead of reacting blindly to that old trigger, you can experiment with introducing new behaviors for that trigger with even stronger rewards. When the donuts show up at work, treat yourself to a good cup of coffee and slightly healthier biscotti instead (if you really really like donuts this isn’t going to work).  Better yet, avoid the strongest morning donut trigger by coming later on Friday after all the good donuts are gone (those stale donuts may be a trigger that you are still suceptible to, especially if you are tired).

This isn’t easy. Change rarely is.

Perhaps another solution to making changes stick is diligence. Be on the lookout for undesired behaviors/actions/triggers. Spot them, then quickly squelch ‘em. Then go back to doing the desired behavior. This works, for a while.

Exercising diligence takes mental energy. You only have limited amounts of willpower to spare and when it is gone, well, things can get ugly. That’s why after a stressful day (when your system 2 is already tired from everything else you used it for) that you can let down your guard in a heartbeat. And then, dammit!…you slip up. You notice you slip up, beat yourself up about it, and vow to be even more vigilant. But you slip up again and…rinse, repeat….  I am not going to continue down this rat hole because it IS depressing.

So how do we get ourselves out of these mental ruts?

Evidence suggests that too much retrospection isn’t good. So don’t beat yourself up when you slip up. Recognize that you’ve slipped up  and try again. Switch things up and keep on trying (identify a better trigger or reward, experiment, try and try again). Don’t give up or give in to a defeatist attitude.

Some good news: Research suggests we are more successful at changing behaviors when we go through life changing events—when we move, retire, graduate from school, go off to college, start a new job, quit a job, marry, get ill—we are more open to change.

But I don’t wait for a big event to effect big changes. I don’t make New Year’s resolutions, either. Instead, I periodically mix things up and try new things just for their novelty. Life is a grand experiment.

But still…research hasn’t yet shown how to sustain new behaviors. Even years after embracing new behaviors, we are prone to slipping up, given the right triggers (think vacations, travel, holidays).

Knowing how we can slip-up is not enough to avoid slip-ups. Santos and Gendler are right.

But believing that you can recover after a slip up is key to getting back on track. And that’s where a positive mindset, determination, the belief that you can learn from mistakes, and resilience go a long way towards making change and making changes stick.

Footnote: The Power of Habit by Charles Duhigg is a good book for those wanting to know more about habits in their life and business.

What good is knowing if it’s not half the battle?

Cognitive psychologists Laurie Santos and Tamar Gendler answered Edge.org’s 2014 question of the year, “What scientific idea is ready for retirement?” by stating we should retire our unquestioned belief that, “Knowing is half the battle.” The phrase, “knowing is half the battle” was popularized by the 1980s G.I. Joe cartoon’s ending message, “Now you know. And knowing is half the battle.”

I am too old to have watched G.I. Joe cartoons…but somehow that meme worked into my brain and has colored my outlook on life. I’m always seeking new information. I want to know and then act based on that knowledge.

Santos and Gendler state that, “Recent work in cognitive science has demonstrated that knowing is a shockingly tiny portion of the battle for most real world decisions.” When it comes to making decisions, knowing about how our brains work isn’t enough to stop us from making irrational decisions, doing things in the short term that undermine our long-term goals, or being biased by seemingly incidental (and unimportant) information. In fact, knowing might even get in the way of taking appropriate action.

But I like to know how things work. I want to believe that if I knew more about how my brain works, that I could make better decisions, act more rationally, and be less influenced by my goofy cognitive biases.

Yet, when I read Santos’ and Gendler’s Edge response and dug into their research, I realized that I have been deluding myself. Knowing is rarely sufficient to overcome our “built-in” brain reactions.

Knowing isn’t intrinsically bad. It just isn’t sufficient to effect change. Yet I still want to know how things work. I still believe knowing is incredibly powerful.

So what good is knowing?

Knowing a name for some thing gives you the vocabulary to talk about it. In my review of Design Patterns, I said, “most importantly, it names these design constructs, allowing teams to share a common vocabulary.”

You aren’t a better designer because you know design patterns. There is so much more to design than knowing a few patterns or applying them to solve a problem. But knowing about patterns enables you to have more informed conversations about what you see in existing designs and what design possibilities there are.

Knowing about system 1 and system 2 thinking as described in Kahneman’s, Thinking Fast and Slow, and how they interact can help you talk about how you behaved the way you did.

Knowing about a thing or concept enables you to see it. Once I knew of the Cargill logo, I spotted it everywhere. On trains, buildings, products. Before that, it was invisible.

Knowing about cognitive biases can help you spot “irrational” decisions.

Knowing about some thing helps you focus attention on it (which can at times be good). But simply increasing awareness and attention won’t guarantee that you won’t miss something else important or avoid making poor decisions.

Simply knowing isn’t enough to eliminate undesirable associations or remove biases. Otherwise, you’d never have knee-jerk reactions and would always accurately weigh risks and rewards when making an important decision.

We can’t change our “system 1” thinking with its automatic fast associations. That’s how we’re wired. Associations are freewheeling, making us extremely susceptible to priming, anchoring, and myriad other cognitive biases.

Knowing increases awareness. Maybe I’m overly optimistic, but I believe increased awareness is the first step towards making any significant change.

Knowing doesn’t prevent the triggering automatic behaviors and reactions. But it helps us shape the words we can use to reflect on our actions and feelings. And then, what?

If we want to behave differently, we need to take some action to avoid tripping up in the future. But since we can’t turn off our automatic reactions, what can we do? Indeed, changing “automatic” reactions is hard.

And that’s the focus of my next blog post.

Beware of Dogma. No. Be aware of dogma

Dogma has several different meanings. I’m going to purposefully split hairs in this post, because I don’t want to attach negative connotations to dogma in a knee jerk fashion. I want to be more thoughtful about my choice of words and my reactions to them.

Here are four meanings for dogma:

“1. an official system of principles or tenets concerning faith, morals, behavior, etc., as of a church.
2. a specific tenet or doctrine authoritatively laid down.
3. prescribed doctrine proclaimed as unquestionably true by a particular group.
4. a settled or established opinion, belief, or principle”
from dictionary.com

At first, these subtle differences in meanings annoyed me. But I wanted to push through that to see what I can learn about dogma. So here goes…

An official set of principles or tenets concerning faith, morals and behavior.
As a software professional, do I have an “official” set of principles and tenets that I believe in?

I have a set of guiding principles and practices for how I work, think about design, write code and tests that I’ve built up over 20+ years of practice. They have become part of how I prefer to operate. I’ve changed and refined them over time, discarding some practices, fine tuning others.

The guiding principles I follow weren’t handed down by authorities. I discovered them working alongside smart people and interacting with thoughtful designers who cared deeply about how they built and implemented software. I wanted to understand how productive people thought and worked, and try to incorporate what I saw as good practices and beliefs into my own beliefs and ways of working.

In the process, I co-wrote two object design books that shared a way of thinking about objects that I still find effective and powerful. Maybe writing books made me an authority. But I also have become a seeker of new and better ways of working. Over the years I have “blended” into my personal set of practices and beliefs about design some powerful ideas of others. This process of incorporating these ways of thinking and problem solving to me feels highly integrative rather than just “accepting” them as unchallenged beliefs or tenets. I have to sort through them, adjust them and then make them part of who I am and what I do. I am not one to blindly accept dogma.

The 3rd definition of dogma has negative connotations: a prescribed doctrine proclaimed as unquestionably true by a particular group.

Hm. I don’t hold many things about software design as being unquestionably true. I find it disconcerting when groups and factions form around the latest truth or discovery. For example, some fervent agile developers I know unquestionably believe that test-first development is the only way to design software. (I’m more of a test-frequent designer by nature). Those who refuse to acknowledge that there are other effective pathways to producing well-written, well-designed, maintainable code are trying to push a dogma of the 3rd meaning.

I find myself questioning any software doctrine that is held as being “universally” true. How presumptuous! There are so many different ways to solve problems and build great software.

I try to keep an open mind. My most strongly held beliefs are ones I should challenge from time to time. To do that, I have to push myself out of my comfort zone. For example, I have discovered a few things by letting go of several strongly held beliefs and performing some interesting experiments: How much code that checks expected behaviors do I really need to keep around to keep software from regressing? How many tests does any organization really need to keep? How many comments do I need in my code? How much of my code should check for well-formed arguments? Is it better to fail fast or fail last? What’s the effect on my code to put in all those checks? What’s the effect of leaving them out.

Not all dogma is handed down from on high or authoritatively laid down…nor is it necessarily bad to hold a common set of beliefs and opinions (the fourth definition of dogma). I’ve been in dysfunctional groups where we couldn’t agree on anything. It was extremely stressful and unproductive.

If as a group we establish and hold a common set of beliefs and practices, then we can just get on with our jobs without all that friction jockeying for who is right and the right way to do things.

But, here’s the rub…if you accept a certain amount of dogma (and I’m not saying what kind of dogma that might be…if you are an agile software developer I am sure you hold certain beliefs on testing, task estimation, collaboration, specification, keeping your code clean, whatever…) be wary of becoming complacent. Dogma needs to be challenged and re-examined from time to time. But don’t toss your current dogma aside on a whim, either. Old beliefs can get stale. But they may still be valid. We need to try out new ideas. But not simply discard older beliefs because shiny new ones are there to distract us.

Why Process Matters

I’ve been working on a talk for Smalltalks 2014 about Discovering Alexander’s Properties in Your Code and Life.

I don’t want it to be an esoteric review of Alexander’s properties.

That won’t satisfy my audience or me.

I want to impart information about how Alexander’s physical properties might translate to properties of our software code as well as illustrate poignant personal examples in the physical world.

But equally important, I want impress upon my audience that process is vital to making lively things (software and physical things). In his, The Process of Creating Life: Nature of Order, Book 2, Alexander states,

“Processes play a more fundamental role in determining the life or death of the building than does the ‘design’.”

Traditionally, building architects hand off their designs as a set of formal drawings for others others to build. Does this remind you of waterfall software development? There isn’t anything inherently wrong with constructing formal architectural drawings…but they never end up reflecting accurately what was built. Due to errors in design, situational decisions based on new discoveries made as things are built, better construction techniques, changing requirements, limitations in tools or materials, a building is never exactly constructed as an architect draws it up.

Builders know that. Good ones exercise their judgment as they make on the spot tactical re-design decisions. Architects who are deeply involved in the building process know that.

Alexander is rather unhappy with how buildings are typically created and suggests that any “living” process (whether it be for building design or software or any other complex process) incorporate the following ten characteristics.

He challenges us software makers to do better, too:

“The way forward in the next decades, towards programs with highly adapted human performance, will be through programs which are generated through unfolding, in some fashion comparable to what I have described for buildings.”

As software designers and implementers we know that nothing is ever built exactly as initially conceived. Not even close. Over the past decade or so we have made significant strides our processes and our tools that enable us to be more effective at adaptively and incrementally building software. My thoughts on some ways we have tackled these characteristics are interspersed in italics, below.

Characteristics of Living Processes

1.Step-by-step adaptive. Small increments with opportunity for feedback and correction.
Incremental delivery, retrospectives, stakeholder reviews
Repetitive incremental design cycles:
Design a little– implement–refactor rework refine–design…
Design/test cycles: Write specifications of behavior, write some code that correctly works according to the specification, test and adapt…
Tests and production code equally valued

2. Whatever the greater whole is always the main focus of attention and the driving force.
Working deployable software, minimally-marketable features

3. The entire process is governed and guided by the formation of living centers (that help each other)
Code with defined boundaries, separate responsibilities, and planned for interconnections

4. Steps take place in a specific sequence to control the unfolding.
We have a rhythm to our work. Whether it is test-first or test-frequent development, conversations with customers to define behavioral “specifications”, or other specific actions. In order to control unfolding we need to understand what we need to build, build it, then refine as we go. And we have tools that let us manage and incrementally build and record our changes.

5. Parts created must become locally unique.
Build the next thing so it fits with and expands the wholeness of what we are building. Consider our options. Refactor and rework our design. Make functions/classes/code cohesive. Bust up things that are too big into smaller elements. Revise.

6. The formation of generic centers is guided by patterns.
We have in mind a high-level software architecture that guides our design and implementation.

7. Congruent with feeling and governed by feeling.
Instead of just making a test pass, see if what you just wrote feels right (or if it feels like an ugly hack). Reflect on how and what we are building. Don’t be merely satisfied with making your code work. How do you feel about what you’ve just built? How do those using your software react to it? How do those who have to maintain and live with your code feel about it?

8. For buildings, the formation of structure is guided by the emergence of an aperiodic grid, which brings coherent geometric order
Software is structured, too…we’ve got to be aware of how we are structuring our code.

9.Oriented by a form language that provides concrete methods of implemented adapted structure through simple combinatory rules
We use accepted “schemas” to create coherent software systems. We have software architecture styles, framework support, and even pattern languages emerging…

10. Oriented by the simplicity transformation, and is pruned steadily
We can consistently refactor and rework our code with the goal of simplifying in order to enable building more functionality. We rebuild to create sustainable software structures. Even if we come back to some old working code and see how to simplify it, we can rework it taking into consideration what we’ve learned in the meantime.

Yet, let’s not be complacent. Agile or Lean or Clean Code or Scrum practices don’t address every process characteristic Alexander mentions. I am not sure that all these characteristics are important for building lively software. Alexander is not a builder of software systems, although he spent a lot of time talking with some pioneers and leaders of the software patterns movement.

Some process ideas of Alexander sound expensive and time consuming. Do we always need to reflect on how we feel about what we code? Sometimes we need to build quickly, not painstakingly. We need to prove its worth, and then refine our software. Our main thought may be on just simply making it work, not how it makes us or others feel. So how do we add liveliness to this quickly fashioned software? What’s a good process for that? Mike Feathers wrote about Working Effectively With Legacy Code, but there is a lot more to consider. Maybe that quickly fashioned software has tests, maybe it doesn’t, maybe some parts have a reasonable structure, and maybe other parts should be tossed.

We often build disposable and hopefully short-lived software. Problems crop up when that code gets rudely hacked to extend its capabilities and live past its expiration date.

There are most likely different processes for creating lively software, based on where you start, where you think you are headed, and how lively it needs to be (not everything needs to be fashioned with such care).

People are continually building new and better tools and libraries. There is a rich and growing ecosystem of innovative open source software. Process matters. I think we have a lot still to learn about building lively software. It is a heady time to be building complex software systems.