Papers and Essays

The complete collection of patterns about transitioning to agile quality described in pattern gist form. A pattern gist is a brief version of the fuller pattern description. The gist motivates the pattern, poses a question about the problem the pattern addresses, and then sketches the pattern solution.

We apply the biological-behavioral concept of an umwelt, which is how an organism perceives and acts within its environment, to the practice of software development. By writing narrative descriptions of our own software umwelts and iteratively discussing and analyzing them, we develop prompts that can elicit reflection on how and why we relate to software in the ways that we do.

This is an about paper—one that we write to gain a better understanding, for ourselves and our readers alike, of the usefulness of a pattern language and what it takes for experience captured in pattern form to be of use to others. In it we look at what it means for individual patterns and collections to be generative and explore some differences between those that are highly generative and those that are not. We ask whether software design and software process patterns can be generative and, if so, what is their potential impact. Finally, we draw some conclusions on how these thoughts on pattern generativity could influence future writing of and support for generative patterns.

Pattern authors ostensibly are experts on the topic their patterns address. Since most software developers don’t share those experts’ underlying design values, practices, and principles—let alone their design context—there’s a disconnect between what is said in pattern descriptions and that which is perceived and understood by pattern readers. As pattern readers, we will be better equipped to grasp the significance of these software patterns if we also gain insights into the design principles and values that they are based on and some reasons why they “work” the way they do. But it is through experience, practice, and reflection, that we deepen the connections between new-to-us patterns and our existing knowledge, and expand our design umwelt.

This essay reflects on my experiences of the past 15 years writing software design patterns and posits that if the patterns community wants to broadly increase pattern literacy, relevancy, and long-term impact, some things need to change. Those changes need to start with pattern writers—like me. Instead of indiscriminately writing ever more patterns, we should focus more on connecting, relating, and refreshing the abundance of existing impactful patterns. Changes also need to be made in how we the community of long-time pattern authors and advocates present and promote patterns to the rest of the world.

As patterns authors, we spend a lot of time writing. Usually our work is driven by what we want to share of experiences, working solutions, and ideas we are excited about. But how often do we think about the readers other than initially stating our intended audience? Who is going to consume our writing, and are we really putting in the effort to reach these readers in the best way possible?

This paper started because we are two authors who plan to make a book from a set of related patterns papers, and in doing so we really want to reach our target audience and provide a practical and useful piece of work. Our approach had two distinct investigation activities. One was to do an informal analysis of books about software processes and methods to gain a better understanding of characteristics and book design practices for this kind of book. The second was to use a combination of interviews and surveys to gather feedback on what readers are looking for when accessing information about software engineering. To better understand our potential readers and how to design our writing for their consumption, we created personas where we incorporated feedback from the reader survey into their personalities and preferences. Finally, we combined our information sources to create guidance for our future work of turning a set of patterns papers into a cohesive whole.

Capturing the wholeness of design solutions in order to effectively communicate them to others can be challenging. We posit that patterns are observable phenomena of design solutions. To represent these phenomena, a pattern author needs to generalize and omit information. Experienced designers are able to unfold the essence of the pattern and generate design solutions based on the information that they do find in a pattern description. Not surprisingly, their personal design heuristics play a central role in this. As they create a design solution, they also liberally apply their pre-existing design know-how and heuristics. But novice designers may have more difficulty. As this folding and unfolding of information and knowledge seems to be quite an abstract concept, we have chosen to make our point by discussing elephants. Like patterns, elephants are an observable phenomenon, a pattern in nature. Many different descriptions, representations, and accounts of elephants exist. Many people claim to know what elephants are. Yet they actually have little or limited knowledge of them. This analogy helps to understand how at the same time we both know and do not know what a thing is.

Architects always make decisions in some context. That context shifts and changes dynamically. Different decision-making strategies are appropriate in different contexts. Architecture decisions are at times made under conditions of time pressure, high stakes, uncertainty, and with too little information. At other times, decision-makers have sufficient time to reflect on the decision and consider alternatives. Understanding context is critical to choosing appropriate approaches to architecture decision making. Naturalistic Decision Making (NDM) explains how people make decisions under real-world conditions. This paper investigates NDM in software architecture and studies architecture decisions in their environment and decision-making context. The research approach includes a case study of large technology organizations consisting of a survey, multiple focus groups, and participant observation. Previous studies that touch on NDM in software architecture have mainly focused on decision-making processes or tools or developing decision models. This paper provides three contributions. First, we build on previous studies by other researchers to produce an in-depth exploration of NDM in the context of software architecture. We focus on Recognition-Primed Decision (RPD) making as an implementation of NDM. Second, we present an examination of the decisions made by experienced architects under conditions that can be considered naturalistic. Third, we provide examples and recommendations that help software architects determine when an NDM approach is appropriate for their context.

Pattern authors intentionally create waypoints—points of interests along a design trail they hope others can traverse. While designers may read others’ design advice—be it patterns or stack overflow replies, the heuristics they’ve personally discovered are equally important. Patterns are just a small part of a much larger body of our design know how. Heuristics, like patterns, can be expressed at various levels. Some are small, simple acts. Others are bigger steps, taken at the beginning of a design journey. This essay explores ways we can grow as designers by becoming more aware of our heuristics, acknowledging the inherent uncertainty in the design process, and learning better ways to articulate and share our heuristics with each other.

Agile development processes are driven from a product backlog of development items. This paper adds three more patterns to our Magic Backlog collection of patterns to build and structure the backlog for an agile development effort using the outcomes of requirements elicitation and analysis. RULES defines who should be allowed to do what in managing the backlog, SHARED DEFINITIONS deals with defining and consistently using clear criteria for core decisions across the backlog team(s), and REMODEL is about restructuring the overall backlog contents.

Many organizations struggle with efficient architecture decision-making approaches. Often, the decision-making approaches are not articulated or understood. This problem is particularly evident in large, globally distributed organizations with multiple large products and systems. The significant architecture decisions of a system are a critical organization knowledge asset, as well as a determinant of success. However, the environment in which decisions get made, recorded, and followed-up on often confounds rather than helps articulation and execution of architecture decisions. This paper looks at aspects of architecture decision-making, drawing from an industry-based case study. The data represents findings from a qualitative case study involving a survey and three focus groups across multiple organizations in a global technology company. Architects in this organization are responsible for multiple products and systems, where individual products can include up to 50+ teams. The impact is not just on others in the system; architecture decisions also impact other decisions and other architects. The findings suggest recommendations for organizations to improve how they make and manage architecture decisions. In particular, this paper notes the relevance of group decision-making, decision scope, and social factors such as trust in effective architecture decision-making.

This paper extends our Magic Backlog Patterns collection with three additional patterns for managing the work of a program– or rather how to deal with coordinating the work of projects which are part of a larger program and where there may be dependencies and shared deployment. While teams within a program may work fairly independently, their work still needs to be coordinated to produce a product. These three patterns, which represent alternative strategies for structuring the backlog of work, are introduced through a story of the correspondence of a business analyst as her hypothetical program moves through different backlog management strategies.

Billy Vaughn Koen, in Discussion of the Method: Conducting the Engineer’s Approach to Problem Solving, defines a heuristic as anything that provides a plausible direction in the solution of a problem, but in the final analysis is unjustified, incapable of justification, and potentially fallible. Software patterns might be considered nicely packaged heuristics in that they provide a context for the problem, and offer plausible solutions along with forces that the designer needs to consider when implementing a solution. Like any heuristic, software patterns come with no guarantees that they will solve the current problem at hand. A dedicated group of authors in the patterns community continues to write patterns, collections of patterns, and more ambitiously weave patterns into pattern languages that attempt to cover paths to solutions in a particular problem space. Are we deluding ourselves about the utility of these efforts? Or is there something important about both the form and use of patterns in the larger context of design heuristics that we need to understand?

Agile development processes are driven from a backlog of development items. This paper looks at how to apply patterns from our Magic Backlog collection through the use of storytelling. We follow a project through various scenarios from the early days through significant growth into a mature setting and then, finally to a disruptive transition as the project becomes part of a program. Our goal is to focus on common project issues that we believe are experienced by many people involved in software development, and how the use of combinations of patterns, including our own as well as other patterns and practices, can help resolve or mitigate these problems.

To achieve quality systems and products, it is vital to enable and infuse quality work throughout the entire process, rather than piling it on at the end. Thus paying attention to when to this, how to do this, and who is involved can increase quality. This paper presents three patterns from the collection of patterns on being agile at quality: System Quality Specialist, Spread the Quality Workload, and Automate As You Go. System Quality Specialists can define, test, and implement system-quality characteristics that are complex or require specialized skills and expertise to get right. Spreading the Quality Workload throughout the development process keeps the team from being overly burdened with quality-related work at any point in time. If you Automate As You Go, this enables teams to streamline their build and testing processes, eliminate tedious or mundane tasks, and allow more time for team members to focus on implementing and testing important system qualities.

Agile development processes are driven from a backlog of development items. This paper adds three new patterns to our collection of patterns to build and structure the backlog for an agile development effort using the outcomes of requirements elicitation and analysis. The Funnel adds new content to the backlog, the Pipeline prepares backlog contents for implementation, and Maintenance performs continued care of backlog contents. The need to formalize the backlog increases with the size of the project.

As organizations transition to agile processes, Quality Assurance (QA) activities and roles evolve. The whole team focuses on quality as they incrementally deliver working software. Incremental delivery provides an opportunity to engage in QA activities much earlier, ensuring that both functionality and system qualities are focused on just in time, rather than too late. Knowing what and when specific qualities need to be paid attention to and checking that they have been appropriately addressed is important. The patterns in this paper extend our previous work with three patterns aimed at increasing quality awareness and expertise: “Quality Checklists,” the “Product Quality Champion,” and “Shadow the Quality Expert.”

As organizations transition to agile processes, Quality Assurance (QA) activities and attention to system quality need to evolve along with the evolution of development practices. Agile quality teams incrementally deliver working software while ensuring that important system qualities are also addressed. In order to pay appropriate attention to system qualities, they need to be visible and included as part of the prioritized work. This paper presents patterns for identifying system qualities and including them on the project roadmap, adding quality-related work items to the project backlog, and creating a quality radiator that communicates the status and goals for delivering system qualities.

Agile development processes are driven from a backlog of development items. This paper describes patterns to build and structure the backlog for an agile development effort using the outcomes of requirements elicitation and analysis. The need to formalize the backlog increases with the size of the project.

As organizations transition to agile processes, Quality Assurance (QA) activities and roles need to evolve. Traditionally, QA activities occur late in the process, after the software is fully functioning. As a consequence, QA departments have been “quality gatekeepers” rather than actively engaged in the ongoing development and delivery of quality software. Agile teams incrementally deliver working software. Incremental delivery provides an opportunity to engage in QA activities much earlier, ensuring that both functionality and important system qualities are addressed just in time, rather than too late. Agile teams embrace a “whole team” approach. Even though special skills may be required to perform certain development and Quality Assurance tasks, everyone on the team is focused on the delivery of quality software. This paper is part two of a series of patterns about Agile QA practices and activities. The patterns in this paper are focused primarily on measuring and monitoring system qualities.

As organizations transition to agile processes, Quality Assurance (QA) activities and roles need to evolve. Traditionally, QA activities occur late in the process, after the software is fully functioning. As a consequence, QA departments have been “quality gatekeepers” rather than actively engaged in the ongoing development and delivery of quality software. Agile teams incrementally deliver working software. Incremental delivery provides an opportunity to engage in QA activities much earlier, ensuring that both functionality and important system qualities are addressed just in time, rather than too late. Agile teams embrace a “whole team” approach. Even though special skills may be required to perform certain development and Quality Assurance tasks, everyone on the team is focused on the delivery of quality software. The patterns in this paper are focused on “breaking down the walls” or removing barriers between people and traditional roles, as this is key for any change within an organization that is transitioning to being more Agile at Quality

As organizations transition to agile processes, Quality Assurance (QA) activities and roles need to evolve. Traditionally, QA activities have occurred late in the process, after the software is fully functioning. As a consequence, QA departments have been “quality gatekeepers” rather than actively engaged in the ongoing development and delivery of quality software. Agile teams incrementally deliver working software. Incremental delivery provides an opportunity to engage in QA activities much earlier, ensuring that both functionality and important system qualities are addressed just in time, rather than too late. Agile teams embrace a “whole team” approach. Even though special skills may be required to perform certain development and Quality Assurance tasks, everyone on the team is focused on the delivery of quality software. This paper outlines 24 patterns for transitioning from a traditional QA practice to a more agile process. Six of the patterns are completely presented that focus on where quality is addressed earlier in the process and QA plays a more integral role.

This column celebrates the 20th year of software patterns. IEEE Software advisory board members teamed up with members of the Hillside Group, a nonprofit organization that promotes the use of patterns and pattern languages, to reflect on the state of the practice and impact of patterns.

A problem frame is a generic, abstract description of the structure of a problem. Using a problem frame requires selecting a candidate frame from a catalog and mapping the specific parts of the problem into the principal parts of the frame. In this paper we present five patterns, one for each of Jackson's original problem frames. We hope that by writing about problem frames as patterns, we can make the frames themselves and their structure more explicit, and so expose problem frames to a wider audience.