Scaling Product Development
Doubling product development output is within the realm of possibility if you build your development machine at the portfolio level to better handle complexity and change. Successful product development companies have a product that establishes their presence. From there, they expand their business, but eventually, productivity plateaus, making it difficult to generate more output or revenue. Product development flows can involve hundreds of people over several years and are usually highly complex. These workflows constantly evolve as the world continues to change and development teams try to keep up. The main barriers to scaling are complexity and change, as well as the need to align plans with operational capabilities. Scaling is all about developing effective development operations. Let’s examine each of these fundamentals: complexity, change, and plans, then discuss how to build what is needed for scale.
The first obstacle to scaling is complexity. We all know that, over time, complexity tends to increase. Our phones used to hang on the walls in our homes with a 20-foot-long cord so you could move around while talking. Today, we have computers in our pockets that can also make phone calls. Businesses like predictability, and sometimes that gets them in trouble because they fail to adapt. One can no longer count on next time being just like last time in product development. The more complex things become, the less predictable doing work becomes. Managing complexity is essential for growth.
Product development is a complex system composed of many interacting parts that together produce behavior that is difficult to predict from the parts alone. Markets and customers are also characterized as complex, so needs and constraints often emerge during product development. To better manage work in these complex systems, more probes are needed within the work system to understand what is happening. (More on complexity theory related to probes here: https://na2.hubs.ly/H04yyWY0 )
Probes can be conversations, meetings, experiments, software, monitors, or metrics that help identify problems. When designing product development workflows, we include probes to facilitate learning. Identifying problems or amplification1 and learning from them is crucial for scaling; workarounds only serve the current program and ultimately harm portfolio performance. Plan for problems to emerge because they will. When probes detect a problem, have a clear process to address it, including escalation paths to leaders who can assist if needed. Keep some team members out of the development flow for flexible capacity to assist when issues arise. Having team members booked at 100% capacity only works if the work is simple and well understood, which is not much of the work that is done today.
We need to recognize that complex systems can’t be fully controlled, no matter how skilled or experienced the workforce is, but we can learn to operate within them by using probes. If we were driving across the country in a car, we wouldn’t rely only on the gas gauge and speedometer probes to reach our destination efficiently. We also need to check the temperature, engine light, tire pressure, and more. Complex product development isn’t predictable, and organizations must learn before, during, and after each program to grow. Knowledge enables scaling, and probes provide opportunities for learning.
Change is the next biggest constraint on scaling development flows. Change can disrupt plans because they assume perfect knowledge, which isn’t realistic. Usually, a buffer is added for issues that come up or emerge during development. Having no buffer in plans today is a bad idea, and it will hurt scalability. Estimating how much buffer to include in plans can be difficult. Leaders often dislike buffers because they delay revenue and increase costs, and it’s easier to manage when things are predictable. From my experience with buffers, I’ve never seen one that was too large. Ultimately, we won’t know how much buffer is needed until the program is finished; that’s the nature of product development. Take your best guess, stop overplanning, and start executing.
The more features added to product plans, the longer development takes, which then creates more opportunities for changes to disrupt plans. Smaller batches or shorter development cycles improve adaptability and help manage change. The gold standard today in terms of release or development cadence comes from the DevOps world. (more on DevOps here: https://na2.hubs.ly/H04DDYp0 )
Their practices have evolved to enable hundreds of thousands of software releases per day, compared to one or two large batch releases a year, which was their historical norm. Reducing the batch size of releases and changing how releases are managed were key to developing and deploying features at this rate. Agile teams also use a practice called Scrum, which typically uses a small two-week batch of work called a sprint, to help manage their work. (more on Agile here: https://na2.hubs.ly/H04DDYj0 )
Small batches of work released incrementally help teams adapt as products and our world evolve. Scrum helps manage change by developing a process that can easily handle emergent work. In both cases, DevOps and Agile evolved as development teams sought to address their struggles in dealing with change and working within complex systems.
The last fundamental factor for scale to consider how plans are created and used. The desire to grow or scale is always a priority for product development companies but caution is advised. If plans are too aggressive, it can turn the organization into a rush hour like environment. With too many programs in development, nothing moves as fast as anyone wants. Plans beyond current work capacity make scaling virtually impossible.
To effectively deliver products that customers value, it is important to accept that all plans are flawed somewhere, but this won’t be obvious until it happens. The primary focus for leaders is to identify where plans are wrong and adjust them, rather than attempting to make the plan right, regardless of reality. During development, insisting on all planned product capabilities going into a product and meeting schedules at any cost in today’s complex environment will limit scalability. This can lead to overburden, resulting in exponential delays if work accumulates too much within a workflow. (more on queues and behavior here: https://na2.hubs.ly/H04DGsp0) View plans as hypotheses to test, not as certainties that are completely predictable.
There were times in my career when our product roadmaps seemed to require more than twice our ability to do work. They may have been able to generate the revenue growth the company desired, but it didn’t seem doable or match our experience. Plan for what you can do, not for what you think you can do or want to do and start digging into the work. The reality is that the real scope of a program isn’t known until it's complete. Leadership needs to build the operations first that can deliver products, rather than setting aggressive or stretch goals for the team to figure out on their own. Planning too early can also set teams up for failure because product requirements from markets and customers will continue to change over time. This can lead to scope creep, cause delays, and ultimately slow down the development of the entire portfolio. Synchronize product requirements with the development work to improve scale. Marketing, architecture, and design all need to be on the same page about the value of customer requirements in order of priority. Another scale limiter is that the most experienced development resources help put together program and portfolio plans, so the more time your team spends planning for the future, the less time these key resources have to work on programs or help their teams do so. Scoping work is becoming increasingly difficult with complexity and change. You can even argue that long-term plans someday will be irrelevant. We tend to be optimistic about what we can deliver, and the development teams want to deliver as much as they can. Be careful not to fall into wishful thinking. Don't be afraid to change plans. Making changes based on new learning is what it takes to be adaptable and sustain a business today.
These foundational concepts set the stage for the design of the system of work that delivers a valued portfolio at scale. Tackling complexity, adapting to change, and developing plans without overtaxing development teams will go a long way to putting your competition in the rear-view mirror.
Scaling Development Operations
To scale a product development business, you must scale the operations. Operations are the roles, responsibilities, and sequence of activities that development teams perform as they create and deliver products to customers. Scaling plans without scaling operations will make things worse, not better. The first and most important step is to understand the current development flow. This will provide much of the data needed to build the portfolio machine that will help your company thrive and survive. The best way to do this is to develop a workflow map for the entire development flow from concept to delivery. Once you have a high-level map of the development flow, stamp it out to show how many programs are in development simultaneously, including legacy products and any other support activities that consume development resources. The connections among programs, teams, and activities are critical to effective scaling. Lastly, identify other connections that may affect the availability of development resources, since people usually don’t spend 100% of their time on projects. Now that all of this has been discovered, connected, and visualized, ask some important questions: Does the current state look scalable? What is the probability that execution will be seamless based on the system design? How did it get like this?
The best way to scale a business is to first deeply understand how all work is done today
There are no shortcuts or silver bullets that will help a company scale. I can’t stress enough how important a shared understanding of the current state among leaders is for scaling output. Leaders, like all of us, see the problems from their own perspective, which isn’t a complete picture. Nobody can know all that information in detail, it's just too much. Developing a large workflow map will provide the full picture, and when leaders look at what all teams do, I guarantee their paradigm on the current state will change.
Initial Design Considerations
First, teams need to understand the customer's point of view from marketing and customer connected teams. The development system should be synchronized with customers. Understanding what, when, and how often customers can integrate new capabilities into their business is critical to be effective. Having an 18-month development cycle while customers release new products every year is problematic. While perfection is unlikely, doing your best to meet their needs is essential, or else return on investment will suffer.
For scale, you should develop a cadenced based synchronized product roadmap to reduce complexity and manage change for your teams and customers. This might involve adjusting what is fixed versus what is variable. The iron triangle in product development represents the three main constraints: scope, time, and cost. In this context, we should assume that "cost" refers to the development resources for a product. Attempting to fix all three constraints simultaneously can lead to problems in today’s environment, as we’ve previously noted; problem-solving is not predictable. The most effective approach for scaling is to fix time and cost, then vary scope. Begin by setting a fixed timeline for a program. Fixing the duration provides clear boundaries for operations, helps synchronize costs and resources across multiple programs, and improves customer engagement. It offers structure for developing product requirements and clarity for decision-making timelines. All teams will then understand exactly when their resources are needed and for how long. While varying scope is challenging, prioritization and limiting scope for development teams are essential for scaling. It’s also beneficial to involve customers closely in this process to gain real-time feedback as the product evolves.
Furthermore, once you establish a consistent cadence for work, it becomes easier to do more with the same resources as they learn within operational boundaries. These constraints are key for learning and scale. For example, if one program plans a month for an activity, another plans three months for the same thing, and yet a third plans for a two month duration, then it becomes extremely hard to scale that process because the process needs to be tailored to the varying scope and timing for each program. When teams can operate every time with the same duration, then teams will find a way to do more with less each time they do it. We are always much better at doing something on the tenth try than we were on our first attempt. This applies to all activities that teams perform for each program. It will quiet the noise of the system, hand-offs between teams become better, synchronizing data flow across programs improves, product development starts to look like a world class orchestra and less like an ant hill you accidentally stepped on. The purpose of this structure is to drive learning, not an effort to control the uncontrollable.
Begin with the end in mind
In the end, a cadenced-based synchronized development flow for a portfolio of products looks something like the picture below. In this example, the development flow is a year long, and key program phases are a quarter in duration. Then they are staggered trying to level out resourcing across programs and teams. Be careful not to couple resources across parallel workflows. The idea that you can matrix people across multiple programs at a time may seem efficient, but it makes scaling difficult and causes more problems than you think. From a cost perspective, in some cases, there may be no choice but minimize coupling of resources across programs; instead, stagger the programs around the fixed resources. Holding the time boundaries in a cadenced based portfolio delivery system is challenging but the key for delivering more with less. Varying the input should be the priority, not moving a cadence deadline to add this one thing. Learning that boundaries are hard lines will lead to the best value delivery system.
Other Considerations
Lead vehicles, or the first product for some new technology, may need their own development flow and timeline, but they need to integrate with the core portfolio cadenced structure once the initial release is complete, if you want to scale effectively. If the new product with the new technology is prioritized as the top priority and the other products draft off that development flow, you end up with an asynchronous development flow that exponentially increases development complexity and limits scalability. In this case, all constraints start to vary, nothing is fixed, and scaling becomes much harder, if not impossible. As complexity increases and change continues, developing products using an asynchronous development flow will only make it more difficult to meet project schedules and scale, while exhausting the entire organization.
Wiring the Winning Organization by Gene Kim and Steven Spear offers numerous perspectives that support scaling.1 The work emphasizes slowification, simplification, and amplification to help teams move away from the danger zone toward the winning zone. Slowification makes problem solving easier by removing it from urgent, chaotic development environments, allowing teams to think, experiment, and learn. This learning is then incorporated back into the delivery system. Simplification techniques include modularization, incrementalization, and linearization. Although modular design has been popular for many years, we should apply the same principles to workflow design. The work system is divided into coherent modules that work together, which not only simplifies the system but also helps contain issues when they occur. The opposite of a modular system is a monolithic work system, in which a change in one area can affect many or all parts of the system. Incrementalization was discussed, referred to as small batch releases, ideally by making one change at a time or delivering one increment of work. Linearization streamlines work by having teams focus on their current tasks and upcoming steps. Development flows across teams in a linear fashion through the workflow. Teams need clear inputs, outputs, and workflows that set them up for success. Amplification, discussed earlier, involves probes that amplify problems. Which technique is applied depends on the problem at hand.
Making the Transition
The cadenced based portfolio structure is the desired future state but it will take many iterations to get there. There is no way to jump directly to a future state. The only way it might work is to shut down the entire system and start up with newly designed processes, but I would not recommend this strategy. The risk is too high, by the time you get new products out of the new development flow there may be no business left to run. The way to think about the transformation is, what do you have to learn to get to that future state? Teams need to keep asking the same question, “Why can’t we get there today?” This exercise will help teams figure out what to do to get closer to the desired state with each iteration.
After designing the desired future state cadence-based flow, this will introduce a new set of challenges. Solving a different set of problems is the point, not an obstacle to making the change. Many teams will say that the new cadence doesn’t work for them, and it probably won’t if they keep working the same way. Cadence and simplification are the priorities, with the focus now on building a scalable foundation by reducing the impact of complexity and change. Many work processes will need to be adjusted to align with that cadence. The initial cadence may also evolve as teams identify constraints. Now, optimization happens at the portfolio level, rather than at the program or team level, which is crucial for scaling.
Ironically, this opportunity to scale product development stems from the complexity and change that have made scaling so difficult. Development systems that rely on better plans, hiring the best people, providing incentives, aggressive goals, inspirational visions, or implementing performance management will struggle. Building the development operations that people can leverage to learn, and having leaders help drive the design of the workflow processes, is the ticket to success. Complicated cross-functional workflows serving complex markets are typically laden with defects. Removing those defects, which can infect as much as half of the work activities development teams perform, is half the battle. The other half is setting up a structured way to learn how to deliver products at scale without burning everyone out. Teams must learn how to do it; there are no shortcuts. If they knew how to do it today, they would just do it. Start building the workflows you need to scale today; it's never too late.
Do something today to improve your work-life balance. You won’t regret it. Have a great day, and good luck with your work-life journey.