The Measurement Paradox

John Miller, SPCT

What is the Measurement Paradox? 

We measure what is not valuable.

We do not measure that which is valuable.

Seems kind of odd – doesn’t it.

Douglas Hubbard states this (and I paraphrase):

We measure what is simple to measure, but valueless in that it does not provide us with information having a high decision analysis value.  We choose to not measure what is valuable (it has a high decision analysis value), but is difficult to measure.  (How to Measure Anything: Finding the Value of Intangibles in Business, Douglas Hubbard)

In addition, when we have no information, we believe we need much information to make a better-informed decision.

Turns out the opposite is true. When we know little, we only need a little information to be better informed. When we know much (or we think we do) we need much more information to make a more informed decision. Read what he says about Emily and Fermi.

These are all part of the Measurement Paradox.

In addition, to me, his discussions on measurement also help me describe how this is a key factor on why 1) teams can get really good at estimating stories in 3-5 sprints (iterations); and 2) why Product Managers (along with the Program Team) can get really good at estimating Features (SAFe® usage of the word) in 3-5 PIs. Noticed I said can. They have to choose to do the estimation technique! They have to care. Or, as Yoda states, "or not".

Hubbard also states, when the estimator chooses not to care, their estimation never gets better!

Does this sound like any teams you know regarding story estimates?

When there is a team whose GACM is permanently set to zero or near zero they will claim they can not estimate.

In Hubbard's workshops he helps people learn how to estimate in the 100's of millions of dollars.

And the teams you are coaching are pushing back with

"We can't estimate this" 

"We don't know"

OR WHATEVER!

(btw - do the above with a very whiny voice and you'll hear your teams and their reasons).

In his book “How to Measure Anything” Hubbard describes how we get much better at estimating (his word is calibrated) after just 5 exercises (uses) of the same estimation system.

Kind of interesting isn’t it.

Yet, in my experience, talking about estimation of stories always gets someone riled up. Kind of like a religious war.

The Leadership you are coaching will want to understand this. Talk to them, help them understand.

Some managers just want hundreds of measures so they feel good having meetings and managing. Let them self-select out.

Want to learn more? Read Hubbard’s book “How to Measure Anything”.

Need to talk about it? Let me know, drop me a note.  John@LeaningAgile.com

Mentoring for a beer.

Kind of like me listening like a bartender does while you commiserate about things!

jm

The 3 Reasons for Value Stream Mapping

The Problem

Value Stream Mapping is a great technique to understand how an organization delivers value and allows the organization to improve this flow of value by optimizing individual steps while still maintaining a systems view.  The technique is a cornerstone of Lean flow-based improvement, however, only following the two commonly recommended steps (Identify, Optimize) falls short of the true potential when it skips the 2nd reason: Organization.  This is where the Scaled Agile Framework (SAFe®) comes in to play, as one of the core constructs of SAFe® is to virtually organize around value delivery. 

(Note: this article documents an approach that is somewhat tangential to the SAFe® prescribed approach: current SAFe® material is based on Karen Martin based thinking; map the operational value streams, find the systems, and map the steps to support those systems.  This guidance works well for understanding business process and gaining visibility, but when used for SAFe® virtual organization purposes it can lead to organizing around current functional silos.  This is due to the fact that most major enterprises have organized around their internal/external systems and subdivided around the various technologies to support that system, leading to the wasteful hand-offs, delays and conflicting priorities.  This article spells out a different pattern for using this powerful technique along with SAFe®)

The Proposed Solution

Value Stream Mapping used in conjunction with SAFe should be organized into three specific usages: Identification, Organization and Optimization, with Organization being the added step in addition to traditional Lean based approaches.

Identification

Identification is the obvious starting point, but there are ‘better practices’ around this first step.  Karen Martin defines a Value Stream as “All of the activities, required to fulfill a customer request from order to delivery (and beyond to cash received)” (2010 Karen Martin & Associates).   Martin extols the virtues of starting at a high level by identifying the actual value stream (“Rooftop View”), and then taking a layered approach to dive down into the details. 

The level of mapping we need to begin to effect change is usually somewhere between the “Rooftop” view and the In The Weeds view.  While we want to eventually understand the deeper aspect of the Tactical portion, we can start to effect change before having this full detailed view.

Dr. Allen Ward discusses the importance of focusing on Operational Value Streams.  “In Lean Thinking, Womack and Jones say that lean companies figure out what value is— what customers actually want— and concentrate on “value streams,” the connected activities that create value.  Conventional companies often get so involved in their internal organization that they lose sight of value and produce waste instead.”  “The operational value stream includes activities converting raw material into products in the hands of customers. It produces high-quality products at the time the customer wants them. Activities are value-creating when they change materials toward the products customers pay for. The development value stream includes activities running from recognizing an opportunity through manufacturing launch.” (Ward, Allen; Sobek, Durward. Lean Product and Process Development, 2nd ed. (highlights added).  This thinking is based on the Toyota concept of value being the “horizontal slice across vertical functions in an organization”. 

To illustrate this concept, think of how we map value streams in a DevOps transformation.  Value Stream Mapping is a critical first step in DevOps improvements, and yet if we were to focus on the tools or ‘systems’ at each step we would not see the true effect of bottlenecks, overloaded people, sign off delays and the like.

Identifying Operational and Development Value Streams

As a veteran of many value stream identification sessions I know all too well how this relatively simple step can really confuse and impede most organizations.  The truth is that the simpler you start this, the better.  My guidance is always to start with a stack of stickies (Post-It’s), pens, and a group of people that have a relatively good understanding of the organization and the value it delivers to its customers.  Some key points are:

·        Don’t Focus on Products: too many organizations are keyed in on the organizations products, or even worse, on their internal applications and systems.  This makes it really tough to find the true value streams.  Instead, focus on how your customer perceives the value you deliver, e.g. what problems do you solve or solutions do you provide to them

·        Identify What You Have: understand that you already have value streams, and you need to avoid the temptation to map out a future state that may be unreachable, or not even the right target.  Identify what you have, even if it’s not a pretty picture.

·        Start with The Fence Posts: What is the value you deliver?  Think of this from the customer standpoint.  A Customer Journey map can sometimes really help in this effort, as it focuses on value as perceived by your customer, and not how you envision it from the inside out.  Once you have this clarity of true value, determine the trigger points to deliver more value.  Is it a new product idea?  A new feature for your mobile game app?  These two points, the Inception and the Delivery, are your fence points to build around.

·        Avoid Writers Block: If you get stuck in an area, move on.  Very often further discovery in other areas will allow you to come back to these initial blockers.  Most Identification workshops start with creating a Value 'Pool', a brainstorm of non-sequenced stickies.  Once we get a number of these on the wall we can start to sequence.

·        Gemba Walks: you will rarely (if ever) have a small group of people in one room that understand the full flow of value in your organization.  In these cases, encourage the group to map what they know and then take a walk to observe and learn how the value really flows (“Gemba” literally means “the Place”).

Once you have sufficient clarity on your Operational value streams (in this case, less is more) you will need to identify the Development value streams, or those interconnected steps you go through to deliver improvements or optimizations to each Operational step.    This is a critical step as these Development stream steps are typically what you will organize around.   Skipping this step will lead to organizing around the processes or systems, which in most large organizations would result in the current functional silos that are responsible for the heavy waste in hand-off’s, lack of visibility, etc. 

Organization

To fully support the SAFe® core values of Alignment and Transparency, we need to be organized around true value delivery steps.  The focus on mapping out each activity or ‘process’ step in a value stream will allow us to then determine the people needed to gather together into an ART to reduce/eliminate the hand-off’s and to create true value delivery-based organizations.  Applying an overlay of the systems impact to this new organization will allow us to see the trade-off’s we are making in architectural integrity, consistency, and perhaps other issues such as system security. 

A former client of mine (Phil Purrrington, now a very successful Agile Coach) described this virtual organization concept as populating a deserted island.  To be successful, we need to bring all the people needed to create a society on to this island.  This would include construction, medical, government, food gathering and preparation, etc.  The same is true with our SAFe® virtual organizations.  In many cases we need to have people from process, legal, compliance, research, etc on the ART to deliver value end to end.  Forming our organization solely around systems will lead to a number of critical roles, skills and people being left out of the ART.  I have experienced this at many customers in the finance, health care, retail and manufacturing verticals, and it invariably leads to greatly reduced ROI from a Lean Agile transformation.

Optimization

Optimization is a critical step in the lean improvement process, but once we have our virtual organization aligned around these value delivery steps we have already eliminated a great deal of the waste in the system just through the alignment and single-purpose/single-piece-flow focus of an ART.  Continuing to re-map the value stream as improvements are made, and optimizing around the largest bottleneck, will be far more effective when the ART is truly organized around value delivery steps.

The Action

Incorporate Value Streams using the first two elements (Identification and Organization) to find virtual organizations to launch trains around.  By this early action of re-organizing around value delivery, many of the current value delivery impediments will be removed.  You can then focus on the remaining Optimization step to further lean out your delivery process.