Author: Mark

Posted on

Toyota Kata: Reflection on Coaching Struggling Learners

The “Five Questions” are a very effective way to structure a coaching / learning conversation when all parties are more or less comfortable with the process.

The 5 Questions of the Coaching Kata

Some learners, however, seriously struggle with both the thinking pattern and the process of improvement itself. They can get so focused on answering the 5 questions “correctly” that they lose sight of the objective – to learn.

A coach, in turn, can exacerbate this by focusing too much on the kata and too little on the question: “Is the learner learning?”

I have been on a fairly steep learning curve* in my own journey to discover how modify my style in a way that is effective. I would like to share some of my experience with you.

I think there are a few different factors that could be in play for a learner that is struggling. For sure, they can overlap, but still it has helped me recently to become more mindful and step back and understand what factors I am dealing with vs. just boring in.

None of this has anything to do with the learner as a person. Everyone brings the developed the habits and responses they have developed throughout their life which were necessary for them to survive in their work environment and their lives up to this point.

Sometimes the improvement kata runs totally against the grain of some of these previous experiences. In these cases, the learner is going to struggle because, bluntly, her or his brain is sounding very LOUD warning signals of danger from a very low level. It just feels wrong, and they probably can’t articulate.

Sometimes the idea of a testable outcome runs against a “I can’t reveal what I don’t know” mindset. In the US at least, we start teaching that mindset in elementary school.

What is the Point of Coaching?

“Start with why” is advice for me, you, the coach.

“What is the purpose of this conversation?” Losing track of the purpose is the first step into the abyss of a failed coaching cycle.

Coach falling over a cliff.

Overall Direction

The learner is here to learn two things:

  • The mindset of improvement and systematic problem solving.
  • Gain a detailed, thorough understanding of the dynamics of the process being addressed.

I want to dive into this a bit, because “ensure the learner precisely follows the Improvement Kata” is not the purpose.

Let me say that again: The learner is not here to “learn the Improvement Kata.”

The learner is here to learn the mindset and thinking pattern that drives solid problem solving, and by applying that mindset, develop deep learning about the process being addressed.

There are some side-benefits as the learner develops good systems thinking.

Learning and following the Improvement Kata is ONE structured approach for learning this mindset.

The Coaching Kata, especially the “Five Questions” is ONE approach for teaching this mindset.

The Current Condition

Obviously there isn’t a single current condition that applies to all learners. But maybe that insight only follows being clear about the objective.

What we can’t do is assume:

  • Any given learner will pick this up at the same pace.
  • Any given learner will be comfortable with digging into their process.
  • Any given learner will be comfortable sharing what they have discovered, especially if it is “less than ideal.”

In addition:

  • Many learners are totally unused to writing down precisely what they are thinking. They may, indeed, have a lot of problems doing this.
  • Many learners are not used to describing things in detail.
  • Many learners are not used to thinking in terms of logical cause-effect.
  • The idea of actually predicting the result in a tangible / measurable way can be very scary, especially if there is a history of being “made wrong” for being wrong.

Key Point: It doesn’t matter whether you (or me), the coach, has the most noble of intentions. If the learner is uncomfortable with the idea of “being wrong” this is going to be a lot harder.

Summary: The Improvement Kata is a proven, effective mechanism for helping a learner gain these understandings, but it isn’t the only way.

The Coaching Kata is a proven, effective mechanism for helping a coach learn the skills to guide a learner through learning these things.

For the Improvement Kata / Coaching Kata to work effectively, the learner must also learn how to apply the precise structure that is built into them. For a few people learning that can be more difficult than the process improvement itself.

Sometimes We Have To Choose

A quote from a class I took a long time ago is appropriate here:

“Sometimes you have to choose between ‘being right’ or ‘getting what you want.’”

I can “be right” about insisting that the 5 Questions are being answered correctly and precisely.

Sometimes, though, that will prevent my learner from learning.

Countermeasure

When I first read Toyota Kata, my overall impression was “Cool! This codifies what I’ve been doing, but had a hard time explaining.” … meaning I was a decent coach, but couldn’t explain how I thought, or why I said what I did. It was just a conversation.

What the Coaching Kata did was give me a more formal structure for doing the same thing.

But I have also found that sometimes it doesn’t work to insist on following that formal structure. I have been guilty of losing sight of my objective, and pushing on “correctly following the Improvement Kata” rather than ensuring my learner was learning.

Recently I was set up in the situation again. I was asked to coach a learner who has had a hard time with the structure. Rather than trying to double down on the structure, I experimented and took a different approach. I let go of the structure, and reverted to my previous, more conversational, style.

The difference, though, is that now I am holding a mental checklist in my mind. While I am not asking the “Five Question” explicitly, I am still making sure I have answers to all of them before I am done. I am just not concerned about the way I get the answers.

“What are you working on?” While I am asking “What is your target condition?,” that question has locked up this learner in the past. What I got in reply was mostly a mix of the problems (obstacles) that had been encountered, where things are now, (the current condition), some things that had been tried (the last step), what happened, etc.

The response didn’t exactly give a “Target Condition” but it did give me a decent insight into the learner’s thinking which is the whole point! (don’t forget that)

I asked for some clarifications, and helped him focus his attention back onto the one thing he was trying to work out (his actual target condition), and encouraged him to write it down so he didn’t get distracted with the bigger picture.

Then we went back into what he was working on right now. It turned out that, yes, he was working to solve a specific issue that was in the way of making things work the way he wanted to. There were other problems that came up as well.

We agreed that he needed to keep those other things from hurting output, but he didn’t need to fix them right now. (Which *one* obstacle are you addressing now?). Then I turned my attention back to what he was trying right now, and worked through what he expected to happen as an outcome, and why, and when he would like me to come by so he could show me how it went.

This was an experiment. By removing the pressure of “doing the kata right” my intent is to let the learner focus on learning about his process. I believe I will get the same outcome, with the learner learning at his own pace.

If that works, then we will work, step by step, to improve the documentation process as he becomes comfortable with it.

Weakness to this Approach

By departing from the Coaching Kata, I am reverting to the way I was originally taught, and the way I learned to do this. It is a lot less structured, and for some, more difficult to learn. Some practitioners get stuck on correct application of the lean tools, and don’t transition to coaching at all. I know I was there for a long time (probably through 2002 or so), and found it frustrating. It was during my time as a Lean Director at Kodak that my style fundamentally shifted from “tools” to “coaching leaders.” (To say that my subsequent transition back into a “tools driven” environment was difficult is an understatement.)

Today, as an outsider being brought into these organizations, my job is to help them establish a level of coaching that is working well enough that they can practice and learn through self-reflection.

We ran into a learner who had a hard time adapting to the highly structured approach of the Improvement Kata / Coaching Kata, so we had to adapt. This required a somewhat more flexible and sophisticated approach to the coaching which, in turn, requires a more experienced coach who can keep “the board” in his head for a while.

Now my challenge is to work with the internal coaches to get them to the next level.

What I Learned

Maybe I should put this at the top.

  • If a learner is struggling with the structured approach, sometimes continuing to emphasize the structure doesn’t work.
  • The level of coaching required in these cases cannot be applied in a few minutes. It takes patience and a fair amount of 1:1 conversation.
  • If the learner is afraid of “getting it wrong,” no learning is going to happen, period.
  • Sometimes I have to have my face slammed into things to see them. (See below.)
  • Learning never stops. The minute you think you’re an expert, you aren’t.

__________________

image* “Steep learning curve” in this case means “sometimes learning the hard way” which, in turn means, “I’ve really screwed it up a couple of times.”

They say “experience” is something you gain right after you needed it.

Posted on

Who is Grading the Questions?

Once again the search terms have revealed an interesting test question. Let’s parse it and see what we can learn.

2. the cycle time to complete a final step on an assembly line is normally distributed, with a mean of 4 minutes and a standard deviation of 1 minute.1) at the end of an 8-hour shift, how many units will have been assembled on average?

The question is about exit cycles – the cadence at the output of a production process of some kind, in this case an assembly line.

Measuring and plotting the variation of exit cycles at the customer end of your production process is a great way to get a handle on how much variation there is.

This question, however, reveals a level of simplistic understanding in the mind of the person who wrote the question of how these things work.

Answering the Question

First, we have to assume that this assembly line (which implies manual work) takes no breaks during their 8 hour shift. We have no information that suggests otherwise.

That means we have 8 hours x 60 minutes = 480 minutes of working time.

Intuitively, if the exit cycles are normally distributed (which implies the distribution is symmetric), with a mean of 4 minutes, then we should be able to simply divide to get the average number of units built during each shift.

480 / 4 = 120 units

Which in the long-haul is close, though it will take a lot of iterations to converge on that. Running about 65 iterations of a random number set with a mean of 4 and standard deviation of 1, I got an average daily production of 120.1, a high of 127 units, and a low of 115.

But Exit Cycle Times Aren’t Normally Distributed

This is a problem with a lot of the “statistics” I see being kicked around today.

This is a histogram of 36 actual exit cycle times from a real production line.

image

The “Cycle Time” bucket numbers on the horizontal axis represent the top of the range. So the high bar says there were 10 cycle times between 25 and 30 seconds.

The mean of the data set is 42, which is actually one of the less frequent values.

The first thing that jumps out is that this data set might have two peaks (be bimodal). I would want a lot more data before I drew this conclusion, however let’s assume it is for the discussion because this is quite common.

A bimodal distribution implies there are two different processes running. If we looked at a run chart of individual cycles, we would probably see one of two things:

  • Clusters of longer times spaced between clusters of shorter times. This would be the case if the line is running lots or batches, then switching over.
  • A repeating cadence like “short-short-short-long, short-short-short-long” which would indicate really good production leveling.

But all of that is a sidebar. The real issue I have with this “homework question” simple: Cycle times are (almost) never normally distributed.

There is a minimum time that can elapse between the completion of one unit and the completion of the next, but there is no theoretical limit on the length of delay. These distributions (almost) always have a tail to the right.

So once again, be very careful about blindly using averages (arithmetic means) unless you understand the whole story.

The Test Question Misleads the Students

And this is my problem with questions like this on exams and homework, especially in classes that purport to be teaching how factories run. Even when using a distribution, the problems almost always default to a symmetric distribution when real world factory-related data sets are rarely symmetrically distributed.

You are teaching your students a simple solution that won’t work in the real world without explaining why.

Posted on

Team Member Saves

image

Now and then one of your team members makes a great save. They catch something that could have caused a defect, an accident, or done harm in some way.

Often we celebrate these saves, sometimes informally, sometimes formally. And that is well and appropriate.

But let’s make sure we are celebrating for the right reasons.

The save isn’t what should be celebrated.

Rather, the celebration should be a big THANK YOU for finding a gap in your process.

Somehow the process is capable of producing a defect, resulting in an accident, or doing harm. Your team member noticed that.

We usually just celebrate correcting the immediate problem.

But What is preventing exactly the same thing from happening tomorrow?*

image

That front-line customer-facing team member is your last line of defense.

They only get an opportunity to make a “save” when every other point in the process has failed to detect the  problem.

Given enough “shots” at this front line team-member, sooner or later, one is going to get through.

What happens then? Is the inverse logic applied? “You should have caught that.”

Perhaps, but where in the process was the problem actually created?

Somewhere, long before this diving catch, there is an instant when the process went from operating safely and defect free to creating an opportunity, an opening, for a problem to pass undetected.

Where and when was that moment?

Or is that how the process normally operates, and we are just lucky most of the time?

Dig in, think about it.

And thank that team member for saving you, but don’t count on it every time.

———-

*Thanks to Craig for this great way to sum up the question.

Posted on

Coaching Kata: Walking Through an Improvement Board

Improver's Storyboard

The Coaching Kata is much more than just asking the 5 questions. The coach needs to pay attention to the answers and make sure the thinking flows.

Although I have alluded to pieces in prior posts, today I want to go over how I try to connect the dots during a coaching cycle.

Does the learner understand the challenge she is striving for?

The “5 Questions” of the Coaching Kata do not explicitly ask about the challenge the learner is striving for. This makes sense because the challenge generally doesn’t change over the course of a week or two.

But I often see challenges that are vague, defined only by a general direction like “reduce.” The question I ask at that point is “How will you know when you have achieved the challenge?”

If there isn’t a measurable outcome (and sometimes there isn’t), I am probing to see if the learner really understands what he must achieve to meet the challenge.

This usually comes up when I am 2nd coaching and the learner and regular coach haven’t really reached a meeting of the minds on what the challenge is.

Is the target condition a logical step in the direction of the challenge?

And is the target condition based on a thorough grasp of the current condition?

I’m going to start with this secondary question since I run into this issue more often, especially in organizations with novice coaches. (And, by definition, that is most of the organizations where I spend time.)

It is quite common for the learner to first try to establish a target condition, and then grasp the current condition. Not surprisingly, they struggle with that approach. It sometimes helps to have the four steps of the Improvement Kata up near the board, and even go as far as to have a “You are here” arrow.

Four steps of the Improvement Kata
(c) Mike Rother

Another question I ask myself is Can I directly compare the target condition and the current condition? Can I see the gap, can I see the same indicators and measurements used for each so I can compare “apples vs apples”?

Along with this is the same question I ask for the challenge, only more so for the target condition:

How will the learner be able to tell when the target is met? Since this has a short-term deadline, I am really looking for a crisp, black-and-white line here. The target condition is either met or not met on the date.

Is there a short-term date that is in the future?

It is pretty common for a novice learner to set a target condition equal to the challenge. If they are over-reaching, I’ll impose a date, usually no more than two weeks out. “Where will you be in two weeks?” Another way to ask is “What will the current condition be in two weeks?”

Sometimes the learner has slid up to the date and past it. Watch for this! If the date comes up without hitting the target, then it is time to reflect and establish a new target condition in the future.

Is the target condition a step in the direction of the challenge?

Usually the link between the target condition and the overall challenge is pretty obvious. Sometimes, though, it isn’t clear to the coach, even if it is clear in the mind of the learner. In these cases, it is important for the coach to ask.

Key Point: The coach isn’t rigidly locked into the script of the 5 questions. The purpose of follow-up questions is to (1) actually get an answer to the Coaching Kata questions and (2) make sure the coach understands how the learner is thinking. Remember coach: It is the learner’s thinking that you are working to improve, so you have to understand it!

(And occasionally the learner will try to establish a target condition that really isn’t related to the challenge.)

Does the “obstacle being addressed” actually relate to the target condition?

(Always keep your marshmallow on top!)

The question is “What obstacles do you think are preventing you from reaching the target condition?” That question should be answered with a reading of all of the obstacles. (Again, the coach is trying to understand what the learner is thinking.) Then “Which one (obstacle) are you addressing now?”

Generally I give a pretty broad (though not infinitely broad) pass to the obstacles on the list. They are, after all, the learner’s opinion (“…do you think are…”). But when it comes to the “obstacle being addressed now” I apply a little more scrutiny.

I have addressed this with a tip in a previous post: TOYOTA KATA: IS THAT REALLY AN OBSTACLE?

It is perfectly legitimate, especially early on, for an obstacle to be something we need to learn more about. The boundary between “Grasp the current condition” and “Establish the next target condition” can be blurry. As the focus is narrowed, the learner may well have to go back and dig into some more detail about the current condition. If that is impeding getting to the target, then just write it down, and be clear what information is needed. Then establish a step that will get that information.

Sometimes the learner will write down every obstacle they perceive to reaching the challenge. The whole point of establishing a Target Condition is to narrow the scope of what needs to be worked on down to something easier to deal with. When I focus them on only the obstacles that relate directly to their Target Condition, many are understandably reluctant to simply cross other (legitimate, just not “right now”) issues off the list.

In this case it can be helpful to establish a second Obstacle Parking Lot off to the side that has these longer-term obstacles and problems on it. That does a couple of things. It can remind the coach (who is often the boss) that, yes, we know those are issues, but we aren’t working on those right now. Other team members who contribute their thinking can also know they were heard, and those issues will be addressed when they are actually impeding progress.

Does the “Next step or experiment” lead to learning about the obstacle being addressed?

Sometimes it helps to have the learner first list what they need to learn, and then fill in what they are going to do. See this previous post for the details: IMPROVEMENT KATA: NEXT STEP AND EXPECTED RESULT.

In any case, I am looking to see an “Expected result” that at least implies learning.

In “When can we go and see what we have learned from taking that step?” I am also looking for a fast turn-around. It is common for the next step to be bigger than it needs to be. “What can you do today that will help you learn?” can sometimes help clarify that the learner doesn’t always need to try a full-up fix. It may be more productive to test the idea in a limited way just to make sure it will work the way she thinks it will. That is faster than a big project that ends up not working.

Posted on

The Lean Plateau

Many organizations trying to deploy lean get great results for the first couple of years, then things tend to stall or plateau. This is in spite of continued effort from the “lean team.”

image

We Still Don’t Have a Lean Culture

This was the comment by the Continuous Improvement director of a pretty large corporation. They had been running improvement events for several years, everyone had pretty much been through one.

Each of the events had made pretty good strides during the week, but the behavior wasn’t changing. Things were eroding behind the events, even though everyone agreed things were better.

It was getting harder and harder to make more progress. They had hit the plateau.

What Causes the Lean Plateau?

While it might not be universal, what I have seen happen is this:

The implementation is led by a small group of dedicated technical experts. They are the ones who are looking for opportunities, organizing the kaizen event teams, leading workshops, and overseeing the implementation of lean techniques.

While this works in the short term, often the last implemented results begin to erode as soon as the lean experts shift their attention elsewhere.

At first, this isn’t noticed because the implementation is proceeding faster than the erosion.

However the more areas that are implemented, the faster the erosion becomes. There is simply more “surface area” of implemented areas.

At some point, the rate of erosion = the rate of implementation, and the lean team’s efforts start to shift from implementing new areas to going back and re-implementing areas that have eroded.

The lean team’s capacity becomes consumed re-implementing, and they spend less and less time going over new ground. They are spending all of their time “spinning the plates” and no time starting new ones.

Key Point: The lean plateau occurs when the level of implementation effort and the rate of erosion reach an equilibrium.

In the worst scenario, sooner or later financial pressures come into play. Management begins to question the expense of maintaining an improvement office if things aren’t getting significantly better on the bottom line. What they don’t see is that the office is keeping things from getting worse, but they aren’t called the “maintain what we have office” for a reason.

Breaking the Lean Plateau

When I was a lean director in a large company, we were confronting this very question. We had a meeting to talk about it, and quickly started blaming “lack of management commitment.”

Leaders Weren’t Stepping Up

In any given area, after education and planning, our last step was always to have a major effort to put flow production into place. Since the performance of the area would be substantially better, we expected the leaders to work hard to continue that performance.

What actually happened in an area was “implemented,” was the line leaders in that area – supervisors, managers, senior managers – weren’t working to look for erosion and correct it.

Instead, when a problem was encountered, they were making some kind of accommodation that compromised flow. The effect of the problem went away, but things had eroded a bit.

What we thought we learned: The weren’t “supporting the changes.”

What we really learned – though it was only realized in hindsight: This is the mechanism of “erosion.”

Flow production is specifically designed to surface small problems quickly. If there is no mechanism to detect those problems, respond, correct, and learn, then the only thing leaders can do is add a little inventory, add a little time, add an extra operation.

As Hirano put it so well decades ago:

All waste is cleverly disguised as useful work.

But Our Current Condition was Incomplete

There were outliers where it was working.

As we talked, we realized that each of us had experience with an outlier – one or two areas that were actually improving pretty steadily. Trying to understand what was different about these bright spots, we looked for what they all had in common. Surprisingly:

  • They were areas with no dedicated improvement teams.
  • They ran few, if any, 5-day kaizen events.
  • They were geographically close to one of us (senior “Directors”).
  • One of us had decent rapport with the area management team.
  • We each had an informal routine with them: We would drop by when we had time, and walk the work area with the area leader. We could discuss the challenges they were facing, how things were operating, go together to the operations concerned, and look at what was happening. We could ask questions designed to “sharpen the vision” of the leader. Sometimes they were leading questions. Most of the time they were from genuine curiosity.
  • By the time we left, there was generally some action or short term goal that the leader had set for himself.

Even though we “lean directors” had never worked together before, our stories were surprisingly consistent.

The Current Condition (Everywhere Else)

aka Dave’s Insight

The next logical question was “If that is what we do, what happens everywhere else? What do the lean staff people do?”

Now we were trying to understand the normal pattern of work, not simply the outcome of “the area erodes because the leaders don’t support the changes.”

Dave confidently stood up and grabbed the marker. He started outlining how he trained and certified his kaizen leaders. He worked through the list of skills he worked to develop:

  • Proficiently deliver the various topical training modules – Waste vs. Value Add; Standard Work; Jidoka; Kanban and Pull;
  • “Scan” an area to find improvement opportunities.
  • Establish the lean tools to be deployed.
  • Organize the workshop team.
  • Facilitate the “Vision”
  • Manage the “Kaizen Newspaper” items
  • etc

and at some point through this detailed explanation he stopped in mid sentence and said something that brought all of us to reality (Please avert your eyes if you are offended by a language you won’t hear on network TV):

“Aw… shit.”

What we realized more or less simultaneously was this:

Management wasn’t engaged because our process wasn’t engaging them.

Instead, our experts were essentially pushing them aside and “fixing” things, then turning the newly “leaned” area over to the supervisors and first line managers who, at most, might have participated in the workshop and helped move things around.

Those critical front line leaders were, at best left with a to-do list of ideas (kaizen newspaper items) that hadn’t been implemented during the 5 days.

There was nothing in the structure to challenge them to meet a serious business objective beyond “Look at how much better everything runs now.” The amount of improvement was an after-the-fact measurement (or estimate) rather than a before-we-begin imperative.

So it really should be no surprise that come Monday morning, when the inevitable forces of entropy showed up, that things started to erode. The whole system couldn’t have been better designed for that outcome.

Why the Difference in Approach?

In retrospect, I don’t know. Each of us senior “lean directors” had been taught, or heavily influenced by, Toyota-experienced Japanese mentors, teachers, consultants.

When we engaged the “outlier” areas, we were following a kinder, gentler version of what they had taught us.

On the other hand, what we were teaching our own people was modeled more on what western consultants were doing. Perhaps it is because it is easier to use forms and PowerPoint for structure than to teach the skills of the conversations we were having.

Implement by Experts or Coached by Leaders

That really is your choice. The expert implementation seems a lot easier.

Unfortunately the “rapid improvement event” (or whatever you call them) system has a really poor record of sustaining.

Perhaps our little group figured out why.

There are no guarantees. No approach will work every time. But a difficult approach that works some of the time is probably better than an easy path that almost never works.

Posted on

Toyota Kata: What is the Learner Learning?

In the language of Toyota Kata we have a “coach” and a “learner.” Some organizations use the word “improver” instead of “learner.” I have used those terms more or less interchangeably. Now I am getting more insight into what the “learner” is learning.

The obvious answer is that, by practicing the Improvement Kata, the learner is learning the thinking pattern that is behind solid problem solving and continuous improvement.

But now I am reading more into the role. The “learner” is also the one who is learning about the process, the problems, and the solutions.

Steve Spear has a mantra of “See a problem, solve a problem, teach somebody.” This is, I think, the role of the learner.

What about the coach?

The coach is using the Coaching Kata to learn how to ask questions that drive learning. He may also be un-learning how to just have all of the answers.

As the coach develops skill, I advise sticking to the Coaching Kata structure for the benefit of beginner learners. It is easier for them to be prepared if they understand the questions and how to answer them. That, in turn, teaches them the thinking required to develop those answers.

Everybody is a Learner

The final question in the “5 Questions” is “When can we go and see what we have learned from taking that step?” It isn’t when can I see what You have learned. It is a “we” question because nobody knows the answers yet.

Posted on

The Root (Cause) Of All Problems

One of my clients has been working with Steve Spear. They shared a great point he made with me, and I want to share it with you:

“The root cause of all problems is ignorance.”

— Steven Spear

I can’t argue with that. If you don’t understand the root cause, you need to learn more about the problem. And to be clear, “problem solving” and “improvement” are learning processes. If you didn’t learn, you didn’t solve the problem, and you didn’t improve anything. At best you suppressed the symptoms.

So… next time you encounter a problem (which is likely as soon as you put your head up from reading this), instead of asking “What should we do?” ask “What do we need to learn about this?” It will set people off in an entirely different (and far more robust) direction.

Posted on

Averages, Percentages and Math

As a general rule I strongly discourage the use of averages and “percent improvement” (or reduction) type metrics for process improvement.

The Problem with Averages

Averages can be very useful when used as part of a rigorous statistical analysis. Most people don’t do that. They simply dump all of their data into a simple arithmetic mean, and determine a score of sorts for how well the process is doing.

The Average Trap

There is a target value. Let’s say it is 15. Units could be anything you want. In this example, if we exceed 15, we’re good. Under 15, not good.

“Our goal is 15, and our average performance is 20.”

Awesome, right?

Take a look at those two run charts below*. They both have an average of 20.

On the first one, 100% of the data points meet or exceed the goal of 15.

Run chart with average of 20, all points higher than 15.

On the one below, 11 points miss the goal

image

But the both have an average 5 points over the goal.

In this case, the “average” really gives you almost no information. I would have them measure hits and misses, not averages. The data here is contrived, but the example I am citing is something I have seen multiple times.

Why? Most people learned how to calculate an arithmetic mean in junior high school. It’s easy. It’s easier to put down a single number than to build a run chart and look at every data point. And once that single number is calculated, the data are often thrown away.

Be suspicious when you hear “averages” used as a performance measurement.

Using Averages Correctly

(If you understand elementary statistical testing you can skip this part… except I’ve experts who should have known better fall into the trap I am about to describe, so maybe you shouldn’t skip it after all.)

In spite of what I said above, there are occasions when using an average as a goal or as part of a target condition makes sense.

A process running over time produces a range of values that fall into a distribution of some kind.

Any sample you take is just that – a sample. Take a big enough sample, and you can become reasonably confident that the average of your sample represents (meaning is close to) the average of everything.

The move variation there is, the bigger sample you need to gain the same level of certainty (which is really expressed as the probability you are wrong).

The more certain you want to be, the bigger sample you need.

Let’s say you’ve done that. So now you have an average (usually a mean) value.

Since you are (presumably) trying to improve the performance, you are trying to shift that mean – to change the average to a higher or lower value.

BUT remember there is variation. If you take a second sample of data from an unchanged process and calculate that sample’s average, YOU WILL GET A DIFFERENT AVERAGE. It might be higher than the first sample, it might be lower, but the likelihood that it will exactly the same is very, very small.

The averages will be different even if you haven’t changed anything.

You can’t just look at the two numbers and say “It’s better.” If you try, the NEXT sample you take might look worse. Or it might not. Or it might look better, and you will congratulate yourself.

If you start turning knobs in response, you are chasing your tail and making things worse because you are introducing even more variables and increasing the variation. Deming called this “Tampering” and people do it all of the time.

Before you can say “This is better” you have to calculate, based on the amount of variation in the data, how much better the average needs to be before you can say, with some certainty, that this new sample is from a process that is different than the first one.

The more variation there is, the more difference you need to see. The more certainty you want, the more difference you need to see. This is called “statistical significance” and is why you will see reports that seem to show something is better, but seem to be dismissed as a “statistically insignificant difference” between, for example, the trial medication and the placebo.

Unless you are applying statistical tests to the samples, don’t say “the average is better, so the process is better.” The only exception would be if the difference is overwhelmingly obvious. Even then, do the math just to be sure.

I have personally seen a Six Sigma Black Belt(!!) fall into this trap – saying that a process had “improved” based on a shift in the mean of a short sample without applying any kind of statistical test.

As I said, averages have a valuable purpose – when used as part of a robust statistical analysis. But usually that analysis isn’t there, so unless it is, I always want to see the underlying numbers.

Sometimes I hear “We only have the averages.” Sorry, you can’t calculate an average without the individual data points, so maybe we should go dig them out of the spreadsheet or database. They might tell us something.

The Problem with Percentages

Once again, percentages are valuable analysis tools, so long as the underlying information isn’t lost in the process. But there are a couple of scenarios where I always ask people not to use them.

Don’t Make Me Do Math

“We predict this will give us a 23% increase in output.”

That doesn’t tell me a thing about your goal. It’s like saying “Our goal is better output.”

Here is my question:

“How will you know if you have achieved it?”

For me to answer that question for myself, I have to do math. I have to take your current performance, multiply x 1.23 to calculate what your goal is.

If that number is your goal, then just use the number. Don’t make me do math to figure out what your target is.

Same thing for “We expect 4 more units per hour.”

“How many units do you expect per hour?” “How many are you producing now?” (compared to what?)

Indicators of a W.A.G.

How often do you hear something like  “x happens 90 percent of the time”?

I am always suspicious of round numbers because they typically have no analysis behind them. When I hear “75%” or “90%” I am pretty sure it’s just speculation with no data.

These things sound very authoritative and it is easy for the uncertainty to get lost in re-statement. What was a rough estimate ends up being presented as a fact-based prediction.

At Boeing someone once defined numbers like this as “atmospheric extractions.”

If the numbers are important, get real measurements. If they aren’t important, don’t use them.

Bottom Line Advice:

Avoid averages unless they are part of a larger statistical testing process.

Don’t set goals as “percent improvement.” Do the math yourself and calculate the actual value you are shooting for. Compare your actual results against that value and define the gap.

When there is a lot of variation in the number of opportunities for success (or not) during a day, a week, think about something that conveys “x of X opportunities” in addition to a percent. When you have that much variation in your volume, fluctuations in percent of success from one day to the next likely don’t mean very much anyway.

Look at the populations – what was different about the ones that worked vs. the ones that didn’t — rather than just aggregating everything into a percentage.

Be suspicious of round numbers that sound authoritative.

_______________

*These charts are simply independent random numbers with upper and lower bounds on the range. Real data is likely to have something other than a flat distribution, but these make the point.

Posted on

The Improvement Kata: Next Step and Expected Result

In the Improvement Kata sometimes it helps to think about the outcome desired and then the step required to accomplish it.

A couple of months ago, I gave a tip I’ve learned for helping a coach vet an obstacle.

Another issue I come across frequently is a weak link between the “Next Step” and the “Expected Outcome.”

In the “Five Questions” of the Coaching Kata we have:

“What is your next step or experiment?” Here we expect the learner / improver to describe something he is going to do. I’m looking for a coherent statement that includes a subject, verb, object here.

Then we ask “What do you expect?” meaning “What do you expect to happen?” or “What do you expect to learn?” from taking that step?

I want to see that the “Expected Result” is a clear and direct consequence of taking the “Next Step.”

Often, though, the learner struggles a bit with being clear about the expected outcome, or just re-states the next step in the past tense.

While this is the order we ask the questions, sometimes it helps to think about them in reverse.

Reverse the Order

Have the learner first, think about (and then describe) what she is trying to accomplish with this step. Look at the obstacle being addressed, and what was learned from the last step.

Based on those things, ask “what do you want to accomplish with your next step?”

The goal here is to get the learner to think about the desired result. Don’t be surprised if that is still stated as something to do, because we are all conditioned to think in terms of action items, not outcomes.

“What do you need to learn?” sometimes helps.

“I need to learn if ______ will eliminate the problem.” might be a reply.

Even a proposed change to the process usually has “to learn if” as an expected outcome, because we generally don’t know for certain what the outcome will be until we try it.

Have the learner fill in the “Expected Outcome” block.

NOW ask “OK, what do you have to do to ______ (read what is in the expected outcome)?”

PDCA Outcome-Activity

That should get your learner thinking about the actions that will lead to that outcome.

A Verbal Test

A verbal test can be to say “In order to ______ (read the expected outcome), I intend to _____ (read the next step.”

If that makes sense grammatically and logically, it is probably well thought out.

Posted on

The Destructiveness of “What Can You Improve?”

“What Can You Improve?”

Leaders often ask “What can you improve?” as an empowerment question. In reality, it may have the opposite effect.

I am coming to the belief that “What can you improve?” (about your job, about your process) is possibly one of the most demotivating, disempowering, destructive questions that can be asked.

“What can you work on?” is another one of many forms this question takes. “How could you improve this process?” is another. What they all have in common is the psychological trap they set.

Now this really isn’t that much of a problem in a company that has a history of transparency in leadership, comfort with discussing the truth, and no need for excuses or justifications. Then again, those companies tend not to ask these questions straight-on.

But the vast majority of organizations aren’t like that. That doesn’t mean they are unkind. Rather, they operate in an environment where truthfully answering this question is difficult at best.

The Psychological Trap

To answer that question with anything other than trying to guess what you want, implies I have:

  • Thoroughly examined my results and the underlying processes.
  • Identified gaps in performance.
  • Know what to do about those gaps.
  • And haven’t done anything about it until you asked.

This puts me in the position of either defending the status-quo, or saying that I need to improve something that is out of my control – someone else’s process needs improvement so I can do better.

Hint: If you are a leader, and you ask a “What can you improve?” question and get an answer like the above – defending the status-quo or pointing to an outside problem –, there is fear in your organization. Justified or not, the person answering is struggling to maintain the impression that everything they can do is being done. Why do they feel the need to do this? Think about it.

This is especially pervasive in support / staff departments with a charter of influencing how other organizations perform, or in those who must work together with line organizations to succeed in their tasks. In industry this might be maintenance, HR, industrial engineering, or even the “improvement office” (who are often not a  beacon of internal efficiency or effectiveness).

A Bit of Background

When I start working with an organization, we usually start with practicing the basic mechanics of the Improvement Kata in a classroom setting. We then follow up immediately with kick-starting some live improvement cycles so we can begin practical application. Classroom learning really doesn’t do much good unless it is applied immediately.

Applying the Improvement Kata is a lot harder in the real world than it is in the classroom. I could go into a tangential rant on why I think our primary and secondary education system makes it harder, but I’ll save that for another day.

Even though I am as adamant as I can be on the importance of the organization identifying challenges for the new improvers / learners, the reality is that most organizations don’t know how to do this, or at least aren’t comfortable with it.*

As a result, the new improvers often struggle to define a “challenge” for themselves.

They guess – because they haven’t yet studied their process (which is the next step once context is established, they haven’t yet established a target condition (which is the step after that), and therefore, they haven’t identified what improvements they must make to get to the challenge state.

And if that guess is something in someone else’s domain, or worse if the “coach” has something else in mind, they are told “That’s not it,” they guess again, and eventually get defensive or give up.

Now – to be clear, this doesn’t happen every time. But I have seen it enough, across multiple organizations in very different domains that it’s a problem. And it is frustrating for everyone when it happens.

I indirectly addressed this topic a long time ago in “How the Sensei Sees.” Now, though I am talking about my own direct observation of the effect. And I am still learning how to deal with the fallout without becoming part of the problem.

It’s not the learner’s problem. It is a leadership problem.

________

*Dave Kilgore at Continental Automotive had the additional insight that it is important for beginners that this challenge should be something important but not urgent so they don’t feel pressured to jump to an immediate solution. This is a good example of “constancy of purpose” – his priority is developing the skill level for improvement first.