Who is Grading the Questions?

Once again the search terms have revealed an interesting test question. Let’s parse it and see what we can learn.

2. the cycle time to complete a final step on an assembly line is normally distributed, with a mean of 4 minutes and a standard deviation of 1 minute.1) at the end of an 8-hour shift, how many units will have been assembled on average?

The question is about exit cycles – the cadence at the output of a production process of some kind, in this case an assembly line.

Measuring and plotting the variation of exit cycles at the customer end of your production process is a great way to get a handle on how much variation there is.

This question, however, reveals a level of simplistic understanding in the mind of the person who wrote the question of how these things work.

Answering the Question

First, we have to assume that this assembly line (which implies manual work) takes no breaks during their 8 hour shift. We have no information that suggests otherwise.

That means we have 8 hours x 60 minutes = 480 minutes of working time.

Intuitively, if the exit cycles are normally distributed (which implies the distribution is symmetric), with a mean of 4 minutes, then we should be able to simply divide to get the average number of units built during each shift.

480 / 4 = 120 units

Which in the long-haul is close, though it will take a lot of iterations to converge on that. Running about 65 iterations of a random number set with a mean of 4 and standard deviation of 1, I got an average daily production of 120.1, a high of 127 units, and a low of 115.

But Exit Cycle Times Aren’t Normally Distributed

This is a problem with a lot of the “statistics” I see being kicked around today.

This is a histogram of 36 actual exit cycle times from a real production line.

image

The “Cycle Time” bucket numbers on the horizontal axis represent the top of the range. So the high bar says there were 10 cycle times between 25 and 30 seconds.

The mean of the data set is 42, which is actually one of the less frequent values.

The first thing that jumps out is that this data set might have two peaks (be bimodal). I would want a lot more data before I drew this conclusion, however let’s assume it is for the discussion because this is quite common.

A bimodal distribution implies there are two different processes running. If we looked at a run chart of individual cycles, we would probably see one of two things:

  • Clusters of longer times spaced between clusters of shorter times. This would be the case if the line is running lots or batches, then switching over.
  • A repeating cadence like “short-short-short-long, short-short-short-long” which would indicate really good production leveling.

But all of that is a sidebar. The real issue I have with this “homework question” simple: Cycle times are (almost) never normally distributed.

There is a minimum time that can elapse between the completion of one unit and the completion of the next, but there is no theoretical limit on the length of delay. These distributions (almost) always have a tail to the right.

So once again, be very careful about blindly using averages (arithmetic means) unless you understand the whole story.

The Test Question Misleads the Students

And this is my problem with questions like this on exams and homework, especially in classes that purport to be teaching how factories run. Even when using a distribution, the problems almost always default to a symmetric distribution when real world factory-related data sets are rarely symmetrically distributed.

You are teaching your students a simple solution that won’t work in the real world without explaining why.

Team Member Saves

image

Now and then one of your team members makes a great save. They catch something that could have caused a defect, an accident, or done harm in some way.

Often we celebrate these saves, sometimes informally, sometimes formally. And that is well and appropriate.

But let’s make sure we are celebrating for the right reasons.

The save isn’t what should be celebrated.

Rather, the celebration should be a big THANK YOU for finding a gap in your process.

Somehow the process is capable of producing a defect, resulting in an accident, or doing harm. Your team member noticed that.

We usually just celebrate correcting the immediate problem.

But What is preventing exactly the same thing from happening tomorrow?*

image

That front-line customer-facing team member is your last line of defense.

They only get an opportunity to make a “save” when every other point in the process has failed to detect the  problem.

Given enough “shots” at this front line team-member, sooner or later, one is going to get through.

What happens then? Is the inverse logic applied? “You should have caught that.”

Perhaps, but where in the process was the problem actually created?

Somewhere, long before this diving catch, there is an instant when the process went from operating safely and defect free to creating an opportunity, an opening, for a problem to pass undetected.

Where and when was that moment?

Or is that how the process normally operates, and we are just lucky most of the time?

Dig in, think about it.

And thank that team member for saving you, but don’t count on it every time.

———-

*Thanks to Craig for this great way to sum up the question.

Coaching Kata: Walking Through an Improvement Board

The Coaching Kata is much more than “asking the 5 questions.” The coach needs to pay attention to the answers and make sure the thinking flows.

Improver's Storyboard

The Coaching Kata is much more than just asking the 5 questions. The coach needs to pay attention to the answers and make sure the thinking flows.

Although I have alluded to pieces in prior posts, today I want to go over how I try to connect the dots during a coaching cycle.

Does the learner understand the challenge she is striving for?

The “5 Questions” of the Coaching Kata do not explicitly ask about the challenge the learner is striving for. This makes sense because the challenge generally doesn’t change over the course of a week or two.

But I often see challenges that are vague, defined only by a general direction like “reduce.” The question I ask at that point is “How will you know when you have achieved the challenge?

If there isn’t a measurable outcome (and sometimes there isn’t), I am probing to see if the learner really understands what he must achieve to meet the challenge.

This usually comes up when I am 2nd coaching and the learner and regular coach haven’t really reached a meeting of the minds on what the challenge is.

Is the target condition a logical step in the direction of the challenge?

And is the target condition based on a thorough grasp of the current condition?

I’m going to start with this secondary question since I run into this issue more often, especially in organizations with novice coaches. (And, by definition, that is most of the organizations where I spend time.)

It is quite common for the learner to first try to establish a target condition, and then grasp the current condition. Not surprisingly, they struggle with that approach. It sometimes helps to have the four steps of the Improvement Kata up near the board, and even go as far as to have a “You are here” arrow.

Four steps of the Improvement Kata
(c) Mike Rother

Another question I ask myself is Can I directly compare the target condition and the current condition? Can I see the gap, can I see the same indicators and measurements used for each so I can compare “apples vs apples”?

Along with this is the same question I ask for the challenge, only more so for the target condition:

How will the learner be able to tell when the target is met? Since this has a short-term deadline, I am really looking for a crisp, black-and-white line here. The target condition is either met or not met on the date.

Is there a short-term date that is in the future?

It is pretty common for a novice learner to set a target condition equal to the challenge. If they are over-reaching, I’ll impose a date, usually no more than two weeks out. “Where will you be in two weeks?” Another way to ask is “What will the current condition be in two weeks?”

Sometimes the learner has slid up to the date and past it. Watch for this! If the date comes up without hitting the target, then it is time to reflect and establish a new target condition in the future.

Is the target condition a step in the direction of the challenge?

Usually the link between the target condition and the overall challenge is pretty obvious. Sometimes, though, it isn’t clear to the coach, even if it is clear in the mind of the learner. In these cases, it is important for the coach to ask.

Key Point: The coach isn’t rigidly locked into the script of the 5 questions. The purpose of follow-up questions is to (1) actually get an answer to the Coaching Kata questions and (2) make sure the coach understands how the learner is thinking. Remember coach: It is the learner’s thinking that you are working to improve, so you have to understand it!

(And occasionally the learner will try to establish a target condition that really isn’t related to the challenge.)

Does the “obstacle being addressed” actually relate to the target condition?

(Always keep your marshmallow on top!)

The question is “What obstacles do you think are preventing you from reaching the target condition?” That question should be answered with a reading of all of the obstacles. (Again, the coach is trying to understand what the learner is thinking.) Then “Which one (obstacle) are you addressing now?”

Generally I give a pretty broad (though not infinitely broad) pass to the obstacles on the list. They are, after all, the learner’s opinion (“…do you think are…”). But when it comes to the “obstacle being addressed now” I apply a little more scrutiny.

I have addressed this with a tip in a previous post: TOYOTA KATA: IS THAT REALLY AN OBSTACLE?

It is perfectly legitimate, especially early on, for an obstacle to be something we need to learn more about. The boundary between “Grasp the target condition” and “Establish the next target condition” can be blurry. As the focus is narrowed, the learner may well have to go back and dig into some more detail about the current condition. If that is impeding getting to the target, then just write it down, and be clear what information is needed. Then establish a step that will get that information.

Sometimes the learner will write down every obstacle they perceive to reaching the challenge. The whole point of establishing a Target Condition is to narrow the scope of what needs to be worked on down to something easier to deal with. When I focus them on only the obstacles that relate directly to their Target Condition, many are understandably reluctant to simply cross other (legitimate, just not “right now”) issues off the list.

In this case it can be helpful to establish a second Obstacle Parking Lot off to the side that has these longer-term obstacles and problems on it. That does a couple of things. It can remind the coach (who is often the boss) that, yes, we know those are issues, but we aren’t working on those right now. Other team members who contribute their thinking can also know they were heard, and those issues will be addressed when they are actually impeding progress.

Does the “Next step or experiment” lead to learning about the obstacle being addressed?

Sometimes it helps to have the learner first list what they need to learn, and then fill in what they are going to do. See this previous post for the details: IMPROVEMENT KATA: NEXT STEP AND EXPECTED RESULT.

In any case, I am looking to see an “Expected result” that at least implies learning.

In “When can we go and see what we have learned from taking that step?” I am also looking for a fast turn-around. It is common for the next step to be bigger than it needs to be. “What can you do today that will help you learn?” can sometimes help clarify that the learner doesn’t always need to try a full-up fix. It may be more productive to test the idea in a limited way just to make sure it will work the way she thinks it will. That is faster than a big project that ends up not working.

The Lean Plateau

Many organizations trying to deploy lean get great results for the first couple of years, then things tend to stall or plateau. This is in spite of continued effort from the “lean team.”

image

We Still Don’t Have a Lean Culture

This was the comment by the Continuous Improvement director of a pretty large corporation. They had been running improvement events for several years, everyone had pretty much been through one.

Each of the events had made pretty good strides during the week, but the behavior wasn’t changing. Things were eroding behind the events, even though everyone agreed things were better.

It was getting harder and harder to make more progress. They had hit the plateau.

What Causes the Lean Plateau?

While it might not be universal, what I have seen happen is this:

The implementation is led by a small group of dedicated technical experts. They are the ones who are looking for opportunities, organizing the kaizen event teams, leading workshops, and overseeing the implementation of lean techniques.

While this works in the short term, often the last implemented results begin to erode as soon as the lean experts shift their attention elsewhere.

At first, this isn’t noticed because the implementation is proceeding faster than the erosion.

However the more areas that are implemented, the faster the erosion becomes. There is simply more “surface area” of implemented areas.

At some point, the rate of erosion = the rate of implementation, and the lean team’s efforts start to shift from implementing new areas to going back and re-implementing areas that have eroded.

The lean team’s capacity becomes consumed re-implementing, and they spend less and less time going over new ground. They are spending all of their time “spinning the plates” and no time starting new ones.

Key Point: The lean plateau occurs when the level of implementation effort and the rate of erosion reach an equilibrium.

In the worst scenario, sooner or later financial pressures come into play. Management begins to question the expense of maintaining an improvement office if things aren’t getting significantly better on the bottom line. What they don’t see is that the office is keeping things from getting worse, but they aren’t called the “maintain what we have office” for a reason.

Breaking the Lean Plateau

When I was a lean director in a large company, we were confronting this very question. We had a meeting to talk about it, and quickly started blaming “lack of management commitment.”

Leaders Weren’t Stepping Up

In any given area, after education and planning, our last step was always to have a major effort to put flow production into place. Since the performance of the area would be substantially better, we expected the leaders to work hard to continue that performance.

What actually happened in an area was “implemented,” was the line leaders in that area – supervisors, managers, senior managers – weren’t working to look for erosion and correct it.

Instead, when a problem was encountered, they were making some kind of accommodation that compromised flow. The effect of the problem went away, but things had eroded a bit.

What we thought we learned: The weren’t “supporting the changes.”

What we really learned – though it was only realized in hindsight: This is the mechanism of “erosion.”

Flow production is specifically designed to surface small problems quickly. If there is no mechanism to detect those problems, respond, correct, and learn, then the only thing leaders can do is add a little inventory, add a little time, add an extra operation.

As Hirano put it so well decades ago:

All waste is cleverly disguised as useful work.

But Our Current Condition was Incomplete

There were outliers where it was working.

As we talked, we realized that each of us had experience with an outlier – one or two areas that were actually improving pretty steadily. Trying to understand what was different about these bright spots, we looked for what they all had in common. Surprisingly:

  • They were areas with no dedicated improvement teams.
  • They ran few, if any, 5-day kaizen events.
  • They were geographically close to one of us (senior “Directors”).
  • One of us had decent rapport with the area management team.
  • We each had an informal routine with them: We would drop by when we had time, and walk the work area with the area leader. We could discuss the challenges they were facing, how things were operating, go together to the operations concerned, and look at what was happening. We could ask questions designed to “sharpen the vision” of the leader. Sometimes they were leading questions. Most of the time they were from genuine curiosity.
  • By the time we left, there was generally some action or short term goal that the leader had set for himself.

Even though we “lean directors” had never worked together before, our stories were surprisingly consistent.

The Current Condition (Everywhere Else)

aka Dave’s Insight

The next logical question was “If that is what we do, what happens everywhere else? What do the lean staff people do?”

Now we were trying to understand the normal pattern of work, not simply the outcome of “the area erodes because the leaders don’t support the changes.”

Dave confidently stood up and grabbed the marker. He started outlining how he trained and certified his kaizen leaders. He worked through the list of skills he worked to develop:

  • Proficiently deliver the various topical training modules – Waste vs. Value Add; Standard Work; Jidoka; Kanban and Pull;
  • “Scan” an area to find improvement opportunities.
  • Establish the lean tools to be deployed.
  • Organize the workshop team.
  • Facilitate the “Vision”
  • Manage the “Kaizen Newspaper” items
  • etc

and at some point through this detailed explanation he stopped in mid sentence and said something that brought all of us to reality (Please avert your eyes if you are offended by a language you won’t hear on network TV):

“Aw… shit.”

What we realized more or less simultaneously was this:

Management wasn’t engaged because our process wasn’t engaging them.

Instead, our experts were essentially pushing them aside and “fixing” things, then turning the newly “leaned” area over to the supervisors and first line managers who, at most, might have participated in the workshop and helped move things around.

Those critical front line leaders were, at best left with a to-do list of ideas (kaizen newspaper items) that hadn’t been implemented during the 5 days.

There was nothing in the structure to challenge them to meet a serious business objective beyond “Look at how much better everything runs now.” The amount of improvement was an after-the-fact measurement (or estimate) rather than a before-we-begin imperative.

So it really should be no surprise that come Monday morning, when the inevitable forces of entropy showed up, that things started to erode. The whole system couldn’t have been better designed for that outcome.

Why the Difference in Approach?

In retrospect, I don’t know. Each of us senior “lean directors” had been taught, or heavily influenced by, Toyota-experienced Japanese mentors, teachers, consultants.

When we engaged the “outlier” areas, we were following a kinder, gentler version of what they had taught us.

On the other hand, what we were teaching our own people was modeled more on what western consultants were doing. Perhaps it is because it is easier to use forms and PowerPoint for structure than to teach the skills of the conversations we were having.

Implement by Experts or Coached by Leaders

That really is your choice. The expert implementation seems a lot easier.

Unfortunately the “rapid improvement event” (or whatever you call them) system has a really poor record of sustaining.

Perhaps our little group figured out why.

There are no guarantees. No approach will work every time. But a difficult approach that works some of the time is probably better than an easy path that almost never works.

Toyota Kata: What is the Learner Learning?

In the Improvement Kata, the “learner” is not only learning the thinking pattern, she is becoming an expert on the process being improved.

In the language of Toyota Kata we have a “coach” and a “learner.” Some organizations use the word “improver” instead of “learner.” I have used those terms more or less interchangeably. Now I am getting more insight into what the “learner” is learning.

The obvious answer is that, by practicing the Improvement Kata, the learner is learning the thinking pattern that is behind solid problem solving and continuous improvement.

But now I am reading more into the role. The “learner” is also the one who is learning about the process, the problems, and the solutions.

Steve Spear has a mantra of “See a problem, solve a problem, teach somebody.” This is, I think, the role of the learner.

What about the coach?

The coach is using the Coaching Kata to learn how to ask questions that drive learning. He may also be un-learning how to just have all of the answers.

As the coach develops skill, I advise sticking to the Coaching Kata structure for the benefit of beginner learners. It is easier for them to be prepared if they understand the questions and how to answer them. That, in turn, teaches them the thinking required to develop those answers.

Everybody is a Learner

The final question in the “5 Questions” is “When can we go and see what we have learned from taking that step?” It isn’t when can I see what You have learned. It is a “we” question because nobody knows the answers yet.

The Root (Cause) Of All Problems

The root cause of all problems is ignorance — Steven Spear

One of my clients has been working with Steve Spear. They shared a great point he made with me, and I want to share it with you:

“The root cause of all problems is ignorance.”

— Steven Spear

I can’t argue with that. If you don’t understand the root cause, you need to learn more about the problem. And to be clear, “problem solving” and “improvement” are learning processes. If you didn’t learn, you didn’t solve the problem, and you didn’t improve anything. At best you suppressed the symptoms.

So… next time you encounter a problem (which is likely as soon as you put your head up from reading this), instead of asking “What should we do?” ask “What do we need to learn about this?” It will set people off in an entirely different (and far more robust) direction.

Averages, Percentages and Math

As a general rule I strongly discourage the use of averages and “percent improvement” (or reduction) type metrics for process improvement.

The Problem with Averages

Averages can be very useful when used as part of a rigorous statistical analysis. Most people don’t do that. They simply dump all of their data into a simple arithmetic mean, and determine a score of sorts for how well the process is doing.

The Average Trap

There is a target value. Let’s say it is 15. Units could be anything you want. In this example, if we exceed 15, we’re good. Under 15, not good.

“Our goal is 15, and our average performance is 20.”

Awesome, right?

Take a look at those two run charts below*. They both have an average of 20.

On the first one, 100% of the data points meet or exceed the goal of 15.

Run chart with average of 20, all points higher than 15.

On the one below, 11 points miss the goal

image

But the both have an average 5 points over the goal.

In this case, the “average” really gives you almost no information. I would have them measure hits and misses, not averages. The data here is contrived, but the example I am citing is something I have seen multiple times.

Why? Most people learned how to calculate an arithmetic mean in junior high school. It’s easy. It’s easier to put down a single number than to build a run chart and look at every data point. And once that single number is calculated, the data are often thrown away.

Be suspicious when you hear “averages” used as a performance measurement.

Using Averages Correctly

(If you understand elementary statistical testing you can skip this part… except I’ve experts who should have known better fall into the trap I am about to describe, so maybe you shouldn’t skip it after all.)

In spite of what I said above, there are occasions when using an average as a goal or as part of a target condition makes sense.

A process running over time produces a range of values that fall into a distribution of some kind.

Any sample you take is just that – a sample. Take a big enough sample, and you can become reasonably confident that the average of your sample represents (meaning is close to) the average of everything.

The move variation there is, the bigger sample you need to gain the same level of certainty (which is really expressed as the probability you are wrong).

The more certain you want to be, the bigger sample you need.

Let’s say you’ve done that. So now you have an average (usually a mean) value.

Since you are (presumably) trying to improve the performance, you are trying to shift that mean – to change the average to a higher or lower value.

BUT remember there is variation. If you take a second sample of data from an unchanged process and calculate that sample’s average, YOU WILL GET A DIFFERENT AVERAGE. It might be higher than the first sample, it might be lower, but the likelihood that it will exactly the same is very, very small.

The averages will be different even if you haven’t changed anything.

You can’t just look at the two numbers and say “It’s better.” If you try, the NEXT sample you take might look worse. Or it might not. Or it might look better, and you will congratulate yourself.

If you start turning knobs in response, you are chasing your tail and making things worse because you are introducing even more variables and increasing the variation. Deming called this “Tampering” and people do it all of the time.

Before you can say “This is better” you have to calculate, based on the amount of variation in the data, how much better the average needs to be before you can say, with some certainty, that this new sample is from a process that is different than the first one.

The more variation there is, the more difference you need to see. The more certainty you want, the more difference you need to see. This is called “statistical significance” and is why you will see reports that seem to show something is better, but seem to be dismissed as a “statistically insignificant difference” between, for example, the trial medication and the placebo.

Unless you are applying statistical tests to the samples, don’t say “the average is better, so the process is better.” The only exception would be if the difference is overwhelmingly obvious. Even then, do the math just to be sure.

I have personally seen a Six Sigma Black Belt(!!) fall into this trap – saying that a process had “improved” based on a shift in the mean of a short sample without applying any kind of statistical test.

As I said, averages have a valuable purpose – when used as part of a robust statistical analysis. But usually that analysis isn’t there, so unless it is, I always want to see the underlying numbers.

Sometimes I hear “We only have the averages.” Sorry, you can’t calculate an average without the individual data points, so maybe we should go dig them out of the spreadsheet or database. They might tell us something.

The Problem with Percentages

Once again, percentages are valuable analysis tools, so long as the underlying information isn’t lost in the process. But there are a couple of scenarios where I always ask people not to use them.

Don’t Make Me Do Math

“We predict this will give us a 23% increase in output.”

That doesn’t tell me a thing about your goal. It’s like saying “Our goal is better output.”

Here is my question:

“How will you know if you have achieved it?”

For me to answer that question for myself, I have to do math. I have to take your current performance, multiply x 1.23 to calculate what your goal is.

If that number is your goal, then just use the number. Don’t make me do math to figure out what your target is.

Same thing for “We expect 4 more units per hour.”

“How many units do you expect per hour?” “How many are you producing now?” (compared to what?)

Indicators of a W.A.G.

How often do you hear something like  “x happens 90 percent of the time”?

I am always suspicious of round numbers because they typically have no analysis behind them. When I hear “75%” or “90%” I am pretty sure it’s just speculation with no data.

These things sound very authoritative and it is easy for the uncertainty to get lost in re-statement. What was a rough estimate ends up being presented as a fact-based prediction.

At Boeing someone once defined numbers like this as “atmospheric extractions.”

If the numbers are important, get real measurements. If they aren’t important, don’t use them.

Bottom Line Advice:

Avoid averages unless they are part of a larger statistical testing process.

Don’t set goals as “percent improvement.” Do the math yourself and calculate the actual value you are shooting for. Compare your actual results against that value and define the gap.

When there is a lot of variation in the number of opportunities for success (or not) during a day, a week, think about something that conveys “x of X opportunities” in addition to a percent. When you have that much variation in your volume, fluctuations in percent of success from one day to the next likely don’t mean very much anyway.

Look at the populations – what was different about the ones that worked vs. the ones that didn’t — rather than just aggregating everything into a percentage.

Be suspicious of round numbers that sound authoritative.

_______________

*These charts are simply independent random numbers with upper and lower bounds on the range. Real data is likely to have something other than a flat distribution, but these make the point.

The Improvement Kata: Next Step and Expected Result

In the Improvement Kata sometimes it helps to think about the outcome desired and then the step required to accomplish it.

A couple of months ago, I gave a tip I’ve learned for helping a coach vet an obstacle.

Another issue I come across frequently is a weak link between the “Next Step” and the “Expected Outcome.”

In the “Five Questions” of the Coaching Kata we have:

What is your next step or experiment?” Here we expect the learner / improver to describe something he is going to do. I’m looking for a coherent statement that includes a subject, verb, object here.

Then we ask “What do you expect?” meaning “What do you expect to happen?” or “What do you expect to learn?” from taking that step?

I want to see that the “Expected Result” is a clear and direct consequence of taking the “Next Step.”

Often, though, the learner struggles a bit with being clear about the expected outcome, or just re-states the next step in the past tense.

While this is the order we ask the questions, sometimes it helps to think about them in reverse.

Reverse the Order

Have the learner first, think about (and then describe) what she is trying to accomplish with this step. Look at the obstacle being addressed, and what was learned from the last step.

Based on those things, ask “what do you want to accomplish with your next step?”

The goal here is to get the learner to think about the desired result. Don’t be surprised if that is still stated as something to do, because we are all conditioned to think in terms of action items, not outcomes.

“What do you need to learn?” sometimes helps.

“I need to learn if ______ will eliminate the problem.” might be a reply.

Even a proposed change to the process usually has “to learn if” as an expected outcome, because we generally don’t know for certain what the outcome will be until we try it.

Have the learner fill in the “Expected Outcome” block.

NOW ask “OK, what do you have to do to ______ (read what is in the expected outcome)?”

PDCA Outcome-Activity

That should get your learner thinking about the actions that will lead to that outcome.

A Verbal Test

A verbal test can be to say “In order to ______ (read the expected outcome), I intend to _____ (read the next step.”

If that makes sense grammatically and logically, it is probably well thought out.

The Destructiveness of “What Can You Improve?”

“What Can You Improve?”

Leaders often ask “What can you improve?” as an empowerment question. In reality, it may have the opposite effect.

I am coming to the belief that “What can you improve?” (about your job, about your process) is possibly one of the most demotivating, disempowering, destructive questions that can be asked.

What can you work on?” is another one of many forms this question takes. “How could you improve this process?” is another. What they all have in common is the psychological trap they set.

Now this really isn’t that much of a problem in a company that has a history of transparency in leadership, comfort with discussing the truth, and no need for excuses or justifications. Then again, those companies tend not to ask these questions straight-on.

But the vast majority of organizations aren’t like that. That doesn’t mean they are unkind. Rather, they operate in an environment where truthfully answering this question is difficult at best.

The Psychological Trap

To answer that question with anything other than trying to guess what you want, implies I have:

  • Thoroughly examined my results and the underlying processes.
  • Identified gaps in performance.
  • Know what to do about those gaps.
  • And haven’t done anything about it until you asked.

This puts me in the position of either defending the status-quo, or saying that I need to improve something that is out of my control – someone else’s process needs improvement so I can do better.

Hint: If you are a leader, and you ask a “What can you improve?” question and get an answer like the above – defending the status-quo or pointing to an outside problem –, there is fear in your organization. Justified or not, the person answering is struggling to maintain the impression that everything they can do is being done. Why do they feel the need to do this? Think about it.

This is especially pervasive in support / staff departments with a charter of influencing how other organizations perform, or in those who must work together with line organizations to succeed in their tasks. In industry this might be maintenance, HR, industrial engineering, or even the “improvement office” (who are often not a  beacon of internal efficiency or effectiveness).

A Bit of Background

When I start working with an organization, we usually start with practicing the basic mechanics of the Improvement Kata in a classroom setting. We then follow up immediately with kick-starting some live improvement cycles so we can begin practical application. Classroom learning really doesn’t do much good unless it is applied immediately.

Applying the Improvement Kata is a lot harder in the real world than it is in the classroom. I could go into a tangential rant on why I think our primary and secondary education system makes it harder, but I’ll save that for another day.

Even though I am as adamant as I can be on the importance of the organization identifying challenges for the new improvers / learners, the reality is that most organizations don’t know how to do this, or at least aren’t comfortable with it.*

As a result, the new improvers often struggle to define a “challenge” for themselves.

They guess – because they haven’t yet studied their process (which is the next step once context is established, they haven’t yet established a target condition (which is the step after that), and therefore, they haven’t identified what improvements they must make to get to the challenge state.

And if that guess is something in someone else’s domain, or worse if the “coach” has something else in mind, they are told “That’s not it,” they guess again, and eventually get defensive or give up.

Now – to be clear, this doesn’t happen every time. But I have seen it enough, across multiple organizations in very different domains that it’s a problem. And it is frustrating for everyone when it happens.

I indirectly addressed this topic a long time ago in “How the Sensei Sees.” Now, though I am talking about my own direct observation of the effect. And I am still learning how to deal with the fallout without becoming part of the problem.

It’s not the learner’s problem. It is a leadership problem.

________

*Dave Kilgore at Continental Automotive had the additional insight that it is important for beginners that this challenge should be something important but not urgent so they don’t feel pressured to jump to an immediate solution. This is a good example of “constancy of purpose” – his priority is developing the skill level for improvement first.

Delivering the Patient Satisfaction Experience

“Our challenge is to improve our patient satisfaction scores.”

This seems to be a fairly common theme as I continue to work in the health care arena.

Background

In the U.S. at least, most major health care operations use one of a couple of major service providers (such as Press Ganey) to survey their patients, and report aggregated patient satisfaction scores to them. Those scores provide a percentile rank of how that facility stacks up against others across various categories. The scores are also made public, and often influence public funding decisions within a region. Thus, they are a big deal.

Chasing the Patient Satisfaction Numbers Doesn’t Work

Here’s the problem. More than a few times I have seen an improver working on a challenge to improve these patient satisfaction numbers. It might be something like “Achieve a 70th percentile score on ___.) with a specific score that has to do with their area.

So far, that’s not a real problem. But what happens next might be.

It is very common to focus solely on the end result, without a lot of thought into the underlying things that drive that result.

Specifically, I have seen more than a couple of cases where a manager is working to directly influence how a patient (customer) will answer the questions on the survey. They parse the question, and try to determine what this word, or that word, actually means to “the patient.” The worst case was trying to introduce fairly heavy handed scripting… “Is there anything I can do for you to be more comfortable?” into every patient interaction.

I certainly can’t speak for the population of patients, but I can say that when I pick up on a scripted phrase, I become very aware of what it is, and it leaves a disingenuous taste.

It’s About the Patient Experience

The patients’ experience is what drives how (and even if) they will answer the questions on these surveys. If their experience was overall favorable, they will be biased to give favorable replies. The opposite is even more true. One bad experience will negatively bias all of their answers.

Here’s the question I ask that sometimes stumps people:

What experience to you want the patient to have?

(If you aren’t in health care, substitute the word “customer” for “patient.”)

If your scores on “Were the staff concerned for my comfort?” are low, think about what experience would give the patient confidence that staff were concerned. Being continuously asked about it with a rote phrase probably isn’t going to do it. But leaving them parked in the hallways with no interaction might be (for example), something that creates discomfort.  (“Comfort” has a psychological, as well as a physical component.) People will put up with a lot of discomfort if they know the higher purpose. It’s hard to make the case for parking the patient in the hallway. That just says “I don’t have anywhere to take you.”

So think deliberately. If everything the patient experienced were something you were doing on purpose, because it contributed to the experience you want the patient to have, what would that look like?

Don’t worry right now about whether that is hard or not. Let go of your internal issues for a while. Just sketch out that awesome “insanely great” patient experience. You don’t have to think of every detail. What are the attributes? What is the flow, from the patient’s perspective – the sequence of events they will experience.

For example, construct a story, told from the patient’s point of view, of coming in for outpatient surgery.

What happens from the time they have their initial consultation until they are on their way home. (And what happens after they get home?) Again, don’t worry about “we can’t do that because…” stuff, we’ll deal with that later.

What experience, what story, would leave the patient with the impression that you are working as a team, that you know what you are doing, that there is a competent process at work to provide safe, effective care and actually care about their experience?

Don’t forget to include your administrative communications in this process – what phone calls do they get? What paperwork do they get? What does crystal-clear billing look like?

Build a block diagram, a story board, of the patients’ ideal flow through the system.

What would a wait-free, smooth flowing experience look like?

Learning From Disney

In Disney theme parks, they make a clear distinction between “On Stage” and “Off Stage.” Their employees (all of them) are referred to as “Cast Members.” Anytime a Cast Member is visible to guests, they are “On Stage.” They are performing. They are part of creating the story, the experience, they want the guest to have.

Meanwhile, behind the scenes, in the tunnels, off stage, are the processes required to create the “On Stage” performance. It’s a show.

The guest experience is designed. Once it is designed, it is created by the process.

Disney’s priorities (in order) are:

  • Safety
  • Courtesy
  • Show
  • Efficiency

Translated, they place putting an a good performance above being efficient. But if pushed, a cast member may break character if required to be courteous. And they will get snippy with someone who persists in doing something unsafe in spite of courteous requests.

What on Earth does this have to do with health care?

Everything. That is if you are trying to create a safe, professional and competent impression to your patients.

What is the Actual Patient Experience?

Now we have a sense of the ideal, it’s time to understand what is really happening. Again, start with the patient’s experience.

What happens at each interaction? What questions are asked? Who asks them? How often are they moved? Where and when are they waiting, and why? 

Use “typical” rather than exceptional cases here. One thing I am seeing is, yes, every case is different but in reality, most are handled within a routine.

Pay attention to the “on stage” part of your process. This is what the patient sees, and what creates their experience.

At the same time, look at the behind-the-scenes “off stage” flow to see what might be causing a less-than-ideal patient flow. For example – The patient’s experience is that he is alone in an exam room waiting, reading Time Magazine for 20 minutes. That is the “on stage” part.

Meanwhile, “back stage” you have a nurse on the phone trying to get the results of tests that were done by another provider. (This is a real-life example.)* (There was also a physician waiting on them!)

Your Processes Create the Patient Experience

(Again, substitute “customer” for “patient” and this becomes an essay for everyone.)

Your Patient Satisfaction scores are driven by the patients’ experience.

The patients’ experience is established by your “on-stage” (patient facing) process.

Your “on-stage” process is the result of your “off-stage” execution.

The people making the improvements need to be challenged, and focused on, creating a specific experience for the patient.

Linking to Policy Deployment

All of that begs the question: Who should make the linkage between process performance and patient satisfaction, because those scores do matter, in a very big way.

Let’s look at this from a policy deployment standpoint.

Certainly Administration (the executives) should be tracking their scores. From their perspective, these are an important (along with patient safety, quality, length-of-stay, financial performance, etc) aspects of how the organization is performing.

They see the overall performance and trends. And they can see how each department is performing.

But the patient’s experience is cross-functional. The patient only sees “the hospital.” He doesn’t see, and doesn’t care, that Admissions, the lab, the Emergency Department, Outpatient Surgery, Environmental Services (who cleans his room) and Radiology are all different departments. The patient doesn’t see, and doesn’t care, that “the clinic” and “the hospital” are separate legal entities.

As part of Policy Deployment, Administration should be establishing operational standards and challenging the Department Directors to meet them. Those standards are based on what Administration believes will move the needle on the patient satisfaction scores. In reality, this is also an experiment. Does this operational standard meet our customer’s expectations?

They also are making sure the Directors are working on the cross-functional interfaces between their departments. (If it isn’t the Directors’ job to do this, whose job is it?)

Key Point: Until you are consistently delivering the product or service, there is little point in trying to change things up. Set a standard, strive to meet it. Once things are somewhat stable, then you can evaluate whether your standard is adequate or not. Think about it… what is the alternative? You have random execution that is randomly working. You don’t know why. You can’t talk to people about performance until they can demonstrate consistent execution.

Summary

Your patient satisfaction scores reflect the experience of the patient.

The patient experience is the outcome of your on stage process performance.

Your on stage process performance is ultimately driven by your back stage process execution.

If you want to improve your patient satisfaction scores, establish the operational standard you want to strive for that you think will improve patient satisfaction.

Then strive to develop a process that meets that operational standard.

THEN you can evaluate whether your process is adequate.

_________

*This was an obstacle in front of a target condition focusing on hitting a standard for “In, Seen and Out” within a specific time frame for routine pre-procedure consultations. They fixed it. Patients no longer have to sit and wait while someone hunts down those test results.