More About Mistake-Proofing

After yesterday’s post about trucks crashing into the famous 11foot8 bridge and mistake proofing, I got the feeling I should drive home my key point that the problem isn’t with the driver, it is with the environment.

As of this writing,  Jürgen has recorded 154 crashes of overheight vehicles into the bridge.

And I’ll put even money that if all of the data were known, this process would pass any test for statistical control and we are getting what we should expect from a stable system. It might not be what we want, but it is what we should expect. (All images are copyright  Jürgen Henn, 11foot8.com)

So addressing the individual incidents probably isn’t a solution. In any case, it is unlikely that any driver will repeat the mistake.* For the TWI folks – this isn’t really a Job Relations type of problem. It might feel like it is, but it isn’t.

In the Factory

I was working with a company with a similar problem. Their inspectors kept missing defects. The response was often to say “I don’t see how she could have missed that!” and even write them up for failure to do something that wasn’t particularly well specified.

But the fact on the ground was, like the bridge, the misses weren’t confined to any particular individuals, any particular shift, any particular anything. People missed things all of the time because the expectations greatly exceeded the limitations of what humans can do for 12 hours (or even one hour). It isn’t an inspection problem. (The reliance on inspection vs. upstream controls is another topic for another day.)

People Work Within a System

It is all too easy to fall into the “bad apple” fallacy and seek out someone who was negligent. It feels good, like we did something about the problem. But the problem will happen again, with someone else. Then I hear frustrated managers start to make disparaging comments about their entire workforce that “doesn’t care” about quality.

I challenged a quality manager to do that inspection job for two hours – not even a complete shift – under the watchful eye of the inspector whose job he was trying to do. Funny – he was a lot slower to assign blame after that experience. He couldn’t keep up.

Deming was pretty clear about the ineffectiveness of exhortation as a way to get better performance. “Be more careful!” might well work for one individual for a short time. “Making an example of someone” might well work for a group for a short time. But there are norms, and the system will return to those norms very quickly. There are simple limits to what humans can focus on and for how long.

The Bridge is a Metaphor

To be clear, the bridge represents a working system, but it is different than what we would find in a company. This is public infrastructure, and the truck drivers that get featured on the videos are not part of a single organization.

This means that you have more control than the city engineers in Durham do. You can establish procedures, ask questions, train people, have them practice, alert them to the Gregson Street Bridge on their route. You can make sure your navigation system routes your trucks around the low bridges. You can support your people so they are less likely to even end up in the situation. All of these are system changes – and that is what it will take to change the outcome.

Change the System: Raising the Bridge

In late 2019 the city of Durham, in coordination with the railroad who owns the bridge, did actually raise the bridge by 8 inches. It is now 11 feet 16 inches (3.76m).

And that is a legitimate approach. Rather than trying to create infallible humans, what can we do to make the system less vulnerable to fallible humans.

While that likely reduced the number of trucks that hit the bridge…


*Caveat: There is one video where a truck seems to avoid the bridge, then circle back around and hit it. And another video where a truck that hit this bridge then proceeded to run into another low bridge with the damaged truck.

KataCon 2020: Billy Taylor on Key Actions

Key Actions vs. Key (Performance) Indicators

Billy Taylor – Photo by Michele Butcher / Lean Frontiers

Another concept Billy brought out in his presentation was the difference between what he calls “Key Actions” (KA) and “Key Indicators” (KI) – often called Key Performance Indicators (KPI).

He actually introduced me (and a couple of other attendees) to the concept the previous evening. (Did I mention that a lot of the rich discussion took place in the lobby bar?)

We use the concept in Toyota Kata, we call them the “process metric” and the “performance metric” but I think Billy’s explanation offers more clarity than I have been able to pull off in the past.

He also ties it back into “what we must practice” to get the outcome we want.

In short, I look at the outcomes (the performance) I want, then ask “What actions, if they were carried out consistently, would give me this performance?” Those are the things that must be tracked, improved, and practiced.

I kind of addressed this concept a few years ago in Delivering the Patient Satisfaction Experience. But I’d like to focus in a little better.

Continuing on the health care theme, a key performance indicator is “hospital acquired infections” – getting sick in the hospital. Everyone agrees that this metric should be as low as possible, ideally zero.

But just tracking the “hospital acquired infections” isn’t going to nudge the needle much. There may be periods when there are improvements if there is emphasis, but year on year these things tend to be frustratingly steady over the long run.

If I ask “What behaviors, what actions, should we take to diminish opportunities for these infections?” then one thing pops right up on top: Anyone interacting with a patient must wash (or sanitize) their hands before doing so. Every. Single. Time. That action alone would have a dramatic and measurable impact.

It is so important that some systems have automated tracking to ensure compliance with this simple rule. (It is amazing to me that, in general, some of the worst offenders are physicians, but that is a rant for another day.)

Key Action: Wash your hands. Key Indicator: Hospital Acquired Infections.

OK – what about industry?

“Our machine downtime is too high. We need to improve our availability.” Key Indicator, but not directly actionable. What actions, if we take them consistently, do we believe are critical to reliable equipment?

Now we can track those. What are the critical-to-reliability things that must be checked every shift? Are they checked? How do you know? Do you track misses?

How about your preventative maintenance schedule?

Is the machine in configuration? Or are there improvised repairs in place? Why?

These are behaviors, actions, that relate directly to the availability of the equipment.

Together, they form a hypothesis: “If we carry out these actions (and know we did), then we predict this KPI will improve.” For this to work, though, we have to test whether or not the actions were carried out AND test whether or not the KPI needle moves over time.

One thing I would add: Focus on what people should do. Not so much on things they should not do. It is a lot easier to get a new habit into place than it is to stamp out an existing one. Working to replace an undesired action with a desired action is a lot easier as well.

The things that keep people from carrying out the Key Actions are obstacles. Now we can engage the Improvement Kata process and get to work.

TWI comes into play as well. “Are we carrying out the actions as we should?” It is all to easy to tell someone to do something and assume they know how, or assume that the way they do it is the way you have in mind. Trust, then verify.

Notes from the 2020 TWI Summit – Part 2

Tyson Ortiz

TWI Job Instruction Card

Tyson zeroed right in on one of the biggest problems with “training” – getting people to adopt the new process or method after we have taught it to them.

Compounding this was that, in his example, the training was TWI Job Instruction – how to train. Tyson took a quick show-of-hands poll and informally confirmed his hypothesis that most people who take the TWI Job Instruction 10 hour course are already engaged in training and teaching.

This means that they have to do more than learn a new habit – one which will feel awkward to them at first. They also have to unlearn their current way of doing things – a way that is likely comfortable and familiar to them. To paraphrase from a slide of mine that seems to keep coming up: This. Is. Hard.

Taking what he has learned from Toyota Kata, Tyson saw the 4 Step Method for what it is: A routine for practice, not the end-all. For that to work, there must be actual practice using the routine. The 10 hour class is telling them about it* – and telling alone is not enough!

What Tyson did was add structured follow-on practice with real work, but not real training where the participants can practice, make mistakes, and learn in a safe environment. Then they move to live environments, but are still being coached. Then they are graduated and put on their own.

Transition of a learner through Recruiting, the 10 hour JI Class, a "safe zone" practice, "real" practice, then graduation.
Clipped from Tyson’s presentation.

Another key is that passing each stage is based on performance, not a time line. It is up to the coach, since the coach is the teacher, and “If the student hasn’t learned, the teacher hasn’t taught.”


*Yes, the class includes demonstrating the four steps – but each participant typically only gets one repetition, hardly enough for us to know that they know.

Roger Bilas

Roger actually built on the theme that Tyson was developing – the process of getting Job Instruction incorporated into the daily routine of the organization.

We often call this “managing change” or more cynically “overcoming resistance” but I think both Roger and Tyson are operating at a much more fundamental and human level. It’s called paying attention to what is causing stress and fear and make sure you deal with it effectively and with empathy.

And it is empathy where Roger begins.

He used the Stanford design school model to experiment his way toward a solution that used the framework of Job Instruction in a way that worked for the particular situation. And isn’t that the whole idea?

The design thinking model steps: Empathize, Define, Ideate, Prototype, Test
Clipped from Roger’s presentation.

As I was listening, I scribbled a note in the margin: “this is Menlo’s model” – the design process that Menlo Innovations. It isn’t really – this model uses different words. But the structure, intent, purpose is the same and is followed by all robust design and product development processes.

Roger was operating in an environment that was unfriendly to paper, had lots of high-variety and low-volume tasks that people had to get right.

Once he understood that he had motivated people in a tough situation, they began working together to develop simple solutions that worked – starting with simple sketches and hand-written notes on laminated cards.

Iterating through, always asking “What small step can we take?” toward the goal, always asking “How can we test that assumption or idea?” they converged on a solution that worked really well.

Not surprisingly, it was very visual and simple, and captured “Key Points” from the Job Breakdown.

There was a lot more good stuff at the TWI Summit. I’ll cover my own keynote separately. And I missed the 3 hour “Experiential” sessions because I was presenting one. And for the afternoon of Day 2 I was attending Oscar Roche’s version of a Toyota Kata class that follows the 5 x 2 hour structure of the classic TWI JI, JR, JM classes.

Thus, the next big thing for me to report on will be KataCon – which will be my next post.

Notes from the 2020 TWI Summit – Part 1

Photo by Michele Butcher of Lean Frontiers

Last week (February 17-20) I attended (and presented at) the TWI and Toyota Kata summits put on by my friends at Lean Frontiers. As always, I took a few notes and I would like to share some of those notes and thoughts with you here.

To be clear, what follows are my impressions and thoughts that were sparked by some of the presentations. I am not trying to be a reporter here, just catch my own reflections.

Martha Purrier

Martha Purrier, a Director of Nursing at Virginia Mason Medical Center in Seattle, talked about “auditing standard work,” though in reality I think her process was more about auditing the outcomes of standard work. More about that in a bit.

My interpretation of the problem: Traditional “audits” are infrequent, and tend to be time consuming for those doing them because there is an attempt to make them comprehensive.

Infrequent checks are not particularly effective at preventing drift from the standard. Instead they tend to find large gaps that need to be corrected. This can easily turn into a game of “gotcha” rather than a process of building habits. What we want to do is build habits.

Habits are built in small steps, each reinforced until it is anchored.

Make it Easy: Short and Simple Checklists

Martha’s organization created short checklists of critical “Key Points” (from TWI Job Instruction) that were critical to the standard they wanted to maintain.

Audit Check Card. Photo from Martha Purrier’s Presentation

As you can see, this is a quick and simple check to see if the contents and organization of a supply cart meets the standard.

But what really caught my attention was how they are triggering the audits.

The Key: Reliable Prompt for Action

This is a pretty typical work task board. There is a row for each person or team. In this case the columns look like they represent days, but they could just as easily represent blocks of time during the day, depending on how granular you want your tracking to be. At some point these start to become a heijunka box, which serves the same purpose.

You can see the yellow bordered audit cards on there. Martha said that when a task is complete, it is moved to a “Done” column that is out of frame to the right.

Here is what is awesome about this: It gives you the ability to “pull” checks according to need.

Do you have a new process that you want multiple people to check during the course of the week? Then put the check card for that task in multiple rows at staggered times.

Do you want to go broad over a group of related checks? Then put different checks on the board.

Who should do the checks? Whoever you assign it to. Totally flexible. Do you want to trigger a self-audit? Then assign the card to the person who does the task being checked, with the expectation that they self-correct.

Do you want to bring a new supervisor up to speed quickly? Assign multiple audits to her, then assign follow-up audits to someone else.

Making it Better: Follow-up Breakdowns

If we don’t want audits to simply become lists of stuff to fix, there has to be some process of following up on why something needed correction.

Martha’s organization introduced a simple check-form that lists “Barriers to Standard Work – (check all that apply)” and provides space to list countermeasures taken.

The lists includes the usual suspects such as:

  • Can’t find it
  • No longer relevant
  • Not enough detail
  • etc.

but also some that are often unspoken even though they happen in real life:

  • Lack of enthusiasm to continue or improve
  • Mutiny
  • Relaxed after training – drift

If a large part of the organization is pushing back on something (mutiny), then the leadership needs to dig in deep and understand why. To continue in our TWI theme, this is a great time to dig into your Job Relations process.

Standard Work vs. “Standards”

In my past post, Troubleshooting by Defining Standards, I made a distinction between defining the outcome you are trying to achieve and, among other things, the way the work must be done to accomplish that outcome.

When I think of “standard work” I am generally looking for a specification of the steps that must be performed, the order for those steps, usually the timing (when, how long) as well as the result. In other words, the standard for the work, not just the outcome or result.

To verify or audit “standard work” I have to watch the work as it is actually being performed, not simply check whether the machine was cleaned to spec.

Now, to be clear, I LOVE this simple audit process. It is an awesome way to quickly follow-up and make sure that something was done, and that the patient or customer-facing results are what we intend. It is flexible in that it can quickly and fluidly be adjusted to what we must pay attention to today.

I realize I am quibbling over words here. And every organization is free to have its own meanings for jargon terms. But when I hear the team “standard work” I am looking for the actual work flow as well as the result. YMMV.

This post got long enough that I am going to let it stand on its own. More to follow.