A Lean Leadership Pocket Card

I was going through some old files and came across a pocket card we handed out back in 2003 or so. It was used in conjunction with our “how to walk the gemba” coaching sessions that we did with the lean staff, and then taught them to do with leaders.

There is a pretty long backstory, some of it is summarized in Earl’s recollection on this old post: Genchi Genbutsu in a Warehouse as well as here: The Chalk Circle – Continued.

A lot has happened, a lot has been learned since then. Toyota Kata has been published, and that alone has focused my technique considerably (to say the least).

Nevertheless, I think the elements on these little cards are valuable things to keep in mind.

With that being said, a caveat: Lists like this run the risk of becoming dogma. They aren’t. There are lots of lists like this out there, and the vast majority are very good. The key here is something that a leader or team member can refer to as a reminder that may bias a decision in the right direction. It is the direction that matters, not the reminders.

Fundamentals

The fundamentals are based on the “Rules-in-Use” from Decoding the DNA of the Toyota Production System, a landmark HBR article by Steve Spear and H. Kent Bowen. The article, in turn, summarizes (and slightly updates) Spear’s findings from his PhD work studying Toyota.

A. All work highly specified as to content, sequence, timing, and outcome.

B. Every customer-supplier connection is simple and direct.

C. The path for every product is simple and direct.

D. All improvements are made using PDCA process.

What we left off, though, is that in each of those rules there is a second one: That all of these systems are set up to be “self diagnostic” – meaning there are clear indications that immediately alert the front line people if:

The work deviates from what was specified.
The connection between a customer and supplying process is anything other than specified.
The path a product follows deviates from the route specified.
Improvements are made outside of a rigorous PDCA (experimental) process.

In other words, the purpose of the rules is to be able to see when we break them, or cannot follow them, so we trigger action.

To put this into Toyota Kata-speak – every process is set up as a target condition that is being run as an experiment – even the process of improvement itself!

Every time there is a disruption – something that keeps the process from running the way it is supposed to – we have discovered an obstacle. That obstacle must first be contained to protect the team members and community (safety) and to protect the customer (quality). Then goes into the obstacle parking lot, and addressed in turn.

If you think about it, the Improvement Kata simply gives us much more rigor to (D).

This ties to the next sections.

Key Leadership Behaviors

Note that this is behaviors. These are things we want leaders to actually strive to do themselves, not just “support.” It was the job of the continuous improvement people to nudge, coach, assist the leaders to move in these directions. It was our job to teach our continuous improvement people how to do that coaching and assisting – beyond just running kaizen events that implement tools.

A. PDCA Thinking

Today we would use Toyota Kata to teach this. But the same structure drove our questioning back then.

B. Four Rules:

1. Safety First

Even though this should be obvious, it is much more common that people are tacitly, or even directly, asked to overlook safety issues for the sake of production. I remember walking through a facility with a group of managers on the way to the area we were going to see. Paul stopped dead in his tracks in front of a puddle on the floor. He was demonstrating just how easy it was for the leadership to walk right past things that should be attended to. And in doing so, they were sending the message – loud and clear in their silence – that having a puddle on the floor was OK.

2. Make a Rule, Keep a Rule

This is a more general instance of Rule #1. But the it is more subtle than it may seem on the surface. Most people immediately interpret this as enforcing organizational discipline, but in reality it is about managerial discipline.

Nearly every organization has a gap between “the rules” and how things really are day-to-day. Sometimes that gap is small. Sometimes it is huge.

Often “rules” are enforced arbitrarily, such as only cases where a violation led to a bigger problem of some kind. Here’s an example: Say your plant has a set of rules about how fork trucks are to be operated – speed limits, staying out of marked pedestrian lanes, etc. But in general the operators hurry, cut a corner now and then. And these violations are typically overlooked… until there is some kind of incident. Then the operator gets written up for “breaking the rules” that everyone breaks every day – and management tacitly encourages people to break every day by focusing on results rather than process.

When we say “make a rule / keep a rule” what we mean is if you aren’t willing to insist on a rule being followed consistently, then take the rule off the books. And if you are uncomfortable taking the rule off the books, then your only option is to develop something that you can stand behind. It might be simple mistake proofing, like physical barriers between forklift aisles and pedestrian aisles. But if you are going to make the rule, then find a way to keep the rule.

Do you have “standard work” documents that are rarely followed? Stop pretending you have standards or rules about how the work is done. Throw them away if you aren’t willing to train to them, mistake proof to them and reinforce following them.

3. Simple is Best

Simply, bias heavily toward the simplest solution that works. The fewest, simplest procedures. The simplest process flow. Complexity hides problems. “Telling people” by the way, is usually less simple than a physical change to the work environment that guides behavior. See above.

4. Small Steps

Again, Toyota Kata’s teaching covers this pretty well today. The key is that by taking small steps, verifying that they work, and anchoring them into practice before taking the next ensures that each step we take has a stable foundation under it.

The alternative would be to make many changes at once in the name of going faster.

We emphasized here that “small steps” does not equal “slow steps.” It is possible to take small steps quickly, and we found that in general doing so was faster than making big leaps. Getting big changes dialed in often required backing out and implementing one thing at a time anyway – just to troubleshoot! See “Gall’s Law” which states:

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.
John Gall, author of Systematics

and sums this up nicely.

C. Ask “Why, what, where, when, who, and how” in that order.

Here we borrowed the sequence from TWI Job Methods. The first two questions challenge whether a process step is even necessary: Why is it necessary? What is its purpose? To paraphrase Elon Musk, the greatest waste of time is improving something that shouldn’t even exist.

Then: Where is the best place? and When is the best time? These questions might nudge thinking about combining steps and further simplifying the process.

And finally we can ask Who is the best person? and “How” is the best method? The key point here is until we have the minimum possible steps in the simplest possible sequence, and understand the cycle times, it doesn’t make sense balance the work cycle or work on improving things.

Come to think about it – perhaps we should ask “How?” before we ask “Who” since improving the method will change the cycle times and may well inform out decisions about the work balance. Hmmm… I’ll have to think about that. Any thoughts from the TWI gurus?

D. Ask Why 5 Times

Honestly, this was a legacy of the times. Unfortunately it suggests that you can arrive at a root cause simply by repeatedly asking “Why?” and writing down the ~~excuses~~ answers that are generated. In reality problem solving involves multiple possible causes at each level, and each must be investigated. I talked about this in a post way back in 2008: Not Just Asking Why – Five Investigations.

E. Go and see.

Go and see for yourself. Taking this into today’s practice, I think it is something that the Toyota Kata community might emphasize a little more. We tend to ask the question “When can we go and see what we have learned…?” but all too often the answer to “What have you learned?” is a discussion at the board rather than actually going and observing. Hopefully the board is close to where the improvement work is being done. Key point for coaches: If the learner can’t show you and explain until you understand, it is likely the learner’s understanding could be deeper.

As You Walk The Workplace:

Check:

perhaps we should have said “Ask…” rather than “Check” but asking and observing are ways to “check.” All of the below are things that the leader walking the workplace must verify by testing the knowledge of the people doing the work.

A. How should the work be done? Content, Sequence, Timing, Outcome

This is another nod to the research of Steven Spear. The key point here is that before you can ask any of the following questions, you have to have a crisp and precise of what “good” looks like. In this paradigm, all processes are target conditions. And as the work is being done, we are actively searching for obstacles so we can work to make the work smoother and more consistent.

In other words, “What should be happening?” and “How do you know?”

Do the people doing the work understand the standard process as it should be done?

A few months ago I went into some depth on this here: Troubleshooting by Defining Standards. That probably isn’t the best title in retrospect, but there are too many links out there that I don’t want to break by changing it.

B. How do you know it is being done correctly?

Today I ask this question differently. I ask some version of “What is actually happening?” followed by “How can you tell?” We want to know if the people doing the work have a way to compare what they are actually doing against the standard.

C. How do you know the outcome is free of defects?

So, question B asks about consistency of the process, and question C asks about the outcome. Does the team member have a way to positively verify that the outcome is defect-free?

D. What do you do if you have a problem?

Again, we are checking if there is a defined process for escalating a problem. And we define “problem” as any deviation from the standard, or any ambiguity in what should be (or is) happening. We want someone to know, and act, on this, and the only way that is going to happen is to escalate the problem.

We want this process to be as rigorous and structured as the value-adding work.

And we want as much care put into designing production process as was put into designing the product itself. All too often great care and a lot of engineering time goes into product design, and only a casual pass is made at designing and testing the process.

Even better if these are done simultaneously where one informs the other.

For Abnormal Conditions:

ACT:

These are actions that the leader must take if she finds something that isn’t “as it should be” in the course of the CHECK questions above. Key Point: These are leadership actions. That doesn’t mean that the leaders personally carry them out, but the leaders are personally responsible for ensuring that these things are done – and checking again.

That is the only way I know of to prevent the process from continuing to erode.

A. Immediately follow up to restore the standard.

If it isn’t possible to get the intended standard into back place, then get a temporary countermeasure into place that ensures safety and quality.

B. Determine the cause of erosion.

We are talking about process erosion here, with the assumption that something knocked the process off its designed standard. Some obstacle has been discovered, we have to better understand what it is – at least enough to get it documented.

C. Develop and apply countermeasure.

Here we may have to run experiments against this newly discovered obstacle and figure out how to make the process more robust.

That is the end of the little card. But I want to point out that we didn’t just hand these out. You got one of these cards after time paired with a coach on the shop floor practicing answering and asking these questions. Only after you demonstrated the skill did you get the card – just as a reminder, not as a detailed reference. This exercise was inspired by a few of us who had experiences “in the chalk circle” especially with Japanese senseis who had been direct reports to Taiichi Ohno.

We piloted and developed this process on a very patient and willing senior executive – but that is another story for another day. (Thank you once again, Charlie. I learned more from you than you will probably ever realize.)

May 28, 2020May 28, 2020

More About Mistake-Proofing

After yesterday’s post about trucks crashing into the famous 11foot8 bridge and mistake proofing, I got the feeling I should drive home my key point that the problem isn’t with the driver, it is with the environment.

As of this writing, Jürgen has recorded 154 crashes of overheight vehicles into the bridge.

And I’ll put even money that if all of the data were known, this process would pass any test for statistical control and we are getting what we should expect from a stable system. It might not be what we want, but it is what we should expect. (All images are copyright Jürgen Henn, 11foot8.com)

So addressing the individual incidents probably isn’t a solution. In any case, it is unlikely that any driver will repeat the mistake.* For the TWI folks – this isn’t really a Job Relations type of problem. It might feel like it is, but it isn’t.

In the Factory

I was working with a company with a similar problem. Their inspectors kept missing defects. The response was often to say “I don’t see how she could have missed that!” and even write them up for failure to do something that wasn’t particularly well specified.

But the fact on the ground was, like the bridge, the misses weren’t confined to any particular individuals, any particular shift, any particular anything. People missed things all of the time because the expectations greatly exceeded the limitations of what humans can do for 12 hours (or even one hour). It isn’t an inspection problem. (The reliance on inspection vs. upstream controls is another topic for another day.)

People Work Within a System

It is all too easy to fall into the “bad apple” fallacy and seek out someone who was negligent. It feels good, like we did something about the problem. But the problem will happen again, with someone else. Then I hear frustrated managers start to make disparaging comments about their entire workforce that “doesn’t care” about quality.

I challenged a quality manager to do that inspection job for two hours – not even a complete shift – under the watchful eye of the inspector whose job he was trying to do. Funny – he was a lot slower to assign blame after that experience. He couldn’t keep up.

Deming was pretty clear about the ineffectiveness of exhortation as a way to get better performance. “Be more careful!” might well work for one individual for a short time. “Making an example of someone” might well work for a group for a short time. But there are norms, and the system will return to those norms very quickly. There are simple limits to what humans can focus on and for how long.

The Bridge is a Metaphor

To be clear, the bridge represents a working system, but it is different than what we would find in a company. This is public infrastructure, and the truck drivers that get featured on the videos are not part of a single organization.

This means that you have more control than the city engineers in Durham do. You can establish procedures, ask questions, train people, have them practice, alert them to the Gregson Street Bridge on their route. You can make sure your navigation system routes your trucks around the low bridges. You can support your people so they are less likely to even end up in the situation. All of these are system changes – and that is what it will take to change the outcome.

Change the System: Raising the Bridge

In late 2019 the city of Durham, in coordination with the railroad who owns the bridge, did actually raise the bridge by 8 inches. It is now 11 feet 16 inches (3.76m).

And that is a legitimate approach. Rather than trying to create infallible humans, what can we do to make the system less vulnerable to fallible humans.

While that likely reduced the number of trucks that hit the bridge…

*Caveat: There is one video where a truck seems to avoid the bridge, then circle back around and hit it. And another video where a truck that hit this bridge then proceeded to run into another low bridge with the damaged truck.

May 27, 2020May 31, 2020

Mistake Proofing – Getting People’s Attention

Besides being a great source of schadenfreude, Jürgen Henn’s website, 11foot8.com offers some great insight in the difficulty of effective mistake-proofing.

Background

The clearance between the road and the railroad bridge at 201 Gregson Street in Durham, North Carolina is officially 11 feet 8 inches (3.55 meters).

A typical rental box truck is about 12 feet high (3.65 meters).

The result:

Penske rental truck smashed under low bridge.

The more astute of you may have noticed the “OVERHEIGHT MUST TURN” sign just above the smashed truck.

Well before the truck approaches the intersection, it passes under a height sensor. If the vehicle is overheight, the OVERHEIGHT MUST TURN sign comes on and starts to flash, and the traffic lights turn red.

While this effort did mitigate the problem, obviously there are drivers who simply do not notice. Lots of them. Jürgen has cameras trained on the intersection and captures over a dozen crashes in a typical year. As a result, his website has attracted international attention. See the short documentary here: http://11foot8.com/about/the-documentary/

Familiar Tasks = “In the Zone”

The human brain is an amazing thing. It also works by deceiving us. It creates the illusion of complete awareness of the things around us when, in reality, we are simply aware of a model our brains have constructed of what we perceive to be there.

It is also an amazing engine at engaging actions based on pattern matching. For example –

In sessions I facilitate, I routinely ask people if they can recall a time when they arrived home after work but realized they didn’t remember driving the route. Nearly every hand in the room goes up. That is pretty amazing because driving is an incredibly complex task. But, assuming you get a license in your mid-teens, by the time you are in your early 20s, most people don’t give it much thought. (There is a reason your insurance rates drop when you turn 25.)

It has become a hard-wired neural pattern, a series of habits, a programmed set of responses that are operating below below the conscious and deliberate thinking part of the brain. This is really good because this process is much faster and more responsive than the alternative. Think about how it felt to be driving when you were just learning – or how it feels to be doing anything new when you are just learning.

The downside to this amazing programming is that things that don’t jar us into consciousness often go unnoticed.

And the more familiar, the more expert, we are with the task at hand, the more likely this will happen.

Levels of Mistake Proofing

I like to talk about three levels of mistake-proofing. Four actually, if you count Zero as a level.

Level 0 – the task must be performed correctly from memory.
Level 1 – there is a discrete task of checking build into the job.
Level 2 – there is an active indication when a mistake is about to be made (or has just been made and there is still time to correct without consequences).
Level 3 – there is active prevention that gets in the way of making the mistake.

The OVERHEIGHT MUST TURN sign is a Level 2. The driver has already passed signs informing him of the bridge clearance about 100m before the bridge.

The height sensors are on the poles just past the speed limit signs.

Low clearance signs about 100m before the bridge. (Google Street View)

Then in the next block, there is another sign telling the driver to turn:

Approaching the last intersection (Google Street View)

And finally the light changes and the OVERHEIGHT MUST TURN sign illuminates.

And still they miss it.

(Direct YouTube link for the email readers: https://youtu.be/I0BJmC6u7MU)

To be clear, the vast majority of truck drivers don’t hit the bridge. But over a dozen a year do. (There were more before the flashing sign was put in.)

“Pay Attention!” = Fundamental Attribution Error

As we watch Jürgen’s videos it is easy to assign character attributes to the drivers who hit the bridge. If only they were better drivers. (The vast majority people who are asked to rate their driving skill will say they are “above average” by the way. Think about that.)

But it is human nature to get lulled into a bit of complacency when engaging a routine task- and driving is a routine task in spite of the complexity.

Now – think about the people in your organization. How many opportunities to they have to make a mistake every day… every hour… in the course of their work? How many of those possible mistakes have serious or expensive consequences?

They are experts at what they do. And they don’t make many mistakes. And they usually catch themselves before there is a problem. But the Law of Large Numbers says “Given enough opportunities, the unlikely is inevitable.”

Can anyone reading this honestly say they have never inadvertently run a stop sign? Yet accidents rarely occur. Now and then there is some panic braking, and rarely there is a collision.

Yet it is really easy to single out the person who:

Performs the work the same way, in the same conditions as everyone else.

Makes the same mistakes, just as often, as everyone else.

Just like everyone else, usually they are caught in time, or other conditions aren’t present for a bad outcome.

But this time everything lined up and BANG – the defective product got missed, or the leak wasn’t noticed before the sump went dry.

Then that person gets singled out for “not following procedure” when nobody follows procedure.

Job Instruction “Key Points” = Opportunity

In a Job Breakdown for TWI Job Instruction we assign a “Key Point” as something the learner must remember specifically because it:

Might result in a hazard – injure the worker or someone else.
“Make or break the job” – if something isn’t done a specific way, the task fails.
Is a “knack” or technique that makes it easier to do.

Those first two are red flags. You are asking your team member to memorize critical to safety and critical to quality tasks. My challenge: How many of those key points can you “mistake proof” out of the job breakdown?

Finally – rather than the sensors and flashing lit signs, there is this approach from “somewhere on the Internet.”

Warning Sign: If you hit this sign, you will hit that bridge.

That is awesome because even if the driver doesn’t see the sign, the BANG! may get his attention in time for it to register and stop. In the Durham, NC example above, apparently this wouldn’t work because there are legitimate reasons for trucks to come down the street up to the intersection just before the bridge. After that, it is really too late unless the driver is already aware that he has to turn.

Oh – and by the way – you can get souvenirs from the Gregson Street Bridge at Jürgen’s store here: https://squareup.com/store/11foot8-dot-com/ 🙂

There is a follow-up to this post here: https://theleanthinker.com/2020/05/28/more-about-mistake-proofing/

August 16, 2018January 9, 2019

Software Rules

“Turn off the radio, Hal.” “I’m sorry, Mark, I’m afraid I can’t do that.”

A couple of months ago I got the news that my 1995 Toyota Truck with 280,000 miles on it would require an engine rebuild plus a bunch of other stuff I wanted it to continue to be a reliable daily driver and for trips to Portland, etc.

I’ll relate the reason I didn’t replace it in-kind with a Toyota Tacoma in another post.

So now, skipping over the rest of the story, I am the owner of a new Jeep JL Wrangler.

One of the things that has happened since my 1995 Toyota was built is the introduction of electronics into vehicles. My truck’s OEM radio had an on/off/volume knob and a station tuner, plus some presets. (And the optional cassette deck.)

Now I have a 7” touch screen with all kinds of features.

One of those features is a radio. But search as I might, I couldn’t figure out how to just turn it off. I could mute it after I started the vehicle. But no matter what settings I had, when I next started the vehicle, the radio dutifully came on. As a last resort, I went to the OEM “Help” page:

US Customer Service – Chrysler Brand Site

Brief Description: How do I turn off the radio?

Comments: Every time I start the vehicle, the radio comes on. How do I stop that from happening? I can’t see any setting to prevent this.

VIN: 1CHJXDG3Wxxxxxx

Mileage: 61

This is the response I got:

On Tue, Jul 10, 2018, 12:52 PM customerassist
<customerassist@chrysler.com> wrote:
Dear Mark,
Thank you for contacting the Jeep Customer Assistance Center.

Radio mode is the default when the radio is booted up. Sadly there is no option to adjust what mode the radio will start in when the vehicle is started. This ensures the radio is in the correct mode to allow for updates each time the vehicle starts.

Thank you again for your email. Should you require additional
assistance, or have any new information to provide, please reply to this email message or call 1-877-I-AM-JEEP (1-877-426-5337).
Sincerely,
James
Customer Service Representative
Jeep Customer Assistance Center
For any future communications related to this email, please refer to the following information:
REFERENCE NUMBER: 3xxxxxxx
EMAIL CASE NUMBER: 3xxxxxx

Well, that was interesting. To be crystal clear, James is doing a great job. It is obvious he read my original query, and he didn’t just send a cut-and-paste response that is all too common. Still… just to make sure I was understanding this correctly, I responded:

Perhaps I wasn’t clear about what problem I am seeking to solve.

I am looking for how to start the vehicle and not have sound automatically start coming out of the speakers. The system does not remember “mute” for example.

So far the only thing I have found is to tune the radio to an empty frequency.

I am certain there is not an intention to force the customer to hear the radio every time the radio is started.

And get a deeper, more personal, response back:

Dear Mark,
Thank you for contacting the Jeep Customer Assistance Center.

Honestly the radio tuned to dead air is the best solution for this
concern I have as yet heard. I can confirm that this is in fact the way the radio was designed and it is most likely operating as intended. The radio will start in radio mode and the volume will be set to mid range to ensure the kids do not leave it cranked up too high for you.

I also feel that they dropped the ball a bit on this one as the radio will default to radio mode even if you were using a connected device when you left the vehicle outside of radio mode.

I have recorded your comments on the file for further review and we thank you for the time and effort it took to bring this information to our attention.

Thank you again for your email. Should you require additional assistance, or have any new information to provide, please reply to this email message or call 1-877-I-AM-JEEP (1-877-426-5337).
Sincerely,
James

James didn’t write design, spec, or write the software. It is clear from his response that if he did, it wouldn’t include this “feature.” And I think I can interpret his response to strongly hint that this isn’t the first time they have heard about this little issue.

So I am thinking now of the mission of Menlo Innovations and my friends there: “End human suffering in the world as it relates to technology.” Perhaps this doesn’t rise to the level of “human suffering” but – seriously?

Kudos, though to James at Jeep Customer Service for his empathetic response that shows he actually read my email. So FCA – you got that part right. Now… let’s talk about the user experience with your software. Looking at the Jeep brand’s core values, “Freedom, Adventure, Authenticity and Passion” I’d say you came up a little short on the “Freedom” part – I just want to be free to turn of the radio. 😉

I promised I’d be writing about leadership and coaching at the end of last post. That is coming, I just wanted to get this one out of the “drafts” list first.

August 2, 2016January 9, 2019

Team Member Saves

Now and then one of your team members makes a great save. They catch something that could have caused a defect, an accident, or done harm in some way.

Often we celebrate these saves, sometimes informally, sometimes formally. And that is well and appropriate.

But let’s make sure we are celebrating for the right reasons.

The save isn’t what should be celebrated.

Rather, the celebration should be a big THANK YOU for finding a gap in your process.

Somehow the process is capable of producing a defect, resulting in an accident, or doing harm. Your team member noticed that.

We usually just celebrate correcting the immediate problem.

But What is preventing exactly the same thing from happening tomorrow?*

That front-line customer-facing team member is your last line of defense.

They only get an opportunity to make a “save” when every other point in the process has failed to detect the problem.

Given enough “shots” at this front line team-member, sooner or later, one is going to get through.

What happens then? Is the inverse logic applied? “You should have caught that.”

Perhaps, but where in the process was the problem actually created?

Somewhere, long before this diving catch, there is an instant when the process went from operating safely and defect free to creating an opportunity, an opening, for a problem to pass undetected.

Where and when was that moment?

Or is that how the process normally operates, and we are just lucky most of the time?

Dig in, think about it.

And thank that team member for saving you, but don’t count on it every time.

———-

*Thanks to Craig for this great way to sum up the question.

July 14, 2016January 9, 2019

Averages, Percentages and Math

As a general rule I strongly discourage the use of averages and “percent improvement” (or reduction) type metrics for process improvement.

The Problem with Averages

Averages can be very useful when used as part of a rigorous statistical analysis. Most people don’t do that. They simply dump all of their data into a simple arithmetic mean, and determine a score of sorts for how well the process is doing.

The Average Trap

There is a target value. Let’s say it is 15. Units could be anything you want. In this example, if we exceed 15, we’re good. Under 15, not good.

“Our goal is 15, and our average performance is 20.”

Awesome, right?

Take a look at those two run charts below*. They both have an average of 20.

On the first one, 100% of the data points meet or exceed the goal of 15.

On the one below, 11 points miss the goal

But the both have an average 5 points over the goal.

In this case, the “average” really gives you almost no information. I would have them measure hits and misses, not averages. The data here is contrived, but the example I am citing is something I have seen multiple times.

Why? Most people learned how to calculate an arithmetic mean in junior high school. It’s easy. It’s easier to put down a single number than to build a run chart and look at every data point. And once that single number is calculated, the data are often thrown away.

Be suspicious when you hear “averages” used as a performance measurement.

Using Averages Correctly

(If you understand elementary statistical testing you can skip this part… except I’ve experts who should have known better fall into the trap I am about to describe, so maybe you shouldn’t skip it after all.)

In spite of what I said above, there are occasions when using an average as a goal or as part of a target condition makes sense.

A process running over time produces a range of values that fall into a distribution of some kind.

Any sample you take is just that – a sample. Take a big enough sample, and you can become reasonably confident that the average of your sample represents (meaning is close to) the average of everything.

The move variation there is, the bigger sample you need to gain the same level of certainty (which is really expressed as the probability you are wrong).

The more certain you want to be, the bigger sample you need.

Let’s say you’ve done that. So now you have an average (usually a mean) value.

Since you are (presumably) trying to improve the performance, you are trying to shift that mean – to change the average to a higher or lower value.

BUT remember there is variation. If you take a second sample of data from an unchanged process and calculate that sample’s average, YOU WILL GET A DIFFERENT AVERAGE. It might be higher than the first sample, it might be lower, but the likelihood that it will exactly the same is very, very small.

The averages will be different even if you haven’t changed anything.

You can’t just look at the two numbers and say “It’s better.” If you try, the NEXT sample you take might look worse. Or it might not. Or it might look better, and you will congratulate yourself.

If you start turning knobs in response, you are chasing your tail and making things worse because you are introducing even more variables and increasing the variation. Deming called this “Tampering” and people do it all of the time.

Before you can say “This is better” you have to calculate, based on the amount of variation in the data, how much better the average needs to be before you can say, with some certainty, that this new sample is from a process that is different than the first one.

The more variation there is, the more difference you need to see. The more certainty you want, the more difference you need to see. This is called “statistical significance” and is why you will see reports that seem to show something is better, but seem to be dismissed as a “statistically insignificant difference” between, for example, the trial medication and the placebo.

Unless you are applying statistical tests to the samples, don’t say “the average is better, so the process is better.” The only exception would be if the difference is overwhelmingly obvious. Even then, do the math just to be sure.

I have personally seen a Six Sigma Black Belt(!!) fall into this trap – saying that a process had “improved” based on a shift in the mean of a short sample without applying any kind of statistical test.

As I said, averages have a valuable purpose – when used as part of a robust statistical analysis. But usually that analysis isn’t there, so unless it is, I always want to see the underlying numbers.

Sometimes I hear “We only have the averages.” Sorry, you can’t calculate an average without the individual data points, so maybe we should go dig them out of the spreadsheet or database. They might tell us something.

The Problem with Percentages

Once again, percentages are valuable analysis tools, so long as the underlying information isn’t lost in the process. But there are a couple of scenarios where I always ask people not to use them.

Don’t Make Me Do Math

“We predict this will give us a 23% increase in output.”

That doesn’t tell me a thing about your goal. It’s like saying “Our goal is better output.”

Here is my question:

“How will you know if you have achieved it?”

For me to answer that question for myself, I have to do math. I have to take your current performance, multiply x 1.23 to calculate what your goal is.

If that number is your goal, then just use the number. Don’t make me do math to figure out what your target is.

Same thing for “We expect 4 more units per hour.”

“How many units do you expect per hour?” “How many are you producing now?” (compared to what?)

Indicators of a W.A.G.

How often do you hear something like “x happens 90 percent of the time”?

I am always suspicious of round numbers because they typically have no analysis behind them. When I hear “75%” or “90%” I am pretty sure it’s just speculation with no data.

These things sound very authoritative and it is easy for the uncertainty to get lost in re-statement. What was a rough estimate ends up being presented as a fact-based prediction.

At Boeing someone once defined numbers like this as “atmospheric extractions.”

If the numbers are important, get real measurements. If they aren’t important, don’t use them.

Bottom Line Advice:

Avoid averages unless they are part of a larger statistical testing process.

Don’t set goals as “percent improvement.” Do the math yourself and calculate the actual value you are shooting for. Compare your actual results against that value and define the gap.

When there is a lot of variation in the number of opportunities for success (or not) during a day, a week, think about something that conveys “x of X opportunities” in addition to a percent. When you have that much variation in your volume, fluctuations in percent of success from one day to the next likely don’t mean very much anyway.

Look at the populations – what was different about the ones that worked vs. the ones that didn’t — rather than just aggregating everything into a percentage.

Be suspicious of round numbers that sound authoritative.

_______________

*These charts are simply independent random numbers with upper and lower bounds on the range. Real data is likely to have something other than a flat distribution, but these make the point.

September 27, 2015January 9, 2019

Prediction Doesn’t Equal Understanding

Lunar Eclipse over Everett, WA. Photo by Mark Rosenthal, © 2015 Sometimes people fall into a trap of believing they understand a process if they can successfully predict it’s outcome. We see this in meetings. A problem or performance gap will be discussed, and an action item will be assigned to implement a solution.
Tonight those of us in the western USA saw the moon rise in partial eclipse.

We knew this would happen because our understanding of orbital mechanics allows us to predict these events… right?

Well, sort of. Except we have been predicting astronomical events like this for thousands of years, long before Newton, or even Copernicus.

The photo below is of a sophisticated computer that predicted lunar eclipses, solar eclipses, and other astronomical events in 1600BC (and earlier). Click through the photo for an explanation of how Stonehenge works:

Photo of Stonehenge — Creative Commons flickr user garethwiscombe

Stonehenge represented a powerful descriptive theory. That is, a sufficient level of understanding to describe the phenomena the builders were observing. But they didn’t know why those phenomena occurred.

Let’s go to our understanding of processes.

The ability to predict the level of quality fallout does not indicate understanding of why it occurs. All it tells you is that you have made enough observations that you can conclude the process is stable, and will likely keep operating that way unless something materially changes. That is all statistical process control tells you.

Likewise, the ability to predict how long something takes does not indicate understanding of why. Obviously I could continue on this theme.

A lot of management processes, though, are quite content with the ability to predict. We create workforce plans based on past experience, without ever challenging the baseline. We create financial models and develop “required” levels of inventory based on past experience. And all of these models are useful for their intended purpose: Creating estimates of the future based on the past.

But they are inadequate for improvement or problem solving.

Let’s say your car has traditionally gotten 26 miles-per-gallon of fuel. That’s not bad. (For my non-US readers, that’s about 9 liters / 100 km.) You can use that information to predict how far a tank of fuel will get you, even if you have no idea how the car works.

If your tank holds 15 gallons of fuel, you’ll be looking to fill after driving about 300 miles.

But what if you need to get 30 miles-per-gallon?

Or what if all of a sudden you are only getting 20 miles-per-gallon?

If you are measuring, you will know the gap you need to close. In one case you will need to improve the operation of the vehicle in some way. In the other case, you will need to determine what has changed and restore the operation to the prior conditions.

In both of those cases, if you don’t know how the car operated to deliver 26 miles-per-gallon, it is going to be pretty tough. (It is a lot harder to figure out how something is supposed to work if it is broken before you start troubleshooting it.)

Here’s an even more frustrating scenario: On the last tank of fuel, you measured 30 miles per gallon, but have no idea why things improved! This kind of thing actually happens all of the time. We have a record month or quarter, it is clearly beyond random fluctuation, but we don’t know what happened.

The Message for Management:

If you are managing to KPIs only, and can’t explain the process mechanics behind the measurements you are getting, you are operating in the same neolithic process used by the builders of Stonehenge. No matter how thoroughly they understood what would happen, they did not understand why.

If your shipments are late, if your design process takes too long, if your quality or customer service is marginal, if the product doesn’t meet customer’s expectations, and you can’t explain the mechanisms that are causing these things (or the mechanisms of a process that operates reliably and acceptably) then you aren’t managing, you are simply directing people to make the eclipse happen on a different day.

“Seek first to understand.”

Dig in, go see for yourself. Let yourself be surprised by just how hard it is to get stuff done.

February 17, 2015January 9, 2019

Scientific Improvement Beyond The Experiment

“How do we deploy this improvement to other areas in the company?” is a very common question out there. A fair number of formal improvement structures include a final step of “standardize” and imply the improvement is laterally copied or deployed into other, similar, situations.

Yet this seems to fly in the face of the idea that the work groups are in the best position to improve their own processes.

I believe this becomes much less of a paradox if we understand a core concept of improvement: We are using the scientific method.

How I Think Science Works

In science, there is no central authority deciding which ideas are good and worth including into some kind of standard documentation. Rather, we have the concept of peer review and scientific consensus.

Someone makes what she believes is a discovery. She publishes not only the discovery itself, but also the theoretical base and the experimental method and evidence.

Other scientists attempt to replicate the results. Those attempts to replicate are often expanded or extended in order to understand more.

As pieces of the puzzle come together, others might have what seems to be an isolated piece of knowledge. But as other pieces come into place around them, perhaps they can see where their contributions and their expertise might fit in to add yet another piece or fill in a gap.

If the results cannot be replicated at all, the discovery is called into serious question.

Thus, science is a self-organized collaborative effort rather than a centrally managed process. All of this works because there is a free and open exchange among scientists.

It doesn’t work if everyone is working in isolation… even if they have the same information, because they cannot key in on the insights of others.

What we have is a continuous chatter of scientists who are “thinking out loud” others are hearing them, and ideas are kicked back and forth until there is a measure of stability.

This stability lasts until someone discovers something that doesn’t fit the model, and the cycle starts again.

How I Think Most Companies Try To Work

On the other hand, what a lot of people in the continuous improvement world seem to try to do is this:

Somebody has a good idea and “proves it out.”

That idea is published in the form of “Hey… this is better. Do it like this from now on.”

We continue to see “standardization” as something that is static and audited into place. (That trick never works.)

What About yokoten. Doesn’t that mean “lateral deployment” or “standardize?”

According to my Japanese speaking friends (thanks Jon and Zane), well, yes, sort of. When these Japanese jargon terms take on a meaning in our English-speaking vernacular, I like to go back to the source and really understand the intent.

In daily usage, yokoten has pretty much the same meaning [as it does in kaizen] just a bit more mundane scope…along the lines of sharing a lesson learned.

Yokogawa ni tenkai suru (literally: to transmit/develop/convey sideways) is the longer expression of which Yokoten is the abbreviation.

Yoko means “side; sideways; lateral. Ten is just the first half of “tenkai” to develop or transmit. Yokotenkai..

If you take a good look at the Toyota internal context, it is much more than just telling someone to follow the new standard. It is much more like science.

How the Scientific Approach Would Work

A work team has a great idea. They try it out experimentally. Now, rather than trying to enforce standardization, the organization publishes what has been learned: How the threshold of knowledge about the process, about a tricky quality problem, whatever, has been extended.

We used to know ‘x’, now we know x+y.

They also publish how that knowledge was gained. Here are the experiments we ran, the conditions, and what we learned at each step.

Another team can now take that baseline of knowledge and use it to (1) validate via experimentation if their conditions are similar. Rather than blindly applying a procedure, they are repeating the experiment to validate the original data and increase their own understanding.

And (2) to apply that knowledge as a higher platform from which to extend their own.

But Sometimes there is just a good idea.

I am not advocating running experiments to validate that “the wheel” is a workable concept. We know that.

Likewise, if an improvement is something like a clever mistake proofing device or jig (or something along those lines), of course you make more of them and distribute them.

On the other hand, there might be a process that the new mistake-proofing fixture won’t work for. But… if they applied the method used to create it, they might come up with something that works for them, or something that works better.

“That works but…” is a launching point to eliminate the next obstacle, and pass the information around again.

oh… and this is how rocket science is done.

Edit to add:

I believe Brian’s comment, and my response, are a valid extension of this post, so be sure to read the comments to get “the rest of the story.” (and add your own!)

August 9, 2014January 9, 2019

Often Skipped: Understand the Challenge and Direction

I’ve been practicing, teaching (and learning) the Toyota Kata for about four years now, and I’m seeing some pervasive patterns that are getting in people’s way of making it work.

The one I’d like to address today is skipping over “Understand the Direction.”

As designed by Mike Rother, the Improvement Kata has four basic steps:

This fact (that there are four steps) is often lost on beginning practitioners. They tend to associate “Toyota Kata” only with the “Five Questions” of the Coaching Kata, which guide Step 4 (above). On the surface, that is understandable, because we hear the coaching cycles a lot more frequently, and they are a hallmark of the kata principle.

But we (hopefully) wouldn’t think of jumping right into the Five Questions until the answer to “What is your target condition?” is something more definitive than “To improve this process.” (We only used to do that, right? hmmm.)

One of the reasons to set a clear target condition is to get away from general “waste safari” improvement efforts, and focus the improver’s attention on what must be done to get to the next level.

Without a sense of direction, it is easy for the improver to see every improvement opportunity (or none of them), and get locked up trying to find a way to fix them all.

It’s frustrating for everyone, because little progress is made, and those improvements that are made have a tough time finding their way to any higher level meaning.

Someone, long ago, pointed out to me that the geometric figure “frustum” seems to have the same Latin root as “frustration.” This is the graphic that comes to mind:

Though the concept of a clear target condition is fairly easy to get across, organizations seem to get locked into a cycle of Steps 2, 3 and 4 without ever going back to Step 1, except in the most general terms.

Here’s what I think happens:

When an organization is just getting started, we’ve commonly told them to pick an area to practice, based on quick cycles and other practice criteria, and get to learning the improvement kata, striving toward a general challenge like one-by-one flow.

That pattern of seeking improvement opportunities gets engrained, and it is hard to shift off it and start to set clear direction and challenges. Further, it is very similar to the “old way” of planning kaizen events where the goal is to “implement lean tools” … like one-by-one flow.

Setting that direction also seems to run counter to the idea of empowered workers are in the best position to know what to work on. (It doesn’t, in fact direction is required for this to really take hold.)

In another sense, though, the Challenge for one level is, in reality, the Target Condition (or at least a major obstacle) for the level above. If I am asking the VP-Operations what his target condition is, I am going to hear things that sound a whole lot like a Challenge for the next level down.

And if the senior leaders don’t have a clear sense of where they are trying to take the organization next… what is their scope of their responsibility?

What to do differently?

(It might help if the leadership team themselves got a coach.)

1) Be clear that your initial sessions are practice drills. You are not yet playing the game, or even a scrimmage. You are flipping tires. Be conscious that this is a transitory learning stage.

2) Establish a target condition for proficiency you are striving for in your practice. Check it weekly against your current condition. This is the job of the advance / steering team.

Challenge Light

For the first pass, it isn’t necessary to dig headlong into hoshin planning, and in many cases, you can issue a compelling challenge with a theme.

Look at what is bugging your customers about your current operation, and challenge yourself to fix it.

For example, one client had lead times and response times that were getting longer and longer, so the challenges issued by the leadership team revolved around lead time and removing “sources of delay.”

Another company had rework and quality escapes going out of control. Their first step was to “grasp the current condition” and identify where these problems were happening.

Each dot represents the origin of a defect or rework. (The colors don’t mean anything, they ran out of red, then orange, then yellow dots.) (You can see exit cycle run charts taped up there as well.)

This gave them a quick picture of where to focus their attention, and which leaders they needed to challenge, and then coach, through the process of improving their quality.

Note that for these guys, integrating the flow of material and information though the plant isn’t a target condition, or even a challenge yet. It is in the vision, but the first priority is simply to ship what the customer ordered, and then to only make the product once to do so.

By the way, profit is measurably up, rework instances have been cut by about 3/4 from the starting point, and they are well on their way.

September 1, 2013January 9, 2019

Checklists: “Do.” vs. “Did you do?”

When operations or steps get omitted, a common countermeasure is to establish a checklist.

A typical checklist has a list of items or questions – sometimes even written in the past tense.

“Was the _______?”

There are a couple of common problems with this approach.

First, the time to actually, physically make the checks is not included in the planned cycle time. This implies we are expecting the team member to review the checklist and remember what she did.

The second issue is that the team member often does remember doing it even if it wasn’t done.

This is human nature, it isn’t a fault or flaw in the individual. It is impossible to maintain continuous conscious vigilance for any length of time. There are techniques that help, however they require some discipline from leaders.

Overall, a checklist that asks “Did you___?” in the past tense is mostly ineffective in practice.

We make things worse when the checklist is used as a punitive tool and we “write up” the team member for signing off on something that, actually, didn’t get done. Most of the time it does get done, but everyone in this system occasionally misses something. Sometimes those errors get caught. This kind of “accountability” is arbitrary at best.

Where checklists work is in “what to do next” mode – referring to the check list, doing one item, checking off that it was done, then referring to the next item on the list. This is how it works in an airplane cockpit.*

CAPTAIN: okay, taxi check.

FIRST OFFICER: departure briefing, FMS.

CAPTAIN: reviewed runway four.

FIRST OFFICER: flaps verify. two planned, two indicated.

CAPTAIN: two planned, two indicated.

FIRST OFFICER: um. takeoff data verify… one forty, one forty five, one forty nine, TOGA.

CAPTAIN: one forty, one forty five, one forty nine, TOGA.

FIRST OFFICER: the uh weight verify, one fifty two point two.

CAPTAIN: one fifty two point two.

FIRST OFFICER: flight controls verify checked.

CAPTAIN: check.

FIRST OFFICER: stab and trim verify, thirty one point one percent…and zero.

CAPTAIN: thirty one point one percent, zero.

FIRST OFFICER: the uh…. engine anti-ice.

CAPTAIN: is off.

FIRST OFFICER: ECAM verify takeoff, no blue, status checked.

CAPTAIN: takeoff, no blue, status checked.

FIRST OFFICER ON PA: ladies and gentlemen at this time we’re number one for takeoff, flight attendants please be seated.

FIRST OFFICER: takeoff min fuel quantity verify. nineteen thousand pounds required we got twenty one point eight on board.

CAPTAIN: nineteen thousand pounds required, twenty one eight on board.

FIRST OFFICER: flight attendants notified, engine mode is normal, the taxi checklist is complete sir.

(This is also how it works when assembling a nuclear warhead, but I can’t tell you that.)

This is also very effective for troubleshooting. For example, I was working with a team in a food processing plant. The obstacle being addressed was the long (and variable) time required to change over a high-speed labeling machine and get it “dialed in” and running at full speed without stops and jams.

Some operators were much better at this than others. We worked to capture an effective process of returning the machine’s settings to a known starting point, then systematically adjusting it for the specific bottle, label, etc. It worked when they were able to slow down enough to use it. That was an instance of “Slow is smooth; smooth is fast.”

The act of reading out load, performing the action, and verbally confirming is very effective when it is actually done that way. Even so, people who are very familiar with the procedure will often take shortcuts. They don’t “need” the checklist… until they do.

Still, you have a sequence of operations, and it is critical that they are all performed, in a specific order, in a specific way.

What works?
I’d say look around.
If you are reading this, you likely have been at least dabbling, and hopefully trying to apply “lean” stuff for a while.

What is a basic shadow board? It is a “checklist” of the tools to confirm they are all there – and a lot faster because missing items can be spotted at a glance. At a more advanced level, companies move away from shadow boards and to having the visual controls outlining what should be where to perform the work.

If you kit parts, you can set them out in a sequence – a “checklist” that cues the team member what order they should be installed.

I could continue to cite examples, but here’s the point.

When things are being left out, there is a high temptation to say “Let’s make a checklist” and sometimes make it worse by saying “…and we’ll have the worker sign it off for accountability.” That is more often than not simply a “feel good” solution. You feel like you have done something, and I’ve even heard “Well, it’s better than nothing.” I’m not sure it IS better than nothing – at least not in very specific conditions.

Instead, you need to study the actual work. Don’t try to ask questions, just stand and watch for a while. (Explain what you are doing to the team member first, otherwise this is creepy. “Hi – I’m just trying to understand some of the things that might get in your way. Do you mind if I just watch for a while without bothering you?”)

What cues the team member which step to perform next? Does he have to know it from memory? Or is there something built into the way the workplace is organized?

Does he end up going back and doing things he forgot?
Does he set out parts and tools in order on his own so he doesn’t forget?
Does he get interrupted, by anyone or anything, that takes him out of his mental zone?
(I go through airport TSA security checkpoints at least twice a week. I have a routine. When the TSA agent tries to “help” by talking to me, my routine gets broken, and that is when I forget stuff.)

If you are coaching someone, it helps if you go there with them, help the see the details by spotting these things and “asking” about them; then taking them to another area and challenging them to see as many of these issues as they can. See who can spot more of them.

What you are seeing are obstacles that impact the team member’s ability to do quality work.

Checklists don’t help remove those obstacles.

___________________

*The checklist transcript here is a cleaned up version of the Cockpit Voice Recorder transcript from Cactus 1549, the US Airways A320 that successfully landed in the Hudson River after multiple bird strikes knocked out both engines. I used it here because it is authentic, and the accident was one where everything went right and no one was seriously injured.