Category: Quality

Posted on

A Lean Leadership Pocket Card

I was going through some old files and came across a pocket card we handed out back in 2003 or so. It was used in conjunction with our “how to walk the gemba” coaching sessions that we did with the lean staff, and then taught them to do with leaders.

There is a pretty long backstory, some of it is summarized in Earl’s recollection on this old post: Genchi Genbutsu in a Warehouse as well as here: The Chalk Circle – Continued.

A lot has happened, a lot has been learned since then. Toyota Kata has been published, and that alone has focused my technique considerably (to say the least).

Nevertheless, I think the elements on these little cards are valuable things to keep in mind.

With that being said, a caveat: Lists like this run the risk of becoming dogma. They aren’t. There are lots of lists like this out there, and the vast majority are very good. The key here is something that a leader or team member can refer to as a reminder that may bias a decision in the right direction. It is the direction that matters, not the reminders.

Fundamentals

The fundamentals are based on the “Rules-in-Use” from Decoding the DNA of the Toyota Production System, a landmark HBR article by Steve Spear and H. Kent Bowen. The article, in turn, summarizes (and slightly updates) Spear’s findings from his PhD work studying Toyota.

A. All work highly specified as to content, sequence, timing, and outcome.

B. Every customer-supplier connection is simple and direct.

C. The path for every product is simple and direct.

D. All improvements are made using PDCA process.

What we left off, though, is that in each of those rules there is a second one: That all of these systems are set up to be “self diagnostic” – meaning there are clear indications that immediately alert the front line people if:

  • The work deviates from what was specified.
  • The connection between a customer and supplying process is anything other than specified.
  • The path a product follows deviates from the route specified.
  • Improvements are made outside of a rigorous PDCA (experimental) process.

In other words, the purpose of the rules is to be able to see when we break them, or cannot follow them, so we trigger action.

To put this into Toyota Kata-speak – every process is set up as a target condition that is being run as an experiment – even the process of improvement itself!

Every time there is a disruption – something that keeps the process from running the way it is supposed to – we have discovered an obstacle. That obstacle must first be contained to protect the team members and community (safety) and to protect the customer (quality). Then goes into the obstacle parking lot, and addressed in turn.

If you think about it, the Improvement Kata simply gives us much more rigor to (D).

This ties to the next sections.

Key Leadership Behaviors

Note that this is behaviors. These are things we want leaders to actually strive to do themselves, not just “support.” It was the job of the continuous improvement people to nudge, coach, assist the leaders to move in these directions. It was our job to teach our continuous improvement people how to do that coaching and assisting – beyond just running kaizen events that implement tools.

A. PDCA Thinking

Today we would use Toyota Kata to teach this. But the same structure drove our questioning back then.

B. Four Rules:

1. Safety First

Even though this should be obvious, it is much more common that people are tacitly, or even directly, asked to overlook safety issues for the sake of production. I remember walking through a facility with a group of managers on the way to the area we were going to see. Paul stopped dead in his tracks in front of a puddle on the floor. He was demonstrating just how easy it was for the leadership to walk right past things that should be attended to. And in doing so, they were sending the message – loud and clear in their silence – that having a puddle on the floor was OK.

2. Make a Rule, Keep a Rule

This is a more general instance of Rule #1. But the it is more subtle than it may seem on the surface. Most people immediately interpret this as enforcing organizational discipline, but in reality it is about managerial discipline.

Nearly every organization has a gap between “the rules” and how things really are day-to-day. Sometimes that gap is small. Sometimes it is huge.

Often “rules” are enforced arbitrarily, such as only cases where a violation led to a bigger problem of some kind. Here’s an example: Say your plant has a set of rules about how fork trucks are to be operated – speed limits, staying out of marked pedestrian lanes, etc. But in general the operators hurry, cut a corner now and then. And these violations are typically overlooked… until there is some kind of incident. Then the operator gets written up for “breaking the rules” that everyone breaks every day – and management tacitly encourages people to break every day by focusing on results rather than process.

When we say “make a rule / keep a rule” what we mean is if you aren’t willing to insist on a rule being followed consistently, then take the rule off the books. And if you are uncomfortable taking the rule off the books, then your only option is to develop something that you can stand behind. It might be simple mistake proofing, like physical barriers between forklift aisles and pedestrian aisles. But if you are going to make the rule, then find a way to keep the rule.

Do you have “standard work” documents that are rarely followed? Stop pretending you have standards or rules about how the work is done. Throw them away if you aren’t willing to train to them, mistake proof to them and reinforce following them.

3. Simple is Best

Simply, bias heavily toward the simplest solution that works. The fewest, simplest procedures. The simplest process flow. Complexity hides problems. “Telling people” by the way, is usually less simple than a physical change to the work environment that guides behavior. See above.

4. Small Steps

Again, Toyota Kata’s teaching covers this pretty well today. The key is that by taking small steps, verifying that they work, and anchoring them into practice before taking the next ensures that each step we take has a stable foundation under it.

The alternative would be to make many changes at once in the name of going faster.

We emphasized here that “small steps” does not equal “slow steps.” It is possible to take small steps quickly, and we found that in general doing so was faster than making big leaps. Getting big changes dialed in often required backing out and implementing one thing at a time anyway – just to troubleshoot! See “Gall’s Law” which states:

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.

John Gall, author of Systematics

and sums this up nicely.

C. Ask “Why, what, where, when, who, and how” in that order.

Here we borrowed the sequence from TWI Job Methods. The first two questions challenge whether a process step is even necessary: Why is it necessary? What is its purpose? To paraphrase Elon Musk, the greatest waste of time is improving something that shouldn’t even exist.

Then: Where is the best place? and When is the best time? These questions might nudge thinking about combining steps and further simplifying the process.

And finally we can ask Who is the best person? and “How” is the best method? The key point here is until we have the minimum possible steps in the simplest possible sequence, and understand the cycle times, it doesn’t make sense balance the work cycle or work on improving things.

Come to think about it – perhaps we should ask “How?” before we ask “Who” since improving the method will change the cycle times and may well inform out decisions about the work balance. Hmmm… I’ll have to think about that. Any thoughts from the TWI gurus?

D. Ask Why 5 Times

Honestly, this was a legacy of the times. Unfortunately it suggests that you can arrive at a root cause simply by repeatedly asking “Why?” and writing down the excuses answers that are generated. In reality problem solving involves multiple possible causes at each level, and each must be investigated. I talked about this in a post way back in 2008: Not Just Asking Why – Five Investigations.

E. Go and see.

Go and see for yourself. Taking this into today’s practice, I think it is something that the Toyota Kata community might emphasize a little more. We tend to ask the question “When can we go and see what we have learned…?” but all too often the answer to “What have you learned?” is a discussion at the board rather than actually going and observing. Hopefully the board is close to where the improvement work is being done. Key point for coaches: If the learner can’t show you and explain until you understand, it is likely the learner’s understanding could be deeper.

As You Walk The Workplace:

Check:

perhaps we should have said “Ask…” rather than “Check” but asking and observing are ways to “check.” All of the below are things that the leader walking the workplace must verify by testing the knowledge of the people doing the work.

A. How should the work be done? Content, Sequence, Timing, Outcome

This is another nod to the research of Steven Spear. The key point here is that before you can ask any of the following questions, you have to have a crisp and precise of what “good” looks like. In this paradigm, all processes are target conditions. And as the work is being done, we are actively searching for obstacles so we can work to make the work smoother and more consistent.

In other words, “What should be happening?” and “How do you know?”

Do the people doing the work understand the standard process as it should be done?

A few months ago I went into some depth on this here: Troubleshooting by Defining Standards. That probably isn’t the best title in retrospect, but there are too many links out there that I don’t want to break by changing it.

B. How do you know it is being done correctly?

Today I ask this question differently. I ask some version of “What is actually happening?” followed by “How can you tell?” We want to know if the people doing the work have a way to compare what they are actually doing against the standard.

C. How do you know the outcome is free of defects?

So, question B asks about consistency of the process, and question C asks about the outcome. Does the team member have a way to positively verify that the outcome is defect-free?

D. What do you do if you have a problem?

Again, we are checking if there is a defined process for escalating a problem. And we define “problem” as any deviation from the standard, or any ambiguity in what should be (or is) happening. We want someone to know, and act, on this, and the only way that is going to happen is to escalate the problem.

We want this process to be as rigorous and structured as the value-adding work.

And we want as much care put into designing production process as was put into designing the product itself. All too often great care and a lot of engineering time goes into product design, and only a casual pass is made at designing and testing the process.

Even better if these are done simultaneously where one informs the other.

For Abnormal Conditions:

ACT:

These are actions that the leader must take if she finds something that isn’t “as it should be” in the course of the CHECK questions above. Key Point: These are leadership actions. That doesn’t mean that the leaders personally carry them out, but the leaders are personally responsible for ensuring that these things are done – and checking again.

That is the only way I know of to prevent the process from continuing to erode.

A. Immediately follow up to restore the standard.

If it isn’t possible to get the intended standard into back place, then get a temporary countermeasure into place that ensures safety and quality.

B. Determine the cause of erosion.

We are talking about process erosion here, with the assumption that something knocked the process off its designed standard. Some obstacle has been discovered, we have to better understand what it is – at least enough to get it documented.

C. Develop and apply countermeasure.

Here we may have to run experiments against this newly discovered obstacle and figure out how to make the process more robust.

That is the end of the little card. But I want to point out that we didn’t just hand these out. You got one of these cards after time paired with a coach on the shop floor practicing answering and asking these questions. Only after you demonstrated the skill did you get the card – just as a reminder, not as a detailed reference. This exercise was inspired by a few of us who had experiences “in the chalk circle” especially with Japanese senseis who had been direct reports to Taiichi Ohno.

We piloted and developed this process on a very patient and willing senior executive – but that is another story for another day. (Thank you once again, Charlie. I learned more from you than you will probably ever realize.)

Posted on

More About Mistake-Proofing

After yesterday’s post about trucks crashing into the famous 11foot8 bridge and mistake proofing, I got the feeling I should drive home my key point that the problem isn’t with the driver, it is with the environment.

As of this writing,  Jürgen has recorded 154 crashes of overheight vehicles into the bridge.

And I’ll put even money that if all of the data were known, this process would pass any test for statistical control and we are getting what we should expect from a stable system. It might not be what we want, but it is what we should expect. (All images are copyright  Jürgen Henn, 11foot8.com)

So addressing the individual incidents probably isn’t a solution. In any case, it is unlikely that any driver will repeat the mistake.* For the TWI folks – this isn’t really a Job Relations type of problem. It might feel like it is, but it isn’t.

In the Factory

I was working with a company with a similar problem. Their inspectors kept missing defects. The response was often to say “I don’t see how she could have missed that!” and even write them up for failure to do something that wasn’t particularly well specified.

But the fact on the ground was, like the bridge, the misses weren’t confined to any particular individuals, any particular shift, any particular anything. People missed things all of the time because the expectations greatly exceeded the limitations of what humans can do for 12 hours (or even one hour). It isn’t an inspection problem. (The reliance on inspection vs. upstream controls is another topic for another day.)

People Work Within a System

It is all too easy to fall into the “bad apple” fallacy and seek out someone who was negligent. It feels good, like we did something about the problem. But the problem will happen again, with someone else. Then I hear frustrated managers start to make disparaging comments about their entire workforce that “doesn’t care” about quality.

I challenged a quality manager to do that inspection job for two hours – not even a complete shift – under the watchful eye of the inspector whose job he was trying to do. Funny – he was a lot slower to assign blame after that experience. He couldn’t keep up.

Deming was pretty clear about the ineffectiveness of exhortation as a way to get better performance. “Be more careful!” might well work for one individual for a short time. “Making an example of someone” might well work for a group for a short time. But there are norms, and the system will return to those norms very quickly. There are simple limits to what humans can focus on and for how long.

The Bridge is a Metaphor

To be clear, the bridge represents a working system, but it is different than what we would find in a company. This is public infrastructure, and the truck drivers that get featured on the videos are not part of a single organization.

This means that you have more control than the city engineers in Durham do. You can establish procedures, ask questions, train people, have them practice, alert them to the Gregson Street Bridge on their route. You can make sure your navigation system routes your trucks around the low bridges. You can support your people so they are less likely to even end up in the situation. All of these are system changes – and that is what it will take to change the outcome.

Change the System: Raising the Bridge

In late 2019 the city of Durham, in coordination with the railroad who owns the bridge, did actually raise the bridge by 8 inches. It is now 11 feet 16 inches (3.76m).

And that is a legitimate approach. Rather than trying to create infallible humans, what can we do to make the system less vulnerable to fallible humans.

While that likely reduced the number of trucks that hit the bridge…

*Caveat: There is one video where a truck seems to avoid the bridge, then circle back around and hit it. And another video where a truck that hit this bridge then proceeded to run into another low bridge with the damaged truck.

Posted on

Mistake Proofing – Getting People’s Attention

Besides being a great source of schadenfreude, Jürgen Henn’s website, 11foot8.com offers some great insight in the difficulty of effective mistake-proofing.

Background

The clearance between the road and the railroad bridge at 201 Gregson Street in Durham, North Carolina is officially 11 feet 8 inches (3.55 meters).

A typical rental box truck is about 12 feet high (3.65 meters).

The result:

Penske rental truck smashed under low bridge.

The more astute of you may have noticed the “OVERHEIGHT MUST TURN” sign just above the smashed truck.

Well before the truck approaches the intersection, it passes under a height sensor. If the vehicle is overheight, the OVERHEIGHT MUST TURN sign comes on and starts to flash, and the traffic lights turn red.

While this effort did mitigate the problem, obviously there are drivers who simply do not notice. Lots of them. Jürgen has cameras trained on the intersection and captures over a dozen crashes in a typical year. As a result, his website has attracted international attention. See the short documentary here: http://11foot8.com/about/the-documentary/

Familiar Tasks = “In the Zone”

The human brain is an amazing thing. It also works by deceiving us. It creates the illusion of complete awareness of the things around us when, in reality, we are simply aware of a model our brains have constructed of what we perceive to be there.

It is also an amazing engine at engaging actions based on pattern matching. For example –

In sessions I facilitate, I routinely ask people if they can recall a time when they arrived home after work but realized they didn’t remember driving the route. Nearly every hand in the room goes up. That is pretty amazing because driving is an incredibly complex task. But, assuming you get a license in your mid-teens, by the time you are in your early 20s, most people don’t give it much thought. (There is a reason your insurance rates drop when you turn 25.)

It has become a hard-wired neural pattern, a series of habits, a programmed set of responses that are operating below below the conscious and deliberate thinking part of the brain. This is really good because this process is much faster and more responsive than the alternative. Think about how it felt to be driving when you were just learning – or how it feels to be doing anything new when you are just learning.

The downside to this amazing programming is that things that don’t jar us into consciousness often go unnoticed.

And the more familiar, the more expert, we are with the task at hand, the more likely this will happen.

Levels of Mistake Proofing

I like to talk about three levels of mistake-proofing. Four actually, if you count Zero as a level.

  • Level 0 – the task must be performed correctly from memory.
  • Level 1 – there is a discrete task of checking build into the job.
  • Level 2 – there is an active indication when a mistake is about to be made (or has just been made and there is still time to correct without consequences).
  • Level 3 – there is active prevention that gets in the way of making the mistake.

The OVERHEIGHT MUST TURN sign is a Level 2. The driver has already passed signs informing him of the bridge clearance about 100m before the bridge.

The height sensors are on the poles just past the speed limit signs.

Low clearance signs about 100m before the bridge. (Google Street View)

Then in the next block, there is another sign telling the driver to turn:

Approaching the last intersection (Google Street View)

And finally the light changes and the OVERHEIGHT MUST TURN sign illuminates.

And still they miss it.

(Direct YouTube link for the email readers: https://youtu.be/I0BJmC6u7MU)

To be clear, the vast majority of truck drivers don’t hit the bridge. But over a dozen a year do. (There were more before the flashing sign was put in.)

“Pay Attention!” = Fundamental Attribution Error

As we watch Jürgen’s videos it is easy to assign character attributes to the drivers who hit the bridge. If only they were better drivers. (The vast majority people who are asked to rate their driving skill will say they are “above average” by the way. Think about that.)

But it is human nature to get lulled into a bit of complacency when engaging a routine task- and driving is a routine task in spite of the complexity.

Now – think about the people in your organization. How many opportunities to they have to make a mistake every day… every hour… in the course of their work? How many of those possible mistakes have serious or expensive consequences?

They are experts at what they do. And they don’t make many mistakes. And they usually catch themselves before there is a problem. But the Law of Large Numbers says “Given enough opportunities, the unlikely is inevitable.”

Can anyone reading this honestly say they have never inadvertently run a stop sign? Yet accidents rarely occur. Now and then there is some panic braking, and rarely there is a collision.

Yet it is really easy to single out the person who:

Performs the work the same way, in the same conditions as everyone else.

Makes the same mistakes, just as often, as everyone else.

Just like everyone else, usually they are caught in time, or other conditions aren’t present for a bad outcome.

But this time everything lined up and BANG – the defective product got missed, or the leak wasn’t noticed before the sump went dry.

Then that person gets singled out for “not following procedure” when nobody follows procedure.

Job Instruction “Key Points” = Opportunity

In a Job Breakdown for TWI Job Instruction we assign a “Key Point” as something the learner must remember specifically because it:

  • Might result in a hazard – injure the worker or someone else.
  • “Make or break the job” – if something isn’t done a specific way, the task fails.
  • Is a “knack” or technique that makes it easier to do.

Those first two are red flags. You are asking your team member to memorize critical to safety and critical to quality tasks. My challenge: How many of those key points can you “mistake proof” out of the job breakdown?

Finally – rather than the sensors and flashing lit signs, there is this approach from “somewhere on the Internet.”

Warning Sign: If you hit this sign, you will hit that bridge.

That is awesome because even if the driver doesn’t see the sign, the BANG! may get his attention in time for it to register and stop. In the Durham, NC example above, apparently this wouldn’t work because there are legitimate reasons for trucks to come down the street up to the intersection just before the bridge. After that, it is really too late unless the driver is already aware that he has to turn.

Oh – and by the way – you can get souvenirs from the Gregson Street Bridge at  Jürgen’s store here:  https://squareup.com/store/11foot8-dot-com/ 🙂

There is a follow-up to this post here: https://theleanthinker.com/2020/05/28/more-about-mistake-proofing/

Posted on

That Broken Bolt is Speaking to You

The factory is running complex automated equipment. At the morning meeting today we heard “machine x was down for broken bolts.” Actually “again.”

Background – the bolts in question resist pressure in molding equipment. The details of how the equipment works aren’t relevant here. This isn’t the first time I have heard of “broken bolts” being the source of downtime.

After the meeting, I saw the production manager on the shop floor. “So, tell me about the broken bolts.” We know each other, he is happy to.

We went to the machine in question. “How do bolts break?” (These days I ask “how?” rather than “why?” because I am interested in the mechanism of failure rather than whatever mistake is being made.

I am asking that question for a simple reason: Grade 8 bolts don’t “just break” in equipment that is properly engineered and assembled as designed. Something, somewhere, is stressing things beyond their limits.

Normalized Deviance

By accepting that “bolts break” and “shafts get stripped” and “hoses fail” we move into the realm of “normalized deviance” where we accept as OK something that actually is a sign of a serious problem.

image

This is no different than accepting that “o-rings burn through sometimes but it hasn’t caused a problem so it must be OK.”

How do Bolts Break?

A few possibilities came up.

  • The bolts might be just a bit too long allowing movement.
  • The might be loose.

“Why is a ‘too long’ bolt even needed in the plant?” – that is still an open question.

“What is the torque spec?”

“… I don’t know.”

Now we are getting somewhere.

Once we hit the threshold of knowledge, we know the next step.

Posted on

Software Rules

“Turn off the radio, Hal.”  “I’m sorry, Mark, I’m afraid I can’t do that.”

image

A couple of months ago I got the news that my 1995 Toyota Truck with 280,000 miles on it would require an engine rebuild plus a bunch of other stuff I wanted it to continue to be a reliable daily driver and for trips to Portland, etc.

I’ll relate the reason I didn’t replace it in-kind with a Toyota Tacoma in another post.

So now, skipping over the rest of the story, I am the owner of a new Jeep JL Wrangler.

One of the things that has happened since my 1995 Toyota was built is the introduction of electronics into vehicles. My truck’s OEM radio had an on/off/volume knob and a station tuner, plus some presets. (And the optional cassette deck.)

Now I have a 7” touch screen with all kinds of features.

One of those features is a radio. But search as I might, I couldn’t figure out how to just turn it off. I could mute it after I started the vehicle. But no matter what settings I had, when I next started the vehicle, the radio dutifully came on. As a last resort, I went to the OEM “Help” page:

US Customer Service – Chrysler Brand Site

Brief Description: How do I turn off the radio?

Comments: Every time I start the vehicle, the radio comes on. How do I stop that from happening? I can’t see any setting to prevent this.

VIN: 1CHJXDG3Wxxxxxx

Mileage: 61

This is the response I got:

On Tue, Jul 10, 2018, 12:52 PM customerassist
<customerassist@chrysler.com> wrote:
Dear Mark,
Thank you for contacting the Jeep Customer Assistance Center.

Radio mode is the default when the radio is booted up. Sadly there is no option to adjust what mode the radio will start in when the vehicle is started.  This ensures the radio is in the correct mode to allow for updates each time the vehicle starts.

Thank you again for your email.  Should you require additional
assistance, or have any new information to provide, please reply to this email message or call 1-877-I-AM-JEEP (1-877-426-5337).
Sincerely,
James
Customer Service Representative
Jeep Customer Assistance Center
For any future communications related to this email, please refer to the following information:
REFERENCE NUMBER: 3xxxxxxx
EMAIL CASE NUMBER:  3xxxxxx

Well, that was interesting. To be crystal clear, James is doing a great job. It is obvious he read my original query, and he didn’t just send a cut-and-paste response that is all too common. Still… just to make sure I was understanding this correctly, I responded:

Perhaps I wasn’t clear about what problem I am seeking to solve.

I am looking for how to start the vehicle and not have sound automatically start coming out of the speakers. The system does not remember “mute” for example.

So far the only thing I have found is to tune the radio to an empty frequency.

I am certain there is not an intention to force the customer to hear the radio every time the radio is started.

And get a deeper, more personal, response back:

Dear Mark,
Thank you for contacting the Jeep Customer Assistance Center.

Honestly the radio tuned to dead air is the best solution for this
concern I have as yet heard.  I can confirm that this is in fact the way the radio was designed and it is most likely operating as intended.  The radio will start in radio mode and the volume will be set to mid range to ensure the kids do not leave it cranked up too high for you.

I also feel that they dropped the ball a bit on this one as the radio will default to radio mode even if you were using a connected device when you left the vehicle outside of radio mode.

I have recorded your comments on the file for further review and we thank you for the time and effort it took to bring this information to our attention.

Thank you again for your email.  Should you require additional assistance, or have any new information to provide, please reply to this email message or call 1-877-I-AM-JEEP (1-877-426-5337).
Sincerely,
James

James didn’t write design, spec, or write the software. It is clear from his response that if he did, it wouldn’t include this “feature.” And I think I can interpret his response to strongly hint that this isn’t the first time they have heard about this little issue.

So I am thinking now of the mission of Menlo Innovations and my friends there: “End human suffering in the world as it relates to technology.” Perhaps this doesn’t rise to the level of “human suffering” but – seriously?

Kudos, though to James at Jeep Customer Service for his empathetic response that shows he actually read my email.  So FCA – you got that part right. Now… let’s talk about the user experience with your software. Looking at the Jeep brand’s core values, “Freedom, Adventure, Authenticity and Passion” I’d say you came up a little short on the “Freedom” part – I just want to be free to turn of the radio.   😉

I promised I’d be writing about leadership and coaching at the end of last post. That is coming, I just wanted to get this one out of the “drafts” list first.

Posted on

Team Member Saves

image

Now and then one of your team members makes a great save. They catch something that could have caused a defect, an accident, or done harm in some way.

Often we celebrate these saves, sometimes informally, sometimes formally. And that is well and appropriate.

But let’s make sure we are celebrating for the right reasons.

The save isn’t what should be celebrated.

Rather, the celebration should be a big THANK YOU for finding a gap in your process.

Somehow the process is capable of producing a defect, resulting in an accident, or doing harm. Your team member noticed that.

We usually just celebrate correcting the immediate problem.

But What is preventing exactly the same thing from happening tomorrow?*

image

That front-line customer-facing team member is your last line of defense.

They only get an opportunity to make a “save” when every other point in the process has failed to detect the  problem.

Given enough “shots” at this front line team-member, sooner or later, one is going to get through.

What happens then? Is the inverse logic applied? “You should have caught that.”

Perhaps, but where in the process was the problem actually created?

Somewhere, long before this diving catch, there is an instant when the process went from operating safely and defect free to creating an opportunity, an opening, for a problem to pass undetected.

Where and when was that moment?

Or is that how the process normally operates, and we are just lucky most of the time?

Dig in, think about it.

And thank that team member for saving you, but don’t count on it every time.

———-

*Thanks to Craig for this great way to sum up the question.

Posted on

Averages, Percentages and Math

As a general rule I strongly discourage the use of averages and “percent improvement” (or reduction) type metrics for process improvement.

The Problem with Averages

Averages can be very useful when used as part of a rigorous statistical analysis. Most people don’t do that. They simply dump all of their data into a simple arithmetic mean, and determine a score of sorts for how well the process is doing.

The Average Trap

There is a target value. Let’s say it is 15. Units could be anything you want. In this example, if we exceed 15, we’re good. Under 15, not good.

“Our goal is 15, and our average performance is 20.”

Awesome, right?

Take a look at those two run charts below*. They both have an average of 20.

On the first one, 100% of the data points meet or exceed the goal of 15.

Run chart with average of 20, all points higher than 15.

On the one below, 11 points miss the goal

image

But the both have an average 5 points over the goal.

In this case, the “average” really gives you almost no information. I would have them measure hits and misses, not averages. The data here is contrived, but the example I am citing is something I have seen multiple times.

Why? Most people learned how to calculate an arithmetic mean in junior high school. It’s easy. It’s easier to put down a single number than to build a run chart and look at every data point. And once that single number is calculated, the data are often thrown away.

Be suspicious when you hear “averages” used as a performance measurement.

Using Averages Correctly

(If you understand elementary statistical testing you can skip this part… except I’ve experts who should have known better fall into the trap I am about to describe, so maybe you shouldn’t skip it after all.)

In spite of what I said above, there are occasions when using an average as a goal or as part of a target condition makes sense.

A process running over time produces a range of values that fall into a distribution of some kind.

Any sample you take is just that – a sample. Take a big enough sample, and you can become reasonably confident that the average of your sample represents (meaning is close to) the average of everything.

The move variation there is, the bigger sample you need to gain the same level of certainty (which is really expressed as the probability you are wrong).

The more certain you want to be, the bigger sample you need.

Let’s say you’ve done that. So now you have an average (usually a mean) value.

Since you are (presumably) trying to improve the performance, you are trying to shift that mean – to change the average to a higher or lower value.

BUT remember there is variation. If you take a second sample of data from an unchanged process and calculate that sample’s average, YOU WILL GET A DIFFERENT AVERAGE. It might be higher than the first sample, it might be lower, but the likelihood that it will exactly the same is very, very small.

The averages will be different even if you haven’t changed anything.

You can’t just look at the two numbers and say “It’s better.” If you try, the NEXT sample you take might look worse. Or it might not. Or it might look better, and you will congratulate yourself.

If you start turning knobs in response, you are chasing your tail and making things worse because you are introducing even more variables and increasing the variation. Deming called this “Tampering” and people do it all of the time.

Before you can say “This is better” you have to calculate, based on the amount of variation in the data, how much better the average needs to be before you can say, with some certainty, that this new sample is from a process that is different than the first one.

The more variation there is, the more difference you need to see. The more certainty you want, the more difference you need to see. This is called “statistical significance” and is why you will see reports that seem to show something is better, but seem to be dismissed as a “statistically insignificant difference” between, for example, the trial medication and the placebo.

Unless you are applying statistical tests to the samples, don’t say “the average is better, so the process is better.” The only exception would be if the difference is overwhelmingly obvious. Even then, do the math just to be sure.

I have personally seen a Six Sigma Black Belt(!!) fall into this trap – saying that a process had “improved” based on a shift in the mean of a short sample without applying any kind of statistical test.

As I said, averages have a valuable purpose – when used as part of a robust statistical analysis. But usually that analysis isn’t there, so unless it is, I always want to see the underlying numbers.

Sometimes I hear “We only have the averages.” Sorry, you can’t calculate an average without the individual data points, so maybe we should go dig them out of the spreadsheet or database. They might tell us something.

The Problem with Percentages

Once again, percentages are valuable analysis tools, so long as the underlying information isn’t lost in the process. But there are a couple of scenarios where I always ask people not to use them.

Don’t Make Me Do Math

“We predict this will give us a 23% increase in output.”

That doesn’t tell me a thing about your goal. It’s like saying “Our goal is better output.”

Here is my question:

“How will you know if you have achieved it?”

For me to answer that question for myself, I have to do math. I have to take your current performance, multiply x 1.23 to calculate what your goal is.

If that number is your goal, then just use the number. Don’t make me do math to figure out what your target is.

Same thing for “We expect 4 more units per hour.”

“How many units do you expect per hour?” “How many are you producing now?” (compared to what?)

Indicators of a W.A.G.

How often do you hear something like  “x happens 90 percent of the time”?

I am always suspicious of round numbers because they typically have no analysis behind them. When I hear “75%” or “90%” I am pretty sure it’s just speculation with no data.

These things sound very authoritative and it is easy for the uncertainty to get lost in re-statement. What was a rough estimate ends up being presented as a fact-based prediction.

At Boeing someone once defined numbers like this as “atmospheric extractions.”

If the numbers are important, get real measurements. If they aren’t important, don’t use them.

Bottom Line Advice:

Avoid averages unless they are part of a larger statistical testing process.

Don’t set goals as “percent improvement.” Do the math yourself and calculate the actual value you are shooting for. Compare your actual results against that value and define the gap.

When there is a lot of variation in the number of opportunities for success (or not) during a day, a week, think about something that conveys “x of X opportunities” in addition to a percent. When you have that much variation in your volume, fluctuations in percent of success from one day to the next likely don’t mean very much anyway.

Look at the populations – what was different about the ones that worked vs. the ones that didn’t — rather than just aggregating everything into a percentage.

Be suspicious of round numbers that sound authoritative.

_______________

*These charts are simply independent random numbers with upper and lower bounds on the range. Real data is likely to have something other than a flat distribution, but these make the point.

Posted on

Scientific Improvement Beyond The Experiment

“How do we deploy this improvement to other areas in the company?” is a very common question out there. A fair number of formal improvement structures include a final step of “standardize” and imply the improvement is laterally copied or deployed into other, similar, situations.

Yet this seems to fly in the face of the idea that the work groups are in the best position to improve their own processes.

I believe this becomes much less of a paradox if we understand a core concept of improvement: We are using the scientific method.

How I Think Science Works

In science, there is no central authority deciding which ideas are good and worth including into some kind of standard documentation. Rather, we have the concept of peer review and scientific consensus.

Someone makes what she believes is a discovery. She publishes not only the discovery itself, but also the theoretical base and the experimental method and evidence.

Other scientists attempt to replicate the results. Those attempts to replicate are often expanded or extended in order to understand more.

As pieces of the puzzle come together, others might have what seems to be an isolated piece of knowledge. But as other pieces come into place around them, perhaps they can see where their contributions and their expertise might fit in to add yet another piece or fill in a gap.

If the results cannot be replicated at all, the discovery is called into serious question.

Thus, science is a self-organized collaborative effort rather than a centrally managed process. All of this works because there is a free and open exchange among scientists.

It doesn’t work if everyone is working in isolation… even if they have the same information, because they cannot key in on the insights of others.

What we have is a continuous chatter of scientists who are “thinking out loud” others are hearing them, and ideas are kicked back and forth until there is a measure of stability.

This stability lasts until someone discovers something that doesn’t fit the model, and the cycle starts again.

How I Think Most Companies Try To Work

On the other hand, what a lot of people in the continuous improvement world seem to try to do is this:

Somebody has a good idea and “proves it out.”

That idea is published in the form of “Hey… this is better. Do it like this from now on.” image

We continue to see “standardization” as something that is static and audited into place. (That trick never works.)

What About yokoten. Doesn’t that mean “lateral deployment” or “standardize?”

According to my Japanese speaking friends (thanks Jon and Zane), well, yes, sort of.  When these Japanese jargon terms take on a meaning in our English-speaking vernacular, I like to go back to the source and really understand the intent.

In daily usage, yokoten has pretty much the same meaning [as it does in kaizen] just a bit more mundane scope…along the lines of sharing a lesson learned.

Yokogawa ni tenkai suru (literally: to transmit/develop/convey sideways) is the longer expression of which Yokoten is the abbreviation.

Yoko means “side; sideways; lateral. Ten is just the first half of “tenkai” to develop or transmit. Yokotenkai..

If you take a good look at the Toyota internal context, it is much more than just telling someone to follow the new standard. It is much more like science.

How the Scientific Approach Would Work

A work team has a great idea. They try it out experimentally. Now, rather than trying to enforce standardization, the organization publishes what has been learned: How the threshold of knowledge about the process, about a tricky quality problem, whatever, has been extended.

We used to know ‘x’, now we know x+y.

They also publish how that knowledge was gained. Here are the experiments we ran, the conditions, and what we learned at each step.

Another team can now take that baseline of knowledge and use it to (1) validate via experimentation if their conditions are similar. Rather than blindly applying a procedure, they are repeating the experiment to validate the original data and increase their own understanding.

And (2) to apply that knowledge as a higher platform from which to extend their own.

But Sometimes there is just a good idea.

I am not advocating running experiments to validate that “the wheel” is a workable concept. We know that.

Likewise, if an improvement is something like a clever mistake proofing device or jig (or something along those lines), of course you make more of them and distribute them.

On the other hand, there might be a process that the new mistake-proofing fixture won’t work for. But… if they applied the method used to create it, they might come up with something that works for them, or something that works better.

“That works but…” is a launching point to eliminate the next obstacle, and pass the information around again.

oh… and this is how rocket science is done.

Edit to add:

I believe Brian’s comment, and my response, are a valid extension of this post, so be sure to read the comments to get “the rest of the story.” (and add your own!)

Posted on

DMAIC and Toyota Kata

A lot of the organizations I deal with have a legacy with Six Sigma, or some x-Sigma variant. If they are now trying to incorporate Toyota Kata as a way to shift their daily behavior, questions arise about how it fits (or might fit, or whether it fits) with DMAIC.

This sometimes comes about when the impetus to embrace Toyota Kata comes from outside the organization, such as an initiative from the corporate Continuous Improvement office. In this case, unless integration with legacy approaches is carefully thought through, Toyota Kata (or whatever else is coming down the pipeline) can easily be perceived as “yet another corporate initiative” or “something else to do” rather than “a new (and hopefully better) way to do what we are already doing.”

My Background

First a disclaimer. My deepest exposure to Six Sigma was during my time as a Quality Director in a large company that had a long history with TQM and then Six Sigma. Thus, I dealt with the Black Belts and Green Belts in the organization, and paid a lot of attention to the projects they were working on.

In addition, every certification project for a new Black Belt came across my desk. Unlike a lot of managers in the chain (apparently), I actually read them, parsed them, and asked questions when I couldn’t follow the story line of the project. (Apparently nobody expected that, but it’s another story.)

I worked with the corporate Master Black Belt, made input into their programs, and did what I could to create a degree of cooperation, if not harmony, between the Six Sigma community and the lean guys.

Thus I am not claiming this is anything new or profound. Rather, this is sharing my own sense of connection between these two approaches in a world where I often find them competing for people’s mindspace.

The Improvement Process Flow

As I observed it, a Six Sigma project was typically organized and conducted as follows:

An area manager, usually a Green Belt, identifies something that needs improving. He assembles a team of stakeholders. He is coached by a Black Belt through*:

Defining the problem and establishing a charter for the team.

Establishing a Measurement that will define progress.

Conducting a thorough Analysis of the process, with a primary focus on sources of variation, especially those which are intertwined with quality issues.

Developing a list or set of Improvements and putting them into place, again focusing primarily on variation in methods, etc. that drive defects.

Establishing a standard to Control the process and keeping it running the new way.

Define, Measure, Analyze, Improve, Control. DMAIC.

In actual practice, it is very similar to the large single-loop “six step problem solving” process I was taught as “the problem solving process that came out of TQM.” That is probably not a coincidence.

DMAIC projects are typically targeted at measurable significant financial payback. Black Belts are trained to find and spend their time on high-payback projects. At least in the company I worked for, there was a minimum payback they had to achieve to get their certification. They also had to demonstrate knowledge of the various high-powered statistical tools.

It makes sense for people steeped in this methodology to ask how it fits in with “short cycles of experiments and improvements” that are the anchor for not only Toyota Kata, but kaizen in general (if you are doing it right).

And, put another way, if the organization is trying to establish a coaching culture using Toyota Kata, is Toyota Kata something different from DMAIC, or do they fit together? (I have actually been asked exactly the same question about Toyota Kata and kaizen(!), but that, too, is another story.)

To give credit where credit is due, this was the topic over lunch last week at a client site whose Six Sigma projects also follow the general structure I outlined above. Jazmin, the continuous improvement leader, was already working through this in her mind, and recognized the linkage right away.

Since the company has an active Six Sigma program, with dozens of projects ongoing, we wanted to find a way to integrate Toyota Kata thinking into what they were already doing vs. introducing yet another separate initiative. (It is easier to “embrace and extend” something you are already doing than bring something brand new into the domain.)

Relating DMAIC to the Improvement Kata

Here is how I relate DMAIC to the Improvement Kata.

Define the Problem =(more or less) Challenge and Direction. This is what we are working on, and why it is important.

Measure and Analyze = Grasp the Current Condition. Six Sigma has a host of powerful tools (which are often used just because they are there … so be careful not to make easy things complicated).

I would point out that if you follow the process in the Improvement Kata Handbook, you are also initially focused on variation in the process. Lean people tend to reduce all variation down to units of time, but in that noise are all sources of variation of the process. Defects, for example, don’t count as a delivery, and so introduce noise into the exit cycles. Machine slowdowns and stoppages, likewise, disturb the rhythm of the process. Like DMIAC, the Improvement Kata, as outlined in the handbook, steers you toward sources of variation very quickly.

Thus, a Six Sigma project team, and an improver following the Improvement Kata are both going to initially look for sources of instability. (Quality First, Safety Always).

At this point, the two diverge a little, but only a little.

Perhaps because DMAIC sounds like a single cycle, a fair number of teams tend to try to Implement a Single Grand Solution. They spend a fair amount of time brainstorming what it should look like, and designing it. Then, once they think they have a solution, they put it into place by establishing new “standards” (in this context, that usually means procedures), training people, and validating that it all works.

Again – a lot of kaizen events do pretty much the same thing, they just might do it faster if it is a classic five-day event.

A few years ago I was on a discussion panel at a conference in Chicago that was very Six Sigma centric. In the various breakout sessions, the Black Belts (who are mostly staff practitioners) universally complained about “management embracing the changes” and not enforcing the new processes. They were frustrated that once they were done, things slipped back.

In other words, once the energy input of the project itself stopped, entropy took over, and things regressed back to the original equilibrium.

And (once again) traditional kaizen events often have the same problem.

Blending Toyota Kata and Six-Sigma Coaching

Think of this conversation between the Black Belt coach and the Green Belt project leader in their daily meeting and check-in:

[preliminary social rituals]

“Just to review, what is the problem you are working on?”

[Green Belt reviews the charter and objective]

“Great, so what is the target condition you are striving for right now?”

[Green belt describes the next intermediate step toward the chartered defined state. He describes how the process will operate when key parts of it are stabilized, for example.]

The Black Belt is listening carefully, and may ask follow-up questions to make sure the target condition is clearly on the path toward solving the defined problem vs. chasing something interesting-but-irrelevant to the issue at hand.

“Good. Can you tell me the last step you took?”

[Green Belt describes a change they made, or some additional data they needed to collect and analyze, or a control measure they have experimented with, etc]

“What did you expect from that step?”

[Green Belt reviews the intent of the action, and what he expected to happen.]

“And what actually happened?”

[Green Belt describes what his data collection and observation revealed about the process, or how well his control measure worked to contain a source of variation, etc…. or didn’t.]

“And what did you learn?”

[Green Belt describes insights that have been gained, especially insights into sources of variation, or the effect of controlling them. He also describes what he might have learned about the tools, or struggled with in applying them.]

“Very good. So what sources of variation or obstacles do you think are preventing you from reaching the target right now?”

[Green Belt describes the current suspects for causes and sources of variation in the process, as well as other issues that may be impeding progress.]

“Which one of those are you addressing now?”

[The Green Belt may well still be working on the last one, or might have it effectively controlled and is now addressing a new one. If he has moved on to a new one, the Black Belt is going to be especially interested in the control mechanism for the sources of variation that has been “eliminated” so it stays that way.]

For example, a follow-up question might be “Can you tell me how you are controlling that? What countermeasures do you have to detect if that variation comes back into play?”

We aren’t limited to SPC of course, and actually would rather have a binary yes/no  need-to-act/don’t-need-to-act signal of some kind.

What we are going here is iterating through sources of variation, and establishing a positive Control on each of them before moving on rather than trying to stabilize everything in one step at the end.

Once he is satisfied that the project team hasn’t “left fire behind them,” then the Black Belt can move on.

“Great. Can you tell me the next step you plan to take, or experiment you’re going to run?”

[Green Belt reviews the next action to either learn more about a source of variation or attempt to keep it controlled.]

“And what result do you expect?”

[If the Green Belt is proposing using a specific Six Sigma statistical cool, the Black Belt is going to be carefully listening, and asking follow-on questions to confirm that the Green Belt understands the tool’s function and limitations, how he plans to use it, what he expects to learn as a result, and why he thinks that specific tool will give him the answers he is looking for.]

“OK, when do you think we can review what you have learned from taking that step”

This interaction is using the Coaching Kata script to develop the Six Sigma skills of the Green Belt project leader. We already have a coaching relationship, so all we are doing here is practicing a technique to make it more effective and more structured.

DMAIC is now more like:

DM [repeat AIC as necessary]

as the team works methodically through the sources of variation as they are uncovered.

The Master Black Belt

At the next level up is typically a Master Black Belt who is generally responsible for mentoring and developing the Black Belts. They typically meet weekly or monthly and review and share progress on projects.

Only now, let’s shift the discussion to reviewing and sharing progress on developing the problem analysis and solving skills of the Green Belts. Remember, the Green Belts are line management, and we want to get them thinking this way about everything.

“Lets review your learning objective for your Green Belt last week.”

“What did he do? What did you expect him to learn?”

“What did he actually learn? Did he make any mistakes?”

“Is he stuck anywhere? What is your plan to give him additional coaching or instruction?”

In other words, they are following DMAIC as well. Except that the “problem” is the skill of the Green Belt and how effectively he is applying that skill to solve his chartered problem.

 

I am really interested in hearing from Six Sigma folks out there about how this resonates, or doesn’t, with you.

————

*Sometimes I observed a Black Belt doing the same thing on his own initiative, leading the project himself. Occasionally I saw a Black Belt acting solo. I am not discussing either of those approaches here, however.

Posted on

Checklists: “Do.” vs. “Did you do?”

When operations or steps get omitted, a common countermeasure is to establish a checklist.

A typical checklist has a list of items or questions – sometimes even written in the past tense.

“Was the _______?”

There are a couple of common problems with this approach.

First, the time to actually, physically make the checks is not included in the planned cycle time. This implies we are expecting the team member to review the checklist and remember what she did.

The second issue is that the team member often does remember doing it even if it wasn’t done.

This is human nature, it isn’t a fault or flaw in the individual. It is impossible to maintain continuous  conscious vigilance for any length of time. There are techniques that help, however they require some discipline from leaders.

Overall, a checklist that asks “Did you___?” in the past tense is mostly ineffective in practice.

We make things worse when the checklist is used as a punitive tool and we “write up” the team member for signing off on something that, actually, didn’t get done. Most of the time it does get done, but everyone in this system occasionally misses something. Sometimes those errors get caught. This kind of “accountability” is arbitrary at best.

Where checklists work is in “what to do next” mode – referring to the check list, doing one item, checking off that it was done, then referring to the next item on the list. This is how it works in an airplane cockpit.*

CAPTAIN: okay, taxi check.

FIRST OFFICER: departure briefing, FMS.

CAPTAIN: reviewed runway four.

FIRST OFFICER: flaps verify. two planned, two indicated.

CAPTAIN: two planned, two indicated.

FIRST OFFICER: um. takeoff data verify… one forty, one forty five, one forty nine, TOGA.

CAPTAIN: one forty, one forty five, one forty nine, TOGA.

FIRST OFFICER: the uh weight verify, one fifty two point two.

CAPTAIN: one fifty two point two.

FIRST OFFICER: flight controls verify checked.

CAPTAIN: check.

FIRST OFFICER: stab and trim verify, thirty one point one percent…and zero.

CAPTAIN: thirty one point one percent, zero.

FIRST OFFICER: the uh…. engine anti-ice.

CAPTAIN: is off.

FIRST OFFICER: ECAM verify takeoff, no blue, status checked.

CAPTAIN: takeoff, no blue, status checked.

FIRST OFFICER ON PA: ladies and gentlemen at this time we’re number one for takeoff, flight attendants please be seated.

FIRST OFFICER: takeoff min fuel quantity verify. nineteen thousand pounds required we got twenty one point eight on board.

CAPTAIN: nineteen thousand pounds required, twenty one eight on board.

FIRST OFFICER: flight attendants notified, engine mode is normal, the taxi checklist is complete sir.

(This is also how it works when assembling a nuclear warhead, but I can’t tell you that.)

This is also very effective for troubleshooting. For example, I was working with a team in a food processing plant. The obstacle being addressed was the long (and variable) time required to change over a high-speed labeling machine and get it “dialed in” and running at full speed without stops and jams.

Some operators were much better at this than others. We worked to capture an effective process of returning the machine’s settings to a known starting point, then systematically adjusting it for the specific bottle, label, etc. It worked when they were able to slow down enough to use it. That was an instance of “Slow is smooth; smooth is fast.”

The act of reading out load, performing the action, and verbally confirming is very effective when it is actually done that way. Even so, people who are very familiar with the procedure will often take shortcuts. They don’t “need” the checklist… until they do.

Still, you have a sequence of operations, and it is critical that they are all performed, in a specific order, in a specific way.

What works?
I’d say look around.
If you are reading this, you likely have been at least dabbling, and hopefully trying to apply “lean” stuff for a while.

What is a basic shadow board? It is a “checklist” of the tools to confirm they are all there – and a lot faster because missing items can be spotted at a glance. At a more advanced level, companies move away from shadow boards and to having the visual controls outlining what should be where to perform the work.

Color coded tool holders.

If you kit parts, you can set them out in a sequence – a “checklist” that cues the team member what order they should be installed.

I could continue to cite examples, but here’s the point.

When things are being left out, there is a high temptation to say “Let’s make a checklist” and sometimes make it worse by saying “…and we’ll have the worker sign it off for accountability.” That is more often than not simply a “feel good” solution. You feel like you have done something, and I’ve even heard “Well, it’s better than nothing.” I’m not sure it IS better than nothing – at least not in very specific conditions.

Instead, you need to study the actual work. Don’t try to ask questions, just stand and watch for a while. (Explain what you are doing to the team member first, otherwise this is creepy. “Hi – I’m just trying to understand some of the things that might get in your way. Do you mind if I just watch for a while without bothering you?”)

What cues the team member which step to perform next? Does he have to know it from memory? Or is there something built into the way the workplace is organized?

Does he end up going back and doing things he forgot?
Does he set out parts and tools in order on his own so he doesn’t forget?
Does he get interrupted, by anyone or anything, that takes him out of his mental zone?
(I go through airport TSA security checkpoints at least twice a week. I have a routine. When the TSA agent tries to “help” by talking to me, my routine gets broken, and that is when I forget stuff.)

If you are coaching someone, it helps if you go there with them, help the see the details by spotting these things and “asking” about them; then taking them to another area and challenging them to see as many of these issues as they can. See who can spot more of them.

What you are seeing are obstacles that impact the team member’s ability to do quality work.

Checklists don’t help remove those obstacles.

___________________

*The checklist transcript here is a cleaned up version of the Cockpit Voice Recorder transcript from Cactus 1549, the US Airways A320 that successfully landed in the Hudson River after multiple bird strikes knocked out both engines. I used it here because it is authentic, and the accident was one where everything went right and no one was seriously injured.