Make Sure Failure = Learning

Take a look at this cool video from Space-X that highlights all of the failures that preceded their successful (and now more or less routine) landing of a recoverable orbital booster rocket. Then let’s discuss it a bit.

(Here is the direct link if you don’t get the embed in your feed: https://www.youtube.com/watch?v=bvim4rsNHkQ)

When we see failure, or even failure after failure, it is easy to forget that learning is rarely linear.

A Culture of Learning

Organizations like Space-X (and their counterparts such as Blue Origin) are in the business of learning. They are pushing the edges of what is known and moving into new territory. For organizations that understand that setbacks, mistakes, failures and the like are an inevitable part of learning, these things – while costly and unpleasant – are regarded as part of the process.

We have seen the same mechanisms in play – a process of experimentation toward progressive target conditions toward a visionary challenge – behind pretty much every breakthrough achievement throughout history.

No Mistakes = No Learning

At the opposite end of the spectrum are organizations with no tolerance for mistakes. They expect everything (and every one) to get everything right every time. They dismiss as incompetent any notion of failure, and attack as weakness any admission of “I don’t know” or “I don’t know how.”

A few years ago, as I was teaching Toyota Kata coaching with a client, a middle manager approached me during a break and said – point blank – that it was not his responsibility to develop his people. “Our policy is to hire competent people, and we expect them to be able to do the job.” He wasn’t the only one to say that, so I built the impression that this belief was, indeed, part of their culture. Needless to say they struggle a bit with getting innovation to happen because they try to mechanize the process.

Mistakes = Tuition

Here’s how I look at it. When a mistake happens – especially one that is expensive – you have paid considerable tuition. Your choice now is to either extract as much learning as you can from the event, or to try to ignore it and move on. The later choice is like paying your tuition up-front, then skipping all of your classes and wonder why you aren’t getting it.

Learning = Adapting to Change

Organizations that manage in ways that regard learning as part of their everyday experience are much more adaptive to changes and surprises than those who just execute their routines every day. The paradox here is that organizations who value learning are generally the most disciplined at following their routines. This discipline makes execution a hypothesis test, and they can quickly see when their process isn’t appropriate and adapt and learn quickly as an organization. They strengthen their routines, and through those routines, embed what they have learned in the organization’s DNA for future generations.

Organizations that figure it out as they go, on the other hand, tend to rely on individuals to adapt, but there is no mechanism to capture that learning beyond the individual or small group. Sometimes there is a “lessons learned” document, but that’s it. Those reports rarely result in the changes in organizational behavior that reflect learning. I suppose the most egregious case would be the loss of the space shuttle Columbia upon re-entry for exactly the same organizational failures that resulted in the loss of Challenger.

Technical vs. Cultural Learning

Space-X is solving a technical problem with science and engineering. I hope (and expect) that as they become more successful they will always be striving for something really hard that will drive them to the next level. Based on what I see publicly, I think that is embedded into their culture by Elon Musk. (But I don’t really know. If anyone from Space-X is reading this, how about getting in touch? I’d love to learn more.)

I expect this works for Space-X because they have a culture of learning.

What doesn’t work, though, is to try to apply technical solutions to transition a rote-execution culture into a learning culture. Changing the culture – the default behaviors and responses of people as they interact – isn’t about improving the mechanics of the work process. You certainly can work on the work processes, but the starting condition is what evolved in the context of the organization’s culture. The mechanics of the “improved” process that we try to duplicate evolved in the context of a learning culture. The ecosystems are different. It is difficult for a lean process to survive in a culture that expects everything to run perfectly and doesn’t have robust mechanisms to turn problems into improvements.

Creative Safety Supply: Kaizen Training and Research Page

Normally when I get an email from a company pointing me to the great lean resource on their web page, I find very little worth discussing. But Creative Safety Supply in Beaverton, Oregon has some interesting material that I think is worth taking a look at.

First, to be absolutely clear, I have not done business with them, nor do I have any business relationship. I can’t speak, one way or the other, about their products, customer service, etc

With that out of the way, I found their Kaizen Training and Research Page interesting enough to go through it here and comment on what I see.

What, exactly, is “PDCA?”

The section titled Kaizen History goes through one of the most thorough discussions of the evolution of what we call “PDCA” I have ever read, tracing back to Walter Shewhart. This is the only summary I have ever seen that addresses the parallel but divergent histories of PDCA through W. Edwards Deming on the one hand and Japanese management on the other. There has been a lot of confusion over the years about what “PDCA” actually is. It may well be that that confusion originates from the same term having similar but different definitions depending on the context. This section is summed up well here:

The Deming Circle VS. PDCA

In August of 1980, Deming was involved in a Roundtable Discussion on Product Quality–Japan vs. the United States. During the roundtable discussion, Deming said the following about his Deming Circle/PDSA and the Japanese PDCA Cycle, “They bear no relation to each other. The Deming circle is a quality control program. It is a plan for management. Four steps: Design it, make it, sell it, then test it in service. Repeat the four steps, over and over, redesign it, make it, etc. Maybe you could say that the Deming circle is for management, and the QC circle is for a group of people that work on faults encountered at the local level.”

So… I learned something! Way cool.

Rapid Change vs. Incremental Improvement

A little further down the page is a section titled Kaizen Philosophy. This section leans heavily on the thoughts / opinions of Masaaki Imai through his books and interviews. Today there is an ongoing debate within the lean community about the relative merits of making rapid, radical change, vs. the traditional Japanese approach of steady incremental improvement over the long-haul.

In my opinion, there is nothing inherently wrong with making quick, rapid changes IF they are treated as an experiment in the weeks following. You are running to an untested target condition. You will likely surface many problems and issues that were previously hidden. If you leave abandon the operators and supervisors to deal with those issues on their own, it is likely they simply don’t have the time, skill or clarity of purpose required to work through those obstacles and stabilize the new process.

You will quickly learn what the knowledge and skill gaps are, and need to be prepared to coach and mentor people through closing those gaps. This brings us to the section that I think should be at the very top of the web page:

Respect for People

Almost every discussion about kaizen and continuous improvement mentions that it is about people, and this page is no different. However in truth, the improvement culture we usually describe is process focused rather than people focused, and other than emphasizing the importance of getting ideas from the team, “employee engagement is often lip-service. There is, I think, a big difference between “employee engagement” and “engaging employees.” One is passive, waiting for people to say something. The other is active development of leaders.

Management and Standards

When we get into the role of management, the discussion turns somewhat traditional. Part of this, I think, is a common western interpretation of the word “standards” as things that are created and enforced by management.

According to Steve Spear (and other researchers), Toyota’s definition of “standard” is quite different. It is a process specification designed as a prediction. It is intended to provide a point of reference for the team so they can quickly see when circumstances force them to diverge from that baseline, revealing a previously unknown problem in the process.

Standards in this world are not something static that “management should make everyone aware of” when they change. Rather, standards are established by the team, for the team, so the team can use them as a target condition to drive their own work toward the next level.

This doesn’t mean that the work team is free to set any standard they like in a vacuum. This is the whole point of the daily interaction between leaders at all levels. The status-quo is always subjected to a challenge to move to a higher level. The process itself is predicted, and tested, to produce the intended quality at the predicted cost, in the predicted time, with the predicted resources. Because actual process and outcomes are continuously compared to the predicted process and outcomes, the whole system is designed to surface “unknowns” very quickly.

This, in turn, provides opportunities to develop people’s skills at dealing with these issues in near-real time. The whole point is to continuously develop the improvement skills at the work team level so we can see who the next generation of leaders are. (Ref: Liker and Convis, “The Toyota Way to Lean Leadership”)

Staging improvement as a special event, “limited time only” during which we ask people for input does not demonstrate respect, nor does it teach them to see and solve those small issues on a daily basis.

There’s more, but I’m going to stop here for now.

Summary

Creative Safety Supply clearly “gets it.” I think this page is well worth your time to read, but (and this is important), read it critically. There are actually elements of conflicting information on the page, which is awesome because it gives you (the reader) an opportunity to pause and think.

From that, I think this one-page summary really reflects the state of “lean” today: There IS NO CANONICAL DEFINITION. Anyone who asserts there is has, by definition, closed their mind to the alternatives.

We can look at “What Would Toyota Do?” as somewhat of a baseline, but ultimately we are talking about an organizational culture. Toyota does what they do because of the ways they structure how people interact with one another. Other companies may well achieve the same outcomes with different cultural mechanisms. But the interactions between people will override process mechanics every time.

Hopefully I created a lot of controversy here.  🙂

Experimenting at the Threshold of Knowledge

The title of this post was a repeating theme from KataCon 3. It is also heavily emphasized in Mike Rother’s forthcoming book The Toyota Kata Practitioner’s Guide (Due for publication in October 2017).

What is the Threshold of Knowledge?

“The root cause of all problems is ignorance.”

– Steven Spear

September 1901, Dayton, Ohio: Wilbur was frustrated. The previous year, 1900, he, with his brother’s help1, had built and tested their first full-size glider. It was designed using the most up-to-date information about wing design available. His plan had been to “kite” the glider with him as a pilot. He wanted to test his roll-control mechanism, and build practice hours “flying” and maintaining control of an aircraft.

But things had not gone as he expected. The Wrights were the first ones to actually measure the lift and drag2 forces generated by their wings, and in 1900 they were seeing only about 1/3 of the lift predicted by the equations they were using.

The picture below shows the 1900 glider being “kited.”. Notice the angle of the line and the steep angle of attack required to fly, even in a stiff 20 knot breeze.  Although they could get some basic tests done, it was clear that this glider would not suit their purpose.

image

In 1901 they had returned with a new glider, essentially the same design only about 50% bigger. They predicted they would get enough lift to sustain flight with a human pilot. They did succeed in making glides and testing the principle of turning the aircraft by rolling the wing. But although it could lift more weight, the lift / drag ratio was no better.

image

What They Thought They Knew

Wilbur’s original assumptions are well summarized in a talk he gave later that month at the invitation of his mentor and coach, Octave Chanute.

Excerpted from the published transcript of Some Aeronautical Experiments presented by Wilbur Wright on Sept 18, 1901 to the Western Society of Engineers in Chicago (Please give Wilbur a pass for using the word “Men.” He is living in a different era):

The difficulties which obstruct the pathway to success in flying-machine construction are of three general classes:

  1. Those which relate to the construction of the sustaining wings;
  2. those which relate to the generation and application of the power required to drive the machine through the air;
  3. those relating to the balancing and steering of the machine after it is actually in flight.

Of these difficulties two are already to a certain extent solved. Men already know how to construct wings or aeroplanes which, when driven through the air at sufficient speed, will not only sustain the weight of the wings themselves, but also that of the engine and of the engineer as well. Men also know how to build engines and screws of sufficient lightness and power to drive these planes at sustaining speed. As long ago as 1884 a machine3 weighing 8,000 pounds demonstrated its power both to lift itself from the ground and to maintain a speed of from 30 to 40 miles per hour, but failed of success owing to the inability to balance and steer it properly. This inability to balance and steer still confronts students of the flying problem, although nearly eight years have passed. When this one feature has been worked out, the age of flying machines will have arrived, for all other difficulties are of minor importance.

What we have here is Wilbur’s high-level assessment of the current condition – what is known, and what is not known, about the problem of “powered, controlled flight.”

Summarized, he believed there were three problems to solve for powered, controlled flight:

  1. Building a wing that can lift the weight of the aircraft and a pilot.
  2. Building a propulsion system to move it through the air.
  3. Controlling the flight – going where you want to.

Based on their research, and the published experience of other experimenters, Wilbur had every reason to believe that problems (1) and (2) were solved, or easy to solve. He perceived that the gap was control and focused his attention there.

His first target condition had been to validate his concept of roll control based on “warping” (bending) the wings. In 1899 he built a kite and was able to roll, and thus turn, it at will.

At this point, he believed the current condition was that lift was understood, and that the basic concept of changing the direction by rolling the wing was valid. Thus, his next target condition was to scale his concept to full size and test it.

What Happened

Wilbur had predicted that their wing would perform with the calculated amount of lift.

When they first tested it at Kitty Hawk in 1900, it didn’t.

However, at this point, Wilbur was not willing to challenge what was “known” about flight.

Instead the 1901 glider was a larger version of the 1900 one with one major exception: It was built so they could reconfigure the airfoil easily.

Impatient, Wilbur insisted on just trying it. But, to quote from Harry Combs’ excellent history, Kill Devil Hill:

“The Wrights in their new design had also committed what to modern engineers would be an unforgivable sin. […] they made two wing design changes simultaneously and without test.4

Without going into the details (get the book if you are interested) they did manage to get some glides, but were really no closer to understanding lift than they had been the previous year.

They had run past their threshold of knowledge and had assumed (with good reason) that they understood something that, in fact, they did not (nobody did).

They almost gave up.

Deliberate Learning

Being invited to speak in September actually gave Wilbur a chance to reflect, and renewed his spirits. That fall and winter, he and his brother conducted empirical wind tunnel experiments on 200 airfoil designs to learn what made a difference and what did not. In the process, as an “oh by the way,” they invented the “Wright Balance” which was the gold standard for measuring lift and drag in wind tunnel testing until electronics took over.

They went back to what was known, and experimented from there. They made no assumptions. Everything was tested so they could see for themselves and better understand.

The result of their experiments was the 1902 Wright Glider. You can see a full size replica in the ticketing area of the Charlotte, NC airport.

I’ll skip to the results:

image

Notice that the line is now nearly vertical, and the wing pointed nearly straight forward rather than steeply tipped back.

What Do We Need to Learn?

Making process improvements is a process of research and development, just like Wilbur and Orville were going through. In 1901 they fell into the trap of “What do we need to do?” After they got back to Dayton, they recovered and asked “What do we need to learn?” “What do we not understand?”

The Coaching Kata

What I have come to understand is the main purpose of coaching is to help the learner (and the coach) find that boundary between what we know (and can confirm) and what we need to learn. Once that boundary is clear, then the next experiment is equally clear: What are we going to do in order to learn? Learning is the objective of any task, experiment, or action item, because they are all built on a prediction even if you don’t think they are.

By helping the learner make the learning task explicit, rather than implicit, the coach advances learning and understanding – not only for the learner, but for the entire organization.

Where is your threshold of knowledge? How do you know?

 

________________________________

1We refer to “the Wright Brothers” when talking about this team. It was Wilbur who, in 1899, became interested in flight. Through 1900 it was largely Wilbur with his brother helping him. After 1901, though, his letters and diary entries start referring to “we” rather than “I” as the project moved into being a full partnership with Orville.

2The Wright Brothers used the term “drift” to refer to what, today, we call “drag.”

3Wilbur is referring to a “flying machine” built by Hiram Maxim.

4I’m not so sure that this is regarded as an “unforgivable sin” in a lot of the engineering environments I have seen, though the outcomes are similar.

The Importance of Prediction for Learning

image

One of the things, perhaps the thing, that distinguishes “scientific thinking” from “just doing stuff” is the idea of prediction: When we take some kind of action, and deliberately and consciously predict the outcome we create an opportunity to override the default narrative in our brain and deliberately examine our results.

The Toyota Kata “Experiment Record” (which also goes by the name “PDCA Cycles Record”) is a simple form that provides structure for turning an “action item” into an experiment.

Why Is It Important to Make a Prediction?

Explicit learning is driven by prediction.

Explicit Learning

“The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’ (I found it!) but ‘That’s funny …’ “

— Isaac Asimov

Curiosity is sparked by the unexpected. “I wonder what that is…”

The only way to have “unexpected” is to have “expected.”

When we consciously and deliberately make a prediction, we are setting ourselves up to learn. Why? Because rather than relying on happening to notice things are a little unusual, we are deliberately looking for them.

Deliberate Prediction: The Key to a “Learning Organization?”

Steve Spear, in his book The High Velocity Edge, makes the case that what all high-performance organizations have in common is a culture of explicitly defining their expected result from virtually everything they do.

He studied Toyota extensively for his PhD work, and discovered that rather than exploiting a “lean tool set,” what distinguished Toyota’s culture was deliberately designing prediction mechanisms into all of their processes and activities. This was followed up by an immediate response to investigate anything that doesn’t align with the prediction.

This is the purpose behind standard work, kanban, takt time / cycle time, 1:1 flow, etc. All of those “tools” are mechanisms for driving anomalous outcomes into immediate visibility so they can say… “Huh… that’s funny. I wonder what just happened?”

The High Velocity Edge extends the theory into a more general one, and we see a common mechanism in other high-performance organizations.

OK… that’s one data point on the higher-level continuum.

 

Building 214

Back in 2009 I wrote about a culture change in a post titled A Morning Market. That story actually took place around 2002-2004, and I have just re-verified (Spring 2017) that it still holds.

But it really wasn’t until this afternoon as I was discussing that story with Craig that it finally hit me. The last step in their problem-solving process was “Verification.” To summarize a key point that is actually buried in that post, they could not say a problem was cleared until they had a countermeasure, and had verified that it works.

What is that? It’s a prediction.

Rather than simply putting in a solution and moving on, their process forced them to construct a hypothesis (this countermeasure will make the problem go away), and then experimentally test that hypothesis.

If it worked, great. If it didn’t work then… “Huh, that’s funny. I wonder what just happened?”

This, in turn, not only made them better deliberate problem solvers, it engaged deliberate learning.

What is critically important to understand here is this: That verification step was not included in the problem solving process they trained on. We added it internally as part of our (then kind of rote) understanding of “What would Toyota do?” But it worked, and I believe added a level of nuance that was instrumental in keeping it going.

 

The Improvement Kata

Mike Rother’s work extends what we learned about Toyota. Going beyond “How do they structure their processes?” he went into “How do they structure their conversations?” (And “How can we learn to structure ours the same way?”)

A hallmark of the Improvement Kata, especially (but not exclusively) the “Starter Kata” around experiments, is a deliberate step to make a prediction, test it, and compare the actual outcome with the prediction.

This, in turn, is backed up in Steve Spear’s HBR articles, especially Learning to Lead at Toyota and Fixing Health Care From the Inside, Today,”  both of which should be mandatory reading for anyone interested in learning about continuous improvement.

 

You are Always Making a Prediction Anyway

Any action you take, anything you do, is actually a hypothesis. You are intending or expecting some kind of outcome.

What time do you leave for work? Why? Likely because you predict that if you leave at a particular time, and follow a particular route, you will arrive by a specified time. You might not think about it, but you have made a prediction.

If you are running to any kind of plan, the plan itself is a prediction. It is saying that “If these people work on these tasks, starting at this time, they will complete them at this later time.” It is predicting that the assigned tasks are the tasks that are required to get the bigger job done.

A work sequence is a prediction. If these people carry out these tasks in this order, we will get this outcome in this amount of time at this quality level.

A Six Sigma project is a prediction. If we control these variables in this way, we will see this aspect of the variation stay within these limits.

An “action item” is a prediction. If we take this action then that will happen, or this problem will be solved.

In all of these cases you don’t know, for sure if it will work until you try it and look for anomalies that don’t fit the model.

But in the difference in day-to-day life is we aren’t explicit about what we expect. We don’t really think it through and aren’t particularly aware when an outcome or result differs from what we expected. We just deal with the immediate condition and move on, or worse, assign blame.

What About Implicit Learning?

The human brain (and all brains, really) is a learning engine. Our experience of learning typically comes from what we perceive as feelings.

Take a look at Destlin Sandlin’s classic “Backwards Bicycle” video here, then let’s talk about what was happening.

 

There is nothing special about a “backwards bicycle.” If Destin (or his son) had no prior experience with a regular bicycle, this would simply be “learning to ride a bicycle.” What makes it hard is that, in addition to building new neural pathways for riding a backwards bicycle, he must also extinguish the existing pathways for “riding a bicycle.”

The Neuroscience of Learning (As I understand it.)

Destin has a clear (very clear) objective (Challenge) in his mind: Ride the bicycle without falling down.

As he tries to ride, he knows if he feels like he is losing his balance then he is about to fall.

He (his brain) doesn’t know how to control the wheel to keep the bike upright as he tries to ride. His arms initially make more or less random movements in an attempt to stay upright. This is instinctive, he isn’t thinking about how to move his arms. (This is what he calls the difference between “knowledge” and “understanding.”)

Whatever neurons were firing to move his arms when he loses his balance are a little less likely to fire again the next time he attempts to ride.

Whatever neurons were firing to move his arms when he stays upright for a little while are a little more likely to fire again the next time he attempts to ride.

This actually starts with increased levels of excitatory or inhibitory neurotransmitters in those neural synapses. No physical change to the brain takes place. But this requires a lot of energy. IF HE PERSISTS, over time (often a long time), the brain grows physical connections in those circuits, making those new pathways more permanent. (It also breaks the connections in the pathways that are being extinguished.)

Destin’s six year old son’s brain is optimized for this kind of learning. He creates those new physical neural connections much faster than an adult does. His brain is set up to learn how to ride a bicycle. His father’s brain is set up to ride a bicycle without thinking too much about it. Thus, Destin has a harder time shifting his performance-optimized brain back into learning mode.

All of this is implicit learning. You have something you want to learn, and you are essentially trying stuff. Initially it is random. But over time, the things that work eventually overpower the things that do not. This is also how machine learning algorithms work (not surprisingly).

 

What does this have to do with prediction?

Destin’s brain is running a series of initially random trials and comparing the result of each with the desired result. The line between a “desired result” and a “predicted result” can be kind of blurry in this type of learning. But what is critical here is to understand that learning cannot take place without some baseline to compare the actual result against. There must be a gap of some kind between the outcome we want and what we got. Without that gap, we are simply reinforcing the status-quo.

The weakness with implicit learning is it can reinforce behaviors and beliefs that correlate with a result without actually causing it. We aren’t actually testing whether our actions caused the outcome. We are just repeating those actions that have been followed by the outcome we wanted whether that is by causation or coincidence.

In the case of something like learning to ride a bicycle, that is generally OK. We may learn things that are unnecessary to stay upright on the bicycle1, but we will learn the things that are required.

In athletics, once the basics are in place, coaches can help shift this learning from implicit to explicit by having you practice specific things with specific objectives.

Moving from Implicit to Explicit

Bluntly, the vast majority of organizations are engaged in implicit, not explicit, learning. They repeat whatever has worked in the past without necessary examining why it worked, or if “now” even is similar to “the past.”

These are organizations that operate on “instinct” and “feel.” That actually more-or-less works as long as conditions are relatively stable. They may do things that are unnecessary but are also doing things that are required.

… Until conditions or requirements change.

When the organization has to accomplish something that is outside of their current domain of knowledge – beyond their knowledge threshold – those anecdotes break down. The narrative of cause-and-effect in our minds is no longer accurate.

That is when it is critical to step back, become deliberate, and ask “Where, exactly, are we tying to go?” and “What do we need to learn to get there?”

The alternative is “just trying stuff” and hoping, somewhere along the way, you get the outcome you want. The problem with that? You’re right back where you were – it works, but you don’t know why.

_______________

1Sometimes we develop beliefs that things we do can influence events that, in reality, we have no control over whatsoever. Once we develop those beliefs, we bias heavily to see evidence they are true, and exclude evidence that they are not true.